End Users involvement (UniCam – Marche Region)

Table of Contents Steps/Activities for end users involvement in the building of the EPBR and SUAP Demonstrators Jun 2016 - end Demonstrator assessment: Final end users validation Feb 2014 - now BPs modeling and refining Apr 2014 - now Dissemination and exploitation activities Jul - Nov 2015 Prototype early evaluation The focus of this presentation is about all the project activities in which the end users have been involved since the beginning. [CLICK] We will scroll these activities to highlight who we involve, retracing our steps from the content production [CLICK] The disseminating events [CLICK] The gathering of end users needs [CLICK] The early evaluation of the prototype [CLICK] the design of a new graphical user interface [CLICK] till the next coming final assessment of the platform Mar 2014 - now Content production and KPIs definition Feb - Jul 2015 Users needs survey and requirements Nov 2015 - Mar 2016 A new GUI

EPBR overview European Project Budget Reporting It refers to the activities that an Italian public research body (its administrative offices) has to put in place in order to manage the administrative and reporting procedures related to the participation to an approved European research project under the FP7. The grant management BP is the reference point, trigged by the notification of project acceptance, and it is composed by related sub- processes (Periodic Report, Final Report, Manage Payment and eventually Manage Amendment). Considering an intra-organization perspective (the process does not cross the border of an italian applicant PA) we implement the case of a partner acting as a Coordinator or a Beneficiary. But first just a brief introduction of the two real use cases on which we have worked. The first case is related to the procedures for the budget reporting among European research projects, seen with an internal perspective of the proponent organization. The analysis activities were carried on, mainly, by the University of Camerino.

SUAP overview Sportello Unico Attività Produttive (One Stop Shop) It refers to the activities that the Italian Public Administrations have to put in place in order to permit entrepreneurs to set up a new company, build a factory or more generally to organize a business activity. To reduce the administrative burden, the entrepreneurs must refer to a single office (SUAP) and the contacts between them and the SUAP offices have to be undertaken completely on- line. Communications among the SUAP office and other third involved offices (in the same or different Public Administrations) are managed by the SUAP office without impacting on the entrepreneurs. It represents a very complex inter-organizational scenario, involving many PAs and it is based on 4 types of BP, declined into more than 110 sub-Processes, with further differences derived from the economic classification of the applicant target, the geographical location and other factors. The SUAP office can be placed inside a municipality or can be aggregated and provided by a consortium of municipalities. SCIA Commerciale (Business Start-up Certified Notification) SCIA Edilizia (Building Start-up Certified Notification) Titolo Unico (Standard request to start business activity) The second case study is related to the complex scenario of the administrative procedures of the public One Stop Shops (we will call them SUAP) who deal with the authorizations for setting up new companies, building factories, implementing business activities, and so on. As i told you it is a very complex scenario, because of the many variables which have to be considered, such as the types of procedures, the tasks and sub-tasks related, the laws and the geographic locations, the actors involved). For this reason a civil servant usually needs a very long apprenticeship and a deep study and practice, before he can become fully operational in a front-end office toward entrepreneurs. Variante Urbanistica (Zoning variances)

SUAP scenario for the demo For the demo we will take into consideration the learner interaction within the LearnPAd platform on the Titolo Unico BP (Standard request to start business activity) and its related tasks (assessing the instance to admit it or not, checking the answers by third parties PAs, managing a service conference meeting, and so on). In particular, for the purpouses of our demo, we will focus on the interaction of different type of users within the platform, in the context of one specific business process (the Standard Request or Titolo Unico, in italian) and some of its related tasks, such as assessing an instance or dealing with other public administrations.

Discuss facts and create abstractions Build formal representation BPs modeling D8.1 Meetings and interviews to analyze and define BPs, organizational roles and other info to be derived as models. TELLERs Tell their story FACILITATORs Discuss facts and create abstractions MODELERs Build formal representation ICT Dept. Coming back, more specifically, to the users involvement, we will go through the activities carried on about the SUAP scenario, as they have been absolutely the most time intensive and time consuming activities above all. The first involving activity has been the analysis of all the information to be derived as formal models. [CLICK] As described in the deliverable 8.1, we used a methodology for interviewing local government bodies [CLICK] with the intermediation of facilitators, able to translate the reality into abstractions [CLICK] that were needed to build more formal representations of this reality INVOLVED PA ACTORS MARCHE REGION: SUAP regional standing Board for the standardization of procedures / PA School / ICT Department / Councillor of “industry and handcrafts” / office for “simplification and liberalization of the business” OTHER LOCAL PAs: Mountain Unions Monti Azzurri (MC), Valmetauro (PU), Urbania (PU) Municipalities of Senigallia (AN), Ascoli Piceno (AP), Sarnano (MC) Territorial Service Centers and Province of Macerata and Pesaro-Urbino

Discuss facts and create abstractions Build formal representation Competence Model WP3-5 Modeling the competences within the European Qualification Framework TELLERs Tell their story FACILITATORs Discuss facts and create abstractions MODELERs Build formal representation PA School SUAP board WORK PHASES WORK ACTIVITIES BPMN Tasks COMPETENCE SKILLS (TO BE ABLE TO) KNOWLEDGE EQF LEVEL LEARNING GOALS Activities related to the execution of work The OSS checks the admissibility of the request: he verifies the completness and if it is formally and technically correct in order to the specific activity Receive Instance Deep check data Gerneral Check Data Send Comfirmation on Receipt Assess Application Verify Application Request Integration Send Rejection Assess administrative and procedural regularity through checking the completness and formal correctness of the request. Analytical skills: Identifying the needs and expectations of users Diagnostic skills: Activate control procedures of data and documentation, identifying any anomalies and irregularities, in according to sectoral laws. Sotfware used by the specific OSS for the formal control. Legislation related to the administrative documentation (445/2000). Regulation on Certified Notification of Starting Activity (Law 241/1990) English language 4 Check regularity/irregularity of the requests proper management of user requests In the same way, as a result from collaborating within the workpackages 3 and 5 [CLICK] we engage public administrations in describing their everyday routine, [CLICK] as it could be finally mapped [CLICK] into work phases and job activities related both to the business process tasks and to the competence model we derived from the EQF, the European Qualification Framework.

KPIs KPIs definition and measurement (organizational and learning goals) WP5-8 TELLERs Tell their story FACILITATORs Discuss facts and create abstractions MODELERs Build formal representation PA School ICT Dept. LEARNING or BUSINESS GOAL KPI-No KPI PRIORITY SOURCE TYPE of KPI Aggregation HOW TO MEASURE UNIT Period LEARNING GUIDANCE Acting Responsibly L1 Acting autonomously on one's own responsibility high KPIcapture.xlsx individual Org Unit % cases performed by a learner without help from experts >=70% >=50% <70% < 50% percentage 30 days try to simulate already performed case without requesting help to experts Intensive use of Learn PAd platform B4 total no of valuable comments LearnPAd system PA total no of valuable comments of members of OU related to all comments provided by members of OU your group need to improve the quality and valuability of the comments provided [CLICK] Another important interaction with our end users has driven to the definition of the learning KPIs and the measurement of the degree of achievement of organizational and learning goals. This was due at the end of this month (so it is not still integrated in the demo you will see today).

D8.3, WP3-4-7-8 WP5-6-8 Other content models refining and text optimization content management: providing documents, metadata and real cases x simulation & recommender – similarity reasoner ICT Dept. SUAP board WHOLE CONSORTIUM Content Providers Content Managers WP5-6-8 In the last phases of our content production activities for the WPs 4-5-6-7-8, we enlarge a bit the audience of public sector bodies, [CLICK] primarily to refine the models and optimize the textual contents we had produced, [CLICK] by arranging further meetings between local PAs and the project partners; [CLICK] secondly to provide real cases documentation [CLICK] in order to feed the knowledge repository [CLICK] both for the simulation practical exercises and for the semantic functionalities of the recommender and its similarity reasoning engine. ICT Dept. PA School

Dissemination and Exploitation WP9 involved actors with tools / events Web/Social media users In another direction, we tried to involve various stakeholders in our dissemination and exploitation activities and plans. [CLICK] We turned to both users of social media and the web [CLICK] and key players representing local and central government bodies, entrepreneurs, academic institutions [CLICK] and the research and education community. But this will be better explained in the afternoon. LEARNPAD WEBSITES (wiki.learnpad.eu – testbed.learnpad.eu – www.learnpad.eu) PARTNER WEBSITES (istitutional eg www.regione.marche.it, thematic eg ecommunity for Marche Digital Agenda) 5-6/6/2014 event “Innovattori”, at Civitanova Marche MC http://www.impresa.marche.it/Ricercaeinnovazione/Innovattori.aspx 10/09/2015 Presentations at practitioner conferences - LearnPAd project presentation to Marche local public administrations attended by 55 persons 14/10/2015 Insertion of the LearnPAd project as a best practice in the digital skills Coalition database managed by AgID - Presidency of the Italian Council of Ministers (http://competenzedigitali.agid.gov.it/) 22/10/2015 Presentation of the initiatives and projects of the Digital Agenda of Marche - supporting the digitalization of Public Administrations - to the students of the course "Jean Monnet" 2015 "The Network, Citizens and Rights" (third edition – module held by prof. S. Di Minco) Regions of Emilia-Romagna, Lazio, Tuscany, Umbria that, with Marche, signed the protocol "Italia mediana" Stakeholders

D1.1, D1.2, D8.1 addendum User needs main results gather from the user elearning needs and competences SURVEY (>100 PA employees interviewed) -> requirements finalization Directors / BP responsibles (Modelers) Back Office – Front office operators (Learners) How to represent BPs ? Available info ? Frequent issues ? How to solve ? After the feedback we received during the previous review, we set up a detailed questionnaire to gather end users needs, expectations and starting points around the main project themes (for istance: training and learning issues, business execution, workflows organization, familiarity with ict tools and so on). [CLICK] We focused on two distinct groups: on one side public officers and managers responsible for the procedings, who are in charge of organizing the working activities (so, in a certain sense, the «modelers» or their advisors) on the other side the employees who deal with the daily working routine, who are the civil servants that need to learn how to move within the processes (so the learners). We collected a lot of useful information from more than 100 interviewed persons. For instance: 1 [CLICK] what are the frequently faced issues when you need to perform actions within the process execution 2 [CLICK] what are the main solutions adopted to solve those problems 3 [CLICK] what is the best intelligible way of representing BPs 4 [CLICK] how often and where knowledge is at your disposal when working at a process task, and so on This material was also merged to finalize the end users platform requirements Further questions: how to identify experts – how to share each own notes or documents – how to mix more than one training tool/approach – how to simplify learning about complex procedures Questionnaire about PA end users learning needs (IT https://docs.google.com/forms/d/1HvjJwiyDpJwoS_Tgqi4R9qVaDXFCunPkg6d5wWzRzFs/viewform - EN https://docs.google.com/forms/d/15FD4s7FwV1tUonslZYRHcySJ4iDEexdbNHrKJfo6u9Y/viewform) and answers/results http://www.slideshare.net/AndreaSergiacomi/learnpad-project-results-for-the-public-survey-on-learning-challenges-in-public-administrations

Platform early validation platform early validation (Discussion on a focus group of 6 experts & Questionnaire for quantitative validation by a group of 7 end-users) usability, UI layout, multimedia, innovative e-learning tech, learning model, interactivity, gamification, … ICT Dept. SUAP board PA School functionalities Focus group End users group #1 From July to November two-thousand-fifteen we had the first platform validation [CLICK] We involved users both from Marche Region and UNICAM and from local PAs. At first we discussed within a focus group of experts about all the relevant aspects of the prototype. Then a group of users tried and tested the platform. [CLICK] At the end they answered a questionnaire about the platform usability and its layout, its learning effectiveness [CLICK] its features, modules and functionalities, and so on The focus group aimed at gathering information about the following items: Usability of the platform / Innovative aspects in comparison with e-learning technologies / Efficacy of the proposed learning model in relation to the end-users profile / Multimedia solutions / Gamification solutions / Interactive solutions / General layout of the interfaces and of the functionalities

Platform early validation - feedback Feedbacks provided by the prototype end users The emerging feedbacks were shared and presented to all the partners, in order to correct errors, to find better solutions to address what the end users required, and to highlight the best innovative features and strenghts that users had appreciated. Feedback shared with the partners https://docs.google.com/spreadsheets/d/1wZMghpZ77Z801euU6FNkkL55OidY4ujjWegPQkF3MLA/edit#gid=0 http://www.slideshare.net/AndreaSergiacomi/learnpad-prototype-early-validation-feedbacks-from-end-users

Web developers / editorial staff User Interface D9.5, WP2-9 user interface design accessibility test ICT Dept. Web developers / editorial staff Some of the requests that emerged from the platform early evaluation were related to the fact that a better Graphical User Interface would have helped the understandability and the interaction with the provided contents and functionalities. [CLICK] For this reason a group of web experts from Marche Region and X-Wiki (developers, designers and people who deal with web content management and editing for web sites) provided a draft of a possible new layout for the platform interface. Recently this new GUI has been partially implemented in the demo environment. [CLICK] They have also made a test of accessibility for the web pages, trying to assure the compliance with the W3C guidelines and with the requirements descending from the italian laws (l. 1/2014).

D8.4 & final review Next steps final validation plan: 30 end users divided into 2 groups Course in presence (G#1) or Platform usage (G#2) + same Questionnaire As we are almost near to the end of the project, we have started to plan the final validation of the platform. The basic idea is to teach the same detailed aspects related to the execution of a SUAP business process, both during a class lesson in presence and using the LearnPAd platform, to two different groups of end users. After the courses, the two groups will have to answer the same questionnaire to evaluate the respective training model effectiveness and their own specific learning improvements. [CLICK] From June and even after the release of the deliverable 8.4 (so, till the next review in October) we will try to involve at least 30 learners, belonging not only to the already so far involved PAs [CLICK] but also to other SUAP, municipalities and local government entities who are collaborating in other regional related projects. [CLICK] In the final validation we are also arranging the participation of a consultant from the Economic Development Minister, who is involved in the SUAP reformation process at national level (Ing. Carlo Apponi). Local Gov Bodies - End users Group #1 Local Gov Bodies - End users Group #2

Any question ? If not we can quickly go on with the next introduction while Guglielmo is setting up the demo.