Lithuanian case: The challenges of user friendly questionnaire and data validation Laura Perevičiūtė

Collecting data Participant’s questionnaire form User friendliness Calculating the indicators Completeness and validation checks

Collecting data Data for output is being collected on the first day of participation Data for immediate result indicators is being collected 28 days after the end of participation Beneficiary collects the questionnaires and enters the data to IT system.

Participant’s questionnaire form Information on a project participant’s gender, status in the labour market, education and belonging to vulnerable groups General information on a project participant (name, contact information, date of birth) Start date Information on project participant’s compliance with the additional criteria

Challenge: a user friendly questionnaire Co-working with NGO‘s to make the questionnaire more understandable (especially household situation questions) A detailed guideline that helps to fill in the questionnaire Closed questions Minimize the scope of information asked Aiming to put as much burden on the IT system as possible

Employment status Employed Self employed (a footnote with examples) Employed on other grounds (employment contracts, civil servant, etc.) Unemployed 6 months 6-12 months More than 12 months Inactive Unemployed student or pupil Unemployed person who is not in education (examples)

Education status Education you already have 1 group: primary (4 grades) or lower secondary education (10 grades) 2 group:upper secondary (12 grades) or professional qualification 3 group: With tertiary education (bachelor, master, doctor degree) Without primary education( a grown up person without primary education) Your current participation in trainings At the moment participating in education programs, trainings, learning courses, etc. At the moment not participating in education programs, trainings, learning courses, etc.

Belonging to vulnerable social groups Everyone at home is unemployed Everyone at home is unemployed and there are dependent children Migrants, people with a foreign background, minorities (including marginalised communities such as the Roma) Disabled A person who doesn‘t have a home or who is at risk at losing it Single parents with dependent children A person who belong in other vulnerable groups (examples) A person who doesn‘t belong to any vulnerable group Not providing information

Challenge: a user friendly quationnaire Date of birth Unemployed for: 6 months 6-12 months More than 12 months Long term unemployed

Calculating the indicators One national level IT system managed by managing authority The micro data is being entered by the beneficiary It may be seen only by the employees of the implementing agency who are coordinating the implementation of a project – data protection issue The IT system calculates indicators: a participant is included if it matches a set of characteristics.

Example (1) Common product indicator „Above 54 years of age who are unemployed, including long-term unemployed, or inactive not in education or training“ includes all participants who match one of these sets of characteristics: Above 54 + unemployed up to 6 months Above 54+ unemployed 6 to 12 months Above 54 + unemployed more than 12 months Above 54 + inactive not in education or training

Example (2) Common product indicator „Long-term unemployed“ includes all participants who match one of these sets of characteristics: Up to 25 years old + unemployed 6 to 12 months Unemployed more than 12 months

Completeness and validation checks Who? When? What is being checked? IT system Beneficiary entering data into IT system Completeness of data Validation tests on micro level In case of erroneous data IT system informs about critical error (the data about a participant can not be saved in the system till the data is corrected. Implementing agency Payment claim check Eligibility of expenditure In case of erroneous data agency returns payment claim to beneficiary and asks to correct the data. If beneficiary does not correct the errors, expenditure may be declared ineligible. Planned on spot checks Data corresponding with the questionnaires In case of erroneous data beneficiary may be asked to correct the data. In case of infringement, expenditure may be declared ineligible and recovered. Participants are connected to expenditure eligibility. Data is being transferred to indicators only after the expenditure is verified by implementing agency Spot checks include evaluating if the data in IT system correspond with data provided in the questionnaires forms Validation tests at aggregate level are being carried out automatically by IT system Second evaluation of data completeness and validation is being implemented while writing annual implementation report

Completeness and validation checks Who? When? What is being checked? Implementing agency Unplanned on spot checks The process of collecting data from participants In case of infringement, a beneficiary may be asked to correct their procedures, a financial correction may be applied. IT system Calculating indicators Validation tests at aggregate level In case of erroneous data IT system informs implementing body about the inconsistencies of the data. If it is not an IT problem then implementing body identifies a specific project that has erroneous data and informs beneficiary about inconsistencies. Managing authority Annual implementation report Completeness of data MA informs implementing body about inconsistencies of the data. Implementing body takes actions described in the step above. Participants are connected to expenditure eligibility. Data is being transferred to indicators only after the expenditure is verified by implementing agency Spot checks include evaluating if the data in IT system correspond with data provided in the questionnaires forms Validation tests at aggregate level are being carried out automatically by IT system Second evaluation of data completeness and validation is being implemented while writing annual implementation report

To sum up Direct contact with the participant User friendly questionnaire Calculating the indicators by combining the data collected Complete and valid data ensured by different checks

Thank you for your attention