Planning for adoption of electronic data collection technologies Session 11 Planning for adoption of electronic data collection technologies UNSD presentation

Overview of the presentation Paper-based vs electronic data collection approaches Advantages and disadvantages of using electronic data collection technologies Transitioning from paper to electronic data collection Decision making Planning considerations Implementing handheld electronic devices for data collection Implementing Internet for data collection

Paper-based vs Electronic collection approach Data Collection Data Entry/Capture Data Coding/Editing Paper-based approach Manual Key entry/computer-assisted OMR/OCR/ICR Manual computer-assisted Data Coding/Editing (partial) Data Collection/Entry/Coding/Editing Electronic collection approach Automated/built-in Manual computer-assisted

Advantages of using electronic data collection technologies Allow for the collection of much of the same information collected via paper-based approaches, AND that of information that cannot be collected using paper-based approaches, such as: GPS co-ordinates, pictures, videos, date/time stamps, etc Improve data quality through built-in consistency checks and logical validation rules, allowing errors and inconsistencies to be resolved at interview time Allow a more accurate progression through the questionnaire due to automated skip patterns and mandatory items. Avoid manual data entry errors; avoid problems due to illegible writing Optimize time spent by enumerator and respondents Improve field management and real time monitoring of enumeration activities (eg. built-in date and time stamp could be useful for supervision) Reduce time and costs by eliminating a separate process for data capture, automated coding, and reducing the volume of post-enumeration data checking. Decrease time between data collection and release of the results.

Advantages of using electronic data collection technologies Reduce costs of data capture, questionnaire printing, storage, and transportation. However, the increased equipment costs may outweigh the savings Allow the provision of on-screen help - for enumerators from the screen of electronic device. A help feature can makes it easier for the enumerators to access definitions or other items needing clarification during the interview. Unlike in paper manual, the help feature in an electronic questionnaire can be linked to each question or a particular term that often needs clarification. Allow computerized case management, capacity to monitor and control over various census operations Can utilize GPS features and digital mapping capabilities. For example: Captured GPS locations can be used as a reference point for future census activities, eg. help track location of data entry EA maps and/or address information can be loaded onto the device and even aerial or satellite photos to help the enumerator find the correct housing units to visit.

Disadvantages of electronic data collection technologies There are a number of risks or drawbacks associated with the introduction of technology that should be considered and managed, otherwise these could lead to increases in expenditure, delays to the census timetable or impacts on the quality of the census. Technology increases dependence on technology providers and introduces new challenges and risks Large investment (eg high equipment costs with limited long-term use) More time needed during the preparation stage (eg. programming application, setting up system, testing) Needs more skilled programmers who are able to do sophisticated programming specific to the mobile device (e.g., Android) and all the necessary components of an electronic questionnaire application. Requires technologically skilled enumerators with more training and field support.

Disadvantages of electronic data collection technologies There are more infrastructure constraints to consider. Electricity needs to be available to charge the devices; which is challenging in areas with limited electricity. Needs a system to transfer the data from the handheld devices in a timely and secure manner. This could be the cellular network, Internet, local area network, or through manual data transfer. Limited Internet availability may cause difficulties in data transmission and other functions of the mobile data capture software application that rely on the Internet. A realistic assessment of Internet connectivity in all areas should be conducted during the planning stage. Data security must be assured during data collection, transfer, and storage. Needs a system for backing up the data to prevent data loss.

Transitioning from paper to electronic data collection Paper-and-Pencil interviewing Handheld devices Paper-based data collection Self-administrated questionnaire Internet

Plans for 2020 round - Southern African countries Name of country Botswana Malawi Namibia South Africa Uganda Zambia Zimbabwe 9 Which methods of enumeration do you plan to use for field based data collection in the 2020 round? 3 Personal face-to-face interview, using electronic questionnaire (CAPI) 2 Personal face-to-face interview, using paper questionnaire, 3 Personal face-to-face interview, using electronic questionnaire (CAPI) 2 Personal face-to-face interview, using paper questionnaire, 3 Personal face-to-face interview, using electronic questionnaire (CAPI), 5 Personal telephone interview, using electronic questionnaire (CATI), 8 Self-completion of paper questionnaire by respondents (delivery/collection by enumerator), 9 Internet self-response option (CAWI), on-line, 10 Internet self-response option (CAWI), off-line (using downloadable form)

Decision making The decision on whether to use electronic data collection technologies should begin in the initial census planning stages -- the decision should be done in consultation with all stakeholders and with a clear understanding of the technology To the extent possible, the determination as to whether digital collection meets the needs of a census operation, should be based on information generated by hands-on experience and field tests; best practices from other statistical collections/organizations/countries could also be useful in making a decision The decision requires taking into consideration a number of critical factors, including: National context size of country in terms of area, population; diversity, socio-econ characteristics; investment in tech required by law? Institutional factors Policy environment, support of government/senior managers, org culture Technological factors (ICT infrastructure, extent of existing investment in tech) Socio-economic factors (literacy, public trust, cultural diversity, etc)

Information required for decision-making Operational aspects # of modes being used; # of days to complete survey; # of enumerators and EAs; complexity of survey (# of questions); Stakeholders’ requirements Resources Cost estimates; total cost of ownership (TCO) Benefits “why” use technology: data quality, improved coverage, timeliness, response burden, efficiency, cost savings, etc. Risks impact on/risks to: schedule (eg. testing plan); budget (positive & negative); changes to business process; field staff considerations; procurement; technical; managerial; impact on other statistical processes; integration into existing systems; etc.) Timetable for procurement, system development, testing, training, enumeration, etc. Plan for reuse/disposition of devices

Planning considerations for electronic data collection In order to ensure the success of the census, it is critical to identify all requirements for carrying out the census using electronic data collection and develop plans early in the census life cycle. Census timetable - generally, more time is needed to develop and test the collection application, set up the data transfer and processing systems, and procure, program, and test the devices. If edit checks are to be added to the application, the edit programming must be completed before the enumeration begins, rather than programming those separately in the post-enumeration stage. Furthermore, time should be allotted for training the enumerators since the training must include the use of the devices. Therefore, it is critical to determine all the steps needed to take place in order to set up an electronic data collection system and allow sufficient time prior to the enumeration in the census timetable.

Planning considerations for electronic data collection Budget Considerations - Using an electronic questionnaire may save costs in the printing of the questionnaire and data capture, but the costs of the electronic equipment may be higher than the savings. Carefully consider all costs of conducting mobile data capture and account for them in the budget. Infrastructure Considerations - Infrastructure issues such as availability of electricity and Internet access can affect the success of mobile data capture. Early in the planning stage, identify places that lack electricity and/or Internet access. If relying on a cellular or Wi-Fi data connection to transmit data, it is important to research the speed of the data transmission and plan accordingly. Mapping the areas without electricity and/ or Internet access is helpful in planning. Plans should be developed for charging and backing up the devices. It is critical to have contingency plans for when electricity and/ or Internet access are not available.

Planning considerations for electronic data collection Questionnaire development process - As with a paper questionnaire, developing an electronic questionnaire is an iterative process. It must be developed, tested, revised, and then tested again, repeating the cycle until the questionnaire works as intended. When designing an electronic questionnaire, it is important to make sure that the subject matter specialists work closely with the programmers so that they do not lose control over the wording, layout, and design of the instrument. Need to build partnerships with: Application development partners Device manufacturer (to provide the devices as per specification) Connectivity providers (to provide connectivity for the device so that the data can be transferred seamlessly to the data center) Capacity building partners (training on using not only the forms and the entire process of data collection but also on the basics of the device and what to do for trouble shooting).

Implementing handheld electronic devices

Considerations for selection of handheld electronic devices Important features that could impact the choice of device include: Portability – weight, size, ease of use and transport Battery life – battery life should be sufficient to perform several hours of fieldwork without recharging Screen size, resolution and visibility – preferable to use a monochrome screen not easily affected by bright sunlight Processors and random-access memory (RAM) – for better performance and rapid navigation, a faster processor and a good amount of storage are needed Stability of the operating system Data storage and backup possibilities – external data cards and flash drives (SD/MMC cards) for adequate storage and for backing up data Connectivity options for data transfer (such as cellular communication, WiFi, Bluetooth, USB and other types of connectivity between devices) Ruggedness and durability in field conditions (such as hot and cold temperatures, rain, sand storms, dust, heavy wind) In-built functions (e.g. GPS receiver, camera, voice recorder, etc) Location accuracy (with respect to geographic location) for fieldwork application, especially if GPS functionality is required Cost

CAPI software - Essential functional features CAPI software packages should be evaluated to assess whether they are robust enough in performance and broad enough in functionality to support a census op. The evaluation criteria for performance assessment of each CAPI software package should include the following desirable characteristics and functionalities: User-friendly development environment for creating, modifying, and updating the survey instrument Simple but powerful interface Data capture and quality control Questionnaire navigation Skipping/branching Case management Data management, transfer and export facilities Support and documentation

Testing process for evaluating CAPI software The test process for evaluating software should include at least the following steps: Obtain test copies; Develop test prototypes for testing ability of CAPI to satisfy key functionality requirements; Detailing implications on the organization’s existing computing environment; Gauging user satisfaction – assess whether the supplier is responsive and provides reliable support mechanism Conduct tests according to previously established criteria; Assess and document upgrade policy; Determine full costing; Produce a report on the evaluation process.

CAPI – build, buy or freeware The build versus buy decision is a critical one - choosing incorrectly could result in an inadequate solution or a poor return on investment Some considerations in any build-versus-buy decision: Will building your own solution deliver a unique advantage that helps you achieve business objectives? What is the time available to enumeration? -- Purchasing a solution eliminates the software development process, leaving only the implementation, testing and deployment phases to complete. Are there sufficient internal programming resources? -- Building your own solution requires a set of developers with the right coding skills and the technical ability to integrate your solution to back-end systems. If NSO does not currently have employees with this skill set, it might make sense to buy or get freeware Total cost - In most cases, implementing a packaged data collection application will be more cost-effective than building it from scratch. When assessing cost it is important to look at Total Cost of Ownership (TCO)

Implementing handheld electronic devices DATA SECURITY, TRANSMISSION, AND STORAGE CONSIDERATIONS An important consideration is how to transmit data securely from electronic devices to a central server. Secure data storage is also critical at every level where data are stored Most electronic devices can store data either locally or on a network. Storing data locally risks data loss if the device is lost or stops working. Storing data on a network requires a way to transfer the data securely from each device to the main server. Some ways to transfer data securely include: Secure Internet connection; Local area network (LAN), a computer network that connects computers and mobile devices within a limited area (such as an office building); Bluetooth or Wi-Fi that connects tablet PC to tablet PC; Cable connection (for example, connecting an enumerator’s tablet PC to her supervisor’s laptop); Wide area network (WAN), or a private network that extends beyond the geographic reach of LAN, but is not connected to the Internet (for example, a dedicated mobile network).

Implementing handheld electronic devices It is important to consider what is feasible in all areas of the country when setting up a system. Reliable Internet may be available in cities, but not in rural areas. It may be necessary to establish more than one system according to infrastructure constraints. For example, data may be uploaded to the server through secure Internet connection in cities, but in rural areas, data may be transferred using cable connection. It is important to allocate time and resources to test the data transfer and storage system as failure to do so may have costly consequences.

Implementing handheld electronic devices COMPUTERIZED FIELD CASE MANAGEMENT One feature of using an electronic questionnaire is that it can be linked to a computerized field case management system. As data from each device are sent to a central database, the data can be used to monitor the progress of the enumeration and identify which households the interviewers need to visit. Quality metrics and audits also can be incorporated in the field case management system. Field operations and programmers should work together to ensure that the case management system meets the needs of the field operations unit and be able to take full advantage of a computerized case management system.

Implementing handheld electronic devices STAFF SKILL REQUIREMENTS Extensive training will be needed for enumerators and supervisors. Training will be needed to familiarize them with the technical aspects of the data collection process, such as how to operate a tablet PC, transmit data, and navigate through the application. Field supervisors may have fewer quality check responsibilities, since most of the completeness and consistency checks will be done by the data collection software. However, they will be required to conduct field case management and reporting using a computerized system. So they need training too. Help desks at headquarters and field technical support staff may be needed to handle technical queries and troubleshoot problems during data collection. In addition, existing programmers may not have the skills to program the questionnaire application and set up extensive data systems, requiring further training or hiring new staff.

Implementing handheld electronic devices Other factors to consider include: Integration with geospatial tools/digital maps to assist enumeration and monitoring Procurement and IT acquisition management (understanding TCO, developing specifications, financial rules/standards ) Logistics for storage, distribution and return of handheld electronic devices and accessories (eg. asset tagging for traceability; transportation; etc.) Device use policy (eg. authorized uses; misuses; proper handling (eg. during inclement weather); linking final payment to enumerators to return of device; etc.) Reuse/disposition of devices

Implementing Internet based data collection

Implementing Internet based data collection The main advantage of Internet-based data capture is that it can be cost effective since enumerators are not used. It can also be used for residents who are hard to reach in person. However, in order for it to be cost effective, there needs to be a large number of people with reliable access to the Internet. Another disadvantage is that a highly sophisticated control system is required to avoid duplications and undercounting and to ensure the security of the data.

Implementing Internet based data collection Requirements for data collection with Internet include: Reliable information and communication technology (ICT) infrastructure High rate of internet penetration/access coverage High literacy rate Public trust and acceptability of internet self-enumeration by the population Making reasonable assumptions and projections for response/take-up rates Availability of address/building/dwelling registers/lists (eg. to organize enumeration, generate unique access codes, avoid duplication, contact respondents, etc.)

Internet collection - data collection application and portal Most of the questionnaire design considerations for CAPI are also applicable to Internet-based data capture – in terms of features and functions of data collection application (navigation, consistency controls, validation, etc.) as well as in terms of the considerations for the design of data collection application Additional requirements include the need to: optimize the application for use on variety of devices, screen resolutions, operating systems, browsers provide instructions for respondents on the portal Provide respondent authentication procedures testing IT infrastructure stability, security and capacity to handle anticipated response loads

Internet collection Contact and communication strategies to motivate Internet response Push/pull methods (Pull: publicity campaigns, incentives; Push: internet first, face-to-face later) Contact approaches (letters, post cards, FAQs brochures, e-mails, texts, phone calls, etc.) Contact approaches for targeting difficult to reach demographic groups and geographic areas Communication/publicity to promote response (eg. targeting, multi-channel outreach, national and local partnerships, awareness campaigns via traditional and new media, etc.) Support to respondents Instructional and contextual help materials for completing online questionnaire, particularly for “difficult” census questions Provide census questionnaire assistance via call centres/toll-free telephone helpline, social media Partnership with community facilities (schools, libraries, other public facilities) to set up PCs and Internet to facilitate response by those without personal Internet access

Conclusions Electronic data collection technologies are being recognized as an option worth considering in the 2020 round of population and housing censuses The adoption of electronic data collection technologies require early planning and through preparation including prototyping, use in small survey projects, and pilot exercises in order to validate all the stages and anticipate alternative solutions As for any new technology-based approach, there is a strong need to seek partnerships Maintaining data security and confidentiality is also of paramount concern Successful adoption of new technologies requires managers to have strong project management skills and ability to anticipate potential challenges and thinking ahead about alternative ways to solve problems