1. datalibweb – Stata module to access micro data
The World Bank
The Poverty and Equity Global Practice
Global Solutions Group on Welfare Measurement and Statistical Capacity
July 2017

2. Motivation Improve access to data databases version control
Organize micro-data in a logical and structured way
version control
Keep track of corrections, modifications or updates of the micro-data
Ability to replicate numbers calculated with different versions from the current one.
knowledge sharing
Ensure knowledge is sharing and results can be replicated
users working with micro-data at the same time
Ensure that all the users are using the same data source

3. What is datalibweb?
It is a global data system and improved version of datalib/datalib2
WEB: User registration and subscription and Management System, Web API
Stata: Core encrypted functions to authenticate registered WB users, identifies and grants access to the microdata from the server based on users’ profiles of survey subscriptions
Stata: Support to modular plugins (i.e. other related functions to process the retrieved data within user’s local machine [such as metadata on the surveys, convert to PPP, merge with different modules and raw data, etc])

4. Benefits of datalibweb
Accessibility
All users have access to the same micro-data and use by default the latest versions
When available, quick access to metadata such as questionnaires, interviewers manual, reports, among others (through Microdata Library)
Institutional memory
Ensures that all results can be replicated
Saves hard-drive space
Nobody needs to have the micro-data on their computer’s hard-drive
It is easy for quality control programing
All do-files that call –datalibweb- might be shared and run among users directly
No changes or modifications needed
No micro-data is lost
Faster communication between different units within the WB

5. Benefits of datalibweb
It is IT-independent
IT does not have to provide permissions to new users.
Users need to register and subscribe to raw/original and harmonized collections they want (one time)
Increase level of security
Users can access the microdata based on their registration and subscriptions to surveys from multiple regional and GP network drives (must access through Stata)
It works in a modular way
Each region, Global Practice is able to contribute their own collections/modules that adapt to specific needs of use
Easy management and reporting
Better management of users (monitoring the usages, and informing users about new and updated database).
Better integration of the system with other Bank applications/databases such as Microdata Library, the IW portal
Regions and GPs maintain their own collection of databases
Each region maintains and curates their databases, and ensure that users always get the latest data (real time).

6. Adoption of datalibweb is growing

7. Adoption of datalibweb is growing

8. How it works Go to FURL: datalibweb
Follow instruction to install the Datalibweb package
Subscribe to the surveys/collections you want to have access to.
Poverty GP colleagues by default have access to the Global Poverty Working Group Database (GPWG-DB).
GPWG-DB is the first global database of 1100 microdata that was used to produce the international poverty indicators and Shared Prosperity, it is maintained by the GPWG with contributions.
Raw/original and harmonized data:
Currently we have about 18 collections (both raw and harmonized), maintained by each regional statistical teams and the global team.
Highly interactive (no code) or deeply nested code
Users can use interactive interface with adaptive guided information or go straight to the code.
Working with vintages/version controls
Better control of vintages for users based on unique Survey Identifier.
Survey identification enables the user to call a specific version or vintage of the survey of choice. For example, TZA_2011_HBS_v01_M refers to “Tanzania - Household Budget Survey 2011“

9. Installing datalibweb
Manual installation:
Get the file from this link:
Copy with replacement all the files into c:/ado/plus, without changing the folder structure
Enter this line in Stata
net install datalibweb, all replace force from("
datalibweb, update(ado)

10. Adaptive guided information for datalibweb users

11. datalibweb users can also use script to access data
Typing multiple entries in “country” option produces an appended dataset of selected countries and year(s) of the chosen collection.
datalibweb, country(ALB ARM) year(2008) type(GPWG) clear
Survey ID: ALB_2008_LSMS_v01_M
Title: Living Standards Measurement Survey 2008
Link to DDI: Survey metadata (microdata portal)
Survey ID: ARM_2008_ILCS_v01_M
Title: Integrated Living Conditions Survey 2008
When raw data is requested, user has an option to choose amongst individual files within the survey.
datalibweb, country(ALB) year(2012) type(ECARAW) clear
Survey ID: ALB_2012_LSMS_v01_M
Title: Living Standards Measurement Survey 2012
Link to DDI: Survey metadata (microdata portal)
1. [datalibweb, country(ALB) year(2012) type(ECARAW) surveyid(ALB_2012_LSMS_v01_M) filename(bookbread.dta)]
...
41. [datalibweb, country(ALB) year(2012) type(ECARAW) surveyid(ALB_2012_LSMS_v01_M) filename(weights_identification.dta)]

12. Future features of datalibweb
User web interface for subscriptions and usage
Regional admin can upload subscription and maintain their data repositories
Much better performance
Local option/getfile to save and reuse the data in your machine while on mission
Interactive explore of the data availability (raw and harmonized data, documentations and dofile)
API for querying different types of files (data, dofile, questionnaire)
Graphical User Interface

13. datalibweb performance
We care a lot about performance of the system and the secured environment of your data!
Getting the catalog and a specific file each might take about 3-7 seconds, conditional on the network location, traffic usage and machine.
We are trying to reduce the time with some caching listings at the cost of real-time listings.

14. Flexible subscription mechanism
User subscriptions will be based on “Types”, which can be:
Data
Documentation
Dofile
Or anything
Within each Type, for example, you can have access to different folders of data. For example under the Data type, you can have “Data\Stata” or “Data\Base” folders, and each folder contains different stages of data harmonization.
Similarly, within Type “Documentation”, users can have access to Doc/Questionnaire, Doc/Technical, or Doc/Reports
Or you can create your own “Type”

15. Web interface for user’s subscription and usage

16. datalibweb from your laptop
Users can run datalibweb while on mission or on planes (no need to have WB intranet network)
Users need to use datalibweb to save the files into your machines – option “getfile”
Users use the “local” option – just add the “local” to the end of your normal code
The system can remember the query “getfile” and update or remove the files at the user’s request

17. datalibweb interactivity
Users can explore the availability of data and documentation for both Raw/original and harmonization, and their own subscription.

18. datalibweb interactivity – view by vintage
Users can explore the availability of data and documentation for both Raw/original and harmonization, and their own subscription.

19. datalibweb interactivity – view by available modules
Users can explore the availability of data and documentation for both Raw/original and harmonization, and their own subscription.

20. datalibweb interactivity – view by available modules
Users can explore the availability of data and documentation for both Raw/original and harmonization, and their own subscription.

21. datalibweb interactivity – view by available modules
Users can explore the availability of data and documentation for both Raw/original and harmonization, and their own subscription.

22. datalibweb Stata GUI – Country view
The graphical user interface will be replacement of Windows Explorer to see what surveys we have across surveys and what type of documentations we can access organized by Types.
It also links with user subscriptions and let users know what they subscribed to.
Users can filter the surveys by various options such as “latest version”, “data only”, “subscription status” or by keywords matched.

23. datalibweb Stata GUI – collection view
The collection view shows all the harmonized collection within a server.
It is organized by the collection type and allow you to see all the countries and vintages within that collection.

24. datalibweb Stata GUI – export survey lists
Users can export the list of surveys in the view to text or clipboard for sharing or to prepare the list of non-subscribed surveys by sub-menu options.

25. datalibweb Stata GUI – Subscription request email

26. SOL – Statistics online (future)

27. Please let us know if you have any question/suggestion.
FURL: datalibweb