Data Upload & Management Second Generation Live Database (LDB) Training Manual Data Upload & Management Dbms_copy - is a tool to copy data between SQL databases DTS - import data from another format into the databases. It’s a SQL tool. Equation Validation - a World Bank tool that is used to check the validity of the equation within the database Data Uploading & Managing

Contents Overview of the Data Input – this section looks at the architecture of the LDB system. Tools Available for Data Input – this section looks at the upload tools that are available for populating the LDB. Dbms_copy - is a tool to copy data between SQL databases DTS - import data from another format into the databases. It’s a SQL tool. Equation Validation - a World Bank tool that is used to check the validity of the equation within the database Data Uploading & Managing

LDB Architecture - Input Web Query LDB Uploading & Managing Tools Web Server Query & Analysis Data Sources LDB Staging Area LDB Production Data Administration (Quality) This diagram shows the input side highlighted. This indicates that the following slides will be dealing with data input. Reports Library Cube Data Uploading & Managing

Introduction As can be seen from the diagram, the LDB system can be divided into three main components of input, core components and output. This training is concerned with the input of data into the LDB system using various tools that are available for this purposes. The purpose of data upload is to upload time series data into the database. This application needs to access your database through an OBDC connection. This connection is created by configuring a Data Source Name (DSN), via the ODBC Data source icon in the Control Panel. Your database administrator must have granted a read and write access to the database through a username and password. The data is entered into the system using Excel based spreadsheets. Dbms_copy - is a tool to copy data between SQL databases DTS - import data from another format into the databases. It’s a SQL tool. Equation Validation - a World Bank tool that is used to check the validity of the equation within the database Data Uploading & Managing

Data Uploading using Excel Spreadsheet Open the LDB folder Open the Upload Tools folder Open the upload spreadsheet named 2gldbuPLOADHF.xls The following functions are available through push buttons on the screen. Clear – clears the whole workspace, for a new upload Update – launches the application Help – shows a short help about the colours used for the series Exit – Quits the application Apply Scale – applies the scale inserted in the scale columns to all the figures to the right. Dbms_copy - is a tool to copy data between SQL databases DTS - import data from another format into the databases. It’s a SQL tool. Equation Validation - a World Bank tool that is used to check the validity of the equation within the database Data Uploading & Managing

Data Uploading using Excel Spreadsheet To enter data into the system Open the upload spreadsheet named 2gldbuPLOADHF.xls Enter the series of data to be uploaded starting at line 3 The values that can used in the system can be obtained using the Custom Management Tool or ProClarity. Please refer to these sections for these values Enter an ‘X’ or Ý’ if you want to replace existing data in the system, otherwise leave it blank Enter the country code under the country heading e.g. ZMB for Zambia Dbms_copy - is a tool to copy data between SQL databases DTS - import data from another format into the databases. It’s a SQL tool. Equation Validation - a World Bank tool that is used to check the validity of the equation within the database Data Uploading & Managing

Data Uploading using Excel Spreadsheet Enter the Sub National area code, otherwise it will be defaulted to GLB Enter the source code if no value is input it is defaulted to 1, which is World Bank Enter the version, if no value is input it is defaulted to 1 which is nominal Enter the owner, if no value is input it is defaulted to DBA Enter the code for the data you want to enter Enter the scale, if no value is input it is defaulted to zero. Dbms_copy - is a tool to copy data between SQL databases DTS - import data from another format into the databases. It’s a SQL tool. Equation Validation - a World Bank tool that is used to check the validity of the equation within the database Data Uploading & Managing

Data Uploading using Excel Spreadsheet To enter the different frequencies of data into the database, the time series needs to be changed. For example, the diagram to the right shows how to enter yearly data starting with 1987, 1988 etc. Dbms_copy - is a tool to copy data between SQL databases DTS - import data from another format into the databases. It’s a SQL tool. Equation Validation - a World Bank tool that is used to check the validity of the equation within the database Data Uploading & Managing

Data Uploading using Excel Spreadsheet To enter semester data, the time series will be changed to say SA0290 meaning the second semester of year 1990 or SA0101 meaning the first semester of the year 2001 The diagram on the right shows an example with the first entry being the first semester of 1990. Dbms_copy - is a tool to copy data between SQL databases DTS - import data from another format into the databases. It’s a SQL tool. Equation Validation - a World Bank tool that is used to check the validity of the equation within the database Data Uploading & Managing

Data Uploading using Excel Spreadsheet To enter quarterly data, the time series will be changed to say QT0302 meaning the third quarter of the year 2002, or say QT0189 meaning the first quarter of the year 1989. The diagram on the right shows an example with the first entry being the first quarter of 1990. Dbms_copy - is a tool to copy data between SQL databases DTS - import data from another format into the databases. It’s a SQL tool. Equation Validation - a World Bank tool that is used to check the validity of the equation within the database Data Uploading & Managing

Data Uploading using Excel Spreadsheet To enter monthly data, the time series will be changed to say MO0901 meaning September 2001, or say MO0102 meaning January 2002. The diagram on the right shows an example with the first entry being the first month of 1992. Dbms_copy - is a tool to copy data between SQL databases DTS - import data from another format into the databases. It’s a SQL tool. Equation Validation - a World Bank tool that is used to check the validity of the equation within the database Data Uploading & Managing

Data Uploading using Excel Spreadsheet To enter weekly data, the time series will be changed to say WK5200 meaning the 52nd week of 2000 or say WK0601 meaning the 6th week of 2001. The diagram on the right shows an example with the first entry being the first week of 1999. Dbms_copy - is a tool to copy data between SQL databases DTS - import data from another format into the databases. It’s a SQL tool. Equation Validation - a World Bank tool that is used to check the validity of the equation within the database Data Uploading & Managing

Data Uploading using Excel Spreadsheet To enter daily data, the time series will be changed to say DY00101 meaning the first day of the year 2001 or DY30088 meaning the day 300 of the year 1988 The diagram on the right shows an example with the first entry being the first day of 2002. Dbms_copy - is a tool to copy data between SQL databases DTS - import data from another format into the databases. It’s a SQL tool. Equation Validation - a World Bank tool that is used to check the validity of the equation within the database Data Uploading & Managing

Data Uploading using Excel Spreadsheet Please note that when you update the database with new values, these values override the existing ones. Enter the values that you want to enter into the spreadsheet as shown in the picture on the right Dbms_copy - is a tool to copy data between SQL databases DTS - import data from another format into the databases. It’s a SQL tool. Equation Validation - a World Bank tool that is used to check the validity of the equation within the database Data Uploading & Managing

Data Uploading using Excel Spreadsheet Enter the Driver Name which is ÁFRLDB’ Enter the User Name which is ‘sa’ Enter the password which ‘pass’ Press ÓK’ Dbms_copy - is a tool to copy data between SQL databases DTS - import data from another format into the databases. It’s a SQL tool. Equation Validation - a World Bank tool that is used to check the validity of the equation within the database Data Uploading & Managing

Data Uploading using Excel Spreadsheet A message should appear saying that the data has been uploaded. This is shown in the picture to the right Dbms_copy - is a tool to copy data between SQL databases DTS - import data from another format into the databases. It’s a SQL tool. Equation Validation - a World Bank tool that is used to check the validity of the equation within the database Data Uploading & Managing

Manipulating Data using the Custom Management Tool The Custom Management Tool can be used to load data into the system. Click on the icon and enter the following data. Enter the Driver Name which is ÁFRLDB’ Enter the User Name which is ‘sa’ Enter the password which ‘pass’ Press ÓK’ Dbms_copy - is a tool to copy data between SQL databases DTS - import data from another format into the databases. It’s a SQL tool. Equation Validation - a World Bank tool that is used to check the validity of the equation within the database Data Uploading & Managing

Manipulating Data using the Custom Management Tool To edit data that is already in the system follow these steps Click on Files and Time Series The screen shown on the right will appear. Select the country that you want to work with If you want to search for a CETS code, type in the code or part of the code in the series field and click in another field The system will return a maximum of number CETS codes that match your search, that are indicated in the ‘Top’ field To see more results, enter the required number or leave the field blank to see all the CETS codes that match your search. Dbms_copy - is a tool to copy data between SQL databases DTS - import data from another format into the databases. It’s a SQL tool. Equation Validation - a World Bank tool that is used to check the validity of the equation within the database Data Uploading & Managing

Manipulating Data using the Custom Management Tool Click on the code that you want to edit. It will return the detailed information at the bottom of the screen. You may then edit the Value, Owner, Source and Version. Once you change one of these the ’Update’ button will be activated. Click on the ’Update’ button The value changed will be updated and can be seen in the list above with the changed value Dbms_copy - is a tool to copy data between SQL databases DTS - import data from another format into the databases. It’s a SQL tool. Equation Validation - a World Bank tool that is used to check the validity of the equation within the database Data Uploading & Managing

Manipulating Data using the Custom Management Tool To create new data in the system follow these steps Click on Files and Time Series The screen shown on the right will appear. Select the country that you want to work with If you want to search for a CETS code, type in the code or part of the code in the series field and click in another field The system will return a maximum of number CETS codes that match your search, that are indicated in the ‘Top’ field To see more results, enter the required number or leave the field blank to see all the CETS codes that match your search. Dbms_copy - is a tool to copy data between SQL databases DTS - import data from another format into the databases. It’s a SQL tool. Equation Validation - a World Bank tool that is used to check the validity of the equation within the database Data Uploading & Managing

Manipulating Data using the Custom Management Tool Click on the ‘New’ button at the bottom right hand of the screen. Enter the CETS code in the series field Select the frequency that you want to use Enter the date in the format shown on the right. If the data is for a year enter in the format 01/01/XXXX Enter the value Enter the source of the data Enter the version Enter the scale Click on the Úpdate’ button to create the data. Click on ÓK’ to exit the console Click on File then Exit to exit the tool Dbms_copy - is a tool to copy data between SQL databases DTS - import data from another format into the databases. It’s a SQL tool. Equation Validation - a World Bank tool that is used to check the validity of the equation within the database Data Uploading & Managing

Manipulating Data using the Custom Management Tool Click on the ‘New’ button at the bottom right hand of the screen. Enter the CETS code in the series field Select the frequency that you want to use Enter the date in the format shown on the right. If the data is for a year enter in the format 01/01/XXXX Enter the value Enter the source of the data Enter the version Enter the scale Click on the Úpdate’ button to create the data. Click on ÓK’ to exit the console Click on File then Exit to exit the tool Dbms_copy - is a tool to copy data between SQL databases DTS - import data from another format into the databases. It’s a SQL tool. Equation Validation - a World Bank tool that is used to check the validity of the equation within the database Data Uploading & Managing

LDB Architecture – Input - Compare Tool Web Query LDB Uploading & Managing Tools Web Server Query & Analysis Data Sources LDB Staging Area LDB Production Data Administration (Quality) This diagram shows the input side highlighted. This indicates that the following slides will be dealing with data input. Reports Library Cube Data Uploading & Managing

Compare Tool To launch the application, open the compare tool file named 2gldbCompare.xls. The application will open in an environment similar to the Excel as shown on the right Dbms_copy - is a tool to copy data between SQL databases DTS - import data from another format into the databases. It’s a SQL tool. Equation Validation - a World Bank tool that is used to check the validity of the equation within the database Data Uploading & Managing

Compare Tool Open the upload file that contains the data you want to check for consistency. Otherwise copy and paste the data including the time series This is demonstrated on the right Dbms_copy - is a tool to copy data between SQL databases DTS - import data from another format into the databases. It’s a SQL tool. Equation Validation - a World Bank tool that is used to check the validity of the equation within the database Data Uploading & Managing

Compare Tool This spreadsheet contains the data that the user wants to check for consistency. To start the consistency check click on the Custom Macro icon. Dbms_copy - is a tool to copy data between SQL databases DTS - import data from another format into the databases. It’s a SQL tool. Equation Validation - a World Bank tool that is used to check the validity of the equation within the database Data Uploading & Managing

Compare Tool An input control screen will appear as shown on the left. Enter the DSN as AFRLDB Enter username as SA Enter the password as PASS Enter the country code as ZAM for Zambia Leave SubNat data as GLB. This is the system default Enter the source as 4 for CSO Zambia Enter the version as 6 for Temporary Leave the owner field as dba Dbms_copy - is a tool to copy data between SQL databases DTS - import data from another format into the databases. It’s a SQL tool. Equation Validation - a World Bank tool that is used to check the validity of the equation within the database Data Uploading & Managing

Compare Tool Enter the required tolerance level as a percentage. This tolerance will be used to check the levels of variance of data that is in the spreadsheet against what is in the database. Enter the tolerance as 10% Enter the header line as 1 Enter the first indicator line as 3 Enter the indicator column as A Enter the scale column as C Enter the data columns as D The second column values can vary depending on where the data is located in the spreadsheet Click OK to begin the compare Dbms_copy - is a tool to copy data between SQL databases DTS - import data from another format into the databases. It’s a SQL tool. Equation Validation - a World Bank tool that is used to check the validity of the equation within the database Data Uploading & Managing

Compare Tool The sheet on the right shows a typical view of the result of the comparison. The green colour means the data is already on the database and is the same Blue means the data is new to the database Yellow means the data is different to that on the database but is within the specified range Red means the data is outside the specified range If you point on any particular cell, a callout box will appear with data on the type of information on the database Dbms_copy - is a tool to copy data between SQL databases DTS - import data from another format into the databases. It’s a SQL tool. Equation Validation - a World Bank tool that is used to check the validity of the equation within the database Data Uploading & Managing

Compare Tool The Compare Tool is a quality management tool. It assists the user in checking that the information which the user is about to load on the database is consistent with what is in the LDB database. The spreadsheet that is used for this purpose is different from the upload spreadsheet and does not load data onto the database. It only checks the data in the compare upload spreadsheet against what is on the database. Changes can then be made to data until they are found to be correct. After this consistency check is carried out the data can be uploaded onto the database in the confidence that it is consistent to defined parameters. As can be seen from the diagram, the LDB system can be divided into three main components of input, core components and output. This training is concerned with checking the consistency of the data that is in a spreadsheet and about to be uploaded into the LDB system. This application needs to access your database through an OBDC connection. This connection is created by configuring a Data Source Name (DSN), via the ODBC Data source icon in the Control Panel. Your database administrator must have granted a read and write access to the database through a username and password. Dbms_copy - is a tool to copy data between SQL databases DTS - import data from another format into the databases. It’s a SQL tool. Equation Validation - a World Bank tool that is used to check the validity of the equation within the database Data Uploading & Managing

Compare Tool To run the compare tool, enter the data that you require into the spreadsheet named 2glDBCompare.xls This is as shown on the right. Dbms_copy - is a tool to copy data between SQL databases DTS - import data from another format into the databases. It’s a SQL tool. Equation Validation - a World Bank tool that is used to check the validity of the equation within the database Data Uploading & Managing

Compare Tool Click on the macro icon and enter the following information Enter the DSN name as afrldb Enter the username as sa Enter the password as pass Enter the country code as zmb Enter the subnat, source, version, owner. Enter the tolerance. This function has explained in the overview of this section. Enter the setup of the information on the spreadsheet in the right hand box. This will depend on the setup of your spreadsheet. Dbms_copy - is a tool to copy data between SQL databases DTS - import data from another format into the databases. It’s a SQL tool. Equation Validation - a World Bank tool that is used to check the validity of the equation within the database Data Uploading & Managing

Compare Tool Click on the macro icon and enter the following information Enter the DSN name as afrldb Enter the username as sa Enter the password as pass Enter the country code as zmb Enter the subnat, source, version, owner. Enter the tolerance. This function has explained in the overview of this section. Enter the setup of the information on the spreadsheet in the right hand box. This will depend on the setup of your spreadsheet. Dbms_copy - is a tool to copy data between SQL databases DTS - import data from another format into the databases. It’s a SQL tool. Equation Validation - a World Bank tool that is used to check the validity of the equation within the database Data Uploading & Managing