1. 5 Copyright © 2009, Oracle. All rights reserved. Defining ETL Mappings for Staging Data

2. Copyright © 2009, Oracle. All rights reserved. 5 - 2 Objectives After completing this lesson, you should be able to do the following: Explain the purpose of a staging area Explore the Mapping Editor interface Define simple mappings between source objects and staging table columns Use the different mapping operators such as the External Table, Filter, Expression, and the Data Generator operator

3. Copyright © 2009, Oracle. All rights reserved. 5 - 3 Lesson Agenda Purpose of a staging area Define OWB mappings –Mapping Editor interface: Grouping, ungrouping, and spotlighting Create External Tables and use it in a mapping –Create and bind and synchronization Use Filter operator and the Data Generator operator Mapping source to target attributes –Use automapper Expression operator –Copy-paste operators, attributes, and groups Set loading type and target load ordering Validate and generate a Mapping

4. Copyright © 2009, Oracle. All rights reserved. 5 - 4 Purpose of a Staging Area A staging area is a common platform. Staging tables share a common database. Staging tables can identify data with a point in time. Data marts/ Warehouse Staging area Sources Relational systems Flat files

5. Copyright © 2009, Oracle. All rights reserved. 5 - 5 Data Staging Area Is the construction site for the warehouse Is required by many implementations Can be composed of operational data stores (ODS), flat files, or relational server tables Is frequently configured as multitier staging Extract Transform Load (Transport) Operational systems Data staging areaWarehouse

6. Copyright © 2009, Oracle. All rights reserved. 5 - 6

7. Copyright © 2009, Oracle. All rights reserved. 5 - 7 Data Warehouse Implementation: Typical Steps Part 1 Part 2 1. Define the source metadata. 2. Define staging area metadata. 3. Define a relational dimensional model. 4. Create process flows. 5. Deploy the mappings. 6. Execute or run the mappings. 7. View the data.

8. Copyright © 2009, Oracle. All rights reserved. 5 - 8 Metadata to Data 1. Initialize and organize the modules. 2. Import the metadata. (This gives you only the data definition.) 3. Use the metadata to define the mapping. 4. Deploy and execute the mapping to load the data.

9. Copyright © 2009, Oracle. All rights reserved. 5 - 9 Defining Staging Area in OWB Create and register an OWB user using the Security node in the Globals Navigator –Optionally create an associated location Create an Oracle module –Either with a new location pointing to the staging user you registered –Or using the location created while registering the user Setting up the staging area is no different from creating any other Oracle module pointing to a database schema. You learned about this in lesson 3.

10. Copyright © 2009, Oracle. All rights reserved. 5 - 10 Lesson Agenda Purpose of a staging area Define OWB mappings –Mapping Editor interface: Grouping, ungrouping, and spotlighting Create External Tables and Use it in a mapping –Create and bind and synchronization Use Filter operator and the Data Generator operator Mapping source to target attributes –Use automapper Expression operator –Copy-paste operators, attributes, and groups Set loading type and target load ordering Validate and generate a Mapping

11. Copyright © 2009, Oracle. All rights reserved. 5 - 11 OWB Mapping Extract, transform, and load (ETL) involves the movement and transformation of data from your sources to your targets. Use OWB mappings to specify which source data objects provide data to which target data objects. Mapping definitions reside in target modules. Warehouse Builder supports both bottom up and top-down design in mapping.

12. Copyright © 2009, Oracle. All rights reserved. 5 - 12 Building a Mapping Identify the Oracle module. Create a mapping. Connect operators using connection lines. Generate the mapping. Add operators, and specify the source and target. Deploy the mapping. Execute the mapping.

13. Copyright © 2009, Oracle. All rights reserved. 5 - 13 Creating a New Mapping Mapping Editor canvas: Here you design the mapping Code templates mappings are created under the Template Mappings node, discussed in Part 2 of the course. You do not create the mappings under this for the Part 1 practices.

14. Copyright © 2009, Oracle. All rights reserved. 5 - 14 Mapping Editor Canvas Existing objects can be directly dragged on the canvas. Operators are contained in the Component Palette. You can drag an operator from this palette to the canvas. Useful tools that help you manage the Mapping Editor canvas real estate.

15. Copyright © 2009, Oracle. All rights reserved. 5 - 15 Tools to Manage the Mapping Editor Canvas Operators Auto Layout Fit In Window Zoom Control: In, Out, Interactive Mapping Debugger toolbar

16. Copyright © 2009, Oracle. All rights reserved. 5 - 16 Grouping and Ungrouping 1

17. Copyright © 2009, Oracle. All rights reserved. 5 - 17 Grouping and Ungrouping 23

18. Copyright © 2009, Oracle. All rights reserved. 5 - 18 Spotlighting in the Mapping Editor

19. Copyright © 2009, Oracle. All rights reserved. 5 - 19 Practice 5-1: Working in the Mapping Editor In this practice, you explore the Mapping Editor interface.

20. Copyright © 2009, Oracle. All rights reserved. 5 - 20 Lesson Agenda Purpose of a staging area Define OWB mappings –Mapping Editor interface: Grouping, ungrouping, and spotlighting Create External Tables and use it in a mapping –Create and bind and synchronization Use Filter operator and the Data Generator operator Mapping source to target attributes –Use automapper Expression operator –Copy-paste operators, attributes, and groups Set loading type and target load ordering Validate and generate a Mapping

21. Copyright © 2009, Oracle. All rights reserved. 5 - 21 Creating Mappings for Flat File Sources You define two mappings: – MAP_STG_CUSTOMERS : Uses an external table as a source to derive data from a flat file. – MAP_STG_GEOGRAPHY : Uses a flat file source directly to extract data. External Table Flat File

22. Copyright © 2009, Oracle. All rights reserved. 5 - 22 What Are External Tables? Flat fileStaging areaData mart Flat fileData martExternal table SQL*LoaderPL/SQL Avoids this extra step Customers_txt CUST_Ext

23. Copyright © 2009, Oracle. All rights reserved. 5 - 23 Creating an External Table

24. Copyright © 2009, Oracle. All rights reserved. 5 - 24 Create External Table: File Selection External tables are defined on top of the flat file structures.

25. Copyright © 2009, Oracle. All rights reserved. 5 - 25 Create External Table: Locations

26. Copyright © 2009, Oracle. All rights reserved. 5 - 26 Configuring External Tables 1 2 3 4 By default this configuration is not required. You should configure this when the data file is different from the sampled file or you want to add many data files.

27. Copyright © 2009, Oracle. All rights reserved. 5 - 27 Practice 5-2: Creating a Staging Module and an External Table This practice covers the following topics: Creating a Staging Module, STAGING_AREA Creating an External Table, CUST_EXT

28. Copyright © 2009, Oracle. All rights reserved. 5 - 28 Including the External Table in a Mapping 1 2

29. Copyright © 2009, Oracle. All rights reserved. 5 - 29 Creating a Staging Table

30. Copyright © 2009, Oracle. All rights reserved. 5 - 30 Create and Bind The STG_CUSTOMERS table operator exists only on the mapping canvas and the table metadata is actually not created. To create the table, right-click the operator and select Create and Bind. Select the location where you want to create the table.

31. Copyright © 2009, Oracle. All rights reserved. 5 - 31 Adding an Attribute If you want to remove an attribute from the table, right-click on the numbered column and select Remove.

32. Copyright © 2009, Oracle. All rights reserved. 5 - 32 Levels of Synchronizing Changes Mapping Design Level Design Time Metadata Level Database Level OutboundInbound Reverse: Re-importRe-deploy

33. Copyright © 2009, Oracle. All rights reserved. 5 - 33 Synchronizing: Outbound and Inbound The LOAD_DATE attribute is only added in the Table operator and not in the table definition. You should synchronize the operator with the table definition. This is Outbound synchronization. If you add an attribute in the table definition and you want to synchronize the operator with it, perform inbound synchronization.

34. Copyright © 2009, Oracle. All rights reserved. 5 - 34 Synchronizing: Outbound and Inbound

35. Copyright © 2009, Oracle. All rights reserved. 5 - 35 Synchronize All in a Mapping

36. Copyright © 2009, Oracle. All rights reserved. 5 - 36 Quiz If you define a new unbound Table operator on the Mapping Editor, you should do the following to actually create the table definition: a.Validate the operator b.Generate the operator c.Create and Bind the operator d.Synchronize the operator

37. Copyright © 2009, Oracle. All rights reserved. 5 - 37 Quiz If you delete an existing table definition and then re-import a more recent version of the table, you must perform the following if the table has been used in a mapping: a.Outbound synchronization b.Inbound synchronization

38. Copyright © 2009, Oracle. All rights reserved. 5 - 38 Lesson Agenda Purpose of a staging area Define OWB mappings –Mapping Editor interface: Grouping, ungrouping, and spotlighting Create External Tables and Use it in a mapping –Create and bind and synchronization Use Filter operator and the Data Generator operator Mapping source to target attributes –Use automapper Expression operator –Copy-paste operators, attributes, and groups Set loading type and target load ordering Validate and generate a Mapping

39. Copyright © 2009, Oracle. All rights reserved. 5 - 39 Using a Filter Operator Filter operator is used to conditionally extract rows. Set the condition in the Filter Condition property in the Property Inspector panel. Click the ellipses (…) to start the Expression Builder.

40. Copyright © 2009, Oracle. All rights reserved. 5 - 40 Using the Expression Builder to Define a Filter Condition

41. Copyright © 2009, Oracle. All rights reserved. 5 - 41 Using a Data Generator Use a Data Generator operator to introduce a sequence, record number, or system date into a mapping. MAP_STG_GEOGRAPHY : In this mapping, a Flat File operator is used as a source.

42. Copyright © 2009, Oracle. All rights reserved. 5 - 42 Lesson Agenda Purpose of a staging area Define OWB mappings –Mapping Editor interface: Grouping, ungrouping, and spotlighting Create External Tables and Use it in a mapping –Create and bind and synchronization Use Filter operator and the Data Generator operator Mapping source to target attributes –Use automapper Expression operator –Copy-paste operators, attributes, and groups Set loading type and target load ordering Validate and generate a Mapping

43. Copyright © 2009, Oracle. All rights reserved. 5 - 43 Mapping Source Columns to the Target Individually mapping the attributes Automatically mapping groups of attributes

44. Copyright © 2009, Oracle. All rights reserved. 5 - 44 Using the Automapper in the Mapping Editor You can map between two operators without expanding them. When you connect attribute groups, use this section When you connect individual source to target attributes. It is keyboard enabled: enter * to get a list of all the attributes in a drop-down list

45. Copyright © 2009, Oracle. All rights reserved. 5 - 45 Lesson Agenda Purpose of a staging area Define OWB mappings –Mapping Editor interface: Grouping, ungrouping, and spotlighting Create External Tables and Use it in a mapping –Create and bind and synchronization Use Filter operator and the Data Generator operator Mapping source to target attributes –Use automapper Expression operator –Copy-paste operators, attributes, and groups Set loading type and target load ordering Validate and generate a Mapping

46. Copyright © 2009, Oracle. All rights reserved. 5 - 46 Expression Operator Expression operator is used to compute and transform a column value. Edit the expression inline

47. Copyright © 2009, Oracle. All rights reserved. 5 - 47 You can copy-paste or cut-paste the following across mappings: Operators Attributes Operator groups Copying Operators Across Mappings

48. Copyright © 2009, Oracle. All rights reserved. 5 - 48 Expression Editing and Copying Copy the target attribute and paste it into the OUTGRP1 of the Expression operator to quickly define an output attribute with the same data type and properties.

49. Copyright © 2009, Oracle. All rights reserved. 5 - 49 Quiz Using the automapper, you can enter * to get a list of all the attributes in a drop-down list: a.True b.False

50. Copyright © 2009, Oracle. All rights reserved. 5 - 50 Lesson Agenda Purpose of a staging area Define OWB mappings –Mapping Editor interface: Grouping, ungrouping, and spotlighting Create External Tables and use it in a mapping –Create and bind and synchronization Use Filter operator and the Data Generator operator Mapping source to target attributes –Use automapper Expression operator –Copy-paste operators, attributes, and groups Set loading type and target load ordering Validate and generate a Mapping

51. Copyright © 2009, Oracle. All rights reserved. 5 - 51 Setting Loading Type

52. Copyright © 2009, Oracle. All rights reserved. 5 - 52 Target Load Ordering Allows loading related targets in a single mapping in order Determines target load order based on foreign keys first, to load master before detail

53. Copyright © 2009, Oracle. All rights reserved. 5 - 53 Overriding the Default Load Order You can change the order using the up and down arrow buttons. Click the mapping canvas to see Target Load Order in the Property inspector panel.

54. Copyright © 2009, Oracle. All rights reserved. 5 - 54 Lesson Agenda Purpose of a staging area Define OWB mappings –Mapping Editor interface: Grouping, ungrouping, and spotlighting Create External Tables and Use it in a mapping –Create and bind and synchronization Use Filter operator and the Data Generator operator Mapping source to target attributes –Use automapper Expression operator –Copy-paste operators, attributes, and groups Set loading type and target load ordering Validate and generate a Mapping

55. Copyright © 2009, Oracle. All rights reserved. 5 - 55 Validating the Mapping Click the Validate tool to verify the metadata definitions and configuration parameters in a mapping before generating or deploying the mapping Validation Warning Validation Success

56. Copyright © 2009, Oracle. All rights reserved. 5 - 56 Generating the Mapping Click the Generate tool to generate the PL/SQL code for the mapping. Note that this code is generated only for review purposes.

57. Copyright © 2009, Oracle. All rights reserved. 5 - 57 Quiz If there are multiple targets in a mapping, you can define an order for loading the targets by using the following mapping property: a.Loading Type b.Target load order

58. Copyright © 2009, Oracle. All rights reserved. 5 - 58 Summary In this lesson, you should have learned how to: Explain the purpose of a staging area Explore the Mapping Editor interface Define simple mappings between source objects and staging table columns Use the different mapping operators such as the External Table, Filter, and Data Generator operator

59. Copyright © 2009, Oracle. All rights reserved. 5 - 59 Practice 5-3 Overview: Defining ETL Mappings for Staging Data This practice covers the following topics: Creating a simple mapping that loads the data from the FILE_GEOGRAPHY_MULTI_CSV file into two staging tables; each table stores the records of a type COUNTRY_REC or REGION_REC. Creating a mapping to load data from the CUST_EXT source external table to the STG_CUSTOMERS staging table. The table contains the records of customers in the UK.

60. Copyright © 2009, Oracle. All rights reserved. 5 - 60