1. Oracle Data Integrator By U. Jahnavi M. Ambreen P. Sashank P. Praneeth B. Manoj kumar

4. Connecting to the Designer module

5. Creating Data Models

7. Click on Selective Reverse tab and check the button Objects to Reverse. We get a list of all the tables present in the database. Click on the button at the right most corner.

9. Creating ODI Model for a Flat File source To create a new model for the flat file target datastore, perform the following steps: 1.Click the Models tab in the left panel of the ODI Designer. Click the Insert Model Foldericon. On the screen that appears set the name to FILES. 2.Right-click the Files folder and select Insert Model. 3.On the screen that appears, enter the values provided in the following table. Click the Reversetab (don't click the Reverse button). Select Global from the Context drop-down list. Click OK.

14. Assigning Primary Key

15. Note: If primary key already exists then skip this step. Expand the target model and target table. Right click Constraints and click Insert Key. Give a name for the constraint in the Description and select the columns, we consider as the key columns for the target table in the Columns tab. Click on Apply and say OK.

16. Creating Projects 1.File->Wizard Give Name 2.Import Knowledge Modules from D:\OraHome_1\oracledi\impexp path. 3.Import -CKM Oracle -IKM Oracle Incremental Update -LKM Oracle to Oracle(@source is DB,@target is DB) -RKM Oracle

18. Creating Interfaces

21. Creating Packages 1.Expand the Project and right click on Packages and click Insert Package. 2.Give a name for the package in the Definition tab and click on Diagram tab. 3.Drag and drop the interfaces and connect the interfaces using the arrows on the top which indicate the success and failure as shown:

23. PROCEDURES Procedure is a set of commands that can be executed by an agent. Procedure is composed of command lines,Every command line may contain two commands that can be executed on a source and on a target.

24. CONTENTS:  Creation of Procedures  Procedure in a Package  Purging of OBIEE using ODI procedures

25. CREATING PROCEDURES

27. Procedure in a Package

28. Purging of OBIEE Using ODI

29. Then Execute the procedure and then open the OBIEE (online) and u find in manage ->cache the cache memory will be empty.

30. ODI VARIABLES A variable is an object that stores a single value. This value can be a string, a number or a date. A variable can be created as a global variable or in a project. Contents: 1) Creating Variables 2) Assigning the value of a variable

31. CREATION OF VARIABLES To create a variable: In Designer Navigator select the Variables node in a project or the Global Variables node in the Others view. Right-click and select Insert Variable. The Variable Editor opens.

32. If you want the variable's value to be set by a query: Select the Refreshing tab.

33. SEQUENCES A sequence is a value that increments automatically when used. A sequence can be created as a global or in a project. Global sequences are common to all projects, whereas project sequences are only available in the project where they are defined.

34. CONTENTS Two types of sequences 1) Standard Sequence 2) Specific Sequence

35. Standard sequence

36. Specific sequence In order to have a specific sequence we need to create a seq.table at the basetable which contains target table name and seq.no as columns. Then specify the table name as created table name and column name as the seq.no which was given in the created table And give the filter condition with the tgt table name. Then link the sequence with the corresponding column in tgt table.

38. MEMOS A memo is an unlimited amount of text attached to virtually any object, visible on its Memo tab. When an object has a memo attached, the memo icon appears next to it.

39. MARKERS: Flags are defined using markers. These markers are organized into groups, and can be applied to most objects in a project or a models. Typical marker groups are: The development cycle (development, test, production) Priorities (low, medium, urgent, critical) Progress (10%, 20%, etc) Markers ca be used Globally and in a project.

40. Introduction to Knowledge Modules Knowledge Modules (KMs) are code templates. Each KM is dedicated to an individual task in the overall data integration process. The code that is generated and executed is derived from the declarative rules and metadata defined in the ODI Designer module. A KM will be reused across several interfaces or models. To modify the behavior of hundreds of jobs using hand-coded scripts and procedures, developers would need to modify each script or procedure. In contrast, the benefit of Knowledge Modules is that you make a change once and it is instantly propagated to hundreds of transformations. KMs are based on logical tasks that will be performed. They don’t contain references to physical objects (datastores, columns, physical paths, etc.) KMs can be analyzed for impact analysis. KMs can’t be executed standalone. They require metadata from interfaces, datastores and models.

41. Types of Knowledge Modules 1) RKM (Reverse Knowledge Modules) The RKM’s main role is to perform customized reverse engineering for a model. The RKM is in charge of connecting to the application or meta data provider then transforming and writing the resulting meta data into Oracle Data Integrator’s repository. a)RKM Oracle : Used to perform reverse-engineering from oracle data models.

42. Types of Knowledge Modules 2) LKM (Loading Knowledge Modules) An LKM is in charge of loading source data from a remote server to the staging area. It is used by interfaces when some of the source data stores are not on the same data server as the staging area. The LKM implements the declarative rules that need to be executed on the source server and retrieves a single result set that it stores in a "C$" table in the staging area. a)LKM File to Oracle (EXTERNAL TABLE) Source - File Staging - Oracle b)LKM File to Oracle (SQLLDR) Source - File Staging – Oracle

43. Types of Knowledge Modules c)LKM File to SQL Source - File Staging – Oracle d)LKM Oracle to Oracle (DBLINK) Source - Oracle Staging - Oracle e)LKM SQL to Oracle Source - Oracle Staging – Oracle f)LKM SQL to SQL Source - Any Staging – Any

44. Types of Knowledge Modules 3)JKM (Journalizing Knowledge Modules) Used to create a journal of data modifications (insert, update and delete) of the source databases to keep track of the changes. Oracle Data Integrator supports two journalizing modes: I) Simple Journalizing tracks changes in individual data stores in a model. JKM: JKM Oracle Simple II) Consistent Set Journalizing tracks changes to a group of the model's data stores. JKM: JKM Oracle Consistent

45. Types of Knowledge Modules 4) IKM (Integration Knowledge Modules) The IKM is in charge of writing the final, transformed data to the target table. Every interface uses a single IKM. When the IKM is started, it loads the data from C$ table into I$ temporary table. There are 2 types of IKMs: those that assume that the staging area is on the same server as the target datastore, and those that can be used when it is not. a) IKM Oracle Incremental Update To load the entire data from source to target Staging - Oracle Target - Oracle b) IKM Oracle Slowly Changing Dimension To apply scd types (type1, type2, type3) Staging - Oracle Target - Oracle

46. Types of Knowledge Modules c) IKM SQL to File Append Staging - Oracle Target - File d) IKM SQL to SQL Append Staging - Any Target - Any

47. Types of Knowledge Modules 5) CKM (Check Knowledge Modules) The CKM is in charge of checking that records of a data set are consistent with defined constraints. The CKM can be used in 2 ways: 1. STATIC_CONTROL 2. FLOW_CONTROL 1. In STATIC_CONTROL mode, the CKM reads the constraints of the table and checks them against the data of the table. Records that don’t match the constraints are written to the "E$" error table in the staging area. 2. In FLOW_CONTROL mode, the CKM reads the constraints of the target table of the Interface. It checks these constraints against the data contained in the "I$" flow table of the staging area. Records that violate these constraints are written to the "E$" table of the staging area.

48. Types of Knowledge Modules a)CKM Oracle b) CKM SQL 6) SKM (Service Knowledge Modules) Used to generate the code required for creating data services. a) SKM Oracle b) SKM SQL

49. Importing Knowledge Modules Expand the Knowledge Modules tab under the Project Folder Right click on the required knowledge module and select import knowledge modules.

51.  Select the appropriate path where the Oracle Data Integrator is installed.  Select the impexp directory.

53. ODI Tools Oracle Data Integrator Tools (also called Oracle Data Integrator Commands) are commands provided for performing specific tasks at run-time. These tasks may be as simple as waiting for a certain time or producing a sound, or as sophisticated as executing ANT Scripts or reading emails from a server. The tools may be added while working with Designer. They may be used into Packages, or into Procedures Commands or Knowledge Modules Commands. It is also possible to use them directly from an OS command line.

54. ODI Tools To use an Oracle Data Integrator Tool into a Package: 1)Open the Package you want to modify. 2)Click the Display Toolbox button in the menu bar of the Package window. 3)In the Toolbox, Select the icon of the Tool you want to use and click into the package window. A step with the tool icon appears in the package. 4)Click the Selection button in the menu bar, then double click this step. 5)A Properties window for this tool appears. Set the value of the parameters in this properties window. 6)Drag and drop other objects like interfaces, procedures,etc and link them in a sequence 7)Click Apply.

56. 1)Metadata OdiReverseGetMetaData This command allows to reverse-engineer the given model in the temporary reverse tables. OdiReverseResetTable The command OdiReverseResetTable can purge these temporary tables for a given model. OdiReverseSetMetaData The command OdiReverseSetMetaData is used to integrate data from these temporary tables into the Repository for a given data model.

57. 2)Oracle Data Integrator Objects OdiDeleteScen Deletes a scenario provided name and version. OdiExportAllScen Exports a group of scenarios from the connected repository to XML export files named SCEN_.xml. OdiExportObject Exports an object from the connected repository to an XML export file. This command reproduces the behavior of the export feature available in the user interface. OdiExportScen Exports automatically in an XML format a scenario contained in the current work repository.

58. 2)Oracle Data Integrator Objects OdiExportWork Exports the work repository to a directory or a ZIP export file. OdiGenerateAllScen Generates automatically a set of scenarios from components contained in a folder or a project, filtered by markers. OdiImportObject Imports the contents of an XML export file to a repository. OdiImportScen Imports automatically a scenario to the current work repository from an XML export file.

59. 3)Utilities OdiStartScen Starts a scenario. OdiUpdateAgentSchedule Forces a agent to recalculate its schedule of tasks. This has the same effect as manually telling the agent to update its schedule in Topology Manager.

60. 4)Internet OdiReadMail Can be used to read emails as well as their attachments from a POP or IMAP internet mail service. OdiSendMail Can send an email through an SMTP server.

61. 5)Files OdiFileAppend Concatenates a set of files into a single file. OdiFileCopy Copies files or a folder into files or a folder on the machine of the execution agent. OdiFileDelete Deletes, based on the file modification time, files or a directory on the machine running the agent which executes it. OdiFileMove Moves files or a directory into files or a directory on the machine of the execution agent. OdiMkDir Creates a directory.

62. 5)Files OdiFileWait Scans the directory DIR and waits for files matching the mask given in PATTERN. When the specified files are found, the action indicated by the parameter ACTION is triggered. The execution agent is in standby mode waiting to receive FILECOUNT files unless the timeout limit, given by TIMEOUT is reached. The directory is scanned every POLLINT milliseconds. OdiOutFile Generates a text file on the machine on which the execution agent is running. OdiUnZip Used to unzip an archive file to to a directory. OdiZip Creates ZIP files from a directory or several files.

63. 6)Event Detection OdiFileWait Scans the directory DIR and waits for files matching the mask given in PATTERN. OdiReadMail Can be used to read emails as well as their attachments from a POP or IMAP internet mail service. OdiWaitForData Waits for a number of rows in a table or a set of tables. This can also be applied to a number of objects containing data, such as views. OdiWaitForTable Waits for a table to be created and populated with a pre-defined number of rows.

64. 7)Changed Data Capture OdiRefreshJournalCount This command refreshes for a given journalizing subscriber the number of rows to consume for the given table list or CDC set. OdiRetrieveJournalData This command retrieves for a given journalizing subscriber the journalized events for the given table list or CDC set. OdiWaitForData Waits for a number of rows in a table or a set of tables. OdiWaitForLogData Waits for a number of modifications to occur on a journalized table or a list of journalized tables. OdiWaitForTable Waits for a table to be created and populated with a pre-defined number of rows.

65. SCHEDULING Scheduling is required to automate the whole loading process with out our involvement in production environment. Scheduling Requirements: Need to create both the Physical and Logical Agents. Need to Configure the ODIPARAMS.bat file Need to create Scenario. Create Scheduling. Launching Scheduler Agent. Note: Agent Creation is done from Topology Manager.

66. CONFIGURING ODIPARAMS BATCH FILE For this go to the installation directory and give: edit odiparams.bat We need to modify the following parameters. ODI_SECU_DRIVER=oracle.jdbc.driver.OracleDriver(Jdbc driver used to connect to master reository) ODI_SECU_URL=jdbc:oracle:thin:@SHARP:1522:DEV01(Jdbc URL used to connect to master repository) ODI_SECU_USER=apps(data base account used to connect to master repository) ODI_SECU_ENCODED_PASS=d,yHhBVmdXKhkqnulNETykhj m(encrypted password of the database account) ODI_SECU_WORK_REP=WREP_DEV01(Name of the work repository to connect) ODI_USER=SUPERVISOR(ODI user used to launch a scenario) ODI_ENCODED_PASS=hZyHzHjVTvdbzORf4uVj2y(ODI user encrypted password)

67. Encoding the Password: Use the following command to encode the password. Agent ENCODE

68. Edit the ODIParams file

69. CREATING SCENARIO Go To appropriate interface or package for which you want to create the scenario and then right click on the object (interface or package) and click on ‘Generate Scenario’

70. Give the Scenario Name and Version.

71. CREATING SCHEDULING Go to the Scenario, in that go to the Scheduling node, Right click on Scheduling and select ‘insert scheduling’

72. Assign the Context, agent, log level. Under the Status section check the active radio button, and Under the Execution section check the radio button Hourly.

73. LAUNCHING SCHEDULER AGENT Launch the scheduler agent by running the following command. agentscheduler "-port=20910" "-v=5" OR Agentscheduler “-name= ” “-port=20910”

74. After running the AgentScheduler we get the status as number of items scheduled for execution.

75. I f So go to the Scheduling Information button and click on it we get the Information like what are the scenarios that are scheduled and at what time they are going to execute