Informatica PowerCenter Performance Tuning Tips Pittsburgh Informatica User Group March 22nd 2016 Gregory Reynolds reynolds.gregory@gmail.com

Topics General Tips Source Qualifiers Lookups Sorters / Aggregators / Joiners Targets Other Options

General Tips Remove “Dead Weight” Set Proper Attributes on All Ports Filter unneeded records in the source qualifier Turn off or remove unused ports Filter and aggregate as soon as possible Trim strings immediately Set Proper Attributes on All Ports Improper data types require behind the scenes conversions or cause loss of data Incorrect data sizes require conversions and may truncate data unintentionally Watch Your Logs Don’t ignore warnings and non-critical errors. It requires a significant amount of resources to write to a log file Choose terse level logging in production to save space and processing

Source Qualifier Balance Work Effort Local Files Use Built-In Options Consider the time it takes the source to do work instead PowerCenter is faster at many things / DBMS is faster at others Local Files When reading from files it is almost always faster to ship the file to the local server before reading it then it is to try and read it with a remote connection Large data files likely would be better off staged before trying to do any complex transformations Use Built-In Options A sort is usually faster here than anywhere in the mapping Same thing for distinct Choose Deterministic Output if possible

Lookup Unconnected Lookups Cache Type Size Matters Filter The Data Using an unconnected lookup multiple times saves all of the cost associated with building the cache except the first time. Cache Type Static cache is the default for a reason. Best for a large amount of lookup records Disabling the cache sends every lookup to the database as an individual request. Works well for a very small amount of lookup requests versus a large table. Persistent cache allows you to reuse a cache across multiple sessions. Warning: it may require a lot of drive space Dynamic cache keeps records in the lookup up to date automatically as new data is sent to the target Size Matters Too small a cache allotment will force large lookups to go to disk Too large a cache will waste available memory that other transformations could use Filter The Data Source data can be filtered without using an override More records filtered mean less time to retrieve and smaller more efficient caches

Sorter, Aggregator, and Joiner Maximum Cost No data will pass through until every record has been read and sorted Large cache usage Session will fail if cache grows larger than capacity Waste Not Want Not Filter out as many unneeded records as is possible before hand Sorted Input Whenever possible use sorted input to save the majority of processing and cache cost from joiners as well as aggregators. Data will flow in incremental chunks if the input is sorted versus waiting until the end, releasing memory and cache requirements as it does so. Simplified Group Bys Grouping by a simple port may save an aggregator having to complete all the grouping before making any final decisions. Smaller Master Set Designate the master data in a sorter as the side that is expected to have fewer rows. Two large data flows will not join as quickly

Targets Rejected Rows Update Else Insert Commit Not… Bulk Loading Rejecting rows from the target database will slow down normal operations by potentially several seconds per record! Look out for unexpected NULL values, bad dates, and invalid keys. Update Else Insert With this option selected, every insert is approximately twice as slow as an update Other options include using a lookup to determine the appropriate action before hand Commit Not… Setting too low a threshold for database commits will require transactions to wait more often while the commit is completing. Setting it too high will cause the database to use an increasing amount of temp space and may cause it to slow down transaction speeds Bulk Loading Bulk loading is a fast “cheat” to be able to push large amount of data into some types of databases at high speed but there can’t any indices on the target table

Other Tuning Options System Competition Workflow Parallelization Session Partitioning Push-down Optimization Data Partitions Buffer Size Indices and Primary Keys System Competition Redesign slow mappings to find a better way