Daily Batch Process for Demand Forecasting 2019-01-16

Uploads and Imports Prepare a roadmap of your batch processes – files, queries, import actions / processes in Anaplan Process Name Import Action Name Import Action Description File / SQL File / SQL Source Source File / SQL Name BAT-01a Product Hierarcy 1.1 LST Row ID Product Data as CSV2L Import into ROWID Product Hierarchy Data SQL EBS ProductHierarchy   1.2 MOD EBS Product Hierarchy CSV2M Import into 'DAT01: Product Hierarchy' BAT-01b Product Properties 1.10 DAT09: Product Properties from EBS Product Properties Import into 'DAT09: Product Properties Data Upload' ProductProperties 1.11 ROWID Product Properties Data CSV2L Import into ROWID Product Properties Data BAT-02 Fulfillment Locations 2.1 LST ROWID Fulfillment Location Data CSV2L Import into ROWID Fulfillment Location Data SBL HubAndMarket 2.2 MOD ROWID Fulfillment Location Data CSV2M Import into 'DAT02: Fulfillment Location Hierarchy' 3.1 LST ROWID Sales Location Data as CSV2L Import into ROWID Sales Location Data FILE MANUAL 3.2 MOD Sales Location Hierarchy as CSV2M Import into 'DAT03: Sales Location Hierarchy' BAT-03 Store Hierarchy Data 3.2.1 LST ROWID Store Hierarchy Data s CSV2L Import into ROWID Store Hierarchy Data BI str_chanl_hier.csv 3.2.2 MOD DATXX: Store Hierarchy Data as CSV2M Import into 'DATXX: Store Hierarchy Data' BAT-04 COSC Order Header 4.1 LST 2016 COSC Order Header CSV2L Import into Transaction Sales Header ordr_hdr_lvl_data_cosc.csv 4.2 MOD 2016 COSC Order Header Data CSV2M Import into 'DAT04: Sales Header Transaction Data' BAT-05 Trans Sales Header Wholesale 5.1 LST Transaction Sales Header Wholesale CSV2L ordr_hdr_lvl_data_whlsale.csv 5.2 MOD Transaction Sales Header Wholesale Data CSV2M BAT-06 Order Header Primary DMA 6.1 LST 2016 Order Header Primary DMA CSV2L ordr_hdr_prmry_dma.csv 6.2 MOD 2016 Order Header Primary DMA CSV2M BAT-07 Transaction Sales Lines 7.1 LST Trans Sales Line CSV2L Import into Transaction Sales Line lne_itm_trnsctn_lns.csv 7.2 MOD DAT6.5 Transaction Sales Data CSV2M Import into 'DAT06.5: Transaction Sales Lines' BAT-08 Ship Date 8.1 LST ROWID EBS Ship Date Jan16-Apr16 CSV2L Import into ROWID EBS Ship Date Data soScheduleShipDate 8.2 MOD DATXX: EBS Ship Date Jan16-Apr16 CSV2M Import into 'DATXX: EBS Ship Data' BAT-09 Internal Sales 9.1 LST ROWID Internal Sales Order Data CSV2L Import into ROWID Internal Sales Data intrnl_sls_ordr_trnsctns.csv 9.2 MOD DATXX: Internal Sales Data CSV2M Import into 'DATXX: Internal Sales Data'

Consistent Naming Script name BAT-01a Product Hierarchy.bat Anaplan Action (Process) name BAT-01a Product Hierarchy Reject log name BAT-01a Product Hierarchy.log AC Process log name BAT-01a Product Hierarchy 2019- 01-16 0800.log

Don’t Rename Data Hub Processes! Use Anaplan Processes as wrappers for import actions. Import job scripts have the name of the Anaplan Process hardcoded in them. It is okay to rename Import Actions. Just don’t change Process names.

Other Tasks Pre-import processing Post-import processing Test for file creation process completion, notify Anaplan admin if upstream job was delayed Copy files to an upload folder. Avoids a potential conflict when file processing is delayed. Set current date Post-import processing Cancelled order purge Delete files from upload folder Email notification of process completion Kill hanging java.exe sessions after 30 minutes (Short-term workaround for AC 1.4 bug)

Anaplan Connect versions (as of Jan 2019) 1.3.3.5 – Most stable, but missing 2 key features. No CA Certificate authentication No jdbc.properties 1.4 – Hangs in one specific type of reject Bug: cmd and java.exe threads left hanging on specific duplicate record reject Workaround: Kill scheduled jobs after 30 minutes and kill all java.exe after batch is done. 1.4.1 Due out Feb 2019. Should be best of all worlds. Okay to have some of each: 1.3.3.5 for BI File uploads and 1.4 for SQL queries Recommend consolidating to 1.4.1 when released.

Anaplan Service Account A service account like svcanaplan@customer.com is used to login to Anaplan in all scheduled batch jobs. This account has an Outlook email box … necessary for setting up the account in Anaplan and for password resets. Anaplan Connect does NOT support SSO. This account’s Anaplan password is maintained in Anaplan and will expire every 90 days. SET AN OUTLOOK REMINDER TO UPDATE PASSWORD APRIL 1 AND REPEATING AT 90 DAY INTERVALS. Store info like login credentials and connect strings for jdbc in a GlobalVariables.bat file on Windows server … single place to update when it expires. Future state: CA Certificate authentication with certificate that expires once a year. More secure, less maintenance. Requires AC 1.4+ only (wait for next release due in Feb).

Anaplan Connect Scripts Set variables unique to this job. File, SQL query, Action, Log Name Get connection info from Globals.bat If SQL, run CreateJDBC.Properties.bat Assemble the OPERATION= line Connection, upload, action, and reject log flags Run AnaplanClient.bat, record the chatter to process log file Check ERRORLEVEL and log for errors, send email if trouble Exit

CreateJDBCProperties.bat Custom script collects connection, credentials, and SQL query information from other files and assembles a short-lived plaintext file for Anaplan Connect’s needs. Globals.bat contains connection and credentials. Intention is to encrypt that file to avoid plaintext password storage. SQLQueries.txt contains the SQL statements* used by jobs: ::QueryName Select ABC, DEF From Table Where GHI<0 ::end * No double quotes allowed in SQLQueries.txt, other special characters (<>=&|) are ok.

CreateJDBCProperties info online A best practices forum post in Community.Anaplan.com contains full details. https://community.anaplan.com/t5/Best-Practices/Windows-cmd-shell-script-for-dynamically-generating-JDBC

Logging Import action rejects (if any) are saved to file in \\<servername>\Anaplan\ErrorLogs\ Process “chatter” while AnaplanClient is interacting with Oracle and Anaplan is captured on the Windows server at D:\Anaplan\ProcessLogs\ Process logs are named with <job name> <date> <start time>.log History in Anaplan

Rejected Record Logs Same information as found on Details tab after a manual import into Anaplan. No rejects or warnings = no file. Reject files are overwritten daily. No retention. Could add Date Time to filename and retain for 30 days, same as Process Logs.

Process Logs Connection SQL query Upload Import – summary stats To List To Module Process logs are retained for 30 days to allow some forensics on past trouble or performance changes.

Scheduling Windows Task Scheduler on Windows server Simple scheduler, limited features. Most data centers have a more robust solution that monitors errors and notifies support team, branches on conditions, handles job dependencies… Setting jobs up can be a significant project. Two primary jobs start at 8 am File import jobs (AC 1.3.3.5) SQL jobs (AC 1.4) These could be merged after AC 1.4.1 is released with bug fix. Cleanup task at 8:30 to delete copies of data files and kill java.exe threads (bug workaround)