What-If Testing Framework Manmadha Ambati Senior Principal Quality Analyst Manhattan Associates Chaithra Rama Technical Lead Quality Engineer

Abstract What-If focuses on tactical and strategic level decision making for transportation and supply chain problem Testing the tool which comprises of modeling, optimizing and simulating your supply chain network would be a challenging task as it involves huge amount of data and variations in usage from customer to customer This paper attempts to provide a generic framework that can be adopted for testing various decision making tools involving huge data set

What is ‘What-If tool’?

What-If Testing Framework Historical Data Data Analysis Data Visualization Algorithm Testing Automation

Optimize with What If tool Historical Data Data is very important for any analysis tool and forms the basis of testing Historical data from various customer segments are collected to form the data set for testing this tool This would create artificial supply chain network specific to customer which will aide in testing the application as used in real time Optimize with What If tool Historical Data Compare Results

Data Analysis Data Analysis forms the basis of testing the Transportation business model application The scenario is simulated with the current setup as the baseline for benchmarking Another scenario is simulated with the new setup via either Optimization performed on scenario or manual manipulation Both these simulations are compared for assessments and comparisons.

Data Visualization Data Visualization forms an effective way of testing the changes in the network post optimization. Maps gives the visual experience of customer stores/Vendors assigned to DC, order schedules, static routes The accuracy of Optimization can be tested on Maps where they can be compared before and after optimization is performed Maps give a very realistic view of how the network is strategically located

Data Visualization with example Before Optimization After Optimization As we can observe the way stores are assigned to DC is changed after the optimization based on the distance, average demand Nearby DC can be made operative to source out to stores around it to make the network more cost effective

Algorithm Testing Algorithms are designed such that the problem is modeled by a set of constraints and best solution is found w.r.t objective function defined Testing the algorithm becomes very crucial as various optimization decisions and solutions provided are validated Sample data set is used to verify the correctness of the algorithm Pair testing is performed to verify the algorithms where the tester and developer test the tool together. Pair Testing Actual Validation Sample data set Outcome Algorithm validated

Automation Unit tests, Customer flows are automated using the Groovy Framework Tests automated are integrated with the Continuous Integration(CI) repository which runs 2 times a day with the latest code check-ins TestNG - A reporting system tool identifies failures and issues immediately Sample set/Customer data set Automation Groovy Scripts Test Results

Measurable Impact Key benefits of the Test Framework Functionality well tested With the CI process, Regression breaks are identified Improved customer experience

References https://www.investcalgaryregion.ca/blog/how-to-use-big-data-supply-chain-analytics http://www.datasciencecentral.com/profiles/blogs/predictive-analytics-in-the-supply-chain https://en.wikibooks.org/wiki/Fundamentals_of_Transportation/Modeling

Author Biography Manmadha Ambati works as Senior Principal Analyst at Manhattan Associates since 12 years. He has 14 years of Industry experience in all phases of software Testing life cycle. Rich experience in Transportation Planning engines and understanding business challenges of transportation. Strong product development expertise using Agile methodologies. He holds interests at various industry interaction platforms UNICOM, Step-In etc. Chaithra Rama works as Technical Lead Quality Analyst at Manhattan Associates since 6 years. She has over 7 years of industry experience as a Software Tester. Good experience in Transportation Planning engines. She has also submitted papers at Step-In summit. Good product development expertise using Agile methodologies.

Questions

Thank You!!!