1. What-If Testing Framework
Manmadha Ambati
Senior Principal Quality Analyst
Manhattan Associates
Chaithra Rama
Technical Lead Quality Engineer

2. Abstract
What-If focuses on tactical and strategic level decision making for transportation and supply chain problem
Testing the tool which comprises of modeling, optimizing and simulating your supply chain network would be a challenging task as it involves huge amount of data and variations in usage from customer to customer
This paper attempts to provide a generic framework that can be adopted for testing various decision making tools involving huge data set

3. What is ‘What-If tool’?

4. What-If Testing Framework
Historical Data
Data Analysis
Data Visualization
Algorithm Testing
Automation

5. Optimize with What If tool
Historical Data
Data is very important for any analysis tool and forms the basis of testing
Historical data from various customer segments are collected to form the data set for testing this tool
This would create artificial supply chain network specific to customer which will aide in testing the application as used in real time
Optimize with What If tool
Historical Data
Compare Results

6. Data Analysis
Data Analysis forms the basis of testing the Transportation business model application
The scenario is simulated with the current setup as the baseline for benchmarking
Another scenario is simulated with the new setup via either Optimization performed on scenario or manual manipulation
Both these simulations are compared for assessments and comparisons.

7. Data Visualization
Data Visualization forms an effective way of testing the changes in the network post optimization.
Maps gives the visual experience of customer stores/Vendors assigned to DC, order schedules, static routes
The accuracy of Optimization can be tested on Maps where they can be compared before and after optimization is performed
Maps give a very realistic view of how the network is strategically located

8. Data Visualization with example
Before Optimization After Optimization
As we can observe the way stores are assigned to DC is changed after the optimization based on the distance, average demand
Nearby DC can be made operative to source out to stores around it to make the network more cost effective

9. Algorithm Testing
Algorithms are designed such that the problem is modeled by a set of constraints and best solution is found w.r.t objective function defined
Testing the algorithm becomes very crucial as various optimization decisions and solutions provided are validated
Sample data set is used to verify the correctness of the algorithm
Pair testing is performed to verify the algorithms where the tester and developer test the tool together.
Pair Testing
Actual Validation
Sample data set
Outcome
Algorithm validated

10. Automation
Unit tests, Customer flows are automated using the Groovy Framework
Tests automated are integrated with the Continuous Integration(CI) repository which runs 2 times a day with the latest code check-ins
TestNG - A reporting system tool identifies failures and issues immediately
Sample set/Customer data set
Automation Groovy Scripts
Test Results

11. Measurable Impact Key benefits of the Test Framework
Functionality well tested
With the CI process, Regression breaks are identified
Improved customer experience

12. References

13. Author Biography
Manmadha Ambati works as Senior Principal Analyst at Manhattan Associates since 12 years.
He has 14 years of Industry experience in all phases of software Testing life cycle. Rich experience in Transportation Planning engines and understanding business challenges of transportation. Strong product development expertise using Agile methodologies.
He holds interests at various industry interaction platforms UNICOM, Step-In etc.
Chaithra Rama works as Technical Lead Quality Analyst at Manhattan Associates since 6 years.
She has over 7 years of industry experience as a Software Tester. Good experience in Transportation Planning engines. She has also submitted papers at Step-In summit.
Good product development expertise using Agile methodologies.

14. Questions

15. Thank You!!!