1. Load and Performance Testing
Case Study and Lessons Learned
Brett H Barnhart, HNI Corp
October 26, 2017
Heartland Oracle User Conference

2. Agenda Executive Summary Introductions and Company Overview
BST (Business Systems Transformation)
Technology Overview
Executive Test Objective and Special Challenges
Approach, Phases, Monitoring, Results
Success and Opportunities
Summary
Q & A

3. Executive Summary
Here is a case study of how HNI a leading office furniture manufacturer performed a performance and load test of its solution, hardware, interfaces, and process orchestration as part of a readiness strategy for system roll-out. Having a complex ERP footprint including EBS, VCP, SOA, BI-Publisher, UC4, Endeca, OBIA (OBIEE), Scan Guns, and other interfaces, the Load Test team faced unique challenges in meeting the objective of Load Testing and Performance testing HNI’s full solution.

4. Brett Barnhart Introduction
Technical Application Manager at HNI Corp since May 2014
Application Integration
SOA, BizTalk, Informatica
Solution Delivery
EBS/VCP and related applications code delivery
Scheduling and Workflow Administration
UC4
Release Execution
Application Administration
18 Yrs Experience with Oracle EBS Development/Management

5. HNI Corporation Overview
HNI Corporation (NYSE: HNI) ( is the second-largest office furniture manufacturer in North America, and the nation's leading manufacturer and marketer of gas- and wood-burning fireplaces.

6. BR1 : Sales and Operations Planning
BST Overview
Business Systems Transformation “BST” project currently in the 3rd Business Release Phase
BR1 : Sales and Operations Planning
Demantra
BR2 : Financials AR AP GL
BR3: Order Management/Supply Chain Planning /Manufacturing
ONT
INV
WSH
BOM
Manufacturing
ASCP
VCP
+ 30 more

7. Technology Overview
Complex ERP implementation involving many different technologies
Oracle EBS (12.2.4)
Oracle VCP
OBIA
BI Publisher
Workflow
Oracle SOA Suite (12c)
Informatica
UC4 (v10)
BizTalk
CyberSource
Sabrix
SalesForce
Dealer Portal
Esker Fax Provider

8. Integration Strategy
With many different technologies in play, there is often overlap between products.
Oracle SOA
Application Integration
Informatica
ETL functionality
BizTalk
EDI processing
Automic UC4
Scheduling and Non-EBS Business Flows

9. Executive level test objectives
Validate hardware capacity
Identify process orchestration constraints
Benchmark application (code, configuration, and data) performance using full BR3 business footprint

10. Special Challenges
There were a number of factors which made this execution especially challenging.
Future transaction volume is unknown
Moving from a legacy system which causes an “Apples to Oranges” comparison
Minimal involvement from HNI resources
HNI team dedicated to completing an Integrated test cycle.
Lack of in-house Load Testing expertise
10 week execution timeframe

11. Used tools to automate processes (HP LoadRunner, UC4, and SOAP UI Pro)
Approach
Brought onboard a team of consultants to perform the LoadTest planning, prep, and execution.
Used tools to automate processes (HP LoadRunner, UC4, and SOAP UI Pro)
Some manual test execution where the transaction volume was low or execution was complex
Once LoadTest environment was available, stage data, and create a “Snapshot restore point”
Run each phase, gather results, flashback, repeat.
Allowed us to run a full test cycle daily.

12. 3 major test phases planned, to be completed 3 times
Normal Load
100% transactional volume
1200 users (600 loaded over a 60 minute duration)
Peak Load
125% transactional volume
1400 users
Stress Load
150% transactional volume
1600 users
Simulate node failure

13. System Network Application Integration Monitoring
CPU, Memory, and Storage Utilization
Network
Bandwidth utilization and latency
Application
LoadRunner script performance (User experience)
Concurrent Program, UC4, and Workflow performance
Integration
SOA benchmarks

14. Learnings
We had many meaningful learnings which have been incorporated into our solution.
We had to adjust the number of OACores to handle projected user load
Number and mix of concurrent manager slots
Orchestration constraints
Code performance issues
Data volume concerns (interface tables and such) that impacted performance.

15. Results - System Application Server utilization was low
CPU and memory utilization < 30%
Database Server utilization was normal
CPU utilization < 30%
Memory utilization < 55%
Database tier handled 4200 transaction per second
Network use was in range
Successfully loaded 700 users under 30 minutes using less than 40MB band width
Loading Users was acceptable
1400 EBS users (950 forms users) with two app tiers
1200 EBS users (800 forms users) with one app tier during “node failure” test

16. Success LoadRunner measuring user experience via forms/html
SOAP UI simulating electronic orders via SOA
System Monitoring
Database “Snapshot” refreshes worked perfectly. (4 hr refresh)
Simulating user jobs and reports through UC4

17. Opportunities Telnet Scripts failed to execute at expected volume
Older version of LoadRunner
EBS LoadRunner license does not include Telnet
Scripting
Data Staging
Data was scattered across many Inventory Orgs or in the wrong “Stage”
Meaningful and Measurable SLAs
Ensure that each SLA can be measured down to the “unit” level
Underestimation of HNI resources required
Understanding of solid EBS fundamentals by the scripters
Make sure caching is enabled in LoadRunner
Don’t log out after each test execution

18. Summary
Performance and Load Testing is a useful methodology to verify changes to your application footprint. By automating user activity, it is possible to validate the performance impact of changes.
In the case of HNI, we were able to validate new production hardware footprint, network bandwidth, process orchestration, integrations, and many of our solution footprints. We fell short on validating our telnet transaction volume.
It is the long term goal at HNI to utilize monthly automated testing to streamline our release testing process as well as validate any design changes to our environment.

19. Brett Barnhart - barnhartb@hnicorp.com
Q & A
Questions?
Brett Barnhart -