1. How to Improve Releasing Efficiency via i18N/L10n Test Automation.
View CART Case Study

2. Beyondsoft Overview: 20 Years’ Footprint to Excellence

3. Beyondsoft Overview: Full-Cycle G11N Services

4. VMware and Beyondsoft
Beyondsoft has worked with VMware since Jan 2014.
Services provided :
G11N Testing Service Provider
Team size Engineers
L10N Engineering Service Provider
Team size Engineers
Marketing Partner
Developing VMware new product solutions for Local Chinese Clients

5. View CART Introduction
View CART is a term that describes View Clients for various platforms.
The Project Objective is to test :
View Client features
Interaction between View Client and Virtual Desktops.
L10N
Keyboard Input Testing: It means user can use native keyboard to input various characters to Virtual Desktop from the client.
Client UI Testing: Except Keyboard Input testing, other testing on client UI are belong to Client UI testing, such as: login, logout, do specified operation via menu, check the strings displayed on UI.
PDP: Product Delivery and Packaging, it is similar with installation testing.

6. Areas where efficiency can be improved
Many similar type of bugs are found in Keyboard Input Testing
Multiple platforms need to be covered
Very important old features must be tested every cycle.
Time required to upgrade each test environment
Manual test is performed during office work hours only
Ever increasing number of new features, and manpower is limited.
Automation is the most suitable way to solve the above issues!

7. Automation Assessment Strategy to select test areas for automationSN
Rationale
Area selected 1
Areas with high possibility of finding bugs
Keyboard UI test 2
Basic and important features
P0 features 3
Areas which require long execution time
Download build
Installation
Deployment
Upgrade
Copy or move large files 4
Areas which are not going to be automated
New, incomplete and unstable features
Non-P0 features
Translation: it means rearrange the order of steps in the manual test cases

8. Automation Assessment Reorder Manual Test Cases for AutomationSN
Tool Selection Rationale
Tools selected 1
Rearrange the test cases into a test set in a specified order
For Android keyboard input, manual cases are arranged by category, such as:
Number keys
Letter keys
Symbol keys
Child keys
All number, letter and symbol keys have child keys:
To simplify automation script development, each child key test case is split into three automation scripts for number keys, letter keys and symbol keys.
This helps to improve the efficiency of :
automation script development
Test execution
Translation: it means rearrange the order of steps in the manual test cases

9. Automation Assessment Automation Tools selection criteriaSN
Tool Selection Rationale
Tools selected 1
Reduce learning curve
Tools which VMware has experience with 2
Reduce creating tools from scratch
Open source automation tools or framework
OS native automation tools
Translation: it means rearrange the order of steps in the manual test cases

10. Automation Tool Selection Tool selection for WindowsOS
Tool
Advantage
Disadvantage
Selected
Windows
Ranorex
Support software, web on IE, Chrome, Firefox, safari, flash.
Good report generation.
Support record and play
Programming language is C#
Less time to deploy and install
Can execute with a executable file.
Commercial License required √
UIAutomation
Support Software only and web on IE.
Programming languages - VB, C#, etc.
Does not support record and play
Need more development base on API
QTP
Support software, web on IE, Chrome, Firefox, safari, flash
Programming language is VB Script
Commercial license required
Need more time to deploy and install

11. Automation Tool Selection Tool selection for MAC and AndroidOS
Tool
Advantage
Disadvantage
Selected
Mac
Apple Script
No License required
Can operate software by properties of controller
For some controllers, need assistance from other tools. √
Sikuli
Can operate any controller by using image
Low recognition efficiency based on coordinates
Need a lot of time to maintain images (eg screenshots, etc)
Android
UiAutomator
Can action by other tool during running.
Can test cross software
Don't resign release apk.
only support android 4.0 and above.
Need identify controller by text for Android 4.0 and 4.1.
Robotium
Cannot operate cross application
Cannot insert action from other tool while running.
Need re-sign APK with debug certification

12. Framework Selection Challenges
Operating software in VM OS
Issue encountered
Beyondsoft solution
Windows
Need change IME
operate notepad in VM
copy test result from VM to local desktop.
Develop a tool to perform these tasks and execute it on VM using STAF
Simulate keyboard input OS
Issue encountered
Beyondsoft solution
Windows
Caps Lock and Num Lock can’t work when send key code from local desktop to VM
Use STAF to simulate keyboard input :
Send Scan Code for Caps Lock
Send Key code for numeric keypad since to Num Lock scan code doesn’t work.
Android
Can’t use key code to simulate all characters/symbols input
Use virtual keyboard key coordinates to simulate keyboard input

13. Automation Development Environment
Ranorex for Windows
UIAutomator for Android
Apple Script for Mac OS
Automation Tools
STAF + Client Scripts + VD side Scripts
Automation Framework
C# -> Windows
JavaScript -> Mac OS
Python & Java -> Android
Program Language
Why we use STAF:
When try to input some special characters via automation from client to VD(virtual desktop), such as: Num lock, it will be failed by unknown reason. So we developed a VD side code, and use STAF to connect the client and VD, the STAF can help to:
Lunch the application in VD, e.g. : Notepad
Lunch input method in VD, and transfer input command from client to VD to simulate the keyboard input.
Copy the txt file from VD to client, then it can verify the actual result on client side.

14. Framework Overview
Client(Local PC)
VM(Remote PC)
Network
Config
Open VM in View Client 1
1. Open VM in View Client in test computer (Action on local desktop)
STAF
2. Change default IME in test computer (Action on local desktop)
Change IME
Automation Test
Controller 2
Copy Assistant Tool 3
Copy Command 3
3. Copy Assistant Tool to VM (Action on remote desktop) 4
Execute Command 4
Change IME
4. Change default IME in VM (Action on remote desktop) 5 5
Open Notepad
5. Open Notepad in VM (Action on remote desktop)
6. Send Key code/scan code to Input to Notepad in VM (Action on local desktop)
Send Key Code / Scan Code to VM 6 7 7
Save Notepad
7. Save Notepad with input content in VM (Action on remote desktop)
8. Copy the saved Notepad from VM to test computer (Action on remote desktop) 8 8
Copy Saved Notepad 8
Copy Command 8 9
Verify Test Result
9. Compare test result in the saved Notepad in test computer (Action on local desktop)
Repeat steps for each case
Report
10. Repeat step 4-8 until test is completed.

15. Efficiency Gained 5 days 12.8%
Note: As more cycles are executed, these numbers will become higher
Days saved for this project
5 days
Efficiency improvement
12.8%