1. GALINA MAKAGONOVA GADGET ON THE BUDGET How to budget, create and operate a Device Test Lab

2. Average size of mobile development team is 20-70 people GAMES MOBILE APPS MOBILE WEB

3. Our Agenda 1. WORKSHOP Device lab requirements Device parameters Planning: location, equipment, software, storage Budget Device lab technician: role description and responsibilities Where to buy devices and overview of issues Lab Maintenance Device usage in Agile development team How to present device lab during negotiations with the clients 2. PRACTICE and Q&A

4. OPTIONS TO EXPLORE OR NO DEVICE LAB?

5. NO DEVICE LAB Developers and testers are using their own devices Application delivered to client without testing Or Using device emulator Or BYOD policy

6. External Test Lab TYPES: 1.Device Rental Labs 2.Crowd solution (using private devices across the globe) 3.Free device lab PRO: no need to invest in device acquisition CONS: 1.Cost:$15-25 per device per usage, $25-$30 per hour device+ tester 2.Confidentiality: your app might be exposed to the market 3.Timeline: Sending out your testers at the end of test cycle 4.No flexibility in OS update/Version 5.Your target devices may not be available 6.Booking in advance 7.No way to reproduce bug on device 8.Developers cannot investigate the bugs and validate features Overview: Possible roadblocks

7. Do You Need Internal Test Lab 1.What kind of mobile application are you developing? Mobile Web? Games? Mobile apps? 2.How many projects do you have at the same time? 3.How many developers are in the company? 4.Are development and QA co-located? 5.Do you have a test department? How many testers? 6.Are you developing the app for Android/iOS market? Or for your clients? 7.Do you have contract obligations to test the app on mobile devices? 8.Agile or Waterfall? 9.When is Device test lab usage required? At the end of development cycle or for everyday usage? Overview: various roadblocks

8. Business Focus 1)Who are the users? 2)Who will pay (how to negotiate with the clients or get management buy-in)? 3)How many devices do you need or could you afford? 4)Device selection/Device matrix 5)Device fragmentation (specifically, Android fragmentation) 6)Managing device inventory 7)What is the budget? 8)Device selection should be relevant for a year, preferably for 2 9)Return of investment: what is the value of your lab?

9. Who Are The Internal Users Of A Mobile Test Lab? 1.Development teams: How many? 2.Developers – how many? 3.Testers 4.Designers 5.Artists 6.Customer support / Publishing 7.Sales 8.User acceptance coordinators

10. Who: Target audience Where to get info: from GD, Sales, Data analytics, GOOGLE 1.Market: countries 2.How secure the application is? Do you need to test hackers attacks? 3.Age and gender of the users Overview: various roadblocks

11. And: Technical requirements Where to get info: from GD, Lead developers, game teams 1.IOS and Android? Android only? iOS only? 2.Any specific hardware requirements (i.e. gyroscope) 3.Screen requirements 4.Memory requirements 5.Graphic processor requirements 6.Other specific requirements 7.Phones and tablets? Or just phones? Or only tablets? 8.Pre-installed applications 9.Application to download Overview: various roadblocks

12. Device Parameters: CPU GPU RESOLUTION CHIPSET SENSORS YEAR RELEASED PRICE MARKET RELEASED

13. Device Lab Planning Centralized or Distributed? Static lab: Round table for testing (devices are co-located, anyone who want to test should come there. Agile world: Devices are assigned per team and per tester, with visibility for the whole team: anyone who needs it could borrow for 1 hour or so. Location Storage Space Tracking tool: I’m using Meraki Cables, cleaning, power supply Inventory Bookkeeping/Accounting Internal Policy Disasters happen Overview: Common mistakes

14. Budget List of devices Cost of devices Cost of tools and supplies Cost of device lab (storage, etc) Cost approval process Lab technician (salary) Who will pay Overview: typical mistakes

15. More About Device Lab Budget Per project Per tester Track device life Maintenance materials Plastic gloves Cables Power supply Charging stations Tape for marking cables and devices SIM cards Budget for testing mobile data

16. Device Lab Technician Keep inventory changes Track lost devices Plan and manage OS updates Root devices if needed Sideload client build Charge all devices Gather devices spread across office Create device test list per app Perform device testing per app Perform compliance testing

17. What To Buy And How Many List devices to cover the technical spec (low end, high end, phones and tablets) How many teams do you have? How many developers per team? Total? How many testers per team? Total? Each QA should have 1 Android and 1 IOS gadget (1 phone, 1 tablet) Each team should have 1 high end, 1 low end (both iOS and Android) Who managed provisioning profile? How many devices are already in? Some samples of requests to address

18. Buying devices *** No contract *** New or used *** Beware of jailbroken devices Amazon E-bay Classifieds Cell phone providers Buying old devices from staff (how much do you want to pay)

19. Lab Maintenance Inventory (locate each gadget bi-monthly) Mark each device, cable and charger Cable replacement Monitor chargers Buy sleeves Monitor upcoming OS changes Monitor new devices on the market Manage device assignments to team members

20. Negotiation Presenting options with cost and risk Cost of device rent per test cycle, availability Outsource – cost per cycle Developers and QA need devices during development cycle In-house, including Cost of device tester Quantity of test cycles List of devices per app requirements *** Who will own devices at the end of development *** Add 1/6 of device budget every 6 months to keep current

21. Workshop List of 30 Devices with larger market share (samples) Operation system diversity Practical exercise: reviewing application requirements and selecting 10 top devices Practical exercise: Working on device lab budget Practical exercise: Managing device test lab Practical exercise: Device compatibility checklist

22. Galina Makagonova Gadget on the budget Thank you very much for attending! See you soon for the hands-on Training. Galina Makagonova QA Director of Roadhouse Interactions, Vancouver, Canada