Perfecto Mobile Automation Advanced Scripting Perfecto Mobile Automation

Adding logic to our scripts Agenda Data Driven Testing Loops Adding logic to our scripts Branching Getting device info Conditions Error policy Execute Script User Functions Comparison Checkpoints This module will show how to make your scripts smart, using branching, conditions and data-tables to create scripts that are data-driven and smart enough to deal with events that can happen on the device without breaking the script.

Goals of this Lesson By the end of this lesson, you will learn how to leverage the advanced scripting techniques shown in this modules, sample script, ‘Advanced Scripting’, making your scripts more robust. The sample script can be found in PUBLIC Scripts/Samples folder/Advanced Scripting.

Data Driven Testing What is it? Application frequently display dynamic data that changes all the time Available seats on flights Account balance Today’s top 10 songs Holding the data outside of the script in an external data-table allows the script to work even if the actual data changes all the time One data-table can be used as the input for multiple scripts A Data Table is a multi-column, multi-row table that contains data. It will allow the Perfecto Mobile system to perform automated commands and in each iteration, the data of the active row of the data table will be used. On the last screen, you saw a running automated script. This script ran a loop with a 2-column data table that contained two URL locations and searched for LOGOs on the device screen. It is recommended that all script data is outside the actual script, in a data-table. Remember that it can be imported into PM as CSV/XML (show sample from KB)

Data Driven Testing Continued How to do it Create a CSV file with the data and upload it to the CQLab Repository as a data-table Create a script variable that refers to the data-table Inside the script, create a loop around the data-table The system will iterate over all the rows, using the data of each row per iteration Tips: The data-table can be provided at run-time (more info in later modules on script execution methods)

Data Tables: How to Create: Using a CSV File The simplest way to create a data table is as a CSV or XML file. Create the title row -- Set the Data Type – and provide the data that you want to use. Then save it as a CSV or XML and upload it to your repository for storage.

Loops enable repeating a set of commands Useful for validating script robustness See the GetSnapshots sample, there is a loop and within it a counter that counts successful iterations, in this way we reach a desired number of iterations despite the fact that a particular device may fail. Loops are essential for quality code, running a script on one device is nice and well, but running it 10 times in a loop, on several devices ensures it is robust and covers the test case properly. Loops enable the user to repeat a set of commands. The repetition can be a repeat of a defined number or a loop on a data table. When repeat is selected, the system will iterate the command according to the defined number. Once again, when data table is selected, the system will perform the commands and in each iteration, the data of the active row in the data table will be used.

Data Tables and Loops – How they Work together Once you create a Data Table, and you save it in your repository, you can use and re-use it in your scripts. When you are ready to add it to your script, you will want to create a Loop and select Data Table as your parameter... BUT, before you can do this, you first need to add the Data Table as a Script Variable and then you can add it to your Loop. We learned about Script Variables back in Lesson 4.

Putting it Together Thus Far… [TRAINER] show the sample script – show the loop – show the Data Table Putting it Together Thus Far Here is an example of Data Driven Testing. We are opening a device and going into a loop. A loop is Data Driven. In this script [show sample script DT] = we are validating the login functionality. When we log into any application, we expect certain behavior: access granted with the proper credentials and conversely, access denied without them. In this script, 3 scenarios were created – Correct U/N and P/W Incorrect U/N with Correct P/W Incorrect U/N and Incorrect P/W These 3 scenarios will deliver certain behaviors…and this is what we are testing. Tomorrow morning, we could be tasked with updating this data table from 3 credentials to 500 or even 5000 and it won’t affect the script in any way, because the data is OUTSIDE the script. This is what Data Driven Testing is all about.

Conditions Branching enables us to change the course of the script according to a condition we define, in other words an IF/Else statement The Condition command is used for branching How to use The command evaluates the line preceding it and according to its success/fail performs the commands in the success/failure The preceding line can be set to catch error policy in cases where there is no failure, just a flow issue (location popup for example) A condition allows the user to change the flow of the script according to a set criteria. It is evaluated according to the result of the preceding function. **Notice that the results used to evaluate the condition can be from any function, including Wait. Ensure that the function to be evaluated is located directly above the condition in the script. It is possible to enter any set of commands inside the condition path including nested conditions and groups. The evaluated command can be used in conjunction with the catch error policy. This will avoid marking it as 'pass' or 'fail' in the report, leaving it as the condition input only.

Conditions Continued When do we need it? for things that may appear and I’ll need to handle (pop up asking for upgrade, click no, pop up allowing location, say yes etc) The type of device mandates a different flow Screen size OS OS version Use “Device info” command to find info on device Events on device The location popup will not always appear, it depends on the device settings, browser cookies etc. A robust script should be able to deal with it IF it happens, and continue if it does not. For this a condition is ideal. The logic is basically “if the the popup occurs, deal with it, if not continue as normal” Device Type In some cases, there will be differences according to the device type. [open the objects sample script and see the condition that deals with a different app identifier in iOS and Android] Device Info The device info command lets us query the device for its properties and then use that data in a condition (the way to do it is through a checkpoint to evaluate the info)

Comparison Checkpoints The comparison checkpoints evaluate a string or number and return true or false status The status can be used as an input for a condition Check out the “getSnapshots” and “Advanced Scripting” samples and see how they are used Comparison Checkpoints are a way to insert logic. It answers the question, “is this string equal to that string?”, “is this number equal to that number?’ In the sample script, a comparison checkpoint was added to determine if the login functionality per credential type, was the correct behavior. In other words, were the login credentials valid? Yes or No. The evaluation is based on credential type, which means that the comparison questioned whether the user name was correct in one comparison. In another, it was the password that was being assessed, and the answers to these requests were listed as comments within the report.

Adding Logic [TRAINER] Go back to the sample script and explain the logic behind it. Show the Data Table. Running in the loop, we first are using the data from the data table to login and click Signin. Here’s where the logic comes in…. The login is not the test itself. We are looking to validate the webpage response to 3 different types of login scenarios. Before we can find out what it is that we want to validate, We first want to ask, ‘what am I looking for’ and check for that. What should the page display, because different kinds of data will be typed into the page. Here is where we use the String Compare Checkpoint. It does the following comparison: It goes to the “USERNAME VALID” field and checks whether it is equal to “YES”. [SHOW THIS] Then, by adding the Condition feature, which will allow the script to react according the result of that comparison, success or failure, the script becomes more robust. If the ‘username valid’ is not valid, another string compare checkpoint queries whether the password is valid, which requires another condition. If both username and password are incorrect, we expect the webpage to state “invalid credentials”. This is a success, because it is the correct behavior in that scenario. The logic behind this is the same as expected if you were to go to a house that is not your own and you insert your key in to the lock. The lock should not open. We have another scenario where the username is invalid, but the password is valid. In this case, there is yet another error message, stating ‘Invalid Login Name’. This is being checked as well. What we are doing is using 3 features – Data Driven testing: which tests the functionality with different types of data in a loop and the amount is unlimited String Compare and Condition: to check on ‘What we are actually expecting to see’ ‘Did we provide correct data or not? And, if it is incorrect, as the two scenarios above, incorrect of what? According to the logic, they were error messages that should have been seen and functionality that we should have been getting. All of this is validated with a checkpoint. Bottom line: 3 features – all covered within this lesson to check a login functionality in a better way that how we did it before. In the previous sample script, all that we did was login with correct credentials, which is fine if all you want to do is test something after the login page. But as a tester, if it’s your job to test the login functionality then you will need to dive a little deeper, as we have done here. We are using various logical features.  

Defines the script flow on error execution cases Error Policy Defines the script flow on error execution cases Ignore – Continue with the script flow even if the command fails. Marked as 'failed' in the report Abort – Terminate the execution of the script and any other calling script Break – Inside a loop, stop the current loop and continue to the next command after the loop. Outside of a loop, acts as 'abort‘ Continue – Inside a loop, stop the current iteration and continue to the next iteration. Outside of a loop, acts as 'abort Catch – Use the result as input to condition statement. Marked as 'Catch' in the report (not success or failure) All commands have an error policy that dictates to them what to do upon failure. The default error policy of most commands is “continue” which means that the script will be aborted. Inside a loop continue jumps to the next iteration. It is possible to change the error policy, here are a few use cases where this may come in handy In the location popup example which we just saw, the popup is not fatal – i.e. the correct way to handle it is with a checkpoint that looks for it, and then if it finds it clicks on it. In this case Catch is the recommended error policy. Catch will be caught by the condition commands, but its results will not affect the script status in the report. It basically says, ‘Do not mark script as a fail’. Therefore the script will not fail, but mark it as a fail at runtime. Validations like text checkpoint are set to “ignore” by default. But if a validation is fundamental to the test case and it fails, there is no point in continuing the script (for example, a login to an account fails, so no value in trying to perform transactions inside the account, so in this case “abort” can be used Perfecto Mobile

Error Policy Examples Error Policy Examples Login to website – Edit Set Text (1st one) – Error Policy – Continue In this example, when entering the credentials, the error policy is continue because if there is a problem with a browser or device, and we don’t get the ability to enter the user name, there is nothing more that we can do. We can’t login, so we can’t validate the credentials and continue with the script. So, we get out of the loop and try again. Validate – String Compare Checkpoint – Error Policy – CATCH The reason for Catch – is simply that we don’t know in advance what type of credential we’re using – because it is data driven. The script works for both valid and invalid credentials, so it will be found in any case. That is why it is a catch, it’s not a mistake. It’s intentional, so that the script doesn’t fail.

Execute Script runs a script from within a script It can be used to execute commonly used function i.e. login to application Can be used in async mode The primary value of this is a “master script” that calls sub scripts, however before using consider the advantages of a user functuion which is in essence a “smarter” execute script. The Execute script function invokes a script located in the CQLab repository. This functionality allows users to include sub-scripts within a script. The script parameters must be entered in the same order with the correct types as defined in the original script. The script will not work otherwise. When using Embedded script execution, the devices used are shared between the parent and child scripts; whereas Sync or Async run separate script executions, and the devices are not shared. In Async script execution, multiple sub-scripts can be invoked and executed simultaneously within a script. As a best practice, rather than manually inputting the parameter details, locate and drag the desired script from the CQLab repository directly into the script.

User Function User functions enable branching by different device criteria (e.g. login script for Android and one for iOS) User functions wrap scenarios where there are differences across devices, maintaining one script for multiple devices. The user function feature is useful when testing the same functionality on multiple devices Execute script simply calls another script, user function is a more sophisticated ability, it allows calling another script based on the type of device the script is running on.

Let’s Create a User Function Together... Step 1: Open the User defined functions tab from the left panel in the Automation tab Step 2:  Select a folder, where the user function will be saved Step 3:  Click the add icon to display the Add User Function dialog box 3.1 Name and Description - Enter the name and description of the user function 3.2 Parameters -  Add run-time parameters individually, OR  Select an implementation script to add its run-time parameters that will be inserted automatically 3.3 Matching rules - Click the + to display the User Function Rule dialog box Step 1: Open the User defined functions tab from the left panel in the Automation tab Step 2:  Select a folder, where the user function will be saved Step 3:  Click the add icon to display the Add User Function dialog box 3.1 Name and Description - Enter the name and description of the user function 3.2 Parameters -  Add run-time parameters individually, OR  Select an implementation script to add its run-time parameters that will be inserted automatically 3.3 Matching rules - Click the + to display the User Function Rule dialog box Perfecto Mobile

Let’s Create a User Function Together... Name and Description – Enter name and description of User Function here. Add Run-time parameters individually or select an implementation script to add its run-time parameters. The latter will be inserted automatically. In this example, we’ve added a device. Step 1: Open the User defined functions tab from the left panel in the Automation tab Step 2:  Select a folder, where the user function will be saved Step 3:  Click the add icon to display the Add User Function dialog box 3.1 Name and Description - Enter the name and description of the user function 3.2 Parameters -  Add run-time parameters individually, OR  Select an implementation script to add its run-time parameters that will be inserted automatically 3.3 Matching rules - Click the + to display the User Function Rule dialog box Click ‘+’ to display the User Function Rule dialog box. Perfecto Mobile

Let’s Create a User Function Together... Select Reference Device Select Criteria. Selecting multiple criteria requires all criteria to be met. In this example: Manufacturer – Apple Platform – iOS OS Version – 8.0 This will be true only if the device is manufacturer by Apple AND is on the iOS platform AND is running OS version 8.0. ** (see below) Step 4: Define criteria from the User Function Rule dialog box  For each test criteria, 4.1 Select reference device 4.2 Select criteria - Selecting multiple criteria requires all criteria to be met. For example, manufacturer Apple, iOS platform  OS 8.0 will be true only if the device is manufactured by Apple AND on the iOS platform AND running iOS 8.0 4.3 Select implementation script - Select an existing implementation script or Click New to create a new implementation script. Step 5: Validate the implementation script parameters Step 6:  Click OK to save the User Function It is possible to add a default rule, that will run for all cases. To implement: open the User Function Rule and select an implementation script without selecting a device. Access your implementation script in one click by using the Edit Script button. Select the implementation script – select an existing one or create a new one. **It is possible to have a default script selected to deal with ‘all other cases’. Simply select a device and script. Do NOT select criteria.** Perfecto Mobile

Let’s Create a User Function Together 1. Validate the implementation script parameters 2. Click OK to save USER Function. It is also possible to add a default rule that will run for all cases. To implement: Open the User Function Rule and select an implementation script without selecting a device. Access your implementation script in one click by using the Edit Script button.

User Function vs. Execute Script User Function in Script vs. Execute Script in Script [TRAINER - open user function – Samples –GoToWebsite] One was created for iOS and one was created for Android. Why was this done? Go into the Android script (highlight Android, click on script icon – last icon above Cancel). This opens the subscript – on Android, we want to Open the Browser, and perform a fresh login – no cache, no history, etc…then Browser Clean, which more or less reinstalls Chrome from scratch onto the device. Explain how the first time you do this, and you go to a website, you are required to accept Google Terms and Conditions, which is why we have that as a checkpoint (show Text Checkpoint) because we get passed that and deal with it. What happens on an iOS device? (Open Subscript in Script repository) – the script looks a bit shorter, because we don’t yet have a Browser Clean functionality for Safari. Explain why you are working with the User Function. There is a difference between platforms – show how you could go to the lowest level and say both operating systems use a browser go-to…but, because we have a better level of functionality in the Android device now, we take advantage of that and wrap it in a user function. Because Mobile is dynamic, Perfecto Mobile will provide the same functionality to Safari later on and we will add that in. There will always be differences between platforms and that is why we simply wrapped that up in a user function and once we fix that, then we fix that in this little sub-script that 50 scripts can refer to. That’s the value of the user function. Why wouldn’t use an Execute Script instead? In order to use Execute Script, I would have needed to create a script like this: [Show this] Device Info – then determine the Operating Systems and Values – insert them into parameters – evaluate the parameters – etc…. That is reason #1 why we didn’t use Execute Script over User Function. User Function actually does all of that right out of the box. The 2nd reason we didn’t use Execute Script is that what happens if suddenly (for example) we have a difference between iOS 7 and iOS 8? If I am using a User Function, all we need to do is –SHOW Edit User Function -> Select Device -> select the device]) and add the function rule for that device. It’s editable. Another powerful reason to use User Functions – you create a User function (Go To Website) for your test case. Any other tester in your team can simply drag and drop that right into their scripts and utilize it. It’s completely reusable.

Browser Clean Supports Android devices and Chrome browser only. Advantages: Removes Cookies, Cache, and previous browsing history. Disadvantages: Reinstalls Chrome, requiring Terms of Service to be accepted to continue navigation to the selected URL. 2 ways to address this pop-up: Write a user function or individual script and add it after the Browser Clean command. Remember that Browser Clean does not have to run on every script every time. It can be run daily or weekly using a scheduler. Browser clean is a feature that removes all cookies, cache and previous browsing history on an Android device. It will perform a fresh install of Chrome. When this occurs, however, it requires then acceptance of Google Chromes Terms and Conditions, before navigating to the URL that you are seeking. This is only used on Android devices when Chrome is the browser of choice. It is not yet supported on iOS devices nor does it affect the default Android browser. There are a couple of things that you can do to get around the Terms and Conditions page, Write a script that accepts Terms and Conditions, which you can run after the Browser Clean command and then continue on with the script....this will need to be written and maintained. The other option is to remember that the Browser Clean command is not necessary all the time and in every script. It will depend on the application and what is needed to be tested. Usually, many customers run Browser Clean on a daily or weekly basis and they don’t need to do it on every test case. Ultimately that is your choice. It can be run by a scheduler so that it can be done automatically.

Resources Data-Tables - https://community.perfectomobile.com/posts/912492-loops-and-datatables Conditions - https://community.perfectomobile.com/posts/912487-condition User Function - https://community.perfectomobile.com/posts/912495-user-function

Thank You