Understanding XML and XPath

Lesson Overview Visual analysis works using the device screen as input, and analyzing it Object analysis retrieves the XML and analyzes it This module includes: Intro to XML The Perfecto Object Spy Working with XPath Correctly identifying objects *This is an optional lesson and recommended for users just getting started with XML and XPath. [Trainer notes: this lesson can be skipped if familiar with XML and XPath.] This is an optional lesson. If you have experience with XML and XPath, this module can be skipped. Please note that this lesson is a high level overview of XML and XPath only. It is recommended that you continue on and review the contents in the Resources link, located in the player for more details.

Prerequisite – STOP! To effectively read the XML documents, Perfecto Mobile recommends the following when working with web elements: Mozilla Firefox - Firebug and FirePath add-ons Google Chrome - XPath Helper add-on Microsoft Internet Explorer - Internet Explorer Developer Toolbar and Bookmarklets Please install before continuing. To effectively read XML documents, Perfecto Mobile Best Practice recommends installing Mozilla Firefox with add-on’s Firebug and Firepath, which will allow us to see the complete object tree, which will allow you to easily identify your objects and write/test XPath. Firebug - https://addons.mozilla.org/en-US/firefox/addon/firebug/?src=search Firepath - https://addons.mozilla.org/en-US/firefox/addon/firepath/ IE Developer Toolbar - http://www.microsoft.com/en-us/download/details.aspx?id=18359 Xpath Helper for Chrome - https://chrome.google.com/webstore/detail/xpath-helper/hgimnogjllphhhkhlmebbmlgjoejdpjl?hl=en

Why is understanding XML and XPath SO important? Native Object analysis relies on XML. Object Analysis retrieves the XML document and analyzes it. However, it this is not enough to identify the object using its location in the object tree. This approach is fragile. It is necessary to use specific XPath expressions to uniquely identify the object. This will enable your object to be found across multiple platforms and devices. Our goal in this lesson: is to provide you with the ability to identify objects correctly and accurately by using unique object properties instead of the location of the object in the Object Tree. Let's do a quick recap of native objects and then jump in to XML and XPath.

RECAP: Native Objects Analysis The Object Tree is an XML document The way to find elements inside XML is through XPath The Resources section at the end contains links to read up on it To properly use Objects, a good grasp of XML/XPath is required. It is well worth the effort to read the resources after going over this module. How it works: -Receives Object tree (XML) from App or Website Main Features: -All objects are exposed by Perfecto Mobile -Objects can be clicked, set or querired Advantages: -Clear logical script -100% accurate -XPath allows defining objects logically resulting in robust, easily maintainable, and cross platform scripts -Object enables reading and analyzing text -Fast performance Disadvantages: -Object tree is abstract and NOT what the user sees -Requires platform support

What is XML? Basic Object Tree In Object-based Scripting The OBJECT TREE is an XML document. What is XML? XML (Extensible Markup Language) is one of the most widely-used formats for sharing structured information (data) today: between programs, between people, between computers and people; both locally and across networks. If you are familiar with HTML, this is very similar. However, XML syntax rules are very strict. XML tools will not process files that contain errors, but instead will give you an error message so that you fix them. This means that almost all XML documents can be processed reliably by computer software. The object input works on an XML document. This means that the objects displayed on your device screen (web or native application) are represented in an XML Document. Once we identify the element using some kind of path, we can apply an action to it, such as click or a request to retrieve its info. Figure 1 (top) A basic Object Tree Figure 2 (bottom) A basic XML document Basic XML Doc

Terms that you will find in an XML document Elements (are also called Nodes) Elements are nodes encased in tags. In the sample above, <vehicles> and <vehicle> are elements. This markup begins with a < and ends with a >. Start Tag: <vehicles> End Tag: </vehicles> An element can have a value, the value is the content between the Start Tag and End Tag. This is a small XML document that we will work with in this lesson. It contains everything that we need to learn. We need to begin by identifying the element or object on the device screen that we would like to work with. The document contains elements, (also called nodes), attributes and their values. Mastering these terms is essential, since they serve as our foundation. ELEMENTS An element can have any name. XML does not limit the names -You can have multiple elements with the same name. -When we come to analyze a long and complex XML file, we will encounter many such duplications. This is why we need to delve deeper and find – the XPath term is ‘query’ – exactly what we are looking for. For example, we just learned that elements have values, so we can therefore specify not only the element name, but also its value.

Terms that you will find in an XML document Attributes Attributes consist of a name/value pair that exists within a start tag. An attribute describes the element in a key="value" format Any element can have zero (0) or more attributes Each attribute must have a value encased in quotation marks. Each attribute has a name and value Example: color="red" and make="Honda" Attributes allow a “description” that goes beyond just the element name. In the last example XML document (on the Elements page), we included one attribute, make, that describes each vehicle element. However, let’s say we want to find all the cars of a particular color? The vehicle node as it is defined here, is simply not good enough; but, if we add a color attribute, to our XML document, we can narrow our search and say “give me a blue vehicle". Now, our slightly longer XML document, listed above, includes another attribute that identifies the color characteristic of each vehicle. Now that we have learned the basics of elements and attributes, and we understand that we need both in order to find our objects, let’s put this all together…click next.

Which is which? For Trainer [ Audience participation] Which are elements? Which are the attributes? Which are the attribute values?

What is XPath? An industry standard XPath is the language for finding information in an XML document. XPath,models an XML document as a tree of nodes. An XPath expression is a mechanism for navigating through and selecting nodes from the XML document. An industry standard XPath is a language for addressing specific parts of an XML document. It is similar to SQL – a language used to communicate with a database You can use it to query an object in an XML document and retrieve its properties It enables easy identification of objects that we may have not been able to identify visually In automation, Xpath allows us to identify the object/objects This lesson will cover XPath basics – why use it? how to use it? And finally how to use it in mobile with the Perfecto Mobile platform. Additional lessons will follow this one, and will describe how to use XPath in your scripts to its fullest potential. XPath is an industry standard, developed by the World Wide Web Consortium (W3C). (reference: http://www.w3.org/TR/xpath-30/) It is a language to query an XML document.

What is XPath? (continued) Using the XML example above, XPath uses path expressions to select nodes or node-sets (<vehicle> or <vehicles>) in an XML document. The path expression is the "What I am looking for". The more accurate the expression, the more robust the result. The Result? The result of a standard XPath expression is a list of all the matching nodes. (XPath is case sensitive) Example: "What am I looking for" = "I am looking for all cars (vehicles) that are made by Toyota“. The result is highlighted above.

Understanding what’s going on So, what’s really going on? The object input works on an XML document The required element is identified by a path Once identified, then we can apply an action on the element Click – on a button, link or other object Set – an edit field with a value Info – to retrieve an element to receive its properties Other… Let’s take a step back, before we get started, and understand how objects work. The object input works on an XML document. This means that the objects displayed on your device screen (web or application) are represented in an XML document. Once we identify the element using some kind of a path, we can apply an action to it, such as a click or a request to retrieve its info.

The object input works on an XML document: Breaking it down - Step 1 The object input works on an XML document: The item, on the device screen, that we want to work with, is called an element or object. Translated into a plain English: "I want to work on this button or link" Then, to complete this action, the system needs to know which object to work on. The first thing we need to do is identify the element or object on the device screen that we want to work with. In plain English, we translate this to say: “I want to click on this button.” This translation is what we will use to create a path to identify this object.

Breaking it down - Step 2 The required element is identified by a path: When you select an object, a default path to the object is generated. The path is hierarchical. It says WHERE the object is located on the page (within the Object Tree). Translated into plain English, it is just like saying: "Go to the 3rd hours, around the first corner, next to the big tree, opposite the store..." Example of the default path generated: //device/window/view[1]/view[1]/view[1]/view[1]/secure[1] Translation of this default path: "Search for the element called secure, under an element called view, under an element called view, under..." Bottom line: the path is LONG, Complicated, and Fragile! One single change to the page, changes the ENTIRE PATH!! The default path that is generated identifies the location of the element in the object tree. Device -> Window -> View -> View -> View -> View -> Secure

Let’s complicate it even more… The same object iOS and Android Let’s complicate it even more… The same object in iOS: //device/window/view[1]/view[1]/view[1]/view[1]/secure[1] in Android: //device/view/group[1]/group[1]/group[2]/group[1]/secure[1] The default path is different in iOS and Android This means that the script will not work on both devices! When we inspect the element on the Android device, we can see that the path is completely different. This means that if we use the default path, our script will not work on both devices. It will not be ScriptOnce!

iOS and Android (continued) What about ScriptOnce™? The default path is different in the iOS and Android apps! iOS //device/window/view[1]/view[1]/view[1]/view[1]/secure[1] Android //device/view/group[1]/group[1]/group[2]/group[1]/secure[1] To make your script work on multiple devices, ScriptOnce, you need to check it on multiple devices. When opening the Object Spy on an iOS and an Android device, we can see that the object’s location in the object tree is different. This means that we cannot use the same default path generated by the Object Spy. Android iOS

Example of Native Object Analysis Let’s say that you want to work on a username edit field. The default path generated by the Object Spy refers to the object, (the username edit field) by its placement on the page. This is the WHERE. Is this a robust way to script this? What’s the solution? Trainer: [Open the discussion – is this a robust way to script this?] Answer is NO The placement of the edit field can change or be different in another platform or device type.(Example: Smartphone and Tablet, iOS and Android) What is the solution? Instead of referring to the "WHERE", we want to focus on the logical function, "WHAT". Regardless of its arbitrary placement on a page, the edit field, named 'Username' will be found, because we are looking for what it is: An edit field named "Username".

Writing an XPath expression What to do? Write a meaningful expression The expression identifies the object by WHAT it is and not just WHERE it is We must identify the object according to its unique characteristics. How does the Username Edit Field stand apart from other edit fields? Identifying the uniqueness of this field, and writing an XPath expression that explicitly refers to what it is, instead of where it is located, will ensure your objects based scripts will require minimal maintenance and are likely to be identical across various devices and platforms. So, what do we need to do? We must identify the object according to its unique characteristics. A real life example: Let’s talk about the WHAT and WHERE Think of a scenario where you need to describe how to find the tomatoes in a grocery store, based on their physical location in the store. In other words, “enter the store, turn left, go straight and pass five isles, make a right after the oranges, and a left at the lemons”. What would happen if something changes (like the oranges moved somewhere else in the store, or the tomatoes moved)? Or, what if you went to a different grocery store? How would you find the location of the tomatoes then? It’s just easier to just say “go to the produce section”. This is the WHAT and not the WHERE.

Writing an XPath Expression How can we identify the object correctly? First, we need to read the XML and find the object within it. Once we understand what makes the object unique, we can write an XPath that will be able to find it no matter where it is in the object tree. A simple trick is to write, in plain English, what you are looking for, and then translate it into XPath.  

Firebug and FirePath (Firefox Add-ons) Why do we need the Firebug? The complete tree is not displayed in the Object Spy Export the XML document into a separate browser tab from the object spy This will enable you to easily identify your objects and write/test XPaths So how do we read the xml? Firebug and FirePath Firefox add-ons are useful extensions when working with web elements. They are recommended to use by Perfecto Mobile.

Working with Firebug and FirePath Element you want to work with Default XPath [TRAINER – show how to use Firebug and Firepath] Reading the XML document Open Firebug (top right of the Firefox browser menu bar) Inspect the element you want to work with by clicking on the Firebug inspector and then on the object in the page The object in the page and the location of it in the tree will be highlighted. Also, a default XPath will be automatically provided. Read the XML, understand what makes the element unique and write an XPath that will be able to find it (more on writing XPath expressions later on) Note: More on how to write XPath expressions will be covered later. Firebug Inspector

The power of using XPath with objects Objects allow creating abstract logical scripts The logic should refer to the specific characteristic of the object To harness this power, custom identification of objects should be used The default path provided in object spy is fragile Using Xpath, we can have a robust cross device script that enables advanced screen analysis, even with dynamic data The elements inside an application or website have a logical meaning a “login” button or “my account” link are not just the third or fourth element, they also have a specific purpose inside the application. Identifying these elements according to what they are, and not just according to where they are will result in a script that is much more robust, readable and valid for multiple devices and platforms. For example, the login button will always stay the login button, another element may be added before it changing the where but the what will remain the same.

Example of Power of Xpath with Objects Let's say that we want to work on a username field. Remember: The default path generated by the Object Spy refers to its location; its placement on the page. The placement of that object can change, or be slightly different in another platform (i.e. tablet vs. smartphone). So, it's important to refer to it by its logical function instead of an arbitrary placement on the page. XPath will allow this to be specified: //[@username] This expression explicitly refers to the input field called 'username'. Advantages: It is easy to read. It is likely to work well with minimal maintenance. It is likely to be identical across devices and platforms.

XPath Expression Syntax Description nodename Selects all nodes with the name "nodename" // Selects nodes in the document from the current node that match the selection no matter where they are //vehicle Selects all vehicle elements no matter where they are in the document //@make Selects all attributes named make //*[@make="Toyota"] Select all attributes named make with value of Toyota //vehicle[@*] Selects all vehicle elements which have any attribute //*[text()=’Corolla’] Select all objects with the text Corolla //vehicle[2] Select the second vehicle element XPath allows us to specify: "Give me an element named vehicle that has an attribute with a value Toyota". XPath enables powerful and complex queries quickly and easily. Below is a list of the most common XPath syntax you will need to understand. Copyright 2013 Perfecto Mobile

Which XPath expression is the correct one? “I am looking for an element called vehicle which has an attribute of make with the value of Toyota” 1. //*[text()=‘Corolla’] 2. //@make=‘Toyota’ 3. //@color=‘blue’ 4. //vehicle[@make=‘Toyota’] Answers: 1. This could work, if.... Your site/app page does not contain any other text reference to Toyota models. This is looking for a specific model name, not the manufacturer of a fleet of automobiles. Otherwise, this may not be unique enough.... 2. This could work, if.... Your site/app page does not contain additional elements (other than 'vehicle'). Or, it contains the same attribute and value pair. (make = Toyota) This may not be unique enough.... 3. This could work, but.... It's not specifically what is being asked for...We are asking for the make of the vehicle, and while the color is unique, it is not correct. 4. This is correct. We are stating very specifically (and uniquely): The element is vehicles The attribute we are looking for (make) and its value, Toyota.

Which XPath Expression is the Correct One? “I am looking for an element called vehicle which has an attribute of color with the value of black. 1. //*[text()=‘Civic’] 2. //@make=‘Honda’ 3. //@color=‘black’ 4. //vehicle[@make=‘black’] Answers: 1. This will not work. This is looking for a specific model name, and not the actual color of the vehicle. What happens if there are multiple black cars? This will not work. 2. This will not work. It is looking for all Hondas, not all black cars. What happens if there is a black Toyota car in the document? It won't find it. 3. This will work, but.... It will pull up anything on the site that has a color attribute that equals black. So....what if this site also sells other items that are black in color? We are specifically asking for a vehicle. While this will work, it could still be more unique. 4. This is correct. We are stating very specifically (and uniquely): The element is vehicles The attribute we are looking for (color) and its value, black.

One More for Good Measure…. “I am looking for all objects with the text ‘Silverado’. 1. //*[text()=‘Silverado’] 2. //@make=‘Chevrolet’ 3. //@color=‘red’ 4. //vehicle[@make=‘Chevrolet’] Answers: 1. This is correct. Your looking for all objects with the text 'Silverado'. The * indicates 'all objects'. Check the others to see why they won't work, if you are not sure. 2. This could work, if.... Your site/app page does not contain any other Chevrolet vehicles. If they do, it will not just pull up the Silverado. It will pull up all Chevrolets. This may not be unique enough.... 3This could work, but.... It's not specifically what is being asked for...We are asking for the text, 'Silverado'. While the Silverado happens to be red, this path will pull up all objects that are red on the page and therefore, this is not correct. 4. This could work, but.... While the Silverado is manufactured by Chevrolet, many other vehicles are as well. So, if you have multiple Chevrolet objects on the page, this will pull up all of them. In addition, if you had multiple elements, besides 'Vehicle', and Silverado was listed within these other areas, you will miss them.

What have we learned? In this lesson, Understanding XML and XPath, we've learned: A high level overview of Objects using XML and XPath. How to recognize XML documents and the terms associated with them: elements and attributes. The importance of creating unique XPath expressions for objects based on what they are, logically, and not where they are located. We've reviewed some of the basic XPath expressions. Perfecto Mobile supports all standard XPath commands. We've introduced you to Firefox's Firebug and Firepath; tools that will aid you in working with XML and finding those unique options for your XPath expressions. Perfecto Mobile Best Practice: Use Firefox to generate a robust XPath and replace the default path generated. [Trainer – Please reiterate:] This lesson, in no way covers the complete teachings of XML and XPath. It will be necessary for you to continue reading on, using the materials provided in the Resource link, to fully understand these formats. In the next few lessons, we'll delve deeper and deeper into Advanced Objects, which will require a more complex understanding of these subjects than we've covered here. Copyright 2013 Perfecto Mobile

Resources Sample site: References http://nxc.co.il/demoaut/pages/xpath1.xml References Working with Object Spy & XPath http://help.perfectomobile.com/article/AA-02514 XPath Tutorial http://www.w3schools.com/XPath/ XML Tutorial http://www.w3schools.com/xml/

Thank You