1. Microsoft Robotics Studio Simulation Kyle Johns Software Development Engineer Microsoft Corporation

2. Presentation Overview Why a simulator? Simulator architecture Using the simulator Entities and services Creating simulator scenes Articulated arms

3. Why a Simulator? Robotics hardware is expensive Hardware can be difficult to debug Hard for a team to work concurrently with limited hardware

4. Why a Simulator? Benefits Low barrier to entry Staged approach Easy prototyping Useful for education Good learning and research tool

5. Why a Simulator? Limitations Lack of noisy data Incomplete or inaccurate models Accurate tuning takes time

6. Simulator Architecture The Simulator Engine Service Implemented as a service Maintains world state Manages input devices 3D rendering using XNA Ageia Physics Simulation Graphical User Interface Editor for modeling and debugging Simulation Engine Service XNA Graphics Library Display Hardware Ageia Physics Engine User Interface / Editor

7. Simulator Architecture Entities Entities are objects in the simulated world Visual component Physics shapes Represent:Cameras Sky and Ground Robot components Motors, sensors Buildings & Furniture Anything visible or physical Simulation Engine Service XNA Graphics Library Display Hardware Ageia Physics Engine User Interface / Editor Sky Ground Robot Camera

8. Simulator Architecture Services associated with entities Services link with some entities Drive motors Gather sensor data Modify the world Simulation Engine Service XNA Graphics Library Display Hardware Ageia Physics Engine User Interface / Editor Sky Ground Robot Camera Drive Service Webcam Service

9. Simulator Architecture Orchestration services Provides the “intelligence” Interprets sensor data Commands motors Can directly control the simulated world Simulation Engine Service XNA Graphics Library Display Hardware Ageia Physics Engine User Interface / Editor Sky Ground Robot Camera Webcam Service Orchestration Service Drive Service

10. Starting the Simulator From the Start Menu From the Control Panel From a manifest From another service

11. Starting the Simulator From a manifest <Manifest xmlns="http://schemas.microsoft.com/xw/2004/10/manifest.html" xmlns:dssp="http://schemas.microsoft.com/xw/2004/10/dssp.html" xmlns:simcommon="http://schemas.microsoft.com/robotics/2006/04/simulation.html" > http://schemas.microsoft.com/robotics/2006/04/simulationengine.html LEGO.NXT.Tribot.SimulationEngineState.xml dssp:StateService

12. Starting the Simulator From another service [DisplayName("Simulation Tutorial 2")] [Description("Simulation Tutorial 2 Service")] [Contract(Contract.Identifier)] public class SimulationTutorial2 : DsspServiceBase { State _state = new State(); // partner attribute will cause simulation engine service to start [Partner("Engine", Contract = engineproxy.Contract.Identifier, CreationPolicy = PartnerCreationPolicy.UseExistingOrCreate)] private engineproxy.SimulationEnginePort _engineServicePort = new engineproxy.SimulationEnginePort();

13. Basic Simulator Operation Navigation Rendering modes Menu options The simulator editor

14. Creating a Mesh Use a 3 rd party modeling tool such as Maya, 3D Studio Max Must export objects to Alias.obj format

15. Entities Represent Hardware Entities usually represent motor or sensor hardware Services can interact with simulation entities or real-world hardware Only a manifest change required Lego NXT Entity Differential Drive Service Lego NXT Drive Service Orchestration Service

16. Editing Entities Cameras Sky and ground Editing shapes Specifying meshes and textures Special entity types

17. Using Services with Entities Each entity can specify a service contract Saving a scene Saves simulator state and all entity state Saves a manifest which will start services Manifest links a service to an entity by specifying a partner which is the entity name

18. Using Services with Entities Entity name specified as partner http://schemas.microsoft.com/robotics/simulation/services/2006/05/simulateddifferentialdrive.html http://localhost/LegoNXTMotorBase simcommon:Entity

19. Using Services with Entities Hierarchical entities Specifying a service for an entity Difference between state and manifest Using the Simple Dashboard service The DifferentialDrive service The LaserRangefinder service The SimulatedWebcam service

20. Making a Scene Add entities to the simulation environment Using the simulation editor Simulation Tutorial 6 Editing simulator state Simulation Tutorial 3 Using a manifest Lego NXT Tribot simulation manifest From a service Simulation Tutorials 1,2,4,5

21. Making a Scene Adding new types of entities Creating a completely new type of entity (such as a legged robot) requires defining a new entity class The entities.cs file in the SDK shows examples of how entity classes are defined This topic is outside the scope of this presentation

22. Making a Scene Adding entities with the editor Turn on the simulation editor (F5) Select Entities->New Entity… Select desired assembly Select entity Select parent entity if desired Specify constructor parameters Optionally edit state

23. Making a Scene Copy entity from one scene to another Load the source scene Turn on the simulation editor (F5) Check the desired entity or entities Select Entities->Save Entities… Load the destination scene Select Entities->Load Entities…

24. Making a Scene Adding entities by editing simulator state Save the scene to an XML file Use a text editor to open the file Copy and paste the desired entity Modify state as necessary Save the file Re-load the scene from the modified file Can also copy entities between scene files in a similar way

25. Making a Scene Loading simulator state from a manifest <Manifest xmlns="http://schemas.microsoft.com/xw/2004/10/manifest.html" xmlns:dssp="http://schemas.microsoft.com/xw/2004/10/dssp.html" xmlns:simcommon="http://schemas.microsoft.com/robotics/2006/04/simulation.html" > http://schemas.microsoft.com/robotics/2006/04/simulationengine.html LEGO.NXT.Tribot.SimulationEngineState.xml dssp:StateService

26. Making a Scene Using the simulation editor Copying an entity between scenes Editing simulator state Loading simulator state from a manifest

27. Making a Scene Adding entities from a service Service creates the scene Specifies simulation engine as partner Creates entities and sets state Inserts entities into simulator Starts associated services Simulation tutorial 2 is a good example

28. Adding Entities from a Service Required references SimulatedBumper.Y2006.M05.Proxy SimulatedDifferentialDrive.2006.M06.Proxy SimulatedLRF.Y2006.M05.Proxy SimulatedWebcam.Y2006.M09.Proxy using drive = Microsoft.Robotics.Services.Simulation.Drive.Proxy; using lrf = Microsoft.Robotics.Services.Simulation.Sensors.LaserRangeFinder.Proxy; using bumper = Microsoft.Robotics.Services.Simulations.Sensors.Bumper.Proxy; using simwebcam = Microsoft.Robotics.Services.Simulations.Sensors.SimulatedWebcam.Proxy; Add a reference to the proxy for each service which will be used.

29. Adding Entities from a Service Set the camera view private void SetupCamera() { // Set up initial view CameraView view = new CameraView(); view.EyePosition = new Vector3(2.491269f, 0.598689f, 1.046625f); view.LookAtPoint = new Vector3(1.873792f, 0.40983f, 0.2830455f); SimulationEngine.GlobalInstancePort.Update(view); }

30. Adding Entities from a Service Add a Sky void AddSky() { // Add a sky using a static texture. We will use the sky texture // to do per pixel lighting on each simulation visual entity SkyEntity sky = new SkyEntity("sky.dds", "sky_diff.dds"); SimulationEngine.GlobalInstancePort.Insert(sky); // Add a directional light to simulate the sun. LightEntity sun = new LightEntity(); sun.Name = "Sun"; sun.Type = LightEntityType.Directional; sun.Color = new Vector4(0.8f, 0.8f, 0.8f, 1); sun.Pose.Position = new Vector3(1.0f, 1.0f, -0.5f); SimulationEngine.GlobalInstancePort.Insert(sun); }

31. Adding Entities from a Service Add the Ground void AddGround() { HeightFieldShapeProperties hf = new HeightFieldShapeProperties( "height field", 64, // number of rows 100, // distance in meters, between rows 64, // number of columns 100, // distance in meters, between columns 1, // scale factor to multiple height values -1000); // vertical extent of the height field. // create array with height samples hf.HeightSamples = new HeightFieldSample[hf.RowCount * hf.ColumnCount]; for (int i = 0; i < hf.RowCount * hf.ColumnCount; i++) { hf.HeightSamples[i] = new HeightFieldSample(); hf.HeightSamples[i].Height = (short)(Math.Sin(i * 0.01)); }

32. Adding Entities from a Service Add the Ground (2) // create a material for the entire field. // We could also specify material per sample. hf.Material = new MaterialProperties("ground", 0.8f, 0.5f, 0.8f); // insert ground entity in simulation and specify a texture SimulationEngine.GlobalInstancePort.Insert( new HeightFieldEntity(hf, "03RamieSc.dds")); }

33. Adding Entities from a Service Add a motor base // use supplied entity that creates a motor base // with 2 active wheels and one caster Pioneer3DX robotBaseEntity = new Pioneer3DX(position); // specify mesh. robotBaseEntity.State.Assets.Mesh = "Pioneer3dx.bos"; // the name below must match manifest robotBaseEntity.State.Name = "P3DXMotorBase"; // Start simulated motor service drive.Contract.CreateService(ConstructorPort, Microsoft.Robotics.Simulation.Partners.CreateEntityPartner( "http://localhost/" + robotBaseEntity.State.Name)); Entity source is provide in entities.cs

34. Adding Entities from a Service Add a laser rangefinder // Create a Laser Range Finder Entity. // Place it 30cm above base CenterofMass. LaserRangeFinderEntity laser = new LaserRangeFinderEntity( new Pose(new Vector3(0, 0.30f, 0))); laser.State.Name = "P3DXLaserRangeFinder"; laser.LaserBox.State.DiffuseColor = new Vector4(0.25f, 0.25f, 0.8f, 1.0f); // Create LaserRangeFinder simulation service and specify // which entity it talks to lrf.Contract.CreateService(ConstructorPort, Microsoft.Robotics.Simulation.Partners.CreateEntityPartner( "http://localhost/" + laser.State.Name));

35. Adding Entities from a Service Insert the new entities // insert laser as child to motor base robotBaseEntity.InsertEntity(laser); // Insert the motor base and its children into the simulation SimulationEngine.GlobalInstancePort.Insert(robotBaseEntity); Child entities are inserted into their parent The composite entity is inserted into the simulation engine The engine initializes the entities

36. Articulated Arms Simulation Tutorial 4

37. Joints and Segments Articulated arms are built from entities (segments) attached by Joints Joints are physics objects Up to 6 degrees of freedom 3 linear and 3 rotational Each degree of freedom can be locked or free

38. Joint Properties A Joint object has the following properties: The degrees of freedom locked or free The entities which are joined The point on each entity where the joint attaches The axis of attachment for each entity The joint normal (specifies joint orientation) An optional drive and spring mechanism for each unlocked degree of freedom

39. Example Joint One rotation degree of freedom unlocked PhysicsJoint jointInstance = null; JointAngularProperties commonAngular = new JointAngularProperties(); commonAngular.TwistMode = JointDOFMode.Free; commonAngular.TwistDrive = new JointDriveProperties( JointDriveMode.Position, new SpringProperties(500000, 100000, 0), 1000000); jointInstance = PhysicsJoint.Create( new JointProperties(commonAngular, null, null)); // joints must be named jointInstance.State.Name = "Joint0";

40. An Articulated Arm Built from multiple segments and joints Each joint may have a different normal and axis

41. Articulated Arm Initialization Segment entities must be initialized before joints can be initialized Requires a 2-step process Good example in entities.cs KukaLBR3Entity

42. Example Link Initialization commonJoint.State.Connectors[0] = new EntityJointConnector( null, new Vector3(0, 1, 0), // joint normal (+Y) new Vector3(0, 0, -1), // joint axis (-Z) new Vector3(0, ARM_LENGTH2, 0)); // entity 0 connection point commonJoint.State.Connectors[1] = new EntityJointConnector( null, new Vector3(0, 1, 0), // joint normal (+Y) new Vector3(0, 0, -1), // joint axis (-Z) new Vector3(0, 0, 0)); // entity 1 connection point ArmLinkEntity link = new ArmLinkEntity( "arm2",// the name of the link entity "lbr3_j1.obj", // the visual mesh for the link entity new Pose(new Vector3(0, MIDPOINT_Y, 0), // initial orientation Quaternion.RotationAxis(new Vector3(1, 0, 0), (float)Math.PI)));

43. The KUKA LBR3 Arm Using Simple Dashboard to move joints

44. Summary Simulator architecture Basic Simulator Operation Modifying and Creating Entities Using Services with Entities Articulated Arm Entities

45. © 2005 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.