Introduction to Mindstorms Educational Robotics Equipment Introduction to Smart Systems Introduction to Mindstorms Educational Robotics Equipment 1

Mindstorms Introduction – Overview Mindstorms NXT is an educational robotics system which enables a vast range of experimental embedded systems and robotics work. The physical devices are quite easy to construct. If Java is used; the programming can also be quite easy. The processor is powerful enough to support the most ambitious control logic. leJOS NXJ is a firmware replacement for Mindstorms NXT. It includes a Java virtual machine, so allows Mindstorms robots to be programmed using Java. (see later) Downloads and info available at: http://lejos.sourceforge.net/ Java Class / method information for Mindstorms sensors: http://lejos.sourceforge.net/p_technologies/nxt/nxj/api/lejos/nxt/package-summary.html LeJOS tutorial available at: http://lejos.sourceforge.net/nxt/nxj/tutorial/index.htm 2 Embedded Systems Programming II Richard Anthony, Computer Science, The University of Greenwich

The Mindstorms computer unit - the “NXT Brick” The NXT brick is the computer unit that runs a program to control the ‘robot’. The NXT brick is designed to actually be part of the ‘robot’ and can have other structural components attached to it. Input sensors and output actuators are attached to the NXT via ‘RJ12’ cables: Up to four input sensors can be attached inc: Touch / Light / Sound / Distance Up to three output actuators can be attached inc: Motors / Lights Direct I/O is also provided on the NXT: A 100 x 64 pixel monochrome LCD display. Four input buttons (can be programmed to have specific functions). A speaker can play sound files (typically one word or short sound clip). A USB 2.0 port for connection to the programming host (a PC). Bluetooth wireless connectivity: Allows loading programs to the NXT wirelessly (instead of USB), Can be used to control ‘robots’ remotely (through phone / PDA etc.), Can be used to provide NXT-NXT communication. Embedded Systems Programming II Richard Anthony, Computer Science, The University of Greenwich

Simplified block diagram of the Mindstorms platform User application program – Java High-level I/O logic e.g. set motor speed and direction LeJOS NXJ (JVM, O/S firmware) MicroController Low-level I/O control e.g. setting specific port bits, timing events, handling interrupts MicroProcessor Sensors (inputs) Output devices I/O control interface (I2C / TWI) Embedded Systems Programming II Richard Anthony, Computer Science, The University of Greenwich

NXT Brick - Technical specification Main microprocessor (runs the application program): 32-bit AT91SAM7S256 256 KB flash memory (for the program), 64 KB RAM (for data, variables etc.) Peripheral control processor (controls the I/O devices): 8-bit ATmega48 microcontroller @ 4 MHz (4 KB flash memory, 512 Bytes RAM) This is not accessible to the programmer – (i.e. you). It is used as a helper for the main processor, (it simplifies the control operations and reduces the complexity of the interface electronics needed). For further technical information see: http://mindstorms.lego.com/Overview/The_NXT.aspx Embedded Systems Programming II Richard Anthony, Computer Science, The University of Greenwich

Main processor - AT91SAM7S256 32-bit ARM7TDMI RISC processor. 256K bytes of embedded high-speed Flash (for program storage) 64K bytes of SRAM (for data, variables). Peripherals include: USB 2.0 Full Speed Device Port, USARTs (serial programmable interfaces), Serial Peripheral Interface Bus (SPI) 4-wire synchronous serial data link, Two-Wire Interface (TWI) – serial interface, 8-channel 10-bit ADC. The maximum clock frequency is 55MHz. Typical core supply is 1.8V (low voltage, good for battery-powered deployment). Embedded Systems Programming II Richard Anthony, Computer Science, The University of Greenwich

Secondary processor - ATmega48 Architecture Secondary processor - ATmega48 Works as a peripheral controller to handle the I/O interfacing on the Mindstorms NXT smart brick. 4K memory, 8-bit processor, 8 ADC channels. Embedded Systems Programming II Richard Anthony, Computer Science, The University of Greenwich

Mindstorms NXT Brick – Output Ports (Hardware implementation using Atmel ATmega48)

Mindstorms NXT Brick – Input Ports (Hardware implementation using Atmel ATmega48) NXT hardware development manual URL: http://www.legolab.daimi.au.dk/DigitalControl.dir/LEGO%20MINDSTORMS%20NXT%20Hardware%20Developer%20Kit/LEGO%20MINDSTORMS%20NXT%20Hardware%20Developer%20Kit.pdf

Mindstorms NXT Brick – Sensor Ports (Input) The NXT has four sensor ports. Input sensors can be connected to any ports, in any combination. Need to identify which type of sensor is connected to each port. This is done in the constructor, when creating a sensor object, e.g. static UltrasonicSensor us = new UltrasonicSensor(SensorPort.S1); static SoundSensor sound = new SoundSensor(SensorPort.S2); static TouchSensor touch1 = new TouchSensor(SensorPort.S3); static TouchSensor touch2 = new TouchSensor(SensorPort.S4); LCD Buttons Sensor (input) ports 1 2 3 4 Embedded Systems Programming II Richard Anthony, Computer Science, The University of Greenwich

Mindstorms NXT Brick – Motor Ports (Output) The NXT has three output ports. Motors can be connected to these ports. Need to identify which motor is connected to each port. This is done by having three separate motor objects (A, B and C), e.g. : Motor.A.stop(); Motor.B.rotate(10); Motor.C.setSpeed(100); (see later for details of these methods) Buttons Motor (output) ports C B A LCD USB Embedded Systems Programming II Richard Anthony, Computer Science, The University of Greenwich

Mindstorms NXT Brick – Battery The NXT has a rechargeable battery pack. This must be charged up with a mains adapter (usually these will be provided already charged). The rechargeable battery clips onto the bottom of the NTX. (Hint: Think about access to the battery, especially the charger port, when building robots – it is time-consuming if you have to dismantle to gain access). Battery Pack Power on charging NXT (face down) Press release catch Charger port Embedded Systems Programming II Richard Anthony, Computer Science, The University of Greenwich

LeJOS NXJ overview leJOS NXJ is a tiny Java Virtual Machine ported to the LEGO Mindstorms NXT brick. leJOS NXJ provides: Object oriented language (Java) Preemptive threads (tasks) Multi-dimensional Arrays, Recursion Synchronization Exceptions Java types including float, long, and String Most of the java.lang, java.util and java.io classes A Well-documented Robotics API and thus enables powerful robotics control applications to be developed easily and quickly. Full Class Hierarchy at http://lejos.sourceforge.net/nxt/pc/api/overview-tree.html Embedded Systems Programming II Richard Anthony, Computer Science, The University of Greenwich

Mindstorms sensors and output devices – Overview Up to four input sensors can be attached inc: Touch (digital), Light (analogue), Sound (analogue), Distance (analogue) ‘Digital’ means a true or false, on / off behaviour. e.g. A switch is either on, or off – there are a finite number of possible states (often 2, but can be more than 2). ‘Analogue’ means a continuously variable range of values. E.g. the temperature in a room can be 20.10023 … degrees. We can never completely accurately state an analogue value as a number. With the mindstorms sensors, the analogue values are converted to digital values so you get a numerical output. For example, distance is converted to a value between 0 and 255. This is a digital representation of the analogue value, it is less accurate, but in this form it can be used in a computer program. Up to three output actuators can be attached inc: Motors, Lights Embedded Systems Programming II Richard Anthony, Computer Science, The University of Greenwich

Mindstorms sensors and output devices – Types Input sensors include: Touch / push switch Light intensity / Colour sensor Sound intensity Distance from an object 4-button keypad on Brick Output devices include: Motors Lights LCD display on Brick Sound output (sounds, tones, words) on Brick Illumination light on Light intensity sensor Communication: Bluetooth Feedback mechanism: Rotation sensor on Servo motor Embedded Systems Programming II Richard Anthony, Computer Science, The University of Greenwich

Mindstorms Sensors – Servo Motor Speed, Rotation angle and Rotation direction can be controlled. Various gears can be used if necessary to change speed and torque ratio. Built-in rotation sensor: Can be used to control the actual distance rotated. Can be used as an input sensor – for example as a dial that is turned. For further technical information see: http://mindstorms.lego.com/Overview/Interactive_Servo_Motors.aspx Embedded Systems Programming II Richard Anthony, Computer Science, The University of Greenwich

Motors – Example interfaces and methods Class Motor public void forward() Rotate forward. public void backward() Rotate backwards. public void reverseDirection() Reverses direction of motor. public void flt() Motor ‘floats’, loses power, not same as stop. public void stop() Motor stops, and resists any further motion. public void rotate(int angle) Motor rotates through angle. public void rotateTo(int limitAngle) Motor rotates to limitAngle; public void regulateSpeed(boolean yes) Turns speed regulation on/off. public void setSpeed(int speed) Sets speed (degrees/second); Up to 900. public interface Tachometer (Abstraction for tachometer built into motors.) int getTachoCount() Read position / count. void resetTachoCount() Set count back to 0. Embedded Systems Programming II Richard Anthony, Computer Science, The University of Greenwich

Receives reflection a short time later Mindstorms Sensors - Distance Sensor (Ultrasonic) Sends out an ultrasonic sound wave and measures how long it takes to bounce back. Speed of sound is approximately 340 meters / second. Based on this known value, the distance can be computed from the time delay. The sensor’s electronic circuits, together with the internal methods of the class, do this computation for you, so you can call a simple method to find out the distance from the robot to an obstacle. Sends sound pulse Receives reflection a short time later For further technical information see: http://mindstorms.lego.com/Overview/Ultrasonic_Sensor.aspx Embedded Systems Programming II Richard Anthony, Computer Science, The University of Greenwich

Distance Sensor – Example methods public int ping() Send a single ping. public int continuous() Continuous ping mode. Enables continuous ping and capture mode. The default operating mode. public int getDistance() Return distance to an object (0 -254, 255 if no object) public int off() Turn off the sensor. public int capture() Capture mode: detect other ultrasonic sensors. A program can evaluate when it is best to make a new measurement which will not conflict with other ultrasonic sensors. Could be used to enable robots to detect each other. public int reset() Restores to default state. After, sensor operates in continuous mode. Example: static UltrasonicSensor us = new UltrasonicSensor(SensorPort.S1); int iDistance; iDistance = us.getDistance(); Embedded Systems Programming II Richard Anthony, Computer Science, The University of Greenwich

Mindstorms Sensors - Touch Sensor This sensor can be used to detect if a robot has crashed into an obstacle, or it can be used as an input switch, for example to start or stop a robot. For further technical information see: http://mindstorms.lego.com/Overview/Touch_Sensor.aspx Embedded Systems Programming II Richard Anthony, Computer Science, The University of Greenwich

Touch Sensor – Example method public boolean isPressed() Check if the sensor is pressed. Returns true if sensor is pressed, false otherwise. Example: static TouchSensor touch = new TouchSensor(SensorPort.S2); if(touch.isPressed() { StopAllMotors(); } Embedded Systems Programming II Richard Anthony, Computer Science, The University of Greenwich

Mindstorms Sensors - Sound Sensor Sound sensor readings are displayed in percent (of value range). The sound sensor can measure sound pressure levels up to 90 dB – about the level of a petrol lawnmower (this would read at 100%). Other examples: • 4-5% A quiet room • 5-10% Conversation some distance away • 10-30% Normal level conversation or music • 30-100% people shouting or loud music For further technical information see: http://mindstorms.lego.com/Overview/Sound_Sensor.aspx Embedded Systems Programming II Richard Anthony, Computer Science, The University of Greenwich

Sound Sensor – Example methods public void setDBA(boolean dba) Set DB or DBA mode. dBA: ‘Adjusted decibels’, frequency sensitivity similar to the human ear. dB: [unadjusted] decibels, all sounds measured with equal sensitivity. May detect sounds that are outside the human hearing frequency range. Parameters: dba - true to set DBA mode, false for DB mode. public int readValue() Read the current sensor value. Returns value as a percentage of maximum sound level. Example: private static SoundSensor sound = new SoundSensor(SensorPort.S4); int iSoundLevel; sound.setDBA(true); iSoundLevel = sound.readValue(); Embedded Systems Programming II Richard Anthony, Computer Science, The University of Greenwich

Mindstorms Sensors - Light Sensor An Analogue input device. Provides numerical output; higher light intensity ► higher number. ‘Sees’ colours as ‘Grey Scale’ values (example below). Example uses: Detect light / dark environments, Detect objects based on their colour, Follow a line. Colours as seen by the eye Colour equivalents in ‘grey scale’ (i.e. as the sensor sees them) For further technical information see: http://mindstorms.lego.com/Overview/Light_Sensor.aspx Embedded Systems Programming II Richard Anthony, Computer Science, The University of Greenwich

Light Sensor – Example methods public void setFloodlight(boolean floodlight) floodlight - true to set floodit mode, false for ambient light public int readValue() Read the current sensor value. Returns value as a percentage of difference between the low and high calibration values. public int readNormalizedValue() Read normalized value. More accurate than readValue(). Value range 0 to 1023. Typical 145 (dark) to 890 (sunlight). public void calibrateLow() Call this when the sensor is reading the low value - used by readValue public void calibrateHigh() Call this when the sensor is reading the high value - used by readValue public void setLow(int low) Set normalized value corresponding to readValue() = 0 public void setHigh(int high) Set the normalized value corresponding to readValue() = 100; Example: private static LightSensor light = new LightSensor(SensorPort.S1, false); light.setFloodlight(true); iValue = light.readNormalizedValue(); Embedded Systems Programming II Richard Anthony, Computer Science, The University of Greenwich

NTX Buttons – Example methods The four buttons are identified as: ENTER, LEFT, RIGHT, ESCAPE. Methods include: public final boolean isPressed() Check if the button is pressed. Returns true if button pressed, false otherwise. public final void waitForPressAndRelease() throws Interrupted Exception Wait until the button is released. public static int waitForPress() Wait for some button to be pressed Returns the ID of the pressed button Example: if (Button.ENTER.isPressed()) { StartRun(); } Embedded Systems Programming II Richard Anthony, Computer Science, The University of Greenwich

NTX LCD (100 x 64 pixel monochrome) – Example methods public static void setPixel(int rgbColor, int x, int y) Set a specified pixel on the screen. public static void drawString(String str, int x, int y) Display a string on the LCD at specified x,y co-ordinate. public static void drawInt(int i, int x, int y) Display an int on the LCD at specified x,y co-ordinate. public static void setDisplay(int[] buff) Write graphics from a Java buffer to the display. public static void clearDisplay() public static void refresh() Update the display. Example: LCD.clearDisplay(); LCD.drawInt(iCount, 2, 3); Embedded Systems Programming II Richard Anthony, Computer Science, The University of Greenwich

Lab (optional for those who have spare time) Mindstorms challenge – work in groups of about 3, to build a remote controlled robot using BlueTooth communication (B/T) There is sample code on the resources website for: B/T on PC B/T on Mindstorms brick In addition there are instructions for using Netbeans for the Java development of the code for the Mindstorms brick, and a short video showing an example remote-controlled robot. The Mindstorms task is not assessed, but very interesting. We recommend that students who are making good progress with the coursework do the Mindstorms challenge. Others should concentrate on the coursework task !

Mindstorms NXT to PC communication via Bluetooth

Establish NXT Connection Method NXT Brick PC Establish NXT Connection Set up input and output streams both ends of the connection Send data to NXT via the streams (could equally be from NXT to PC) then flush the stream Continually check the streams for new data, then flush the streams once the data has been received Interpret received data to perform desired action (e.g. when ‘u’ character received, move robot forward) Close the streams then the connection when done

Important LeJOS method signatures for Bluetooth 1 NXT Only Class Bluetooth public static NXTConnection waitForConnection(int milli_timeout, int IO_mode) Class NXTConnection public DataInputStream openDataInputStream() public DataOutputStream openDataOutputStream() public void close()

Important LeJOS method signatures for Bluetooth 2 PC and NXT Class DataOutputStream public void writeByte(int byte) public void flush() //flush the stream ready for the next byte public void close() Class DataInputStream public boolean available() //check if there is any data waiting in the input stream – normally checked within a loop public byte readByte()

Important LeJOS method signatures for Bluetooth 3 PC Only Class NXTCommFactory public static NXTComm createNXTComm(int IO_mode) // create the NXTComm object Class NXTComm public NXTInfo[] search("NXT") // search for available bricks public void open(NXTInfo info, int IOMode) //establish Conn’ public void close() // close connection Class NXTInfo public NXTInfo NXTInfo(int IO_mode, String Brick_name, String NXTMAC) // create NXTInfo class with the MAC // address for your specific brick

Important notes It is possible to upload your code into the brick via Bluetooth However in the lab use the USB cable because there will be multiple bricks in the lab and your code could be uploaded on your friends’ brick if Bluetooth is used for this part of the process Whilst it is possible to search for all NXT devices to create a connection, use the MAC address of your brick to ensure connection to it and not to another brick in the lab, i.e. use: new NXTInfo(NXTCommFactory.BLUETOOTH,"NXT",NXTMAC);

The Mindstorms Challenge Build a robot from the Mindstorms kit and control its movement from your PC keyboard via Bluetooth. (There is some useful sample code on ‘Richards Resources Website’)