1. Lecture Series on Android Programming Lecturer: Prof.Luqun Li (liluqun@gmail.com)liluqun@gmail.com Teaching Assistants: Fengyou Sun, Haijun Yang, Ting Sun Chapter 10 Programming 3D Graphics with OpenGL

2. Shanghai Normal University 2 Contents 2 2 Defining Shapes 3 3 Drawing Shapes 4 4 Applying Projection and Camera Views 5 5 Adding Motion 1 1 Building an OpenGL ES Environment 6 6 Responding to Touch Events

3. Shanghai Normal University 3 Building an OpenGL ES Environment Implement a GLSurfaceView and a GLS urfaceView.Renderer GLSurfaceView is a view container fo r graphics drawn with OpenGL GLSurfaceView.Renderer controls wh at is drawn within that view GLSurfaceView is for a full-screen or near-full scre en graphics view TextureView is used to incorporate OpenGL ES gra phics in a small portion of their layouts

4. Shanghai Normal University 4 Declare in the Manifest To use the OpenGL ES 2.0 API : If your application uses texture com pression :

5. Shanghai Normal University 5 Create an Activity Android applications that use Open GL ES have activities just like any o ther application that has a user inte rface. A minimal implementation of an acti vity that uses a GLSurfaceView

6. Shanghai Normal University 6 Create an Activity public class OpenGLES20 extends Activity { private GLSurfaceView mGLView; @Override public void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); // Create a GLSurfaceView instance and set it // as the ContentView for this Activity. mGLView = new MyGLSurfaceView(this); setContentView(mGLView); }

7. Shanghai Normal University 7 Build a GLSurfaceView Object A GLSurfaceView is a specialized v iew where you can draw OpenGL ES graphics. The actual drawing of objects is con trolled in the GLSurfaceView.Ren derer that you set on this view.

8. Shanghai Normal University 8 Build a GLSurfaceView Object class MyGLSurfaceView extends GL SurfaceView { public MyGLSurfaceView(Context co ntext){ super(context); // Set the Renderer for drawing on the // GLSurfaceView setRenderer(new MyRenderer()); }

9. Shanghai Normal University 9 Build a GLSurfaceView Object // Create an OpenGL ES 2.0 context setEGLContextClientVersion(2); // Render the view only when there is a //change in the drawing data setRenderMode(GLSurfaceView.REN DERMODE_WHEN_DIRTY);

10. Shanghai Normal University 10 Build a Renderer Class Three methods in a renderer called by th e Android system : onSurfaceCreated() - Called once to se t up the view's OpenGL ES environment. onDrawFrame() - Called for each redra w of the view. onSurfaceChanged() - Called if the geo metry of the view changes, for example when the device's screen orientation cha nges.

11. Shanghai Normal University 11 Build a Renderer Class public class MyGL20Renderer implements GLSurfaceView. Renderer { public void onSurfaceCreated(GL10 unused, EGLConfig co nfig) { // Set the background frame color GLES20.glClearColor(0.5f, 0.5f, 0.5f, 1.0f); } public void onDrawFrame(GL10 unused) { // Redraw background color GLES20.glClear(GLES20.GL_COLOR_BUFFER_BIT); } public void onSurfaceChanged(GL10 unused, int width, int height) { GLES20.glViewport(0, 0, width, height); }

12. Shanghai Normal University 12 Contents 2 2 Defining Shapes 3 3 Drawing Shapes 4 4 Applying Projection and Camera Views 5 5 Adding Motion 1 1 Building an OpenGL ES Environment 6 6 Responding to Touch Events

13. Shanghai Normal University 13 OpenGL coordinate system OpenGL assumes a square, uniform coordinate system and, by default, happily draws those coordinates ont o your typically non-square screen a s if it is perfectly square. To solve this problem, you can appl y OpenGL projection modes and ca mera views to transform coordinate s so your graphic objects have the c orrect proportions on any display.

14. Shanghai Normal University 14 OpenGL coordinate system Default OpenGL coordinate system (left) mapped to a typical Android d evice screen (right)

15. Shanghai Normal University 15 Define a Triangle OpenGL ES allows you to define dra wn objects using coordinates in thre e-dimensional space. So, before you can draw a triangle, you must defin e its coordinates. Define a vertex array of floating point numbe rs for the coordinates Write these coordinates into a ByteBuffer

16. Shanghai Normal University 16 Define a Triangle class Triangle { private FloatBuffer vertexBuffer; // number of c oordinates per vertex in this array static final int COORDS_PER_VERTEX = 3; static float triangleCoords[] = { // in counterclockwise order: 0.0f, 0.622008459f, 0.0f, // top -0.5f, -0.311004243f, 0.0f, // bottom left 0.5f, -0.311004243f, 0.0f // bottom right }; // Set color with red, green, blue and alpha // (opacity) values float color[] = { 0.63671875f, 0.76953125f, 0.2 2265625f, 1.0f };

17. Shanghai Normal University 17 Define a Triangle public Triangle() { // initialize vertex byte buffer for shape coordinates ByteBuffer bb = ByteBuffer.allocateDirect( // (number of coordinate values * 4 bytes per float) triangleCoords.length * 4); // use the device hardware's native byte order bb.order(ByteOrder.nativeOrder()); // create a floating point buffer from the ByteBuffer vertexBuffer = bb.asFloatBuffer(); // add the coordinates to the FloatBuffer vertex Buffer.put(triangleCoords); // set the buffer to read the first coordinate vertexBuffer.position(0); }

18. Shanghai Normal University 18 Define a Square Define the vertices in a counterclock wise order for both triangles that re present this shape, and put the valu es in a ByteBuffer. Use a drawing list to avoid defining the two coordinates shared by each triangle twice

19. Shanghai Normal University 19 Define a Square class Square { private FloatBuffer vertexBuffer; private ShortBuffer drawListBuffer; //number of coordinates per vertex in this array static final int COORDS_PER_VERTEX = 3; static float squareCoords[] = { -0.5f, 0.5f, 0.0f, // top left -0.5f, -0.5f, 0.0f, // bottom left 0.5f, -0.5f, 0.0f, // bottom right 0.5f, 0.5f, 0.0f }; // top right private short drawOrder[] = { 0, 1, 2, 0, 2, 3 }; // order to draw vertices

20. Shanghai Normal University 20 Define a Square public Square() { // initialize vertex byte buffer for shape coordinates ByteBuffer bb = ByteBuffer.allocateDirect( // (# of coordinate values * 4 bytes per float) squareCoords.length * 4); bb.order(ByteOrder.nativeOrder()); vertexBuffer = bb.asFloatBuffer(); vertexBuffer.put(squareCoords); vertexBuffer.position(0); // initialize byte buffer for the draw list ByteBuffer dlb = ByteBuffer.allocateDirect( // (# of coordinate values * 2 bytes per short) drawOrder.length * 2); dlb.order(ByteOrder.nativeOrder()); drawListBuffer = dlb.asShortBuffer(); drawListBuffer.put(drawOrder); drawListBuffer.position(0); }

21. Shanghai Normal University 21 Contents 2 2 Defining Shapes 3 3 Drawing Shapes 4 4 Applying Projection and Camera Views 5 5 Adding Motion 1 1 Building an OpenGL ES Environment 6 6 Responding to Touch Events

22. Shanghai Normal University 22 Initialize Shapes Initialize and load the shapes in the onSurfa ceCreated() public void onSurfaceCreated(GL10 unused, EGLConfig config) {... // initialize a triangle mTriangle = new Triangle(); // initialize a square mSquare = new Square(); }

23. Shanghai Normal University 23 Draw a Shape You must define the following: Vertex Shader - OpenGL ES graphics code f or rendering the vertices of a shape. Fragment Shader - OpenGL ES code for ren dering the face of a shape with colors or text ures. Program - An OpenGL ES object that contai ns the shaders you want to use for drawing one or more shapes.

24. Shanghai Normal University 24 Draw a Shape Define basic shaders private final String vertexShaderCode = "attribute vec4 vPosition;" + "void main() {" + " gl_Position = vPosition;" + "}"; private final String fragmentShaderCode = "precision mediump float;" + "uniform vec4 vColor;" + "void main() {" + " gl_FragColor = vColor;" + "}";

25. Shanghai Normal University 25 Draw a Shape Compile OpenGL Shading Language (GLSL) code public static int loadShader(int type, String shad erCode){ // create a vertex shader type //(GLES20.GL_VERTEX_SHADER) // or a fragment shader type //(GLES20.GL_FRAGMENT_SHADER) int shader = GLES20.glCreateShader(type); // add the source code to the shader and compil e it GLES20.glShaderSource(shader, shaderCode); GLES20.glCompileShader(shader); return shader; }

26. Shanghai Normal University 26 Draw a Shape In order to draw your shape, you must compile the shader code, add them to a OpenGL ES program object and then link the program.

27. Shanghai Normal University 27 Draw a Shape public Triangle() {... int vertexShader = loadShader(GLES20.GL_VERTEX_SHADER, vertexSh aderCode); int fragmentShader = loadShader(GLES20.GL_FRAGMENT_SHADER, fra gmentShaderCode); // create empty OpenGL ES Program mProgram = GLES20.glCreateProgram(); // add the vertex shader to program GLES20.glAttachShader(mProgram, vertexShader); // add the fragment shader to program GLES20.glAttachShader(mProgram, fragmentShader); // creates OpenGL ES program executables GLES20.glLinkProgram(mProgram); }

28. Shanghai Normal University 28 Draw a Shape Drawing shapes with OpenGL ES re quires that you specify several para meters to tell the rendering pipeline what you want to draw and how to draw it. Since drawing options can vary by s hape, it's a good idea to have your shape classes contain their own dra wing logic.

29. Shanghai Normal University 29 Draw a Shape public void draw() { // Add program to OpenGL ES environment GLES20.glUseProgram(mProgram); // get handle to vertex shader's vPosition memb er mPositionHandle = GLES20.glGetAttribLocation (mProgram, "vPosition"); // Enable a handle to the triangle vertices GLES20.glEnableVertexAttribArray(mPositionHa ndle); // Prepare the triangle coordinate data GLES20.glVertexAttribPointer(mPositionHandle, COORDS_PER_VERTEX, GLES20.GL_FLOAT, fals e, vertexStride, vertexBuffer);

30. Shanghai Normal University 30 Draw a Shape // get handle to fragment shader's vColor mem ber mColorHandle = GLES20.glGetUniformLocation (mProgram, "vColor"); // Set color for drawing the triangle GLES20.glUniform4fv(mColorHandle, 1, color, 0); // Draw the triangle GLES20.glDrawArrays(GLES20.GL_TRIANGLES, 0, vertexCount); // Disable vertex array GLES20.glDisableVertexAttribArray(mPositionHa ndle); }

31. Shanghai Normal University 31 Draw a Shape Call draw() from within your rende rer’s onDrawFrame() method

32. Shanghai Normal University 32 Contents 2 2 Defining Shapes 3 3 Drawing Shapes 4 4 Applying Projection and Camera Views 5 5 Adding Motion 1 1 Building an OpenGL ES Environment 6 6 Responding to Touch Events

33. Shanghai Normal University 33 Applying Projection and Camera Views Projection - This transformation adj usts the coordinates of drawn objec ts based on the width and height of the GLSurfaceView where they ar e displayed. Camera View - This transformation adjusts the coordinates of drawn ob jects based on a virtual camera posi tion.

34. Shanghai Normal University 34 Define a Projection @Override public void onSurfaceChanged(GL10 unu sed, int width, int height) { GLES20.glViewport(0, 0, width, height); float ratio = (float) width / height; // this projection matrix is applied to obj ect coordinates // in the onDrawFrame() method Matrix.frustumM(mProjMatrix, 0, -ratio, r atio, -1, 1, 3, 7); }

35. Shanghai Normal University 35 Define a Camera View @Override public void onDrawFrame(GL10 unused) {... // Set the camera position (View matrix) Matrix.setLookAtM(mVMatrix, 0, 0, 0, -3, 0f, 0f, 0f, 0f, 1.0f, 0.0f); // Calculate the projection and view transformat ion Matrix.multiplyMM(mMVPMatrix, 0, mProjMatrix, 0, mVMatrix, 0); // Draw shape mTriangle.draw(mMVPMatrix); }

36. Shanghai Normal University 36 Apply Projection and Camera public void draw(float[] mvpMatrix) { // pass in the calculated transformation matrix... // get handle to shape's transformation matrix mMVPMatrixHandle = GLES20.glGetUniformLoca tion(mProgram, "uMVPMatrix"); // Apply the projection and view transformation GLES20.glUniformMatrix4fv(mMVPMatrixHandle, 1, false, mvpMatrix, 0); // Draw the triangle GLES20.glDrawArrays(GLES20.GL_TRIANGLES, 0, vertexCount);... }

37. Shanghai Normal University 37 Triangle Transformed Triangle drawn with a projection an d camera view applied.

38. Shanghai Normal University 38 Contents 2 2 Defining Shapes 3 3 Drawing Shapes 4 4 Applying Projection and Camera Views 5 5 Adding Motion 1 1 Building an OpenGL ES Environment 6 6 Responding to Touch Events

39. Shanghai Normal University 39 Adding Motion OpenGL ES provides additional capa bilities for moving and transforming drawn objects in three dimensions o r in other unique ways to create co mpelling user experiences.

40. Shanghai Normal University 40 Rotate a Shape private float[] mRotationMatrix = new float[16]; public void onDrawFrame(GL10 gl) {... // Create a rotation transformation for the triangle long time = SystemClock.uptimeMillis() % 4000L; float angle = 0.090f * ((int) time); Matrix.setRotateM(mRotationMatrix, 0, mAngle, 0, 0, -1.0 f); // Combine the rotation matrix with the projection and ca mera view Matrix.multiplyMM(mMVPMatrix, 0, mRotationMatrix, 0, m MVPMatrix, 0); // Draw triangle mTriangle.draw(mMVPMatrix); }

41. Shanghai Normal University 41 Enable Continuous Rendering public MyGLSurfaceView(Context context) {... // Render the view only when there is a c hange in the drawing data //setRenderMode(GLSurfaceView.RENDE RMODE_WHEN_DIRTY); // comment out for auto-rotation } Unless you have objects changing without any user interaction, it’s usually a good idea have this flag tur ned on.

42. Shanghai Normal University 42 Contents 2 2 Defining Shapes 3 3 Drawing Shapes 4 4 Applying Projection and Camera Views 5 5 Adding Motion 1 1 Building an OpenGL ES Environment 6 6 Responding to Touch Events

43. Shanghai Normal University 43 Responding to Touch Events The key to making your OpenGL ES application touch interactive is expa nding your implementation of GLSu rfaceView to override the onTouc hEvent() to listen for touch events.

44. Shanghai Normal University 44 Setup a Touch Listener @Override public boolean onTouchEvent(MotionEvent e) { // MotionEvent reports input details from the to uch screen // and other input controls. In this case, you are only // interested in events where the touch position changed. float x = e.getX(); float y = e.getY(); switch (e.getAction()) { case MotionEvent.ACTION_MOVE: float dx = x - mPreviousX; float dy = y - mPreviousY;

45. Shanghai Normal University 45 Setup a Touch Listener // reverse direction of rotation above the mid-line if (y > getHeight() / 2) { dx = dx * -1 ; } // reverse direction of rotation to left of the mid-line if (x < getWidth() / 2) { dy = dy * -1 ; } mRenderer.mAngle += (dx + dy) * TOUCH_SCALE_FACT OR; // = 180.0f / 320 requestRender(); } mPreviousX = x; mPreviousY = y; return true; }

46. Shanghai Normal University 46 Set Render Mode public MyGLSurfaceView(Context co ntext) {... // Render the view only when there is a change in the drawing data setRenderMode(GLSurfaceView. RENDERMODE_WHEN_DIRTY); }

47. Shanghai Normal University 47 Expose the Rotation Angle Add a public member and declare th is public variable as volatile public class MyGLRenderer impl ements GLSurfaceView.Rendere r {... public volatile float mAngle;

48. Shanghai Normal University 48 Apply Rotation public void onDrawFrame(GL10 gl) {... // Create a rotation for the triangle // long time = SystemClock.uptimeMillis() % 4000L; // float angle = 0.090f * ((int) time); Matrix.setRotateM(mRotationMatrix, 0, mAngle, 0, 0, -1.0f); // Combine the rotation matrix with the projection and ca mera view Matrix.multiplyMM(mMVPMatrix, 0, mRotationMatrix, 0, m MVPMatrix, 0); // Draw triangle mTriangle.draw(mMVPMatrix); }

49. Shanghai Normal University 49 Responding to Touch Events Triangle being rotated with touch in put

50. Shanghai Normal University 50 Summary This class walks you through the basics o f developing applications that use OpenG L, including setup, drawing objects, movi ng drawn elements and responding to to uch input. The example code in this class uses the OpenGL ES 2.0 APIs, which is the recom mended API version to use with current Android devices. For more information ab out versions of OpenGL ES, see the Open GL developer guide.