Augmented Reality Painting Application Capstone Project Presentation Supervisor Ph.D. Phan Duy Hung Team member Nguyen Dinh Vu (SE01988) Phan Dinh Trung (01798) Nguyen Huu Chuc (SE01964) Dau Phuc Chuong (SE02033) Le Dang Dat (SE02381)

OUTLINE 1. Introduction 2. Project Management 3. Requirement Specification 4. Architecture Design 5. Implementation 6. Testing 7. Conclusion 8. Demo 9. Q&A 2

1. INTRODUCTION 3

1.1 Project Infomation Name: Augmented Reality Painting Application Project Manager: Nguyen Dinh Vu Timeline: 12/05/ /08/2014 Project Name: PaintAR Type: Mobile application Category: Education, Entertainment Product 4

What did we do? Please watch a video 5 Image source: Sara Robertson on FlickerSara RobertsonFlicker

1.2 Basic Concepts Augmented Reality? 6 Image source: Nokia HereNokia Here Mix together Augmented content View of real world

1.2 Basic Concepts Image processing? 7 Image source: WikipediaWikipedia Corner detection3D Projection

1.3 Existing applications Tình Chocopie 8 Image source: Google PlayGoogle Play just consuming the available, not personalized contents

9 Image source: Google PlayGoogle Play 1.3 Existing applications TINE Kuer AR poor coloring experience limit the creativity limit color choices

1.3 Existing applications ColAR Mix 10 Print. Color. Play. Image source: ColAR Mix GalleryColAR Mix Gallery

1.4 Proposals 11 PaintAR

1.5 Out of scopes Website In-app purchase Interactive 12

2. PROJECT MANAGEMENT 13

2.1 Process model 14 Incremental Development This diagram is referred from Sommerville, I. (2007), Software Engineering This diagram is referred from Pfleeger, S. L., & Atlee, J. M. (2006). Software Egineering: Theory and Practice

2.2 Roles and Responsibilities 15

2.3 Tools and techniques 16 FilesOfficesDevs

2.4 Project Schedule 17

2.5 Communication  Kick off meeting: 12/5/2014  Working face to face: 4 days per week at library  Meeting with supervisor: once per week 18

2.6 Risk Management IDDescriptionsProbabi lity EffectSolution 5 Team do not have much knowledge in some technical problem NormalSerious - Study technical problem early - Ask supervisor for solution 6 Drawing taking too much effort NormalSerious - Find help from outside - Change to simpler prototype 19 Risk Sample

3. REQUIREMENT SPECIFICATIONS 20

3.1 User Characteristics 21  Reading Skill  Curious  Short Attention  Boys and Girls  Parents Image source: Emran Kassim on FlickerEmran KassimFlicker

3.2 Functional Requirement 22 Use case diagram

Play Workflow 23

3.3 Non-functional Requirement 24 Tutorial User Interface Usability

3.3 Non-functional Requirement 25 Error rate < 5% Handle Input Reliability

3.3 Non-functional Requirement 26 Ready in 1 minute Process in 20 seconds Performance

3.3 Non-functional Requirement 27 Cross-Platform Portability

4. ARCHITECTURE DESIGN 28

4.1 Proposed Architecture 29

4.2 Frameworks 30 Detection and tracking Qualcomm Vuforia ™ Image source: VuforiaVuforia

4.2 Frameworks 31 Vuforia Development Process

4.2 Frameworks 32 Rendering Engine: Unity3D Image source: UnityUnity

4.3 Component-based Architecture Definition: 33 a software engineering approach focuses on the decomposition of the design into individual components that expose communication interfaces

4.4 Component-based style of PaintAR Assets 34

4.4 Component-based style of PaintAR Scene 35

Example: PlayScreen Design 36 ElementDescription Back button When user click, go to previous Screen. Help button When user click, go to HelpScreen. Share button When user click, take a screen shot and show a list to share it with other apps in device. Camera button When user click, take a screen shot and save it to device’s gallery under a folder named “PaintAR” Menu button Show/hide the above buttons AR View Area Where user play with the app. The below layer is the camera view, on top is the 3D model. Help ButtonBack ButtonShare ButtonCamera ButtonMenu Button AR View Area

Example: MainScreen Game Objects 37 Game Object QuantityBrief Description MainCamera 1 From Unity. A Camera is a device through which the player views the world. In this case, it will pass the GUI Element to user. Background Image 1Background Image is a GUITexture that have the responsibility to display the background and buttons. The Scripts for the button will be attached to it, too.

Example: MainGUIBehaviour Script 38 MainGUIBehaviour Purpose A class to display and control behaviour of two buttons: Play and Help Game Object BackgroundImage AttributesTexturetexBackground The texture of background TexturetexPlay The texture of button Play TexturetexHelp The texture of button Help Member Functions voidOnGUI () Show background and buttons in MainScreen voidbtnPlay () The Play button if press will go to PlayScreen voidbtnHelp () The Help button if press will go to HelpScreen

Example: ImageTarget Class Diagram 39

Example: ImageTarget Sequence Diagram 40

5. IMPLEMENTATION 41

5.1 ImageTarget – 3D model Workflow 42

5.2 UV mapping by Maya 43

5.3 Create a target using Vuforia 44

5.4 PlayScreen Process Flow 45

Target Screen Coordinates World coordinates geographic 3D Screen coordinates camera 2D 46 Vector3 screenPoint = Camera.main.WorldToScreenPoint(targetPointInWorld);

Quadrilateral to rectangle transformation 47 Solution: Aforge.NET library

6. TESTING 48

6.1 Scope of testing Unit test By Developer Manual Method Integration test – system test Combine Need test cases Acceptance test By children 49

6.2 Test Plan  Test Stages 50 Types of TestStages of Test Unit Test Integration test Acceptanc e Test Function Test XXX User Interface test XXX Regression test XXX

6.2 Test Plan  Test Items 51 ComponentFeature nameFunction to be tested User Interface Screen Element The display of screen element Function Augmentation Image Target is recognized and 3D model are displayed. 3D model stick with image target on screen. Color 3D model The border is display when processing the Target. 3D model’s color matched with Target Switching Target The function of Augmentation and Color 3D after switching target.

6.3 Test Milestone 52 Milestone TaskStart DateEnd Date Increment 124/06/201404/07/2014 Unit Testing24/06/201430/06/2014 Integration Testing27/06/201404/07/2014 Increment 217/07/201431/07/2014 Unit Testing17/07/2014 Integration Testing24/07/201431/07/2014 Increment 301/08/2014 Unit Testing01/08/201407/08/2014 Integration Testing04/08/201414/08/2014 Acceptance Testing17/08/201419/08/2014

6.4 Test Cases 53 GUI test cases example

6.5 Test Report 54

7. CONCLUSION 55

7.1 Discussions  Disadvantage: 56 Idea Not the first Not the best App Performance Interactive

7.1 Discussions  The bright side: 57

7.2 Lesson Learned  Technique: Understanding how to make an AR app  Teamwork: Improve teamwork skill and communication  Management: Keeping inspiration is the most important. 58

8. DEMO 59

9. Q&A 60

THANKS FOR YOUR ATTENTION! 61