James Walsh Supervisor: Prof. Bruce Thomas Wearable Computer Lab School of Computer and Information Science

Motivation Literature Research Question Implementation User Study Revisions Pilot Study Contributions  Motivation  Literature Review  Research Question  Problem Review and Implementation  User Study  Reviewed Visualizations  Pilot Study  Contributions

Motivation Literature Research Question Implementation User Study Revisions Pilot Study Contributions  Large structures (such as bridges) require regular manual maintenance inspections  Time consuming and expensive  Areas often inaccessible  Why can’t we try and automate this?

Motivation Literature Research Question Implementation User Study Revisions Pilot Study Contributions  Internal and external temperatures, humidity and corrosion logged against current time  Measures corrosion via resistance changes across 5 metal strips in the box  Interpreting data is still unintuitive and time consuming, loss of context

Motivation Literature Research Question Implementation User Study Revisions Pilot Study Contributions  Review of current research in related fields

Motivation Literature Research Question Implementation User Study Revisions Pilot Study Contributions  Overlay virtual information on the real world ◦ ‘Supplement’ with additional information/functionality  Mix between the real world and virtual reality Milgram et al p. 283

Motivation Literature Research Question Implementation User Study Revisions Pilot Study Contributions  Wearable computer  Head-mounted display (HMD)  Full six degrees-of- freedom  Third generation backpack (Piekarski and Thomas, 2003)

Motivation Literature Research Question Implementation User Study Revisions Pilot Study Contributions  ARQuake Thomas et al., 2000 Quake first person shooter in AR  ARWeather Heinrich et al., 2008 Simulate weather conditions outdoors  X-Ray Vision Piekarski, 2009 See through walls using cameras  ARVino Piekarski et al., 2005 Used to visualize GIS data in viticulture  Current applications require Tinmith source code for development

Motivation Literature Research Question Implementation User Study Revisions Pilot Study Contributions  Most relevant Tinmith application in the visualization of GIS data in AR  Enables the understanding of data in a visually intuitive manner Piekarski et al., 2005

Motivation Literature Research Question Implementation User Study Revisions Pilot Study Contributions  SiteLens White et al., 2007, White and Feiner, 2009 Handheld AR system for visualizing CO 2 levels (not real time)  Handheld Symbian System Gunnarsson et al., 2006 Visualize and interpolate Zigbee humidity sensors (real time) White et al., 2007Gunnarsson et al., 2006

Motivation Literature Research Question Implementation User Study Revisions Pilot Study Contributions  Main representation of data is numerical  Loss of data due to context  Only a single attribute in each representation ◦ What happens when we have multiple attributes?

Motivation Literature Research Question Implementation User Study Revisions Pilot Study Contributions What are a set of suitable techniques for visualizing outdoor environmental corrosion in mobile augmented reality?

Motivation Literature Research Question Implementation User Study Revisions Pilot Study Contributions  Set of AR environmental corrosion visualizations that have been shown to be effective  Previous invisible environmental issues can be observed  System for quickly directing users to the location of possible problems  Portable solution – user can navigate the structure whilst viewing detailed information

Motivation Literature Research Question Implementation User Study Revisions Pilot Study Contributions  Visualization system is independent of the AR system being used  Could be run inside any OpenGL application  ‘Dumb’ interface – minimal data is required to be passed to the plug-in  Easy to develop with minimal background knowledge of the Tinmith AR system

Motivation Literature Research Question Implementation User Study Revisions Pilot Study Contributions  Iterative development cycle Analysis Implementation EvaluationDesign

Motivation Literature Research Question Implementation User Study Revisions Pilot Study Contributions  Evaluation of visualizations is based on user feedback  For effective evaluations, users asked to provide quantitative and qualitative data (North 2006)  Two evaluations, one for each iteration

Motivation Literature Research Question Implementation User Study Revisions Pilot Study Contributions  AR user interfaces active area of research  Temporal Nature  Use of Phidgets enables intuitive data navigation by ‘scrolling’ to select date/time  Phidgets’ position represented onscreen with ‘virtual sliders’

Motivation Literature Research Question Implementation User Study Revisions Pilot Study Contributions

 Ability to view more information should it be required – ‘drill down’  Draw attention to areas of interest  ‘Interpolate’ between two or more sensors  Show the changes over time

Motivation Literature Research Question Implementation User Study Revisions Pilot Study Contributions  3 visualizations – 2 show all attributes, other used for interpolation between points  Upon focus, visualizations ‘flip’ for more detail  Plasma designed to be complimentary to other representations Box RepresentationGauge RepresentationPlasma Representation

Motivation Literature Research Question Implementation User Study Revisions Pilot Study Contributions Corrosion Humidity Internal Temp External Temp

Motivation Literature Research Question Implementation User Study Revisions Pilot Study Contributions  14 participants (20-31 years)  Identified what they thought each sensor attributed represented ◦ Any corrections for intended meaning were made  Asked to read values for all 3 representations using 4 simulated sensors R Building D Building Sensor 20m 62m 39m 16m Participant Sensor

Motivation Literature Research Question Implementation User Study Revisions Pilot Study Contributions  For distances less than 40m: ◦ Gauge shown to enable precise reading ◦ Box and Gauge could not be read beyond 40m  Possible limitation of the low resolution display used  Box and/or Plasma preferred for indicating areas of interest  Colour scales (Box/Plasma) ineffective, even for very approximate readings

Motivation Literature Research Question Implementation User Study Revisions Pilot Study Contributions  ‘Traffic light’ Plasma effect ◦ Values broken into 3 ranges – good, ok, bad ◦ Simple identification using green, yellow, red colours  Solves objectivity of colour scales and reading at large distances  ‘Drill down’ functionality provided in status bar and dialog box  ‘Live’ graph shows attribute past/future trends from currently selected time

Motivation Literature Research Question Implementation User Study Revisions Pilot Study Contributions

 5 participants (all participated in first study)  2 components; outside evaluation for visualizations and indoor evaluation of Phidget control panel  Asked for views and opinions on changes since user study, any other comments

Motivation Literature Research Question Implementation User Study Revisions Pilot Study Contributions  All participants strongly preferred the enhanced Plasma  Graph thought to be very useful in identifying trends  Participants felt the need for both the Plasma and Gauge representations (depending on circumstances)  Control panel very intuitive for navigating data, ◦ ‘awkward’ for navigating graph.  Suggested to incorporate weather data in graph

Motivation Literature Research Question Implementation User Study Revisions Pilot Study Contributions  Set of demonstrated visualizations ◦ Shown to be effective in representing corrosion ◦ Visualize multiple attributes simultaneously ◦ Complimentary visualizations and system functionality for ‘drill down’ capability  Previously invisible environmental issues can be observed in context  Approximate interpolation between points  Portable solution – user can navigate the structure whilst viewing detailed information

Motivation Literature Research Question Implementation User Study Revisions Pilot Study Contributions GUNNARSSON, A.-S., RAUHALA, M., HENRYSSON, A. & YNNERMAN, A. (2006) Visualization of sensor data using mobile phone augmented reality. Proceedings of the 5th IEEE and ACM International Symposium on Mixed and Augmented Reality. IEEE Computer Society. HEINRICH, M., THOMAS, B. H., MUELLER, S. & SANDOR, C An augmented reality weather system. Proceedings of the 2008 International Conference on Advances in Computer Entertainment Technology. Yokohama, Japan: ACM. MILGRAM, P., TAKEMURA, H., UTSUMI, A. & KISHINO, F. Year. Augmented reality: a class of displays on the reality-virtuality continuum. In: DAS, H., ed., 1995 Boston, MA, USA. SPIE, NORTH, C Toward Measuring Visualization Insight. IEEE Comput. Graph. Appl., vol. 26, pp PIEKARSKI W., THOMAS, B., "Through-Walls Collaboration," IEEE Pervasive Computing, vol. 8, no. 3, pp , July-Sept PIEKARSKI, W. & THOMAS, B. H Interactive augmented reality techniques for construction at a distance of 3D geometry. Proceedings of the workshop on Virtual environments Zurich, Switzerland: ACM.

Motivation Literature Research Question Implementation User Study Revisions Pilot Study Contributions PIEKARSKI, W Through-Walls Collaboration. In: BRUCE, H. T. (ed.). PIEKARSKI, W., THOMAS, B. H. & KING, G. R. Year. ARVino : outdoor augmented reality visualisation of viticulture GIS data. In, IEEE Computer Society. THOMAS, B., CLOSE, B., DONOGHUE, J., SQUIRES, J., BONDI, P. D., MORRIS, M. & PIEKARSKI, W ARQuake: An Outdoor/Indoor Augmented Reality First Person Application. Proceedings of the 4th IEEE International Symposium on Wearable Computers. IEEE Computer Society. THOMAS, B., CLOSE, B., DONOGHUE, J., SQUIRES, J., BONDI, P. D., MORRIS, M. & PIEKARSKI, W. (2000) ARQuake: An Outdoor/Indoor Augmented Reality First Person Application. Proceedings of the 4th IEEE International Symposium on Wearable Computers. IEEE Computer Society. THOMAS, B., DEMCZUK, V., PIEKARSKI, W., HEPWORTH, D. & GUNTHER, B. (1998) A wearable computer system with augmented reality to support terrestrial navigation. 2nd International Symposium on Wearable Computers. Pittsburgh, Pennsylvania, IEEE. WHITE, S., MOROZOV, P. & FEINER, S. (2007) Imaging for Insight: Site Visit by Situated Visualization. ACM Computer/Human Interaction. San Jose, California. WHITE, S. & FEINER, S. (2009) SiteLens: situated visualization techniques for urban site visits. Proceedings of the 27th international conference on Human factors in computing systems. Boston, MA, USA, ACM.

Motivation Literature Research Question Implementation User Study Revisions Pilot Study Contributions This presentation should be viewed in conjunction with the minor these available at /wiki/Walsh-minorthesis/wiki/Walsh-minorthesis from Mid-October, 2010.