Anna Mason, David Mountain and Jonathan Raper Surveying by motion tracking: modelling 3D subterranean landscape from video imagery Requirement for the degree of MSc in Geographic Information Science. These are a few of the organisations that contributed in different ways… funding and advice (BCRA), software (2d3) and camera parts (DogCam Sport) Anna Mason, David Mountain and Jonathan Raper

Study Areas 1. Requirements 3. Field Work 2. Equipment Tests 3 study areas throughout the UK. One chalk mine and two caves formed by dissolution of limestone. Gathered requirements, tested equipment in an easily accessible environment, and then tested the process in the real world. 3. Field Work 2. Equipment Tests

Conditions a caver might experience… Dark Wet Confined Foggy Dirty Represents what cavers and their equipment might experience underground. The challenging nature of caves means that they are underground do not benefit from investment in GPS, GNSS such as the Galileo, or wireless positioning systems. Conventional cave surveying typically requires repetitive measurements Or alternatively requires costly laser scanner and range finder equipment. In this context, I felt that survey techniques that could use handheld cameras would be worthy of investigation.

Camera Tracking and Line Surveying Bullet camera Line Survey Records: Cave passage dimensions 3D shape for vertical and horizontal transects Reliable visual reference Records: Absolute distance, bearing and inclination Vertical and horizontal dimensions Field sketches of shape Line surveying is a conventional cave surveying technique. Replacing the need for field measurements (click) and sketches (click). The data also provides an accurate digital record of the surveyed environment. The objective of the research was to investigate whether a synergy of camera tracking with line surveying, could provide a significant step forward in the efficiency and cost effectiveness of cave surveying. The scope of the project was not to model an entire specific cave system but to explore the use of camera tracking software as a viable and/or valuable means of doing so. – or to explore a methodoogy

Camera tracking with 2D3’s Boujou4 Calculates optimal estimates of camera rotation and movement in relation to objects as they are tracked throughout an image sequence Offers: Automatic tracking or interactive tracking Tracking with environmental markers or without The idea that lead to the research was to investigate software familiar with scenes from Harry Potter and Lord of the Rings 2D3’s Boujou4 is camera tracking software, which is a type of motion tracking software (and is also known by the term matchmoving software). Its developed in the film making industry The result is essentially a 3D survey generated from 2D video imagery. The next slide will demonstrate how the software works, but it is important to know that commercial software offers complete automatic camera tracking where no camera information or scene information is required, or alternatively the option to intervene with interactive tools and constraints. In this research environmental markers, such as this PSYOP marker, were used for one purpose, to test accuracy. They were not used to track the following scenes. Cluster of 3D points on a marker

Data captured using bullet camera & Boujou4, Chislehurst Caves 2D view b) Bend caused by correction of lens distortion 2D Feature Tracking In general, the matchmoving process tracks 2D points of interest as they move through as image sequence to produce optimal estimates of camera movement and rotation (shown in figure A). The information gathered during this process is used to calculate the depth of objects in the scene. The result, illustrated in figure B and C, are a 3D coordinate point cloud and 3D animation of a camera representing points of interest in relation to the real-world camera (Dobbert 2005; Fitzgibbon and Zisserman, 2003). B is a 2D view of a single frame, C is a 3D view of a section of cave. … so this example is at Chislehurst caves where we developed the data collection process and equipment. Good illustrative purposes but remember it is a mine not a cave… c) 3D view User defined coordinate plane Gap indicating a passage junction

Camera tracking might not always be effective Smooth curved surface If there are few points for a corner detection algorithm to detect Could use: Scene manipulation before filming Interactive feature tracking eg. POIs, survey data, CAD model Camera information Colour correction of imagery – not recommended, but may make difficult imagery useable In challenging scenes where there isn’t much to track, for example the smooth surfaces of Swine Hole’s passage at Peak Cavern, its difficult to track camera rotation and movement in a scene as it changes through a sequence of images. There are potentially a few ways we could give the softare a helping hand by Adding markers to a scene prior to filming as one shown in the slide previously Interactive feature tracking. Experimented with a CAD model of a simple table but this could just have easily been a caving helmet or piece of laminated paper with a marker on it. We experimented with these options, but again they were not used in the results. Giving the software camera information so it has fewer constraints to calculate which can speed up processing time. We found using a fixed lens instead of a zoom lens gave the program less to calculate and this made it easy to work out the focal length (6mm), so here were two things we could tell the program. In addition, we knew the CCD type by checking each camera manual. This is one option we did add as it seemed reasonable to expect a user to know this giving they plan to use the camera for surveying. How can we tell what a good solve looks like?

How can you tell? Virtual camera positions do not reflect the real world camera’s true direction or path travelled Noisy camera path High reprojection errors Spikes Noisy camera path Cameras in odd places According to matchmoving guidelines a good solve doesn’t look like this. Spikes, camera in odd places, noise, etc (Click) It looks like this (Slide 8 – camera tracking Gaping Gill) a straight line, camera moving from the direction the camera was filmed from eg. Not good

Camera tracking using Boujou4 First Frame Last frame point Start Camera path Camera path follows the direction and path the real world camera travelled. Camera path noise realistically reflects the terrain

3D Polyface Meshes a) 3D view of a mesh in Boujou4. b) The same mesh mapped to the line survey results (red and blue lines) in GIS.

3D modelling, Chislehurst Caves Results Scale differences range 0.2cm to approximately 20.3cm. Mean of four measurements was 13.3cm. Within 10-15% of the exact measurements suggested as acceptable when sketching cave passage shape (Day, 2002). Upwood Air Shaft Highclere Air Shaft 3D meshes imported into GIS software and georeferenced Accuracy of scale was tested on a sample coordinate point cloud using a small sample of ground truthed measurements. The scale differences ranged from 0.2cm to approximately 20cm. Only a very small sample of points was measured due to the challenge of accurately identifying markers in this environment, but when considered in proportion to height and width, these measurements were found to fit well within 10-15% guideline suggested by a British Caving Research Publication on cave surveying for sketched cave profiles. © Crown Copyright Ordnance Survey 2008 Mine passage connecting Upwood Shaft to Highclere Shaft at Chislehurst, Bromley

Cave survey sketches Vs 3D meshes Club Rooms, Air Shafts, Gaping Gill, Chislehurst Chislehurst North Yorkshire Field Sketches Vertical Cross Sections Slide 11 illustrates a comparison of shape. Field sketches at survey stations look similar to the sections of mesh below. More so at Chislehurst than Gaping Gill because of the pattern inside the shape which represent rocks. The passage at Chislehurst was open and high with few features in the centre, in contrast to Gaping Gill which was very low and full of rocks. This relates to the fact that its less easy to recognise nice simple shapes with more varied terrain. Detailed assessment of positional accuracy was not possible as there was insufficient control data to adequately establish a 3D model for comparative analysis. Mainly as we often could not access inclination data due to lack of equipment. (Reverse back to slide) - Bottom right image illustrates the size of Gaping Gill main chamber. Shallow half ellipse Arched ellipse Typical star shaped phreatic cave morphology

Study Areas 1. Requirements 3. Field Work 2. Testing Equipment LIDAR data exists for 2 of the sites (which is why we chose them) but was incomparable as it mapped large main chambers which were too impossible to light and film. However, the potential exists to generate 3D digital representations of entire connected cave systems by merging LIDAR data of main chambers with the matchmove data of interconnecting passages. 3. Field Work 2. Testing Equipment

Camera Tracking and Line Surveying Bullet camera Line Survey Records 3D shape for vertical and horizontal transects Reliable visual reference Records absolute distance, bearing and inclination measurements The software isn’t a Commercial of the Shelf (COTS) solution to cave surveying as camera tracking cannot measure distance and direction, but even from this short study the limitations identified were manageable and the technique complementary to the cave surveying technique of line surveying. Research indicates that future software developments may overcome this. Advantages: The cave passage is surveyed in greater detail, resulting in more accurate representation of shape and overall volume. (Instead of only a single measurement at each end of a cave, measurements are recorded wherever there are trackable features. This may allow for more accurate representation of shape and measurements of variables such as volume). Camera tracking means fewer measurements and sketches are needed in the field to record passage shape, or vertical and horizontal measurements. Accurate representation of shape Accurate measurement of dimensions /volume Accurate measurement of relative scale

Benefits to GIS and Caving New source of 3D data Low cost accurate surveying technique compatible with GIS, CAD Animation, AR, VR and Visualisation software Camera tracking research, eg. Exploration rovers in hostile environments Robotic surgery Navigation Animation Virtual TV Further Studies Comparison with laser ranging and finding equipment Compatibility with cave surveying software Test more equipment, eg. infrared New source of 3D data compatible with GIS formats (CAD, animation software, visualisation software and basically anywhere you can use coordinate triplets X, Y, Z) Not many people have laser ranging and finding equipment. In contrast this type of survey is potentially available to anyone anywhere with access to a video camera and lighting with average resolution and brightness (of course higher is better). GIS will benefit from camera tracking research in other areas. So what else are people doing with camera tracking software? Camera tracking is being used to navigate autonomous vehicles and robots through terrain above ground and below ground on Mars (Mars Exploration Rovers) as well as earth. (add picture) It’s being studied as a positioning system in areas where GPS might be unavailable. Research published in 2007 reported that a process based on a camera tracking system could maintain locational accuracy of a person at 0.5% and 1% accuracy over an area of 500 metres indoors and outdoors. In the future, autonomous navigation system research might be applicable to many areas such as Location Based Services (LBS) and navigation systems for vision impaired people, or wearable robotics (Davison, 2003 and 2007). Lastly, this research times well with fusion of CAD/GIS enabling better handling of more detailed datasets. What now? Plan to continue filming caves and test the results using tracking software eg Voodoo (freeware). Use the data with cave surveying software to assess viability and compatibility. To refine the data collection process, different cameras were used in almost every trial. Now we’ve identified what equipment worked best, it would be great to reuse the equipment and process in more trials, so the effect of equipment on performance could be ruled out. Suggestions for further study (the top few) Comparison with GPS has been done but GPS is not available underground, so comparison with laser ranging and finding equipment is one area.

Limitations References Image Quality Camera lens and light must be tailored to fit the scene: 6mm fixed lens and 6degrees Scale must be defined by the user Use h or d or a marker Optimal camera moves create parallax Dolly rather than pan Tracking corners Begin/end with survey stations Price ? References DAY, A.J., (2002) Cave Surveying – A guide to the equipment, techniques and methodology of the BCRA system, Cave Studies Series No.11. Buxton, UK: British Caving Research Association (BCRA) DOBBERT, T. (2005) Matchmoving: The invisible art of camera tracking, Sybex Inc., Alameda, CA HARTLEY, R. and ZISSERMAN, A. (2004) Multiple View Geometry in Computer Vision (Second Edition), Cambridge, UK: Cambridge University Press

Acknowledgements BCRA, 2D3, David Mountain and Jonathan Raper (City University), John Farrer (Peak Cavern), Terry Hunt (Chislehurst Caves), Craven Potholing Club (Peak Cavern), Adam Evans, Rod Legear, Paul Chambers Thank you