HYPACK® Multibeam Training Seminar PERFORMANCE TESTING HYPACK ® MULTIBEAM COURSE HYPACK® Multibeam Training Seminar 4/6/2017 4/6/2017 1 1

HYPACK® Multibeam Training Seminar Performance Testing Questions We Might Ask About Our Multibeam Data: How repeatable is the Multibeam? Out to ‘what’ Beam Angle can I use? How does it compare with Single Beam? Two Performance Tests in MBMAX Help to Answer Beam Angle Test – Shows multibeam repeatability vs. beam angle. Single Beam Test – Compares multibeam vs. single beam depths. Compare results to standards. Bottom Type Hard Soft Maximum Outlier 1 ft 1 ft Max. Mean Bias <0.1 ft <0.2 ft 95% confidence depth < 15 ft ±0.5 ft ±0.5 ft depth 15 – 40 ft ±1.0 ft ±1.0 ft depth > 40 ft ±1.0 ft ±2.0 ft From Table 3-1: Minimum Performance Standards for Corps of Engineers Hydrographic Surveys 4/6/2017 HYPACK® Multibeam Training Seminar

HYPACK® Multibeam Training Seminar Reference Survey Location Small area with a very flat, uniform bottom. The flat surface is to prevent DGPS position errors from influencing the results. Using lots of overlap and a narrow swath we get the best possible estimation of depth. Flat, uniform reference area. Preparation and Survey Two sets of seven (7) parallel lines. Separation = Water Depth. Reference MTX with 1’ x 1’ cells. Survey at low or high tides to minimize errors. (SV cast immediately before survey) Survey the 14 Reference lines. Survey the Check lines (middle lines NS and EW). Mike, I can find where the Corps Manual states that the data be “binned at 1’ x 1’ cell sizes” but cannot find where it states that there has to be a minimum of 3 soundings per cell. It states that “the data in each cell are then thinned using the average depth of all the depths in a cell”. Then it goes on to show the Shuman’s Performance Test that used a 2’ x 2’ cell size with 17 soundings per cell, but did state that, that could be too large and “might be questioned and further evaluated as the the cause” of the biases being constant around +0.1’. Therefore, that is why I have added the following slides. Survey grid. 4/6/2017 HYPACK® Multibeam Training Seminar

HYPACK® Multibeam Training Seminar Processing Reference Survey to Surface XYZ Load reference survey into MBMAX. Apply tide and SV corrections. Filter to +/- 45 degree angle limit. Edit out remaining flyers. Use the reference MTX file for Area Based Editing. Save cell average (minimum 3 soundings per cell) to Reference XYZ. Full Swath Check Lines Load check survey into MBMAX. Edit without angle limits. Start the Beam Angle Test from the Tools menu. Final reference surface is quite flat. Depth range = 51’ to 52’. 4/6/2017 HYPACK® Multibeam Training Seminar

HYPACK® Multibeam Training Seminar Creating your Reference Matrix Choose Cell Sizes that will Maximize your Full Data Set 4/6/2017 4/6/2017 HYPACK® Multibeam Training Seminar 5 5

Points per Cell Statistics 1 2 3+ Which is better?? My opinion is that the 1.5 x 1.5 cells give a more “well-rounded” look and the full Reference Area, where the smaller cells may bypass large “fliers” that are in a cell with less than 3 Soundings. This would be similar to a CUBE Surface. Reference Matrix with 1’ x1’ Cells Reference Matrix with 1.5’ x1.5’ Cells 4/6/2017 4/6/2017 HYPACK® Multibeam Training Seminar 6 6

HYPACK® Multibeam Training Seminar 4/6/2017 4/6/2017 HYPACK® Multibeam Training Seminar 7 7

Beam Angle Test Difference Statistics: Reference Minus Check. Comparison Tab is Overall Result X Axis is beam angle. Y Axis is depth difference. X’s show bias. O’s show repeatability. Maximum difference at 95% confidence. Details Tab is Result at Specific Angle Histogram is the distribution. Max Outlier = Max difference. Mean Difference shows bias between reference and the check lines. 95% confidence = 2 * Standard Deviation. 4/6/2017 4/6/2017 HYPACK® Multibeam Training Seminar 8 8

HYPACK® Multibeam Training Seminar Case Study: S/V Dobrin Successful Beam Angle Test: Results for NY District S/V Dobrin fall within USACE performance standards for survey data collected in 40’ on a hard bottom. Results give system repeatability. Other methods must be included to find system accuracy; bar check, lead line comparison, etc. Requirement Dobrin Result Max Outlier < 1.0 Ft 0.5 Ft Mean Difference < 0.1 Ft 0.0 Ft 95% Confidence 0.3 Ft USACE EM 1110-2-1003 manual HYPACK® Multibeam Training Seminar 4/6/2017

Performance Test in Action Pat asked for example with induced heave. 4/6/2017 HYPACK® Multibeam Training Seminar

Example Performance Test Result Interferometric Multibeam 4/6/2017 HYPACK® Multibeam Training Seminar

Example Performance Test Result Beamforming Multibeam 4/6/2017 HYPACK® Multibeam Training Seminar

SINGLE BEAM VS. MULTIBEAM HYPACK ® MULTIBEAM COURSE HYPACK® Multibeam Training Seminar 4/6/2017 4/6/2017 13 13

HYPACK® Multibeam Training Seminar Single Beam Test Do single beam and multibeam soundings match up? They better! Single beam check lines over the reference surface will give us a multibeam vs. single beam comparison. HYPACK® Multibeam Training Seminar 4/6/2017

Steps to the Single Beam Test Check Lines: Raw Data Single Beam Editor Edited All Format Export XYZ Format Reference.mtx MBMAX Pat suggested a flow chart. Launch Single Beam Test from Tools menu (MBMAX Phase 3) Reference.xyz Single Beam Test 4/6/2017 HYPACK® Multibeam Training Seminar

Single Beam Test Running the Test Results Select Single Beam Test from the Tools menu in MBMAX. Open the Reference XYZ file. Results Once again using difference statistics = Reference minus Check. Histogram shows the distribution and bias. Max Outlier = Max Difference. Mean Difference = Bias. This is the main point of the test! Standard Deviation (Sigma): Spread of the difference distribution. 95% Confidence = 2*Sigma. 95% of the differences fall within 2*Sigma. Not bad. Bias less than 0.1 foot! 4/6/2017 HYPACK® Multibeam Training Seminar 16 16

Sample: Single Beam Test 4/6/2017 HYPACK® Multibeam Training Seminar