1. ALERT2 Protocol Performance
National Flood Workshop
Houston, Texas
October 2010
ALERT2 Protocol Performance
Don Van Wie, Telos Services
R. Chris Roark, Blue Water Design, LLC

2. ALERT2 Implementations
ALERT2 Concentration Protocol is in production in Overland Park/KCMO for repeater-to-base path
ALERT2 is operating in parallel with ALERT on repeater-to-base leg in Urban Drainage and Flood Control District
SLIDE1
In OP-KCMO, ALERT2 is in production in Concentrator mode, transporting messages from the 3 repeaters to three different base stations.
In UDFCD Denver area, ALERT2 is now fully operational, working in parallel with ALERT2 from 5 repeaters to the base station.

3. Studies UDFCD: ALERT v ALERT2 side-by-side comparisons
8 months, 2 million ALERT messages analyzed
Overland Park: Logging at repeater sites permits exact determination of ALERT2 losses
Early June storms with partial logging
Full month analysis for September 2010
SLIDE 2
In the Denver system, we are collecting both an ALERT stream and an ALERT2 stream into a base station computer, so we are able to directly compare the performance of ALERT with ALERT2 over the same paths. We have collected and analyzed over 2 million records in the past 8 months.
In OP/KCMO, only ALERT2 operates on the path from repeaters to base. However, we have logging computers installed at each repeater site, so we can directly compare what was received at the repeater from the gages with what was sent out by the ALERT2 Modulator. We can also directly compare what was sent from each repeater to what was received at the base station.
We have some data from intense storms in early June, and have analyzed a complete dataset for the month of September

4. Purpose of testing
Demonstrate reliability of hardware in normal operating environment
Quantify the relative performance of ALERT/ALERT2
Verify or and adjust the operating parameters based on operations at a production scale
SLIDE3
Our goals for the testing are
Confirm the reliability of the hardware in an operational, production scale environment
Quantify the performance of A2 over existing, known RF paths and make quantitative comparisons with ALERT performance under operational conditions.
Use what is learned to fine tune ALERT2 operating parameters

5. The Bottom Line No hardware failures
ALERT2 data loss “over the air” is typically less than 5 reports per 10,000 over any path that is suitable for ALERT
Relative ALERT2 performance improves as traffic rates increase
RF-only Preamble has been extended 55 msec
Slot variance study shows 0.5 second slots are possible
SLIDE 4
We’ll jump to the bottom line before we bury ourselves in the data…
There have been no hardware failures
ALERT2 losses are limited to the “over the air” failures, and range in the single digits per 10,000 reports over any radio path that will work for ALERT
ALERT2’s relative performance improves greatly as the traffic rate goes up and contention losses increase on the ALERT path. The increase in ALERT2 losses during storm events, if it occurs, is relatively slight.
We learned from observing performance that the RF preamble to the A2 message should be extended to improve reliability of detection at the receiver.
We analyzed variance in slot times and learned that the time of transmission can be controlled tightly enough to permit the use of 500 msec slots for gage reports.
Now for the details

6. HARdware performance 10 BWD Modulator/Encoders
6.5 years of unit time accumulated
4 BWD Demodulator/Decoders
4.75 years of combined operation
About half of equipment is in unconditioned environment
NO Failures!
SLIDE 5
We have accumulated a significant amount of operating time with the new hardware version. There are 10 modulators now in operation, with a combined service time of 6.5 years. There are 4 receivers operating with a combined time of 4.7 years. NO issues with hardware performance or reliability, No problems with firmware.

7. Data Composition and Summary Statistics
Udfcd overview
Data Composition and Summary Statistics
Repeater
Start Date
ALERT Records
ALERT2 Records
Difference (A2-A)
% Difference
Blue Mountain
8/1/2010
328,319
334,064
5,745
1.75%
Smoky Hill
2/23/2010
771,716
781,508
9,792
1.27%
West Creek
478,256
489,544
11,288
2.36%
Lee Hill
4/9/2010
178,764
183,148
4,384
2.45%
Gold Hill
5/10/2010
250,250
261,083
10,833
4.33%
Total
2,007,305
2,049,347
42,042
2.09%
SLIDE 6
A2 is performing every bit up to expectations. In the UDFCD system, we have processed more than 2 million messages on each of the ALERT and ALERT2 channels
In 2 million ALERT transmissions, we have received 42,042 more A2 than ALERT, or 2.09%
The gains vary by repeater. Those with the best paths, such as Smoky Hill, show the lowest gain, and conversely, those with the weakest radio path show the highest gains. The differences in A2 losses over these paths are not great. This suggests that for ALERT, more messages are lost to contention on weak paths than more solid ones. This confirms something we have observed in studying ALERT losses – one message often survives a collision, and it is most often the stronger message.
The bottom trace shows the continuity of data collection over the last 8 month study period.
Not included in the ALERT:ALERT2 comparison is over 45,000 reports from an A2 weather station that reports directly to the system receiver over a 25 mile obstructed path. It has lost three A2 packets out of more than 11,000

8. Alert To alert2 direct comparison
SLIDE 7
In the UDFCD analysis, we took a sample of reports from each of 3 repeater sites and laid the ALERT and ALERT2 records out side by side.
We counted the records that were in the A list but not the A2 list, and vice-versa, and that gave us an accurate count of how many were lost on each channel. This was during a quiet period with little or no rainfall.
Again, the losses for both A and A2 were greater on the weaker radio paths.
Across the 3 repeaters and reports, there were 1560 ALERT reports lost and 6 errors introduced which accounted for 2.08% of the data.
A total of 10 reports were lost on the ALERT channel for a loss rate of 1.3 per 10,000
Another way of assessing the improvement is that the loss ratio was 156 to one in favor of ALERT2
ALERT
ALERT2
Repeater
Sample
Missed by
Alert %
Errors
Lost
Alert2
Blue Mtn
25,000
445 1
1.78%
0.004%
Smoky Hill
149
0.60% 4
0.016%
Gold Hill
966 5
3.88%
0.020%
75,000
1,560 6
2.08% 10
0.013%

9. Recent Overland Park performance
SLIDE 8
This is a comparable study covering a quarter million reports from the Overland Park system during the month of September, this year.
We used the log of reports sent from the repeater sites to compare the data received at each of two base stations in Overland Park.
The losses at the Fire Training Center site are 127 reports out of about 295,000, or 4 in 10,000.
The losses at the City Hall site are much higher, This led us to a spectrum analysis at the receiver, where we found severe RF interference. Filtering will be added to this site.
Summary of September 1-28, 2010
Tx From:
Johnson Co
Blue Valley
Century Twrs
Totals/Avg
Rpts Sent
72678
101091
120940
294709
OP City Hall
Lost Rpts
2326
2785
2774
7885
% Loss
3.20%
2.75%
2.29%
2.68%
Fire Training Cntr 31 75 21
127
0.04%
0.07%
0.02%

10. June 8 o.p. storm alert losses
SLIDE 9
There were some severe storms in June in the KC area for which we had hoped to collect detailed data. Our logging processes were just getting underway, and there were a number of glitches in retrieving data. We have data for the Blue Valley Repeater to OPCH path only, and we know the OPCH site to be impaired, so the numbers are of limited value.
During the June 8 storm, traffic reached 3300 reports per hour briefly at Blue Valley (just 1 of 3 repeaters affected).
The total ALERT losses during the peak of the storm reached 55%
The ALERT2 losses reached more than 3% in the peak period; much of this was the radio interference at the site; we can’t reliably assess how much rain fade was an issue.

11. alert2 performance in rain event
SLIDE 10
This summer was unusually quiet in the Denver area, and the peak reporting rate was about 2000 reports per hour.
At this level of activity, ALERT losses rose to about 15%, while ALERT2 losses remained in the single digits.
The plot shows the traffic rate in red, and the performance improvement of A2 over Alert in green. As expected, the improvement roughly follows the traffic rate.
For 6 hours of storm period, ALERT 2 reduced loses by more than 98%, while the ratio of ALERT loss to ALERT2 loss was 52:1
There were only 11 messages lost by ALERT2 during this period, so the performance improvement is essentially a proxy for the ALERT contention losses.

12. Comparison of losses – june 11-12

13. Smoky hill traffic – alert2 only
SLIDE 12
This is a plot of the ALERT2 traffic from Smoky for the entire season. The few small events we had show as spikes. There was continuity of data except for one brief base station outage.

14. Smoky hill traffic – alert on alert2
SLIDE 13
If we plot the ALERT traffic on top of the ALERT 2 traffic, we see that there is an excellent matchup
You can consistently see the ALERT2 peaks sticking out above the ALERT. The bigger the peak, by and large, the bigger the difference in the spikes…showing once again that the ALERT2 improvement is proportional to the traffic rate.

15. Alert2 performance v traffic load
SLIDE 11
We lumped all 1 hour periods that had the same traffic level, then plotted the ALERT2 performance improvement against the traffic level. It shows the steady increase in relative performance of ALERT2 over ALERT.
ALERT losses can be modeled using the Poisson equation. The upper red line models losses assuming that both participants in a collision are destroyed. This would be true in a copper wire application, but in an RF environment we have seen that one may survive.
However, at least one must always be lost, and that is the lower green line. The actual losses should fall somewhere between those two lines.
In our study, the losses were about 2% at 500 reports per hour, but the minimum according to the model is about 4%. We believe the difference is because at background, no rain reporting, most of the traffic is from weather stations that send out timed reports with several messages concatenated. There are even efforts made to keep these timed reports from interfering with each other, so the losses come in lower than theoretical.
As the traffic increases with single message rain reports, that effect is diluted out, and we see the losses beginning to fall within the model limits. We expected to see some larger traffic numbers to work with, but we were careful not to wish for them.

16. Variance from expected slot time
SLIDE 14
We used the all the OP September data to study how much jitter occurs in the timing of the Encoder-Modulator transmission. The variation here determines the portion of the slot that can be used for actual transmission.
This plot has a dot for each transmission from each of the three repeaters. The x axis covers 2 days; the Y axis is +/-100 msec.
The variation from expected slot time is almost always within 20 msec. One outlier is shown here at about 25 msec.

17. Outliers in GPS CLOCK Setting
SLIDE 15
However, when we expand the Y axis we see that there was one outlier in the month where the variance was in the range of 600 msec for 30 minutes. This was the result of a single bad setting from the GPS on its regular 30 minute check. Clearly the drift in the modulator clock is small and we need to deal with the possibility of getting a bad time read from the GPS

18. Coming improvements to gps synchronization algorithm
To eliminate false setting by anomalous readings:
Limit correction based on plausible drift rate
Lengthen time in lock before taking readings
SLIDE 16
This is one of the lessons from the massive data collection effort.
The forthcoming version of the GPS time setting algorithm will include limiting the amount of correction from any single reading to a reasonable limit, given the time since the last correction.
There is also some indication that waiting a few seconds to read the time after lock is achieved yields more accurate time readings. The next version will include this feature

19. Implications for channel capacity
Lengthened preamble affects capacity
Affects first block only
Impact diluted as traffic increases
Size of required deadband controls available slot time; 100 msec will be adequate
500 msec slots can be used for most gage sites
Capacity of 1 channel is 120 gages, 331 KBytes/ hour
2 sec slot (repeater) can carry 630 Kbytes/hour or 157,500 ALERT Messages
SLIDE 17
We also found that lengthening the preamble time by 40 milliseconds improved performance. We now have a more accurate picture of what the data capacity will be, given the change in length of the first block and the amount of time needed to allow for slot time variance.
A deadband of 100 msec will be adequate, so there will be 400 msec of available transmission time if we use ½ second slots for gage transmissions. This will allow a two block transmission which will be adequate for sites with a rain gage and stage gage.
With 1 minute latency, a single channel could support 120 gage sites, and would have a theoretical capacity of 331 Kbytes per hour
Using a 2 second slot for repeaters yields a capacity of 630 Kbytes per hour, or the equivalent of 157,500 ALERT messages per hour.

20. What’s next… Ready for new ALERT Concentrator applications
Low power repeater for 2011
Vendors are working on gage applications
Implementation of Protocol Application layer

21. Thank you