1. HydroTel/iLink/WebServices Overview
Mention the title of the presentation
AUGUST 2019

2. Intro slide
HydroTel v New features Some uncommonly used new-ish features Current Developments iLink WebServices/IOT
Overview of presentation
New features in hydrotel since last time – some many might know, others not, some boring
Things we’re working on and where we’re headed
Talk about status and stability of latest version
HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

3. SQL Server changes
HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

4. SQL Server changes Data type ‘datetime’ Storage Size – 8 bytes
SQL Server data types
Data type ‘datetime’
Storage Size – 8 bytes
Accuracy - Rounded to increments of .000, .003, or .007 seconds
Deprecated
Data type ‘datetime2’
Storage size – 7 bytes
Accuracy nanoseconds
HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

5. SQL Server changes
SQL Server 2016 compatibility change
SQL Server 2016 started implicitly converting datetime to datetime2 (default precision 7) Literal strings still kept their 3 digit milliseconds Old: New:
HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

6. SQL Server changes
Solution: Temporary – Set database compatibility to SQL Server 2014 or lower Permanent – No more datetime types SampleBuf table recreated to use datetime2
HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

7. SQL Server changes
New Stored Procedure [sp_HydroTel_Sys_AddToSampleBuf] Benefit for users – Performance? Benefit for developers - Abstraction and maintenance
HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

8. Image Importer New Transfer Agent plugin for image files
HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

9. Site creation from Config
HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

10. Site creation from Config
HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

11. Site Exports Historically HydroTel exports have been point-centric
Many small files created
HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

12. Site Exports Site Exports can be created at the Site level
Supported formats so far:
HydroTel XML
Hydstra CSV
Generic/Plain CSV
Still maintains lastexportedDT for each individual point
HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

13. SMS Agent restart tweak
Changes made in latest HydroTel version to alleviate modem woes - SMS Agent now more willing to restart instead of shutting itself off Modems still can be a source of frustration Consider moving to HTTP SMS Service
HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

14. Reporting Custom-built SQL-based reporting Example from dewlp
HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

15. Reporting Custom-built SQL-based reporting Example from dewlp
HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

16. Reporting Custom-built SQL-based reporting Example from dewlp
HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

17. iLink New development focused on iRIS 270 features
Real-time diagnostic logging
SDI-12 commands
Calibration graph(s)
Configuration…
HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

18. iRIS270 Configuration structure
Sending/retrieving from the logger
Historically, an iRIS logger configuration is stored as a ‘blob’ in memory and was sent & retrieved from the logger in its entirety as a data structure
HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

19. iRIS270 Configuration structure
Sending/retrieving from the logger
iRIS270 uses a tag and delta system:
Every configuration item is assigned a unique “tag” identifier (enumeration)
Tags are mapped to configuration items
Configuration is sent and received from the logger as a set of “deltas”
Only sends items that have changed since last time
HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

20. WebServices
Recap from last time
HydroTel Web Services driver Get data in SOS XML format, EcoConnect2/from NIWA etc. Set up as a “logger” in HydroTel
HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

21. Web Services
Today
Still demand for WebServices as a means of retrieving data from a provider Strong leaning towards JSON as the data format of choice Similar data formats and endpoints but not similar enough to create a generic driver to work for everything
HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

22. Waterwatch.io
Sensors push data to the cloud – viewable on the waterwatch website
Provides a REST API for retrieving data
Login to website to get an API key and secret token (secure!)
Well documented and simple API
HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

23. WaterWatch.io List Sensors
HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

24. WaterWatch.io List Measurements
8d0900f1a481/sensors/ /measurements?startTime= &endTime=
HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

25. Waterwatch Driver for HydroTel
Set up as a logger in HydroTel No need to worry about all the API requests or JSON responses Put the logger in a schedule and the rest will take care of itself
HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

26. MQTT Message Queuing Telemetry Transport
MQTT is a simple messaging protocol, designed for constrained devices with low-bandwidth
The MQTT protocol is based on the principle of publishing messages and subscribing to topics, or "pub/sub".
The technology enabling Facebook messenger
HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

27. MQTT Publish / Subscribe
Each subscriber receives messages for topics they have subscribed to
A publisher sends messages for one or more ‘topics’
HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

28. MQTT
Publish / Subscribe
The publisher does not know about subscribers and is not responsible for delivering messages to them A ‘broker’ manages subscriptions and forwards published messages
HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

29. MQTT
Publish / Subscribe
The Publisher:
Connects to the broker via the Internet and notifies it that it has a message to publish on a particular topic.
The Subscriber:
Connects to the broker to subscribe to topics
A continuous process ‘listens’ to the broker to see if messages have been published to the topics of interest
house/room1/main-light
house/room1/alarm
house/garage/main-light
house/main-door
HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

30. MQTT
HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

31. MQTT
Benefits
Fast
Secure - TLS encryption is used to secure the message during transport.
The broker service is managed by someone else – e.g. Azure, AWS, others
No more files
High availability, scalability
HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

32. HydroTel MQTT Agent
Prototype / Early development stage New Object Type – MQTT Sensor Auto subscribe based on Point IDs
HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

33. Live Demo Show – Agent config Object Config Point Config Phone APP
Data
HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

34. MQTT Agent
Cons / Further work needed
There is no standard for the format of MQTT messages, or topic identifiers
The content of a pub/sub message can contain anything
binary
loosely structured text (CSV, TOA5)
highly structured text (JSON, XML)
Support for multiple brokers
Other protocols – HTTPS
Without standards each receiver will be responsible for interpreting data provided to them.
Parsing Data Formats
Mapping Site IDs
Mapping Parameter IDs
Data publishers would still need to coordinate changes with subscribers
HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

35. Future HydroTel Client
HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

36. Future
Web Server
HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

37. Thank You
Jared Lindsey