HydroTel/iLink/WebServices Overview Mention the title of the presentation AUGUST 2019

Intro slide HydroTel v5.2019.1 New features Some uncommonly used new-ish features Current Developments iLink WebServices/IOT Overview of presentation New features in hydrotel since last time – some many might know, others not, some boring Things we’re working on and where we’re headed Talk about status and stability of latest version HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

SQL Server changes HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

SQL Server changes Data type ‘datetime’ Storage Size – 8 bytes SQL Server data types Data type ‘datetime’ Storage Size – 8 bytes Accuracy - Rounded to increments of .000, .003, or .007 seconds Deprecated Data type ‘datetime2’ Storage size – 7 bytes Accuracy - 100 nanoseconds HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

SQL Server changes SQL Server 2016 compatibility change SQL Server 2016 started implicitly converting datetime to datetime2 (default precision 7) Literal strings still kept their 3 digit milliseconds Old: New: HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

SQL Server changes Solution: Temporary – Set database compatibility to SQL Server 2014 or lower Permanent – No more datetime types SampleBuf table recreated to use datetime2 HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

SQL Server changes New Stored Procedure [sp_HydroTel_Sys_AddToSampleBuf] Benefit for users – Performance? Benefit for developers - Abstraction and maintenance HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

Image Importer New Transfer Agent plugin for image files HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

Site creation from Config HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

Site creation from Config HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

Site Exports Historically HydroTel exports have been point-centric Many small files created HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

Site Exports Site Exports can be created at the Site level Supported formats so far: HydroTel XML Hydstra CSV Generic/Plain CSV Still maintains lastexportedDT for each individual point HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

SMS Agent restart tweak Changes made in latest HydroTel version to alleviate modem woes - SMS Agent now more willing to restart instead of shutting itself off Modems still can be a source of frustration Consider moving to HTTP SMS Service HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

Reporting Custom-built SQL-based reporting Example from dewlp HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

Reporting Custom-built SQL-based reporting Example from dewlp HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

Reporting Custom-built SQL-based reporting Example from dewlp HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

iLink New development focused on iRIS 270 features Real-time diagnostic logging SDI-12 commands Calibration graph(s) Configuration… HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

iRIS270 Configuration structure Sending/retrieving from the logger Historically, an iRIS logger configuration is stored as a ‘blob’ in memory and was sent & retrieved from the logger in its entirety as a data structure HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

iRIS270 Configuration structure Sending/retrieving from the logger iRIS270 uses a tag and delta system: Every configuration item is assigned a unique “tag” identifier (enumeration) Tags are mapped to configuration items Configuration is sent and received from the logger as a set of “deltas” Only sends items that have changed since last time HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

WebServices Recap from last time HydroTel Web Services driver Get data in SOS XML format, EcoConnect2/from NIWA etc. Set up as a “logger” in HydroTel HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

Web Services Today Still demand for WebServices as a means of retrieving data from a provider Strong leaning towards JSON as the data format of choice Similar data formats and endpoints but not similar enough to create a generic driver to work for everything HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

Waterwatch.io Sensors push data to the cloud – viewable on the waterwatch website Provides a REST API for retrieving data Login to website to get an API key and secret token (secure!) Well documented and simple API HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

WaterWatch.io List Sensors https://api.waterwatch.io/v1/organisations/9db2aaf0-8d46-11e9-bc9e-8d0900f1a481/sensors/ HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

WaterWatch.io List Measurements https://api.waterwatch.io/v1/organisations/9db2aaf0-8d46-11e9-bc9e- 8d0900f1a481/sensors/20000036/measurements?startTime=1560981244199&endTime=1564418652000 HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

Waterwatch Driver for HydroTel Set up as a logger in HydroTel No need to worry about all the API requests or JSON responses Put the logger in a schedule and the rest will take care of itself HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

MQTT Message Queuing Telemetry Transport MQTT is a simple messaging protocol, designed for constrained devices with low-bandwidth The MQTT protocol is based on the principle of publishing messages and subscribing to topics, or "pub/sub". The technology enabling Facebook messenger HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

MQTT Publish / Subscribe Each subscriber receives messages for topics they have subscribed to A publisher sends messages for one or more ‘topics’ HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

MQTT Publish / Subscribe The publisher does not know about subscribers and is not responsible for delivering messages to them A ‘broker’ manages subscriptions and forwards published messages HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

MQTT Publish / Subscribe The Publisher: Connects to the broker via the Internet and notifies it that it has a message to publish on a particular topic. The Subscriber: Connects to the broker to subscribe to topics A continuous process ‘listens’ to the broker to see if messages have been published to the topics of interest house/room1/main-light house/room1/alarm house/garage/main-light house/main-door HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

MQTT HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

MQTT Benefits Fast Secure - TLS encryption is used to secure the message during transport. The broker service is managed by someone else – e.g. Azure, AWS, others No more files High availability, scalability HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

HydroTel MQTT Agent Prototype / Early development stage New Object Type – MQTT Sensor Auto subscribe based on Point IDs HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

Live Demo Show – Agent config Object Config Point Config Phone APP Data HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

MQTT Agent Cons / Further work needed There is no standard for the format of MQTT messages, or topic identifiers The content of a pub/sub message can contain anything binary loosely structured text (CSV, TOA5) highly structured text (JSON, XML) Support for multiple brokers Other protocols – HTTPS Without standards each receiver will be responsible for interpreting data provided to them. Parsing Data Formats Mapping Site IDs Mapping Parameter IDs Data publishers would still need to coordinate changes with subscribers HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

Future HydroTel Client HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

Future Web Server HydroTel etc. | Author: Jared Lindsey | Creation date: August 2019

Thank You Jared Lindsey