1. Emerging Technologies and It's Use
Implementing fast-track methodologies in IoT-based technology development for the meteorological community: A multidisciplinary approach
By: Bienifer Vinluan

2. About the presentation
This presentation will try to address the following questions:
What are the changing user demands?
What are driving these changes? and
Why should we care?

3. Changing user demands (cont'd)
To begin, our legacy systems today still serve our our community very well, but with an information rich society and availability of mobile devices, our existing systems will not cope with this new generation of expectations. As an example, our observation system architecture looks like this:

4. Existing System Infrastructure
Value Add to
AWS Data
Fully Manual
Coop Observer
Manual Observations
Message Switch
AWS One Minute Data
OMD or NVP format
CMSS WA NT
QLD
NSW
VIC SA
TAS HO
RTDB (NEONS)
Climate
Aviation
Forecasting
International
Regional Products
HO & Regional Computing Systems
AWS One Minute Data
External Product – Web/FTP

5. Internet and our users
This system is robust, highly available and provides the required product/output for public and commercial consumption.
But it has its constraints, there's a growing expectation that data can be provided to everyone, everywhere on everything??
Our new generation of users will only recognise the internet, smart devices and use of:
And we can't easily make end-to-end system changes without huge investments and project overhead…

6. Adopting Emerging Technology
We need to look into different ways of developing and delivering those products fast, up-to-date and can be easily consumed by our users.
So., we dived into the Internet of Things (IoT), in particular we looked at MQTT an IoT protocol.

7. MQTT protocol
Extremely simple and lightweight binary messaging protocol, Designed for constrained devices and low-bandwidth, high-latency or unreliable networks; Ensures reliability and a degree of assurance of message delivery, via its inbuilt three level QoS (Quality of Service) delivery model;
IBM developed with open source community support;
From here we did a proof-of-concept to delivery data through a data broker that has capability to publish and subscribe..

8. MQTT – Data Broker System
Push Sensor One Minute/ One second Data
Subscribe to ASOS-CS 1 minute data
Applications
Publish Sensor One Minute Data
Publish Calculated Data
Subscribe to ASOS-CS 1 minute data
HO & Regional Computing Systems
MQTT WA
Subscribe to Sensor One Minute Data NT
QLD
NSW
VIC
Push calculated Data, met products SA
TAS
Subscribe to products or ASOS-CS one minute data HO
External Clients

9. Key objectives of the Data Broker
The key aim in this stack implementation was to publish as many sources of relevant Bureau information to the broker. This information will include:
1 minute AWS messages from the Bureau’s 700 AWS;
3rd Party AWS data;
Relevant model output and forecasts;
Satellite graphics or images; and,
Customised products.

10. Technology Stack
Measurement devices (devices such as Raspberry Pi’s or telemetry devices using the MQTTX protocol );
A Publish /Subscribe broker that forms the core of our information exchanges;
The use of IoT friendly web and HTTP methods, MQTTX and web sockets;
Visualisation tools;
Node.js - event driven, non-blocking server side JavaScript, optimized throughput and scalability;
Web sockets - full-duplex communication;
NoSQL, Time Series and Key Value data stores (use of Influx Db);
Geo-hash technologies;
Meteor and React – JavaScript based, enables rapid application development for web, mobile, desktop and building reusable components;

11. Our approach to Development
When new trends come up we look, we assess, we think, we evaluate…
But this traditional process has been proving to take just too long.
By the time we complete any
working system the technology
has become obsolete….
and by then we've invested
too much time/effort and $$$

12. Our approach to Development(cont'd)
So we need to think differently…. When we started to tap into the pub/sub platform using MQTT as an IoT protocol we needed to do a very rapid prototype to prove that it can be integrated into our existing architecture… In doing this we have to form team that is Agile – we started to adopt the Agile Framework..

13. Our approach to development (cont'd)
We formed a multidisciplinary team from various areas – IT Developers, Testers, System Administrators, Engineers and Project Management and worked on this project for 3 months.
We worked under an
Agile Devops environment
The outcome is positive – we are able to deliver a number of features including:
Data provisioning through pub/sub platform;
Data visualisation;
Download Features;
Basic alerts and thresholds – per AWS rule.

14. Challenges This didn't get through without challenges such as:
The need to prove that the technology is scalable and can be easily extended at a notice.
The organisation is still in its early stages of being Agile ready.
The ongoing cost and maintenance has not be fully considered.

15. Conclusion
To address changing user demands in this information rich society we now live in and continue to be relevant in the community we need to have a two-fold strategy:
That will allow us to tap into the new space of emerging technology.
2. To develop products and deliver them in an agile fashion. We need to be quick, fast or we will miss opportunities.

16. Questions?
Thank you for listening…. Bienifer Vinluan