Combining the strengths of UMIST and The Victoria University of Manchester 1 Middleware support for Decision Support Tools (DSTs) in water engineering John M. Brooke Kashif Khan, Robert Haines School of Computer Science, The University of Manchester EGI Community Forum, Manchester, April th April 2013

Combining the strengths of UMIST and The Victoria University of Manchester 2 Scheme of Presentation 1.Introduction 2.High Level Architecture 3.Overview of Architectural Components  Hydraulic Simulation Component  Data Storage Component  Data Acquisition Component  Optimization Component  High Performance Computing (HPC) Component  Field Interface Component 4.Prototype DST for Field Engineers

Combining the strengths of UMIST and The Victoria University of Manchester 3 Network operations has typically three phases: a.Planning decisions based on the available knowledge b.Turning decisions into actions c.Actions change system state and form the basis of further decisions Dynamic or Current Contextual Knowledge 1. Planning Decisions2. Field Actions3. System State Modifies Affects Combined Knowledge Make use of ResultsChanges Phases in Performing Network Operations

Combining the strengths of UMIST and The Victoria University of Manchester 4 Motivation Access to dynamic contextual knowledge is necessary for informed decisions But it is remained tacit in most of the distributed infrastructure, particularly for performing network operations in the field Lack of situation awareness about the on-going events:  Customized tools operating on static datasets  E.g. static GIS datasets, SOPs libraries and simulation models Lack of communication among the workforces to coordinate their activities:  Teams work separately in an integrated dynamic system  There exist issues of ordering and concurrency as decisions can impact each other

Combining the strengths of UMIST and The Victoria University of Manchester 5 High-Level Objective The primary objective is to help in establishing a Cyber-Physical System (CPS) for Monitoring and Control of Distributed Infrastructure

Combining the strengths of UMIST and The Victoria University of Manchester 6 High Level Architecture

Combining the strengths of UMIST and The Victoria University of Manchester 7 Wireless Sensor Network Data Acquisition & Processing Agent TinyOS API Real-time raw data Data warehouse Filtered/Processed Data Extended Simulation Toolkit Conventional Computational Model Grid Computing Resources Asynchronous Messaging Queues Concurrency Control Mechanism REST based API Sensors Update Scheduling of Future Decisions Alerts Queries/Results Dynamic and Predictive DST Dynamic and Predictive DST Alerts Queries, Planned Future Actions, Alert Subscriptions Results/Alerts GIS Data Web based Clients Optimization Toolkit Adjusted/Calibrated Parameters HPC Toolkit

Combining the strengths of UMIST and The Victoria University of Manchester 8 Overview of Architectural Components

Combining the strengths of UMIST and The Victoria University of Manchester 1. Hydraulic Simulation Component EPANet toolkit is selected a Hydraulic Network Solver  Freely available with complete source code  Used both in academia and industry  Robust sets of equations and simple hydraulic solver based on Gradient Method Enhancement made to EPANet toolkit are as follows:  The toolkit is made accessible to different operating systems, such as, Linux, Unix, Solaris etc.  The toolkit is transformed into another high level Perl language o The new toolkit is called Perl-EPANet o Allows accessing the toolkit under: − Web applications and services − Grid applications and services 9

Combining the strengths of UMIST and The Victoria University of Manchester 1. Hydraulic Simulation Component (Contd.)  Using the approach followed it can easily be converted to Java, Tcl, Python, C# etc.  Perl-EPANet is extended with novel functions and constant for supporting dynamic Extended Period Simulations (EPS).  The novel functions and constants enable the Perl-EPANet toolkit to derive the hydraulic simulations based on time series data available in the database.  Can be configured to incorporate data from: o SCADA (Status and setting of links and tanks levels) o Human Sources (Field Observations and Planned Future Actions) o WSNs (adjustments in uncertain variables to operate in online mode) 10

Combining the strengths of UMIST and The Victoria University of Manchester 1. Hydraulic Simulation Component (Contd.) Configuration can be done using toolkit’s constants, or using the XML configuration file The novel hydraulic functions are achieved using wrapper based approach instead of modifying the EPANet code directly  Less development cost  More robust 11

Combining the strengths of UMIST and The Victoria University of Manchester 2. Data Storage Components The information, coordination and communication hub for all components in the architecture Based on RDBMS, hence  Large amount of data can be stored.  Data can be normalized into several tables to avoid redundancy.  Faster retrieval of data using features like indexing, views etc.  Fault tolerance and load balancing Concurrent access of data by core components of the architecture. Decoupling of the entities producing the data from the entities consuming the data. Support for varied data formats, such as, geospatial and XML based data storage and retrieval. Optimize queries and retrieval for different kind of data formats. Security of data 12

Combining the strengths of UMIST and The Victoria University of Manchester 3. Data Acquisition Component Responsible for data gathering from sensor based sources, such as, SCADA and WSNs Responsible for improving data quality based on user defined rules Responsible for storing the received time series data in respective place in the RDBMS Current implementation is in Java (write once, use anywhere support ) and is based on:  Receiving data from TinyOS based sensor nodes o TinyOS and TinyDB middleware have been extended to support hydraulic and water quality sensors o The acquisition component can receive data from real motes or TOSSIM simulator  Receiving data by processing a CSV file 13

Combining the strengths of UMIST and The Victoria University of Manchester 4. Optimization Component Water network models have a number of uncertain parameters/variables  Water Demands (the most dynamic parameter)  Pipe roughness Coefficients  Pipe diameters  Valve status and settings, etc. Often calibrated using short term field data (01 week) Parameters can be adjusted on the basis of real-time hydraulic data, such as, pressures and flow rates, received from WSNs Evolutionary Computing methods (e.g. Genetic Algorithm) are commonly used to solve inverse problem for adjusting uncertain variables 14

Combining the strengths of UMIST and The Victoria University of Manchester 4. Optimization Component (Contd.) Flow Chart of Demand Prediction without M5 Predictor 15 Flow Chart of Demand Prediction with M5 Predictor

Combining the strengths of UMIST and The Victoria University of Manchester 4. Optimization Component (Contd.) 16

Combining the strengths of UMIST and The Victoria University of Manchester 5. HPC Component HPC component is based on Task-Pool paradigm Implemented in Java Responsible for communicating Tasks to several distributed servers Servers can be running on local clusters, or/and can also excess Grid computing resources Task Queue receive tasks in FIFO order Scheduler sub component distributes task to servers and receive the response Optimization component working with HPC allows the evaluation of calibration problem by running:  multiple scenarios concurrently  with different grouping of decision variables  With different number of hydraulic sensors  With different selection of uncertain variables 17

Combining the strengths of UMIST and The Victoria University of Manchester 5. HPC Component (Contd.) 18

Combining the strengths of UMIST and The Victoria University of Manchester Visualization Interface Component Based of REST (Representational State Transfer) Architectural Style of developing web services. Benefits are:  Client Server: Interaction based on protocol, hence, loosely coupled  Uniform Interface: Simplicity and Implementation independence  Code On Demand: Visibility, Reliability, Extensibility  Layered System: Low coupling  Client Side Caching: Efficiency and User Experience Web 2.0 can be used to develop dynamic clients  Google Maps API/OpenLayers are used to develop GIS functionality  Ajax are used to perform bi-direction asynchronous communication with the server  Using Ajax GIS data and simulations results can be downloaded into the clients and queries/decisions can be sent to the server  Document based approach i.e. XML is used for the exchange for messages

Combining the strengths of UMIST and The Victoria University of Manchester Network Analysis Screen 20

Combining the strengths of UMIST and The Victoria University of Manchester Hydraulic Analysis Screen 21

Combining the strengths of UMIST and The Victoria University of Manchester An Advance User Interface in DST 22

Combining the strengths of UMIST and The Victoria University of Manchester DST for Lightweight Devices 23

Combining the strengths of UMIST and The Victoria University of Manchester 24 Results of the predictive tool The graph shows the results of predicted DMFs for 24 hours (at time step of 01 hour) and compares with the DMFs available in the original data. The DMFs are predicted based on the SCADA and WSN data by the genetic algorithm.. Accuracy in Predicting the Demand Multiplication Factors (DMFS) using sensor data

Combining the strengths of UMIST and The Victoria University of Manchester Future Plans Field trials with water companies Upgrading simulation software to be natively dynamic rather than via wrappers. Extension of approach to other network engineering, e.g. smart electrical grid. Investigation of different models for supplying necessary computational resource, public vs private, grid or cloud interfaces, linking mobile and distributed computing. Thanks for your attention 25