Implementing the WPS Standard A Case Study for Dissemination of Coastal and Marine Tools GIMA Master Thesis Joost Boerboom

Welcome GIMA Master Thesis Carried out at Deltares in Delft 27 mei 2018 Welcome GIMA Master Thesis Carried out at Deltares in Delft February – August 2013 Committee: dr. Fedor Baart dr.ir Gerben de Boer drs. Marian de Vries prof.dr.ir. Peter van Oosterom

Contents Introduction Case 1 – Tide Forecasting Service Case 2 – Interactive Dredge Planning Tool Discussion Conclusion Recommendations Questions

1. Introduction 27 mei 2018 Dynamic areas, data collection valuable Introduction – Case 1 – Case 2 – Discussion – Conclusion – Recommendations Dynamic areas, data collection valuable -> numerical and physical modeling -> dissemination of models and boundary conditions

Remote Processing 1+1=2 1+1=? 1+1=? 1+1=2 1+1=2 27 mei 2018 Introduction – Case 1 – Case 2 – Discussion – Conclusion – Recommendations 1+1=2 1+1=? 1+1=? 1+1=2 1+1=2

Web Processing Service 27 mei 2018 Web Processing Service Introduction – Case 1 – Case 2 – Discussion – Conclusion – Recommendations OGC Standard for (operating) remote processes No limitations to: Script language Data types Etc. Bridge between client and server

27 mei 2018 Why Standards Introduction – Case 1 – Case 2 – Discussion – Conclusion - Recommendations

Web Services WPS WCS WFS WMS Online medium for: Processing Coverages 27 mei 2018 Web Services Introduction – Case 1 – Case 2 – Discussion – Conclusion – Recommendations WPS WCS WFS WMS Online medium for: Processing Coverages Features Rasters

Using Web Services Browser 27 mei 2018 Using Web Services Introduction – Case 1 – Case 2 – Discussion – Conclusion – Recommendations Browser http://dtvirt5.deltares.nl/wps/?request=Execute&service=wps&version=1.0.0&identifier=constituents&datainputs=[constituent=M2] Different requests: GetCapabilities DescribeProcess Execute

Web Service Advantages 27 mei 2018 Web Service Advantages Introduction – Case 1 – Case 2 – Discussion – Conclusion – Recommendations Always the latest version of data / software No more downloading everything Interoperability

Goal Goal of the research: 27 mei 2018 Goal Introduction – Case 1 – Case 2 – Discussion – Conclusion – Recommendations Goal of the research: “expose complex processes that feed coastal and marine models via a Web Processing Service” Deltares’ goals: “provide a path for existing Matlab tools to be used in next generation user interface” “use boundary conditions from large datasets”

27 mei 2018 Research Questions Introduction – Case 1 – Case 2 – Discussion – Conclusion – Recommendations Can complex processes that feed coastal/marine models be exposed via a Web Processing Service? Can a Web Processing Service be used as a wrapper service for non domain-specific processing services? Which adaptations could be made to the WPS standard to improve the possibilities for application of the standard?

Methodology Literature study 27 mei 2018 Methodology Introduction – Case 1 – Case 2 – Discussion – Conclusion – Recommendations Literature study Practical work in the sense of, for Deltares, relevant case studies

System Setup PyWPS Python based Open Source 27 mei 2018 System Setup Introduction – Case 1 – Case 2 – Discussion – Conclusion - Recommendations PyWPS Python based Open Source Relatively small developer community Deltares test server - http://dtvirt5.deltares.nl/wps/?

2. Case 1 – Tide Forecasting Service 27 mei 2018 2. Case 1 – Tide Forecasting Service Introduction – Case 1 – Case 2 – Discussion – Conclusion – Recommendations What: Global tide forecasting service Why: Relevant for Deltares and expected to be a popular service

Tide Forecasting What is the tide: The tide is an extremely long wave 27 mei 2018 Tide Forecasting Introduction – Case 1 – Case 2 – Discussion – Conclusion – Recommendations What is the tide: The tide is an extremely long wave Astronomical influences Water levels are location specific Required for forecasting: Constituents Amplitudes Starting point in time How to get the inputs: - Long period measurements

Tidal Constituents K1 and M2 – Lunar diurnal 27 mei 2018 Introduction – Case 1 – Case 2 – Discussion – Conclusion – Recommendations K1 and M2 – Lunar diurnal

Satellite Altimetry 27 mei 2018 Introduction – Case 1 – Case 2 – Discussion – Conclusion – Recommendations

Different Services Different services have been developed 27 mei 2018 Different Services Introduction – Case 1 – Case 2 – Discussion – Conclusion – Recommendations Different services have been developed - Base of the script is similar

3. Case 2 – Interactive Dredge Planning Tool 27 mei 2018 3. Case 2 – Interactive Dredge Planning Tool Introduction – Case 1 – Case 2 – Discussion – Conclusion – Recommendations What: Existing online tool made by Deltares/Ecoshape to forecast environmental effects of dredging operations Why: Show that WPS is also able to ‘wrap’ existing (legacy) services

27 mei 2018 Working of IDPT Introduction – Case 1 – Case 2 – Discussion – Conclusion – Recommendations

27 mei 2018 Method of Wrapping Introduction – Case 1 – Case 2 – Discussion – Conclusion – Recommendations Have the WPS service post input files and read the outputs

IDPT via WPS Limited input Less user friendly compared to the original 27 mei 2018 IDPT via WPS Introduction – Case 1 – Case 2 – Discussion – Conclusion – Recommendations Limited input Less user friendly compared to the original Identical results

4. Discussion PyWPS Small community Slow development 27 mei 2018 4. Discussion Introduction – Case 1 – Case 2 – Discussion – Conclusion – Recommendations PyWPS Small community Slow development Tide Forecasting Service Expected limited accuracy (better dataset available) Too slow IDPT Wrapper No status support Timeouts Slow, just like the original one Dataset resolution 0.25 degree Tide Run time about 15 seconds IDPT run time over a minute without statussus

Performance Issues Both services are slow, why? 27 mei 2018 Performance Issues Introduction – Case 1 – Case 2 – Discussion – Conclusion – Recommendations Both services are slow, why? Tide forecasting service: Reading 400MB dataset Interpolating the whole dataset instead of a few points IDPT Wrapper: Limited performance of the original one Saving, loading, reading files via FTP (slow) Only 4 points required for interpolation ftp is the best use for a web service like idpt, it is too slow (write/read in about 3 seconds), better have a single server communication through internal memory

27 mei 2018 5. Conclusion Introduction – Case 1 – Case 2 – Discussion – Conclusion – Recommendations Can complex processes that feed coastal/marine models be exposed via a Web Processing Service? Yes!

27 mei 2018 5. Conclusion Introduction – Case 1 – Case 2 – Discussion – Conclusion – Recommendations Can a Web Processing Service be used as a wrapper service for non domain-specific processing services? In case of IDPT, yes! For every existing service this can be different and compeletly depends on the working of the service.

27 mei 2018 5. Conclusion Introduction – Case 1 – Case 2 – Discussion – Conclusion – Recommendations Which adaptations could be made to the WPS standard to improve the possibilities for application of the standard? Based on the practical experience gathered during the thesis work: - Improve status support, continuous communication - Implementation of a ‘stop’ request - More possibilities for dynamic processes/interfaces

Overall WPS is very broad applicable 27 mei 2018 Overall Introduction – Case 1 – Case 2 – Discussion – Conclusion – Recommendations WPS is very broad applicable (single) WPS processes are somewhat static WPS will not speed up existing processes WPS allows great interoperability Static processes can be made more dynamic by coupling different processes and application of asynchronous processing where output files can be read/used before the process is finished completely

6. Recommendations Tide forecasting service: Performance improvement 27 mei 2018 6. Recommendations Introduction – Case 1 – Case 2 – Discussion – Conclusion – Recommendations Tide forecasting service: Performance improvement Accuracy improvement Path/vector input in GML IDPT Wrapper: More input parameters Graphical interface Status support

Fast Response! Tide forecasting service: Improved performance 27 mei 2018 Fast Response! Introduction –Case 1 – Case 2 – Discussion – Conclusion – Recommendations Tide forecasting service: Improved performance Improved input options IDPT Wrapper: Fixed status support PyWPS: Number of developers increased Tide forecast service replies within a second - Offers more input types like multipoint vectors, long paths

27 mei 2018 Questions?

Thank you for your attention Contact: joostboerboom@gmail.com 06 41921002 http://nl.linkedin.com/in/joostboerboom Drinks: Tomorrow, 20:00u Sportparklaan 16, Heemstede