Capacity Building for the Agriculture Service eStation Presentation SADC THEMA Agriculture Service Capacity Building for the Agriculture Service eStation Presentation By: Thembani Moitlhobogi System Engineer AMESD SADC THEMA

SADC Regional Training, Pretoria 2011 A Brief History History Beginnings at European Commission's Joint Research Centre (JRC, Ispra, Italy)‏ - First version developed at JRC: central repository for satellite data (low resolution) on Africa, and their time series processing. - JRC setup an African-Caribbean-Pacific Observatory. One mandate is to write environmental reports → A component is added to the eStation to allow a group of expert to exploit eStation data and collaborate in the same report (wiki-like)‏ eStation stands for Environmental Station SADC Regional Training, Pretoria 2011

SADC Regional Training, Pretoria 2011 A Brief History → a strong involvement of African partners is desired → the system was designed to support user's contribution: regional configuration User's are welcome to share their experience and knowledge: community building: http://eStation.amesd.org SADC Regional Training, Pretoria 2011

Architecture overview 192.168.0.2 Data synchronisation 192.168.0.5 192.168.0.6 The eStation is made of two PCs: PS stores data and makes processings on those data. EMMA is a web server hosting visualisation applications and collaborative report assistant application. EMMA has also a synchronised version of PS data and database Data storage & DB Redundant data & DB SADC Regional Training,Pretoria 2011

SADC Regional Training, Pretoria 2011 Users' involvement Users can design their own processing chain and put them into production, distribute their own processing Software component modularity allows building users' specific processing - use provided software components to build up your own products - get data from various sources (Eumetcast, ftp, web services), including systems hosted in our premises (ex. Smb, ntfs...)‏ The eStation is made of software “bricks” allowing you to build new processing chains. The idea is to allow expanding the capacity of the system to fulfill each institution needs. Another important idea is to build a community where experience, production, tips and tricks and data can be exchanged SADC Regional Training, Pretoria 2011

AMESD station components An AMESD station is made of An EumetCast receiving station (tellicast): data acquisition via EumetCast → see MFI training eStation's Processing Station (PS)‏ Export data format to geotif to a self organized archive (acquired data)‏ Automatically perform processing (generated data). User's can add new processing on purpose Grant access to the data (acquired and generated) through LAN eStation's Environmental Mapping and Monitoring for Africa (EMMA): optimize team work for writing environmental report Display data as maps with team's predefined options Export/import maps, import other illustration in user's portfolio Web-browser based application to design and write reports SADC Regional Training, Pretoria 2011

Technical specification of the eStation eStation on 2 identical PCs (PS and EMMA), each with DELL MT Optiplex 780 Intel dual core, 2.93 GHz RAM: 4GB DDR3 HD1: 1TB (data)‏ HD2: 160Gb (system)‏ Display: DELL E2210 22'' widescreen OS: Linux Ubuntu 8.04 Ethernet switch Netgear FS 608 (8 ports)‏ UPS: DELL UPS 1000W SADC Regional Training, Pretoria 2011

Architecture overview Basic ideas All open source, only freeware The eStation is a system not a software: it does not replace GIS desktop software Adapted to AMESD needs: recognize data formats (DevCoCast, MSG), get data from EumetCast, automatic installation for AMESD PCs Users can apply customisation: region of interest and write their own specific processing chains Update mechanism (dissemination by EumetCast)‏ Most daily maintenance can be done through a graphical interface Mostly scripts based: all codes are readable (bash, python, xml)‏ SADC Regional Training, Pretoria 2011

SADC Regional Training, Pretoria 2011 PS System View SADC Regional Training, Pretoria 2011

SADC Regional Training, Pretoria 2011 EMMA System View EMMA EMMA: reads data stores on PS, display them as maps, ready to insert into reporter Reporter: web browser based tool, to design and write reports. Multiple simultaneous authors (wiki like), maintain many styles of reports, archive issues. reporter SADC Regional Training, Pretoria 2011

PS Automation EumetCast, FTP, web services New data ? No, scan again yes scheduler Get data Get layer eStation input directory Data collected? derived layer triggers Import to geotiff, rename, organize Processings scripts ingest layer Data storage Compute new data SADC Regional Training, Pretoria 2011

SADC Regional Training, Pretoria 2011 PS Automation Data flow example: SPOT/VEGETATION NDVI Africa NDVI is made every 10 days at VITO/Belgium Send by EumetCast (vgt4africa_S10_NDVI.zip)‏ PS get it, extract the zip file, transform hdf files into geotiff with internal compression, rename files 20100301_NDVI.tif and 20100301_SM.tif Update statistics about those files: annual average, dekad average, min, max, std, and others All data (acquired and computed) stored on the eStation and shared (SMB and EMMA)‏ SADC Regional Training, Pretoria 2011

SADC Regional Training, Pretoria 2011 PS Data \\192.168.0.4\data\processing SADC Regional Training, Pretoria 2011

SADC Regional Training, Pretoria 2011 PS Data SADC THEMA data examples: SADC_THEMA_NDVI_201001010000_SADC_V1.bz2 Contents:- SADC_THEMA_NDVI_201001010000_SADC_V1.tif SADC_THEMA_NDVI_20100101_meta.txt THEMA_NDVI_201001010000_SADC_quicklook.png SADC Regional Training, Pretoria 2011

PS Install Chain Config Chain configuration to be sent to administrators by email/ftp Configuration package to be installed on PS to enable the eStation to receive get SADC_THEMA* products from the acquisition machine and also to visualize on EMMA. Format: SADC_THEMA_NDVIdiffavg.deb SADC Regional Training, Pretoria 2011

SADC Regional Training,Pretoria 2011 EMMA Visualisation SADC Regional Training,Pretoria 2011