Solar Orbiter SOWG 5| Richard Carr| Low Latency Pipeline Hosting and Delivery Architecture | Slide 1 ESA UNCLASSIFIED – For Official Use Solar Orbiter SOWG 5 Low Latency Pipeline Hosting and Delivery Architecture Richard Carr, ESAC, July 2014

Solar Orbiter SOWG 5| Richard Carr| Low Latency Pipeline Hosting and Delivery Architecture | Slide 2 ESA UNCLASSIFIED – For Official Use Objective and Format Recap reasoning and proposal from guidelines document Address any questions from instrument teams Questions from SOC

Solar Orbiter SOWG 5| Richard Carr| Low Latency Pipeline Hosting and Delivery Architecture | Slide 3 ESA UNCLASSIFIED – For Official Use Background Reasoning Advantages of Virtual Machines 1.Lifetime of platform. a.Not a short mission. 2.Greatly simplifies deployment. a.At SOC. b.But also multiple copies in development/testing. 3.Symmetrical mechanism across instruments. 4.Cloneable, notably for diagnostics.

Solar Orbiter SOWG 5| Richard Carr| Low Latency Pipeline Hosting and Delivery Architecture | Slide 4 ESA UNCLASSIFIED – For Official Use Objectives Essentially attempt to define 3 interfaces 1.How the LL pipelines will retrieve the necessary input data. a.Including auxiliary inputs. 2.How, where and in what form the LL pipelines will store the generated products. 3.How, and in what form, each instrument team will deliver the pipeline software to the SOC.

Solar Orbiter SOWG 5| Richard Carr| Low Latency Pipeline Hosting and Delivery Architecture | Slide 5 ESA UNCLASSIFIED – For Official Use Objectives Further interfaces… 1.External control of pipelines. a.How to ‘kick’ etc. 2.Diagnostic. a.Logging b.Potential remote access to clone.

Solar Orbiter SOWG 5| Richard Carr| Low Latency Pipeline Hosting and Delivery Architecture | Slide 6 ESA UNCLASSIFIED – For Official Use Specific Objectives What we (at least the SOC) want from an architecture 1.Uniformity across instruments. 2.Data fetching external to pipeline. a.Partly derived from 1. b.Allows simpler control of pipelines, and coordinated SOC/MOC data interactions. 3.Appropriate products. (See further slide). 4.Allow post-processing. a.Again, as uniformly as possible. 5.Testability.

Solar Orbiter SOWG 5| Richard Carr| Low Latency Pipeline Hosting and Delivery Architecture | Slide 7 ESA UNCLASSIFIED – For Official Use Specific Objectives Appropriate products 1.Allow crude checks of instrument performance and science data quality. 2.Allow instruments implementing selective data downlink to choose the telemetry to retrieve. 3.Allow the improvement of pointing profile and/or retargeting when tracking solar features.

Solar Orbiter SOWG 5| Richard Carr| Low Latency Pipeline Hosting and Delivery Architecture | Slide 8 ESA UNCLASSIFIED – For Official Use Objective/Constraint Relative Isolation of Pipelines Constraint on SOC due to computer infrastructure security concerns. But also meets various objectives Architectural simplicity. Configuration control. Understood that diagnostic mechanisms needed.

Solar Orbiter SOWG 5| Richard Carr| Low Latency Pipeline Hosting and Delivery Architecture | Slide 9 ESA UNCLASSIFIED – For Official Use Proposal

Solar Orbiter SOWG 5| Richard Carr| Low Latency Pipeline Hosting and Delivery Architecture | Slide 10 ESA UNCLASSIFIED – For Official Use 1.Virtual Appliances. a.x86. 2.OVF delivery. 3.OS supported by VMWare. 4.Licencing a.In principle responsibility of instruments. b.Possible exceptions for practical reasons. 5.I/O Exclusively to disk. a.NFS baseline with other support if essential (CIFS/iSCSI) Proposal

Solar Orbiter SOWG 5| Richard Carr| Low Latency Pipeline Hosting and Delivery Architecture | Slide 11 ESA UNCLASSIFIED – For Official Use 6.Processing Batch = Pass Volume. a.Preceding pass data also available. 7.Statelessness. 8.Request mechanism. 9.Output. Proposal

Solar Orbiter SOWG 5| Richard Carr| Low Latency Pipeline Hosting and Delivery Architecture | Slide 12 ESA UNCLASSIFIED – For Official Use Proposal

Solar Orbiter SOWG 5| Richard Carr| Low Latency Pipeline Hosting and Delivery Architecture | Slide 13 ESA UNCLASSIFIED – For Official Use Proposal

Solar Orbiter SOWG 5| Richard Carr| Low Latency Pipeline Hosting and Delivery Architecture | Slide 14 ESA UNCLASSIFIED – For Official Use Proposal

Solar Orbiter SOWG 5| Richard Carr| Low Latency Pipeline Hosting and Delivery Architecture | Slide 15 ESA UNCLASSIFIED – For Official Use SOC Deployment Context

Solar Orbiter SOWG 5| Richard Carr| Low Latency Pipeline Hosting and Delivery Architecture | Slide 16 ESA UNCLASSIFIED – For Official Use Testing Not Yet Addressed

Solar Orbiter SOWG 5| Richard Carr| Low Latency Pipeline Hosting and Delivery Architecture | Slide 17 ESA UNCLASSIFIED – For Official Use Anything not yet answered? ? Instrument Team Questions

Solar Orbiter SOWG 5| Richard Carr| Low Latency Pipeline Hosting and Delivery Architecture | Slide 18 ESA UNCLASSIFIED – For Official Use 1.How similar can LL Pipeline be to science pipeline? Just cut- down version? 2.Which OS do you envision using? 3.How much software are you inheriting from previous instruments? 4.Which virtualisation infrastructure are you accustomed to using/is in use at your institutions? SOC Questions