Scenario use cases Szymon Mueller PSNC

Agenda 1.General description of experiment use case. 2.Detailed description of use cases: 1.Preparation for observation (settings and scheduling):  Principal Investigator  Telescope Operator  Correlator Operator 2.Running the observation:  Principal Investigator  Telescope Operator  Correlator Operator 3.Post-observation 3.Open questions. WP7 meeting– Poznan –

General description Each long-term observation requires a set of specific steps, by each involved actors: ▫ Principal Investigator (PI) – writes observation proposal, prepares schedule, sends the schedule to CO and TOs, monitors execution of the observation, oversees whole experiment ▫ Telescope Operator (TO) – receives schedule of the observation, is responsible for configuring his antenna, communicates with PI during experiment, monitors if everything is ok with the telescope ▫ Correlator Operator (CO)– receives schedule of the correlation, is responsible for configuring correlation software, oversees the correlation process WP7 meeting– Poznan –

General use case - preparation WP7 meeting– Poznan – Principal Investigator Logs in Writes a proposal Creates VEX schedule Kiwi Portal Schedules observation NorthStar Proposal submission tool Vlbeer FTP

General use case - preparation WP7 meeting– Poznan – Telescope Operator Logs in Checks his schedules Converts the VEX file Kiwi Portal Prepares telescope settings Vlbeer FTP

General use case - preparation WP7 meeting– Poznan – Correlator Operator Logs in Checks his schedules Chooses correlator Kiwi Portal Sets correlation settings Vlbeer FTP

WP7 meeting– Poznan – PITOCO Starts Mark5/FSStarts correlation Requests fringe plots Adjusts telescope settings Requests telescope adjustment Monitoring Error on telescope side (e.g. antenna parked) Gathered data is not valid Mark incoming data as invalid

Preparation for observation- PI Logs into the Kiwi portal, using his or her credentials. To write a new proposal: user chooses link in his portal and is forwarded to NorthStar Proposal Submission Tool. (no support from Kiwi portal for writing proposals – task forwarded to external service). After receiving confirmation from Proposal Committee (PC) the portal, with interaction from the user, creates schedule using NRAO SCHED program, and prepares a VEX file. VEX file is then uploaded by the portal to vlbeer FTP for further revision by scientists. (can the SCHED tool be used to automatically generate VEX from PC response?) User can access vlbeer server using portal, and download (“import”) schedules stored there. User can list all of his created schedules in a special tab „My Schedules” and see statuses of the corresponding experiments (created, accepted, scheduled, running, finished). Kiwi portal allows visualization of the VEX file (showing each location which is involved in the experiment). WP7 meeting– Poznan –

Preparation for observation - PI When scientists responsible for review will accept VEX schedule (or upload modified VEX file) PI will receive a notification in the portal system, that one of his schedules is ready to be scheduled. PI then can start scheduling process: ▫ New experiment is created in the portal ▫ Experiment is scheduled to start just before date specified in schedule file, ie. 30 minutes – 1 hour (is this time dependant on the telescopes involved?) ▫ All involved Telescope Operators (TO) and Correlator Operators (CO) are informed about the schedule via and notifications in the portal system PI can monitor scheduled experiment at any time, see which telescopes are ready to conduct the observation etc. WP7 meeting– Poznan –

Preparation for observation - TO Logs into the portal, using his or her credentials. Checks if there are any experiments scheduled for his telescope (this can be done by visiting “My Schedules”). If there is a new schedule, user requests to open it ▫ Schedule is downloaded by the portal from vlbeer server ▫ Drudg is used to obtain Procedure and Snap file for this station. Portal opens created files and requests user to verify settings. If possible validation can be done automatically (via settings specific for this particular telescope). After confirming the settings, telescope task is marked as ready. Main experiment (PI) is informed that this telescope is ready to perform observation. WP7 meeting– Poznan –

Preparation for observation - CO Logs into the portal, using his or her credentials. If there is incoming experiment scheduled: ▫ VEX file is downloaded ▫ Correlator task is marked as ready to be executed. Main experiment (PI) is informed about it. Chooses which software correlator will be used (SFXC/DiFX) Pre-correlation operations (calculate clock errors, etc) are done automatically by the portal. User can select fringe finder and/or time interval to use. Correlation VEX file is created. (can this be done long before correlator should start or is it a part of a calibration at the start of the correlation?) CO must also set correlation parameters, according to the chosen correlator Portal helps the user with generating machinefile and correlator control file. With all files prepared, correlator task is marked as ready for execution. WP7 meeting– Poznan –

Running the experiment After preparation phase is done whole system awaits just before experiment is about to start. Portal is scheduled to send notification to involved actors (PI, TO, CO) and their tasks. After experiment starts, each of the involved user is able to monitor and interact with their respective tasks, as well as communicate between each other about the experiment which is taking place. WP7 meeting– Poznan –

Running the experiment – PI Before experiment begins PI can see status of each telescope task. Telescope tasks which are prepared are marked as READY, if FS and Mark5 unit for a telescope is already scheduled, task status is SCHEDULED. If telescope operation has started, its task is RUNNING. Correlator task status is also visible to PI – he can monitor how many correlation nodes are spawned, correlation progress, other measurable correlator parameters (Will correlator software provide such information?) Can request to observe fringe plots: ▫ Select the baselines for which to create fringe plots ▫ Fringe plots are displayed (max delay – 2 mins) ▫ Allows to communicate with TO and fixing the problem during runtime (Are fringe plots created by the correlator on the cluster?) WP7 meeting– Poznan –

Running the experiment – PI During experiment can receive notifications: ▫ From TO - if there is an error on telescope side, and data send might not be valid ▫ From CO - if correlation process is encountering problems Can also send notifications, for example that fringe plot suggest there should be changes made to telescope settings (send to TO). WP7 meeting– Poznan –

Running the experiment – TO ~ 1hr before experiment Mark5A/jive5a started on Mark5 unit and schedule started on FS (Will it be available through station control software? Or should TO start it manually?). Main experiment is informed that telescope operations has been scheduled. TO might request portal to see fringe plots, to ensure that collected data is valid, and all settings are correct (Same question as for PI – where are fringe plots created in this case? Are those the same fringe plots as for PI?) During experiment many measurable parameters must be taken (temperature of the rack, weather conditions, etc.). Portal will contact external sensors to gather needed data if possible, otherwise user will be required to enter them manually. All gathered measures will be stored. (Will station control software/WP5 provide such API to access sensors) WP7 meeting– Poznan –

Running the experiment - CO Allows the CO to monitor the status of the correlation (depends on the software used for correlation). In portal CO will be able to communicate with underlying correlation task and request detailed info about its status. If correlation software will allow interaction (manual load balancing, changing settings of nodes used etc.) during running correlation, these functionalities will be available in the portal. Receives information from main experiment (PI) that some data might be invalid (i.e. strong wind). WP7 meeting– Poznan –

Post-observation Main experiment (PI) operation informs TO and CO tasks that correlation should finish. TO stop all locally started applications for experiment – the rest is automatically ended by TO task. Correlator output is transformed: ▫ Images are generated using various tools (j2ms etc.) ▫ Verification of generated images ▫ Upload to EVN data archive Correlator output is shipped to selected correlation center (JIVE/ICRAR/?). After transfer is complete, CO is informed about it. All log data is stored in associated repository. PI can request to recheck status and logs of the concluded experiments, as well as download archived data. WP7 meeting– Poznan –

Open questions 1 What is the CO role during correlation process – what parameters of the correlation process he can change? Will correlator provide necessary API for that? What information about the observation should be kept after observation has finished? WP7 meeting– Poznan –