1. WBL/PA - 1 NHSC OT1 Data Processing Workshop for New Users THE NASA HERSCHEL SCIENCE CENTER OVERVIEW William B. Latter NHSC Deputy Director

2. WBL/PA - 2 NHSC OT1 Data Processing Workshop for New Users NASA Herschel Science Center: NASA Herschel Science Center: The US Community’s Herschel Support Center

3. - page 3 February 2011 PACS NASA Herschel Science Center Objective and Scope of the NASA Herschel Science Center The NHSC has been established at the Infrared Processing and Analysis Center (IPAC) at the California Institute of Technology under support from NASA. NHSC began operations in 2001 and will continue to operate throughout the lifetime of the Herschel Mission, including the post- cryogen archival phase. The primary mission of the NHSC is to support the US-based astronomical community in exploiting the Herschel science opportunity, at a level equivalent to that provided to the European Community by ESA’s Herschel Science Centre.

4. - page 4 February 2011 PACS NASA Herschel Science Center NHSC Charter Ensure the necessary resources and tools are available to the US scientific community to take advantage of the scientific capabilities of the observatory in a timely manner. Ensure availability of Herschel science data for US archival research on timescales equivalent to ESA. Act as an interface between the ESA Herschel Project and the US-based scientific user community. Advocate US community needs with the HSC and ESA Project. Provide the US astronomical community with technical support throughout all phases of the Herschel mission, from pre-launch through to the transition to archival phase. Manage the US data analysis funding for the US user community. In Summary: Provide the US-based astronomical community with support in its use of Herschel equivalent to that provided by the Herschel Science Centre in their support of the European Community.

5. - page 5 February 2011 PACS NASA Herschel Science Center To fulfill its charter, the NHSC has established strong working ties with the ESA Herschel Science Centre and the Instrument Control Centres. The NHSC is participating in many activities to enhance our US community support, including: –Providing critical support for HSpot. –Participate directly in Instrument Control Centre activities. –Participate in Data Processing system development. NHSC has knowledge in all areas of Herschel science operations and data processing, stays up-to-date with frequent visits (both directions) NHSC and the Herschel Project

6. - page 6 February 2011 PACS NASA Herschel Science Center MAXIMIZING US COMMUNITY SUPPORT Working with the Instrument Control Centres and HSC – Improve our ability to provide knowledge to US-based observers. Acquire an intimate knowledge of calibration and data processing. Provide tools and support for observers that will maximize return from the observatory. NHSC has knowledge in all areas of Herschel science operations and data processing. Co-location The NHSC has been posting resident liaisons to the three Instrument Control Centres during initial checkout to the start of routine operations. Constant contact will be maintained throughout mission. The NHSC has posted a resident liaison to the HSC at ESAC in Spain for the duration of the cryogenic mission.

7. - page 7 February 2011 PACS NASA Herschel Science Center Proposal grades Herschel Ground Segment ICC MOC HSC S/C ICC @MOC TC frames TC packets ICC@ICC HOTA C General Community TM frames Instruments TM packets Hersch elScien ce Team Guidance, priorities NHSC NHSC/HSC Liaison Scientist provides a direct functional link here. Many NHSC instrument group members work as part of ICCs in a functional way

8. - page 8 February 2011 PACS NASA Herschel Science Center Herschel Observer Support

9. - page 9 February 2011 PACS NASA Herschel Science Center ESA HERSCHEL OBSERVER SUPPORT The ESA Herschel Science Centre (HSC) manages the Herschel Science Mission Proposals will be submitted to the HSC with the Herschel Proposal Submission Tools. Official Herschel Documentation is provided by the HSC. All proposals will be evaluated by the Herschel Observatory Time Allocation Committee (HOTAC). Scheduling and execution of all observations will be done via the HSC. Initial data processing and data archiving will be done by the HSC and observer data will be released by the Herschel Archive administered by the HSC. HSC webpages - herschel.esac.esa.int

10. - page 10 February 2011 PACS NASA Herschel Science Center US-based Proposers/Observers will be supported by the NASA Herschel Science Center (NHSC). Contact us first – We work closely with HSC NHSC OBSERVER SUPPORT

11. - page 11 February 2011 PACS NASA Herschel Science Center NHSC SUPPORT TASKS

12. - page 12 February 2011 PACS NASA Herschel Science Center NHSC Support Tasks ✔ Provide resources and tools to help US Community exploit Herschel science through scientists and software engineers embedded in project at all levels (resources include workshops, web-tutorials and on-line videos etc). ✔ Administration of data analysis grants for US-based observers. ✔ Preparation and support of Herschel-specific data reduction software. ✔ Observation planning support, including workshops and AOR revision and including support in the submission of observing proposals ✔ Advocate US user needs within HSC and ESA, including membership of Data Processes sing Advocacy Group (DPUG), and hosting of NHSC User Group ✔ Program tracking and data retrieval ✔ Facilitate US-based Community Engagement including Science with Herschel ✔ Provide facilities for special needs, such as large-scale computing

13. - page 13 February 2011 PACS NASA Herschel Science Center Recent Support Activities-Community Support Workshops - Post-launch NHSC SDP DP Workshop (Jan 2010) - Post-launch NHSC Data School (June 23-30 2010) - OT1 Proposal Planning Workshop (June 3-4 2010) Scientist Visits - Almost continuous flow since Nov 2009 OT Liaison Activation - Most teams have found NHSC liaisons useful OT Wiki sites for teams and mail lists AAS Meetings: Booth and Special Session Support - Washington DC and Miami FL 2010 + Seattle 2011 Helpdesk Support: Considerable Traffic (Statistics later) Phone/WebEx support: of remote individuals with problems OTKP Scheduling AOR Advocacy

14. - page 14 February 2011 PACS NASA Herschel Science Center Our first NHSC Data Processing Workshop in 2009 Visitors to NHSC getting personal assitance with early data processing

15. - page 15 February 2011 PACS NASA Herschel Science Center Data Processing visits have been extremely popular but probably don’t scale to large numbers of users We are exploring a number of new ways of reaching large numbers of users. Some visits may still be offered.

16. - page 16 February 2011 PACS NASA Herschel Science Center The NASA Administrator Charles Bolden visit the NHSC Booth at the Washington DC AAS meeting and hear about new results! Nanyao Lu and Philip Appleton meet Charles Bolden Jan 2010

17. - page 17 February 2011 PACS NASA Herschel Science Center RESOURCES AND TOOLS of the NHSC

18. - page 18 February 2011 PACS NASA Herschel Science Center Helpdesk: Main initial contact point with NHSC (heavily used) NHSC Science and Developer Staff are primary resource - well embedded/integrated with Project at many levels Workshops - Planning Workshops - DP Workshops/Data Schools - Feedback very positive - Mini workshops on specialist topics (i.e. Mapping or Post-processing tools) Science Wiki Pages provide direct support to quick links and hot topics (http://nhscsci.ipac.caltech.edu or http://herschel.caltech.edu)http://nhscsci.ipac.caltech.eduhttp://herschel.caltech.edu New Online Web-tutorials and Video tutorials on specific topics - Pipeline walk-throughs (PACS) and online Videos are being developed (See some examples at this workshop) Access to Remote Large Memory High-end Computing (later in talk and by B. Ali in more detail NHSC Liaisons : (For OTKP had one NHSC person per team—discuss in feedback session for OT1 groups) Existing Resources and Tools Existing Resources and Tools

19. - page 19 February 2011 PACS NASA Herschel Science Center NHSC Helpdesk http://nhsc.ipac.caltech.edu/helpdesk Has been fully operational since Jan 2007 NHSC Response: -Provide initial OST acknowledgement -Ticket forwarded immediately to expert who responds directly within 2 business days -In rare case that we are unable to assist, we open and follow-up with European help desk ticket. Average response time < 1 business day (often ~ 2-5 hrs) Helpdesk is used for email communications to user groups OST has access to the European helpdesk which uses the same system. - We monitor their tickets and they monitor ours. Spitzer Science Center has adopted the same helpdesk system

20. - page 20 February 2011 PACS NASA Herschel Science Center Post-launch Data processing Workshops and Data Schools Post-launch Data processing Workshops and Data Schools 1 ) Aimed primarily at active users of Herschel 2) In 2010 we had both an introductory workshop (January 2010) and and more advanced Data School geared towards reduction of user data in the workshop. We plan a similar Data School in the Summer of 2011. 3) The Data School featured access to a high speed LAN which connected to several large "compute” machines with large memory. This enabled large maps to be processed on-site. These facilities will soon be available for external users too. 4) Workshops provide users with the ability to work closely with US and ESA experts to learn the basic skills necessary to reduce the Herschel data, and to develop ties with the NHSC so that you can gain help quickly when you data is available for processing.

21. - page 21 February 2011 PACS NASA Herschel Science Center RECENT SLIDE OF WORKSHOP Our First Post-launch Data Processing Workshop January 2010

22. - page 22 February 2011 PACS NASA Herschel Science Center NHSC Communications Channels with Community Science Wikipages and including announcements and Hot links to important time-critical information and calibration hints http://nhscsci.ipac.caltech.edu Helpdesk + Knowledgebase. Telecons and PI Phone Calls (personal contact seems appreciated). Rapid response advocacy for programs like TOO which require quick turnaround. Semi-Regular eNewsletter -23 eNewsletters have been sent out since Sept 2006. NHSC Instrument-based Liaisons Conference Booths/Fliers/AAS Splinters -We have been a constant presence at AAS meetings for several years -Booth includes a 1/4 scale model of Herschel. - We often offer special “Splinter” sessions (like the DP Splinter in Seattle) as well as organize Herschel-dedicated Poster and Oral talks.

23. - page 23 February 2011 PACS NASA Herschel Science Center Web-pages Public and science wikis http://herschel.caltech.edu http://nhscsci.ipac.caltech.edu

24. - page 24 February 2011 PACS NASA Herschel Science Center NHSC Remote Computing Resources for US-based Observers

25. - page 25 February 2011 PACS NASA Herschel Science Center Support Use Cases Bottom line: Observers need this for one of two reasons: * They don't want to invest in HIPE installation, configuration and maintenance most likely because they only have a small program and not enough resources or time. * They simply do not have enough computing power (e.g. a 64 GB machine) Other benefits include: – Provide local user support at the NHSC Visiting scientists work directly with NHSC staff members. Pipeline and mapping jobs that require large amounts of memory and processing speed. Provide resources for workshops. Include other astronomical software along with Herschel: SAOImage DS9, XPA, Fv, Starlink (Gaia, SPLAT,…), SExtractor, IRAF, and PyRAF. Be able to use different versions of Herschel software.

26. - page 26 February 2011 PACS NASA Herschel Science Center Example 1 HIPE is a new system and users without a significant stake in Herschel time will want an up to date and useable configuration they can use to reduce their data. A user has 20 hours of PACS scan map observations on a set of regions on the sky that (s)he wishes to fully process to a scientifically valid final mosaic suitable for science analysis. The user does not have the necessary computing infra-structure to carry out the processing and/or does not wish to invest the time and resources to establish said infra- structure on her/his local institute. The user is additionally unable or unwilling to travel to NHSC to work internally with the local compute machines. The remote access facility allows such users to work on a preconfigured machine to efficiently reduce their data with NHSC support available during the remote sessions via the usual channels.

27. - page 27 February 2011 PACS NASA Herschel Science Center Example 2 Large compute machines. PACS generates data at ~1GB/hour. Processing requires a multiplier of ~8x on this; thus, even short PACS observations quickly move in the realm of requiring large compute facilities. For users who are not interested in upgrading hardware to 64-100GB RAM, the NHSC remote compute facilities are a way to access a fully configured Herschel data reduction facility with enough RAM to process even large PACS maps.

28. - page 28 February 2011 PACS NASA Herschel Science Center Remote Support Architecture Fundamentals

29. - page 29 February 2011 PACS NASA Herschel Science Center Fundamentals (cont.) Current estimate is up to 25 virtual machines can be available at a time (pending load testing). Secure and low overhead. Virtual machine is simply destroyed after work is completed. Plenty of memory, plenty of power. Both local and remote support available.

30. - page 30 February 2011 PACS NASA Herschel Science Center Status The system is undergoing tests now. We expect the system to be “live” in the next few weeks. PRELIMINARY Web Page Description: https://nhscsci.ipac.caltech.edu/sc/index.php/CompSupport/ExternalUsers

31. - page 31 February 2011 PACS NASA Herschel Science Center On-line Tutorials and Videos describing key topics including walk-through of pipelines (especially for large-use instruments like PACS—see examples in this workshop) Use of interactive conferencing software sessions with remote sites Facilitate development of healthy user interest groups (inc. possibly building on the ideas of the DPIGS in Europe?) Develop or facilitate creation of "Forums" to share solutions? Some Ideas we are exploring for reaching a larger numbers of users: Please think about these and bring up in the feedback sessions Some Ideas we are exploring for reaching a larger numbers of users: Please think about these and bring up in the feedback sessions

32. - page 32 February 2011 PACS NASA Herschel Science Center There are various channels: NHSC and HSC Helpdesks (US users should work with the NHSC Helpdesk) NHSC User Panel (NUP) : Current Chair : Margaret Meixner STScI Committee make-up and recent reports—see https://nhscsci.ipac.caltech.edu/sc/index.php/Main/NUP Herschel User Group (HUG): European User Group led by Rob Kennicutt http://herschel.esac.esa.int/HUG.shtml DPUG DATA Processing User Group: Deals with DP Issues for whole Herschel project. Although containing mainly Herschel experts and developers, the membership is expanding to include users. DPIGL: Data Processing Interest Lists : Led by HSC but so far have not developed very far: see http://herschel.esac.esa.int/Data_Processing.shtml bottom of pagehttp://herschel.esac.esa.int/Data_Processing.shtml HOW TO GET INVOLVED IN HERSCHEL ISSUES?

33. - page 33 February 2011 PACS NASA Herschel Science Center You—as users can help us build synergy with other projects GBT and eVLA Deep Radio Astronomy ALMA AND CCAT sub-mm astronomy from ground Spitzer Warm Mission Deep IR Imaging from Space Spitzer Warm Mission Deep IR Imaging from Space JWST Mid-IR from Space SOFIA mid/Far IR from aircraft SOFIA mid/Far IR from aircraft HERSCHEL SPICA/BLISS Far-IR from Space SPICA/BLISS Far-IR from Space WISE IR Surveys HERSCHEL

34. - page 34 February 2011 PACS NASA Herschel Science Center NEWS: The Upcoming AO for Open Time 2 (AO2) AO2 will likely be Issued: ~ March-May 2011 (TBD) AO1 DEADLINE: likely ~June-July 2011 (TBD) OT2 will likely will have less hours available than in OT1 (OT1 involved 6600 hrs).

35. - page 35 February 2011 PACS NASA Herschel Science Center THE END The Data is Flowing!