1. ISTA 520/420 Midterm AJ Jicha Ryan Jicha Ian Kaufer AJ Stangl Roy Zacharias “Astrocats” - Group 3

2. Agenda AstroCats Staff Project Summary Project Goals Results Data Processing Runtime Obstacles Future Goals

3. AstroCats Staff Project ManagersIan Kaufer, Roy Zacharias Chief ArchitectAJ Jicha System AdminsIan Kaufer, AJ Jicha Development/Test ManagerAJ Stangl Release ManagerAJ Stangl Chief ProgrammerRyan Jicha Technical WriterIan Kaufer, Roy Zacharias

4. Project Summary Carl Herzenrother - requesting images processed using HPC for efficiency Input: FITS files Process: Python and Bash scripts Generate modified FITS file Generate Astrometrica config file Linux, HPC, makeflow, Astrometrica, Astrometry.net Output: Modified FITS image file Astrometrica.cfg file for associated FITS image

5. Project Goals Sort files with varying extensions (i.e. fit, fitz,.zip) into a common directory for processing Process each.fits file with solve field in a parallelized manner on HPC Modify each new.fits file with calculated parameters from solve field (Field center values) Create Astrometrica configuration file with modified field rotation (PA) and calculated focal length to match each new.fits file Streamline process for easy replication by client on future datasets

6. Project Work Flow

7. Results Scripts to process images and metadata functional.makeflow script to process FITS images functional Leveraging HPC/HTC and workers to process many tasks functional Output files located in the following directories: /gsfs1/xdisk/ikaufer/midterm/midterm-Carl-Hergenrother/output/ Generate new FITS images functional Generate.cfg files for FITS images functional Generate list of unsolved/error files functional

8. Data Processing Runtime Benchmark: 40 workers, 222 jobs: 2 hours 51 minutes Desired time: 1,500 workers, 9,570 jobs: 2 hours 29 minutes Actual time: 524 workers, 9,570 jobs: X hours Y minutes

9. Obstacles Issues running makeflow.fits metadata in a single huge line Three of the five group members did not have any standard HPC hours Lack of uniform file extension naming scheme Random.jpeg and.mp4 files Spaces within file names

10. Example Obstacle Findings

11. Future Goals Implement procedure to handle multiple pixel scale values Unify all parts of scripts into a single script for ease of use

12. Questions?