1. The Thinking Telescopes Project, RAPTOR, and the TALONS Communication System

2. Machine Learning GENIE, ML Classifiers, Anomaly Detection Context Knowledge Record of Sky variability (Virtual Observatories), Massive Distributed Disk Array Robotic Hardware Wide-Field Sky Monitoring Rapid Response Telescopes, Rapid Response Telescopes, Real Time Pipeline Thinking Telescopes An Engine for Discovery in the Time Domain Goal is to Integrate Three Components

3. System Adaptability: Querying the Sky Traditional Approach Hard Wired to find specific artifacts and phenomena For example---not in previous frame, not in sky catalog, and no parallax For example---not in previous frame, not in sky catalog, and no parallax Thinking Telescope Monitoring of persistent sources for important changes in real time Adaptive processing Machine learning Anomaly detection and automated classification find more like this

4. Machine Learning Automated identification of artifacts and transients in direct and difference images. Automated identification of artifacts and transients in direct and difference images. Automated classification of celestial objects based on temporal and spectral properties. Automated classification of celestial objects based on temporal and spectral properties. Real time recognition of important deviations from normal behavior for persistent sources. Real time recognition of important deviations from normal behavior for persistent sources.

6. Memory and Context http://skydot.lanl.gov

7. Raptor: Sky Monitoring with Both Eyes Open Wide-field imaging system monitors ~1300 square-deg with resolution ~35 arcsec and limiting magnitude of R~13 th in 60 seconds. ( like the rod cells of the retina ) Wide-field imaging system monitors ~1300 square-deg with resolution ~35 arcsec and limiting magnitude of R~13 th in 60 seconds. ( like the rod cells of the retina ) Each array has a fovea telescope with limiting magnitude of R~16.5 (60 sec), resolution of ~7 arcsec and Gunn g (or r) filter. Provides color, better resolution, and faster cadence light curves (cone cells of fovea) Each array has a fovea telescope with limiting magnitude of R~16.5 (60 sec), resolution of ~7 arcsec and Gunn g (or r) filter. Provides color, better resolution, and faster cadence light curves (cone cells of fovea) Rapidly slewing mount places the fovea anywhere in the field in <3 seconds. (rapid eye movement). Rapidly slewing mount places the fovea anywhere in the field in <3 seconds. (rapid eye movement). Two identical arrays are separated by ~38 km. (stereoscopic vision) Two identical arrays are separated by ~38 km. (stereoscopic vision)

16. Best Solution – A Distributed Sensor Network Initial work done by military and contractors to support war fighters. Initial work done by military and contractors to support war fighters. Sensor elements are self sustaining and autonomous. Sensor elements are self sustaining and autonomous. Sensor elements gather data on environment independently. Sensor elements gather data on environment independently. Data is communicated back to a central location and collaboratively processed. Data is communicated back to a central location and collaboratively processed. Working together these elements should provide a better overall picture of the environment, than single-point sensors. Working together these elements should provide a better overall picture of the environment, than single-point sensors. Ultimate Goal – To make decisions or gain knowledge based on information fused from distributed inputs

17. Collaborative Signal Processing Signal Processing Sensing Modalities event data nodes … Data Gathering Event Detection Decision Making ENVIRONMENT GENERAL CONCEPT FOR DISTRIBUTED SENSOR NETWORK Signal Processing Sensing Modalities data event

18. The Distributed Sensor Network Idea Applied to the RAPTOR System event data nodes … Data Gathering Event Detection Decision Making data event

19. DSN Qualifications DSN Qualifications In General Applied to an Astronomical System In General Applied to an Astronomical System Full scalability. Any number of systems coming and going Full scalability. Any number of systems coming and going Fault tolerance. System dropouts, weather, instrument failure, etc. Fault tolerance. System dropouts, weather, instrument failure, etc. Mosaic coverage. L arge area combined imagining Mosaic coverage. L arge area combined imagining Depth of data. Multiple instruments on same object and/or a variety of instrument sensitivities Depth of data. Multiple instruments on same object and/or a variety of instrument sensitivities Temporal coverage. Data covering continuous observations Temporal coverage. Data covering continuous observations

20. TALONS Components Monitor: run from any subscribers computer Monitor: run from any subscribers computer Shows real-time activity of client systems and Central Shows real-time activity of client systems and Central Central: Central: Provides connection point for clients to Provides connection point for clients to transmit and/or receive. Provides security for connections. Provides security for connections. Provides cooperative analysis. Provides cooperative analysis. Relay from outside networks to clients. Relay from outside networks to clients. Logs activity. Logs activity. Filters information to and from clients. Filters information to and from clients. Issues activity requests or alerts via sockets and e-mails. Issues activity requests or alerts via sockets and e-mails. Client: resides on each client computer Client: resides on each client computer Provides connections back to server receive and/or transmit. Provides connections back to server receive and/or transmit. Monitors connections and repairs or notifies as necessary. Monitors connections and repairs or notifies as necessary. Filters incoming information based on previous Filters incoming information based on previous activity, interests, operational capability of client system. activity, interests, operational capability of client system. Logs activity on client Logs activity on client Prepares data for transmission Prepares data for transmission Decodes data for response Decodes data for response Injector: accessed with client Injector: accessed with client Provides a method for manual alert generation Provides a method for manual alert generation Provides method for manual response follow-up Provides method for manual response follow-up

21. TALONS Monitor TALONS Injector

22. The Communication Packets All Packets Packet Size All Packets Packet Size Identifier byte What type, who sent 1 int Identifier byte What type, who sent 1 int Data Bytes How much data to follow 1 int Data Bytes How much data to follow 1 int GCN GCN Header packet (as above) Header packet (as above) GCN Alert Data All data (as per GCN packet info) 40 int GCN Alert Data All data (as per GCN packet info) 40 int TALONS TALONS Header Packet (as above) Header Packet (as above) TALONS Data – 9 int TALONS Data – 9 int Target Follow-up Requests (Alerts) Target Follow-up Requests (Alerts) Target of Opportunity Requests Target of Opportunity Requests Follow-up responses Follow-up responses Status Status Header Packet (as above) Header Packet (as above) Status Data – Instrument or Observatory Status 5 int Status Data – Instrument or Observatory Status 5 int NOTE: All int values are 32 bit

23. Breakdown of Packet Information GCN Packets – Information varies per satellite. Packet details at http://gcn.gsfc.nasa.gov/sock_pkt_def_doc.html TALONS Packets TID –Trigger ID (Bit field definitions, encoded & decoded at client) Imagine Instrument or Observatory systems ID Follow-up or Initial spotting Either Suspected or Confirmed target type (Nova, SN, etc.) Known or new target object 1 TrigNO 2 TID 3 TOO 4 Time 5 RA 6 DEC 7 Mag 8 MagErr 9 EOF TrigNo - Trigger (or Alert) Number For Initial Spotting, returns a 0 to Central and Central assigns a new value. For follow-up observations, Trigger number is passed as the event identifier TOO – Target of Opportunity (Bit field definitions, encoded & decoded at client) Works with TID above to define type details Defines whether this packet is for a TOO Identifies if the event is transient (approaching, or receding from event. Requested Observation Type (Spectra, Photon Counting, Any, All, etc) Requested Observation Range (FIR, Gamma, Radio, etc.). Requested imaging durations (How many seconds?) Time - Time imaged expressed in TJD RA - Target Coordinate DEC - Target Coordinate Mag - Magnitude of target MagErr - Error in Magnitude measure NOTE: RA and DEC errors are to come }

24. Summary TALONS has been in operation now for three years, supporting the RAPTOR system TALONS has been in operation now for three years, supporting the RAPTOR system TALONS can easily grow to support any number of additional robotic and manual telescopic systems TALONS can easily grow to support any number of additional robotic and manual telescopic systems TALONS works well in concert with GCN. No additional coding necessary to receive GCN. TALONS works well in concert with GCN. No additional coding necessary to receive GCN. The TALONS Client library can quickly and easily be added to any existing telescope operation software and can quickly be configured to support the interests of the user. The TALONS Client library can quickly and easily be added to any existing telescope operation software and can quickly be configured to support the interests of the user. The information packets are flexible and can be changed to suit needs or additional packet types can be added as needed. The information packets are flexible and can be changed to suit needs or additional packet types can be added as needed.