1. The Role of Computer Vision in Astronomy
Rob Fergus
New York University
Pietro asked to speak briefly about how computer vision is being used in astronomy.

2. Overview
Virtually all our knowledge about the universe derived from measurements of photons
Usually as images
Big astronomy project is $50M-200M
But only 1-2% of this on software
Just discovering techniques from computer vision & machine learning
Now these are two areas which we don’t think of as being closely related. But its worth realizing that the vast majority of our knowledge of the universe comes from looking at images of the sky. Correspondingly, astronomers spend serious money to capture the best images – space-based telescopes >$1Bn. But the interesting thing from our point of view is that they just don’t have the same emphasis on software. However, this is changing as we can discuss later. As a community, they are just discovering methods from vision and ML and so there is a lot of low-hanging fruit from our perspective.

3. Astrometry.net Input: image of sky Output:
Absolute position
List of objects
Geometric hashing (quads of stars)
Lamdan & Wolfson [ICCV’88]
Widely used by pros & amateurs
Lang, Hogg, Mierle, Blanton & Roweis, [The Astronomical Journal, Vol. 137, 2010]

4. Removing Atmospheric Distortions
Ground-based telescopes look through atmosphere
Blind (online) estimation of atmospheric distortion and true image
Far better than “lucky” imaging (current approach)
Hirsch, Harmeling, Sra & Schölkopf , [Astronomy & Astrophysics 2011]
Hirsch, Sra, Schölkopf & Harmeling, [CVPR 2010]

5. Exoplanet Imaging Want to image planets around other stars
Need contrast ratio >1010 for Earth-like planets
Diffraction in telescope
Light from star obscures planet
Deconvolution problem
Big assistance from optical design
Planet
Zimmerman et al., [Astrophysical Journal, 2009].
Big assistance from  Plus clever
Oppenheimer and Hinkley, [Annual Review of Astronomy and Astrophysics, 2009].
Crepp et al., [Astrophysical Journal, Vol. 729, 2011].

6. Galaxy / Star Classification
Star vs Galaxy
[Sloan Digital Sky Survey]

7. Galaxy / Star Classification
Distinguish stars from galaxies
SVM-based models
Smith et al. [A & A, Vol. 522, 2010]
Generative model of galaxies
Lang et al. [In preparation]
Stars
Galaxies
Data
Model

8. Future Directions Unified Bayesian model
Propagate uncertainty from pixels
Physics- informed priors
Low signal to noise regime is typical
Already use Bayesian modeling for high-level cosmological models
Data is far simpler than natural images
Funded by NSF CDI

10. Cosmology
Bayesian approaches to fitting high-level cosmological models

11. Cosmic Ray Classification
Raw image from Hubble Space Telescope: