Astrometric Calibration OmegaCen Astrometric Calibration Parallel Processing Dataserver Erik Deul

Astrometric Calibration Gnomonic projection Forward (a,b) -> (x,h) Backward (x,h) -> (a,b) 04/12/2003 OmegaCen - UK/Vista meeting

Astrometric Calibration Single field Polynomial description x(x,y) = a + S bij xi yj h(x,y) = c + S dij xi yj 04/12/2003 OmegaCen - UK/Vista meeting

Astrometric Calibration Overlap fields Independent set of params/field Dependence of b and d with field x(x,y) = a + SS bijk xi yj f k h(x,y) = c + SS eij xi yj f k Spatial terms Chebychev polynomes Pn+1 – f Rn(f) + Pn-1(f) = 0 04/12/2003 OmegaCen - UK/Vista meeting

Astrometric Calibration Errors Centering errors dx = c1 +b1h + x (c1 x + f1h) dh = f1 +b1 x + h (c1 x + f1h) Aberration dx = c2 +a1 x + ½ c2 ( x2 + h2) dh = f2 +a1 h + ½ f2 (x2 + h2) Refraction dx = k0 (1 +x2 + h2)/(1 +xo x + ho h) ( x0 - x) dh = k0 (1 +x2 + h2)/(1 +xo x + ho h) ( h0 - h) Const: independent of object position First order: rotation 1” -> 0”02 Second order: 1”-> 0”02 displacement 1o off-axis Const: displacement direction dependent First order: scaling parameter Second order: 1”-> 0”006 displacement 1o off-axis Const: displacement First order: smaller in scale Second order: slightly larger than aberration 04/12/2003 OmegaCen - UK/Vista meeting

Astrometric Calibration Least Squares Pairing Extracted objects with reference catalog USNO-A2 100 sources (RMS 0”3) GSC 8 source (RMS 0”3) Tycho 1 source (RMS 0”03) Hipparcos 0.1 source (RMS 0”003) PPM 0.3 source (RMS 0”1) Extracted objects in overlap Internal extraction precision RMS 0.1 pixel (< 0”03) Minimization Sum of squared differences weighted with positional precision knowledge Iterate Kappa-Sigma clipping > 4s excursions Remove erroneaus pairrings Minimize cenering errors 04/12/2003 OmegaCen - UK/Vista meeting

Astrometric Calibration Example DENIS strip 30o long 10’ wide 180 frames Specifics 1” pixel RMS 0”3 04/12/2003 OmegaCen - UK/Vista meeting

Astrometric Calibration Example WFI dither 5 pointings 8’ x 15’ wide 8 frames pp Specifics 0”26 pixel RMS 0”1 04/12/2003 OmegaCen - UK/Vista meeting

Astrometric Calibration Example WFI dither Tracing pointing Tracing scaling 04/12/2003 OmegaCen - UK/Vista meeting

Astrometric Calibration Example WFI Dither PSF anisotropy 04/12/2003 OmegaCen - UK/Vista meeting

OmegaCen - UK/Vista meeting Parallel Processing Embarrassingly parallel 32 CCD’s -> 32 CPU’s N frames -> N CPU’s Few places in pipeline need synchronization Astrometric calibration overlap solution Parts of swarping Processing I/O bound Few floating point operations per pixel (few GHz) Many pixels, limited bandwidth (1 Gb/s) Administrative burden Bind data to machine (disk) Weight processor (net distance server, cpu speed, ..) Status WEB page 04/12/2003 OmegaCen - UK/Vista meeting

OmegaCen - UK/Vista meeting Parallel Processing Inter process communication WEB technologies Python methods can be called Python objects can be passed Exceptions are thrown and caught XMLRPC Similar to WEB services like SOAP Implicit interface (no WSDL) Authentication user/passwd mechanism No site limitations Could run as GRID machine AstroWise specific 04/12/2003 OmegaCen - UK/Vista meeting

OmegaCen - UK/Vista meeting Parallel Processing Scheduler – Servers - Clients Client Scheduler Server Server Server Server Server 04/12/2003 OmegaCen - UK/Vista meeting

OmegaCen - UK/Vista meeting Parallel Processing Scheduler – Servers - Clients Client OmegaCen Scheduler Server Any Place Server Server Server Server HPC-Groningen 04/12/2003 OmegaCen - UK/Vista meeting

OmegaCen - UK/Vista meeting Parallel Processing Example server class class TaskRequestHandler(xmlrpcserver.RequestHandler): def call(self, method, params): dispatch = {'kill': self.kill, 'weightTask': self.estimate_cost, 'runTask': self.run_task, 'run_task': self.run_task, 'estimate_cost': self.estimate_cost} if method not in dispatch: print 'Server does not contain XML-RPC procedure', method return 0 print "Dispatching:", method, "at", time.ctime(time.time()), "on", self.server.port return dispatch[method](*params) 04/12/2003 OmegaCen - UK/Vista meeting

OmegaCen - UK/Vista meeting Parallel Processing def run_task(self, task): """Execute the task Arguments: task -- A pickled task This method takes a pickled task as argument and performs the following operations: 1. unpickle the task 2. do task.execute() 3. pickle the task 4. return the pickled task The task's execute method is expected to store the results in the task's state. It is also expected to handle it's own errors. """ try: task = pickle.loads(task) task.execute() task = pickle.dumps(task) return task except Exception, e: print e return 0 Example server tasks def estimate_cost(self, task): '''Try to determine the cost of a task ''' try: task = pickle.loads(task) return task.estimate_cost() except Exception, e: print e return 0 04/12/2003 OmegaCen - UK/Vista meeting

OmegaCen - UK/Vista meeting Parallel Processing WSDL Client class A: def do_nothing(self): pass def do_good(self, *args, **kwargs): return args, kwargs def do_bad(*args, **kwargs): raise RuntimeError('Things went really bad') a = A() t = Task(a, 'do_nothing') t.execute() print t.result, t.exception t = Task(a, 'do_good', 1, 2, b=3) t = Task(a, 'do_bad', 1, 2, b=3) ts = TaskSequence([Task(A(), 'do_good', 1), Task(A(), 'do_good', 2), Task(A(), 'do_good', 3)]) ts.execute() for t in ts.tasks: 04/12/2003 OmegaCen - UK/Vista meeting

OmegaCen - UK/Vista meeting Data Server Peer-to-peer server mechanism Like napster/kaaza Local data owner Caching service Ensemble of data servers Serve data for pipeline processing WEB based interface wget enabled Python class Storage, methods retrieve store 04/12/2003 OmegaCen - UK/Vista meeting

OmegaCen - UK/Vista meeting Data Server OmegaCen USM Napoli 04/12/2003 OmegaCen - UK/Vista meeting

OmegaCen - UK/Vista meeting WEB VO Distributed processing Parallel processing, WEB services Data Server, WEB repositories Current VO Cone search Simple Image Access Remapping Montage, Swarp? Future VO Are there commonalities Web Services 04/12/2003 OmegaCen - UK/Vista meeting

OmegaCen - UK/Vista meeting Data Server Filenaming convention Context Type | Possible values (not exhaustive) ----------------------------------------------------- Project | Calibration, Science, Survey, Personal, Other Owner: | Name (pipeline, develop, user, ...) Instrument | OmegaCam, MegaCam, INT, WFI, ... Strategy: | Standard, Deep, Freq, Other Mode: | Stare, Jitter, Dither, Other Time: | Date and Time Filter: | Number, Name Chip: | Number Type: | Raw, (Master) Bias, (Master) Dome/Twilight Flat, | (Master) Flag, Col/Hot pix, | (Master) Weight, (Reduced) Science Proc | MJD Extension | fits, cat, … The Filename name structure could then be setup as follows: [Project]-[Owner]-[Instrument]-[Startegy]-[Mode]-[Date]-[Time]-[Filter]-[Chip]-[Process Level]-[Process Type]-[File Type]-[MJD].[extension] Filling in most known possible values to a maximum of three distinctive characters: [Cal|Sci|Sur|Per|Oth]-[Pip|Dev|EV]-[Ome|Meg|INT|WFI]-[Sta|Dee|Fre|Oth]-[Sta|Jit|Dit|Oth]-[YYYY:MM:DD]-[HH:MM:SS]-[#]-[#|Nam]-[Mas|Oth]-[Dom|Twi|Red|Cold|Hot]-[Raw|Bias|Flat|Flag|Wei|Sci|Pix] Unique identifier GID from Oracle DB numeric 04/12/2003 OmegaCen - UK/Vista meeting