1. Instrument Control SW Pipeline DRS CLIP
„“Common Library for Image Processing“

2. Overview Problem More complex Instruments.
Need for Quicklook procedures.
Data Reduction Software (DRS) normally includes these routines for quality control.
Figure source: P. Biereichel
As instruments become more complex, there is growing need for having better image processing capabilities available in the instrumentation software. Astronomers want to get feedback during the acquisition of the field before the start acquiring data (quicklook) and apply corrections in case the target is shifted. One typical example is the so-call through–slit process where astronomer want to correct the position of the telescope to put the target at the center of the slit.

3. Overview Definition and Goals
CLIP is foreseen as the new standard online image processing tool for the VLT Instrumentation Software.
Goals
To provide an image processing toolkit, flexible and well integrated with the INS SW.
To share the same image processing algorithms whenever possible between INS SW and pipeline DRS
To implement some specialized image processing routines which are generally required during the instrument acquisition process.
CLIP is a project especially developed for VLT second generation instrument and is foreseen as the new standard tool for online image processing.

4. Overview Package Image Processing Library C++ VLTSW Library
clipm extending CPL
C++ VLTSW Library
clipv
Tcl VLTSW Library
clipvTcl
Real Time Display
rtdc
Image Acquisition Simulator
clipias
Template modules
xxclipv, xxclipm
CLIP is a package composed of several modules:
Image processing Library
as a collection of image processing routines especially designed to support the acquisition of the instruments.
C++ VLTSW Library: C++ programmatic interface to access the image processing libraries from C++ applications.
A set of Tcl commands extending the Tcl Language to access the image processing libraries from instrument templates, panels and RTD applications.
Real Time Display: to facilitate the visualization of the images and the user interaction.
Image Acquisition Simulator: a test tool to validate the image processing routines on a development environement.
Templates Modules: Providing the starting point to extend CLIP for instrument specifics requirements.

5. Image Processing Library
Goals
Cover common image processing needs, for example
star centroiding and object characterisation,
image alignment,
focusing etc.
Serve as temporary development platform (until routines are accepted for CPL),
thus providing flexibility by separating DFS and INS SW development cycles
Thus instrument-only
The core of the CLIP package is the image processing library “clipm”. It is intended to provide image processing functions that are typically used in the instrument control. Since it is an own library, its development cycles can be shorter than the one from CPL, and reactions to requirement updates and sudden needs can be quicker.
Therefore it is solely intended to be used in the control software, and not in the pipelines.

6. Image Processing Library
Functions
Image Analysis
Mathematics
Full bad pixel support
Background Estimation
Point-Source Centroiding
Fitting
Barycentre
Alignment
Cross-correlation
Transformation
Aperture Characterisation
Variously shaped objects
Automatic region of interest
Optimised results
Statistics in- & outside
Focusing (pending)
Linear Regression
Random Number Generation
Gaussian
Poisson 2d-pattern
Pattern Alignment
Different degrees of freedom
Transformation
This is an overview of the currently implemented functionality in the clipm.
The interface is in C, as the one from CPL.
It provides a full error history including messages, which can be copied into the VLT control stack.
The interface uses CPL objects that reside in memory, so that the user has the full flexibility over where he gets the data from (for instance from shared memory or from disk).
Interface
ANSI-C
Error Message History
CPL Objects in Memory

7. C++ VLTSW Library Features Main Features:
Direct access of images from shared memory
C++ wrappers for some CPL objects
Automatic initialization and cleaning up of CPL
Verbose and logging facilities
Byte swapping and data conversion.
VLTSW standard error handling
- CLIP does not need to wait until the OS save the images saved on disk. It can access them directly from shared memory. The next slide will explain how this works.
CLIP provides some wrappers exploiting the C++ capabilities like operators, copy constructors, inheritance, etc.
CLIP provide five levels of verbose data and this data can be logged.
Byte swapping and data conversion. CPL only supports three primitive data types (INT, FLOAT and DOUBLE) so for systems generating images of different type we need to convert the data. This is the case of Fiera where the generated images are of type short so they need to be converted to integer. Images coming from the detector might be in a different byte order so CLIP is doing silently the byte swapping.

8. C++ VLTSW Library Shared Memory Shared Memory RTD App. Clipv Server
Client Mode
Server Mode
Shared
Memory
Write
RTD
App.
Read
Register
Notify
Clipv
Server
Create/
Remove
Acq.
Process
RTD
Server
Notify
Register/
Notify
Create/
Write
Notify
Access to shared memory is based in the RTD image event library. RTD forwards the image events containing the image size, shared memory ID, byte order and other information to all clients registered.
In the same way, processes can create a shared memory area and send the image events to the RTD server which will notify the clients.
Shared
Memory
Clipv
Client
Read
Image Events:
Size of the image
Data type
Shared memory ID
Last event flag

9. C++ VLTSW Library Example
// Initialize the connection to the RTD server registering
// as the CCS process name and attaching to the image events
// generated by the acquisition process.
clipvRTD_EVT_RECV *rtdEvtRecv;
rtdEvtRecv = new clipvRTD_EVT_RECV(ccsGetMyNameInt(), ”fierab”);
rtdEvtRecv->Attach();
// Receives the image event and store it in the internal
// structure.
rtdEvtRecv->ReceiveImageInfo();
More examples in the test directory of module clipv
This code is to initialize the connection to the RTD Server, attach to the camera events and receive the events forwarded to the clients. Automatic events notification shall be implemented using evh as is shown in the example of module clipv.

10. Tcl VLTSW Library Functions TCL Commands: Others Services
General Purpose, e.g. verbose, version, subcommands, etc.
RTD Image Events, e.g. rtd_attach, rtd_detach, rtd_image_wait, etc.
Image Basic Operations, e.g image_add_scalar, image_add, etc.
Image Statistics, e.g. image_get_min, image_get_mean, etc.
Image Processing Recipes, e.g. image_center_gauss, image_slit, etc
Total: over 40 Tcl Wrappers
Others Services
Image Store
Keep images in memory across different exposures.
Store images from different sources: file and shared memory.
Data by Reference (based in BLT)
Currently the TCL library contains over 40 commands which can be divided in general purpose, image events, image basic operations, image statistics and at last but not least the image processing recipes.
Internally the image store is STL map which is indexed by the name given to the image.
Facility to pass vectors by reference

11. Real Time Display Display image in real-time
Like a normal RTD
Display superimposed graphics.
Ovals,
Lines,
Rectangles
Crosses
Text
User interaction.
Pickup Object
Get Rectangle
Real Time Display for CLIP is a new tool for the VLTSW that allows to create RTD applications with the panel editor. It can be used to replace the instrument second RTD used for acquisition. It simplify the interaction with the templates since it can be embedded in pop-up panel removing the CCS messages with the RTD.

12. Image Acquisition Simulator
Commands: SETUP, START, STOP.
Modes: BIAS, SKY and FILE
Supported sky objects: Gaussian star
Supported star parameters: pos, sigma, maxflux and gaussian norm.
shell> msgSend “” clipiasControl_IR SETUP “-function CIAS.MODE SKY CIAS.SKY.STAR.NUM 3 CIAS.SKY.STAR1.POSX 300 CIAS.SKY.STAR1.POSY 512 CIAS.SKY.STAR1.MAXFLUX CIAS.SKY.STAR1.SIGMAX 6 CIAS.SKY.STAR2.POSX 200 CIAS.SKY.STAR2.POSY 100 CIAS.SKY.STAR3.POSX 700 CIAS.SKY.STAR3.POSY 700 CIAS.SKY.STAR3.SIGMAX 8”
shell> msgSend “” clipiasControl_IR START “”
CLIP Image Acquisition Simulator is a simple and configurable tool allowing the testing of the instrument online image processing routines in a development environment.
The simulator registers to the RTD server and generate images of certain characteristics defined in a configuration file that the user could overwrite. It permits the generation of images with Gaussian stars and allows the usage of FITS files giving enough flexibility for the offline tests.

13. Image Acquisition Simulator
Configuration
CIAS.DET1.CAM "IR"; # Detector camera name
CIAS.DET1.TYPE "2"; # VISIBLE=1 and INFRARED=2
CIAS.DET1.WIDTH ; # Detector width
CIAS.DET1.HEIGHT ; # Detector height
CIAS.DET1.DATATYPE -32; # FLOAT=-32 SHORT=16
CIAS.DET1.MINBIAS 116; # Detector minimum bias
CIAS.DET1.MAXBIAS 129; # Detector maximum bias
CIAS.DET1.DELAY ; # Detector exposure delay
CLIP Image Acquisition Simulator is a simple and configurable tool allowing the testing of the instrument online image processing routines in a development environment.
The simulator registers to the RTD server and generate images of certain characteristics defined in a configuration file that the user could overwrite. It permits the generation of images with Gaussian stars and allows the usage of FITS files giving enough flexibility for the offline tests.

14. Template Library Interface
Instrument Templates (updated version of tpl)
Asynchronous exposure wait
New class: tplCLIP
New set of classes to create graphics in the Real Time Display
New concept for instrument acquisition.
Tpl, the instrument template library has been updated for CLIP and will be available in the next VLTSW release. This should be the
The new features include an asynchronous exposure wait and a new classes like tplCLIP which is intended to simplify the interface with TCL commands from templates.

15. Template Library Interface Example # object declaration tplCLIP clip
# Attach to the image events from camera
clip Attach fiera
set expId [obs Setup "-function $funcSetup"]
# start the exposure
obs Start [fiera Subsystem] $expId
set cmdId [obs WaitAsync $expId "-detId [fiera Subsystem]"]
# Wait until image is shared memory
clip Wait fiera
# Compute mean value from shared memory
tplLog "Image MeanLevel: [clip MeanLevel fiera]“
# Check the reply from asynchronous wait
obs RecvReply $cmdId
Tpl, the instrument template library has been updated for CLIP and will be available in the next VLTSW release. This should be the
The new features include an asynchronous exposure wait and a new classes like tplCLIP which is intended to simplify the interface with TCL commands from templates.

16. Extendability Requirements & Templates
Requirements to be served: instrument specific needs:
control software-only image processing tasks, and/or
shared functions for pipeline and control software
Template modules:
Interface Templates (C++ and Tcl)
Image Processing Library Template
Some instruments will require special image processing functions, like for example spectral image reconstruction for KMOS.
Requirements can be either image processing that is ONLY used in the control software, or functions which are used in the pipeline and should be shared with the control software.
For them there are template modules, to a) implement the image processing functions, and b) to implement the interfaces for the control software in C++ and Tcl.

17. Extendability Image Processing - Development Strategy Pipeline World
Consortium
ESO
Repositories
Observatory
CVS
Pipeline
World
DRL Developer
Pipeline DRL
Pipeline Responsible
Pipeline Integration
Slave <xx>clipm
<xx>clipm DRL (Master)
Copy
DFS CLIP Responsible
There are two different software worlds: control software, and scientific data reduction.
Normally, a data reduction library (DRL) developer from a consortium writes the reduction functions, stores them, and the pipeline responsible from ESO prepares the pipeline for integration.
In the case of shared image processing functionality, the DRL developer will put the shared functions into a separate software module, which is the xxclipm library template (here called master). This will then be installed together with the pipeline, if the pipeline uses these functions.
A responsible person from ESO takes the module, and tests it. When the tests are passed, he copies it into the VLT SW repository, from where the INS developer uses it for the integration of the control software. The INS developer is also the one, who integrates the interfaces to the image processing functions in C++ and Tcl.
Of course, during development phase, the DRL and INS developers may (and in reality will) be in contact, and circumvent this procedure for testing. Nonetheless, for the operational control software, only the tested version will be allowed. Therefore, early testing and communication with the ESO CLIP responsible is essential.
CMM
INS
World
INS Developer
<xx>clipv (Wrappers)
INS / RTD
Integration

18. Extendability Image Processing - Adding new functions (1)
cpl_image *xxclipm_something( cpl_image *img_in,
double some_param) {
cpl_image *img_out = NULL;
XXCLIPM_TRY
XXCLIPM_TRY_CHECK_AUTOMSG( img_in != NULL,
CPL_ERROR_NULL_INPUT);
XXCLIPM_TRY_CHECK_AUTOMSG( some_param > 1.5,
CPL_ERROR_ILLEGAL_INPUT);
/* do some processing here */
XXCLIPM_TRY_CHECK_ERROR_STATE(); }
XXCLIPM_CATCH
cpl_image_delete(img_out); /* clean up if error happened */
img_out = NULL;
return img_out;
This code shows the structure of the C functions that are to be implemented:
- The macros XXCLIPM_TRY and XXCLIPM_CATCH are not mandatory but help structuring the code, and take care for the CPL error handling.
For instance, the macro XXCLIPM_TRY_CHECK_ERROR_STATE() checks whether an error exists (that was set by a called function) and whether this occurred after the use of XXCLIPM_TRY (a pre-existing error should be ignored).
The XXCLIPM_CATCH – block is executed in the case that a new error exists, and serves as a place for doing clean-up.
The XXCLIPM_TRY_CHECK_* macros also put meaningful messages into the error history.
Furthermore, there is another macro (XXCLIPM_ERROR_RECOVER_TRYSTATE() ), which can revert to the initial state without touching pre-existing errors.

19. Extendability Image Processing - Adding new functions (2)
cpl_error_code xxclipm_something( cpl_image *img_in,
double *out_value) {
cpl_image *img_tmp = NULL;
XXCLIPM_TRY
XXCLIPM_TRY_CHECK_AUTOMSG( img_in != NULL,
CPL_ERROR_NULL_INPUT);
XXCLIPM_TRY_CHECK_AUTOMSG( out_value != NULL,
/* do some processing here */ }
XXCLIPM_CATCH
if (out_param != NULL)
*out_param = -1;
cpl_image_delete(img_tmp); /* delete temporary stuff in any case */
return XXCLIPM_ERROR_GET_NEW_SINCE_TRY();
Here a similar example shows a function that return a CPL error code. The macro XXCLIPM_ERROR_GET_NEW_SINCE_TRY() retrieves the latest one, wile ignoring pre-existing errors.

20. Extendability TCL – Implementation Steps
Retrieve template module from CMM archive
shell> cmmCopy xxclipv
Compile template module to make sure that everything is fine before to start customization
shell> cd xxclipv/src ; make clean all
Create the instrument specific module from the CLIP template.
shell> clipvCreate <prefix> <INS ID>
TCL Library provides classes for the parameter handling of the TCL commands. Each parameter is an object having associated the option name, alias, description, default value, range for the values and a flag which is used to indicate if the parameter is optional or mandatory and if it has arguments.

21. Extendability TCL – Parameter Handling
Predefined classes for main built-in types: int, double, logical and string
Class for handling the list of parameters objects.
clipvTCL_ARG_STR par_mode('m', "mode",
"alignment mode",
"SHIFT",
"SHIFT SHIFT_ROBUST UNIT EQUAL FREE",
(clipvTCL_ARG::OPTIONAL_ARG|
clipvTCL_ARG::MANDATORY_VAL_ARG));
parameter option (one character)
parameter alias (string)
parameter description (string)
parameter default value (value of the same type)
Parameter range (list of allowed values)
parameter flags (valid class flag)
TCL Library provides classes for the parameter handling of the TCL commands. Each parameter is an object having associated the option name, alias, description, default value, range for the values and a flag which is used to indicate if the parameter is optional or mandatory and if it has arguments.

22. Extendability TCL - Adding new commands
// Declare the command in the constructor
NewSubCmd(string("image_app") ,this, &xxclipvTCL_APP::CmdApp);
vltINT32 xxclipvTCL_APP::CmdApp(int argc, char *argv[]) {
clipvLOG_3(("xxclipvTCL_APP::CmdApp - begin"));
// defining subcommand parameters
clipvTCL_LINE_ARG tclArg(&par_file, &par_scalar, NULL);
// parsing subcommand parameters
if (tclArg.ParseArgs(_interp, argc, argv) != TCL_OK)
clipvTCL_ERR_RETURN(tclArg.Usage(tclArg.GetStrError()));
// adding image processing here
clipvLOG_3(("xxclipvTCL_APP::CmdApp - end"));
return(TclSetResult("XXXX sample command executed !")); }
This code shows the way how to add a new TCL command. Commands must be registered in the constructor class and a dedicated method will implement the code for the command.

23. Status & Outlook Status CLIP Version 1.0 with VLTSW release 2009
Instruments using CLIP
HAWKI: Artificial gap and Jitter offsets
XSHOOTER: Cross-correlation
KMOS: Image reconstruction and aligning
SPHERE: Quick look and Image Processing
MUSE?
Outlook
Address instruments which still use MIDAS

24. Now a short DEMO