1. John Hill ATLAS SCT Week2 October 2002 SCT ROD DAQ status and schedule John Hill University of Cambridge (with significant input from Tom Meyer)

2. John Hill ATLAS SCT Week2 October 2002  This talk summarises progress in the last few months (in essence since the SCT/Pixel ROD workshop at UCL in July) in:  RodModule (mainly taken directly from a talk by Tom Meyer – to whom apologies for any mistakes I’ve introduced)  internal ROD software (ie. in the DSPs – taken from presentations by Douglas Fergusen)  anything else (in passing…)  There is also an attempt at a possible short term schedule towards getting a working DAQ of some sort.  Main problem remains a severe shortage of useful manpower – and there must be a concern that it may be too late to find new recruits with enough expertise already, to make any practical difference.

3. John Hill ATLAS SCT Week2 October 2002 RODModule Context RodModule is the class library providing driver routines for the ROD VME module.

4. John Hill ATLAS SCT Week2 October 2002 Present Status  ROD being used for this development has been working in Ames (Iowa State) since 1 August.  Debugging has covered most of the basic functionality.  Create a primitive and add it to a primitive list  Send a primitive list and retrieve the reply buffer if it exists  Read an unsolicited text buffer  Background information:  VmeModule is the base class from which RodModule is derived.  The layer “below” this is called VmeInterface – it provides a system-independent interface to the VME hardware. VmeInterface has been in use for several months by the Pixels group in the testbeam and by me in low level tests at Cambridge.

5. John Hill ATLAS SCT Week2 October 2002 Methods Inherited from VmeModule   initialize()  Reset the ROD (bit 7 of FPGA Control Register 2)  Read ROD serial number  Initialize Master HPIC register and read it back to verify  Wait for MasterDSP init  Read reply buffer addresses  Initialize state variables  ? reset()  Reset the ROD (bit 7 of FPGA Control Register 2)  Clear IDRAM  Reinitialize state variables  ? status()  Under development  Creates a RodStatus object  For now just print to standard output  What information do we want?, Levels of info?  How to display it? Key:  = well tested; ? = untested; ! = known problems

6. John Hill ATLAS SCT Week2 October 2002 Constructor  RodModule( unsigned long baseAddr, unsigned long mapSize, VmeInterface & ourInterface, long slot, long numSlaves);  There are no copy or assignment operators.  The constructor creates a VmePort object and registers it with its VmeInterface object.

7. John Hill ATLAS SCT Week2 October 2002 Shared Header Files  The C-code DSP programs and the TestStand program use shared header files to specify addresses and bit assignments. We would like to use these same headers for C++.  We want to specify these details in only one place and propagate the information.  In C, these are set with #define statements.  C++ can use C headers, but it is more in keeping with C++ principles to declare the constants as const unsigned long variables. This lets the compiler catch any attempts to change them, and it does not expose the code to global text substitutions from #define statements.  I have wrapped the C headers with C++ header files that expose only data of interest to RodModule and use const unsigned long values. This also isolates C++ code from changes in C headers, and hides C- only parts of the header.

8. John Hill ATLAS SCT Week2 October 2002 RodModule: HPI Access Methods //! Load a 32-bit value into an HPI register.  void hpiLoad(const unsigned long hpiReg, const unsigned long hpiValue); //! Fetch a 32-bit value from an HPI register.  unsigned long hpiFetch(const unsigned long hpiReg);

9. John Hill ATLAS SCT Week2 October 2002 RodModule: DSP-space Access Methods //! Read to and write from Master DSP address space  unsigned long mdspSingleRead(unsigned long address);  void mdspSingleWrite(unsigned long address, unsigned long value); ! void mdspBlockRead(unsigned long address, unsigned long buffer[], long count); ! void mdspBlockWrite(unsigned long address, unsigned long buffer[], long count); Note: multi-block transfers are not yet tested. //! Read to and write from a slave DSP memory address ? Unsigned long slaveSingleRead(long slaveNum, unsigned long dspAddr); ? void slaveSingleWrite(long slaveNum, unsigned long address, unsigned long value); ? void slaveBlockRead(long slaveNum, unsigned long address, unsigned long buffer[], long count); ? void slaveBlockWrite(long slaveNum, unsigned long address, unsigned long buffer[], long count);

10. John Hill ATLAS SCT Week2 October 2002 RodModule: DSP Reset Methods //! Reset the Master DSP via FPGA Control Reg 2  void resetMasterDsp(); //! Reset a Slave DSP via FPGA Control Reg 2 ? void resetSlaveDsp(long slaveNumber); //! Reset all DSPs via FPGA Control Reg 2 ? void resetAllDsps();

11. John Hill ATLAS SCT Week2 October 2002 RodModule: Primitives //! Send a primitive list  void sendPrimList(RodPrimList *list); //! State machine to handle primlist dialog with ROD  PrimState& primHandler(); //! Delete reply list retrieved by primHandler  void deleteOutList();

12. John Hill ATLAS SCT Week2 October 2002 RodModule: Text Buffers //! Get a text buffer  void getTextBuffer(char * buffer, long length); //! State machine to handle text dialog with ROD  TextBuffState textHandler(); Note: reading a wrapped-around buffer not yet tested //! Clear the text buffer when done  void clearTextBuffer();

13. John Hill ATLAS SCT Week2 October 2002 RodModule: Utilities //! Calculate a checksum (bit-wise XOR)  long checkSum(const unsigned long *sourceArray, const long wordCount); //! Go to sleep for a specified number of milliseconds  void sleep(const double milliSecs); //! Reverse the endian-ness of a data word if a flag is set  unsigned long endianReverse32(const unsigned long inValue); These methods may be moved to a RodUtilities class in the future.

14. John Hill ATLAS SCT Week2 October 2002 RodPrimitive Class  A simple class to hold Primitive objects  Data members:  long m_length  long m_index  long m_id  long m_version  long *m_body   Methods: Only the usual data accessor methods (“get” and “put”).

15. John Hill ATLAS SCT Week2 October 2002 RodPrimList Class  Derived from the C++ Standard Template Library (STL) list class. This gives us almost all the functionality we need for free.  Data members:  unsigned long m_index  Unsigned long m_version  unsigned long *m_buffer  unsigned long m_listLength  Methods (other than data accessors)   unsigned long numWords()   unsigned long checkSum()   void bufferBuild()   void clear()

16. John Hill ATLAS SCT Week2 October 2002 RodOutList Class  Contains reply lists generated by the MDSP in response to an associated primitive list.  Structure is the same as a PrimList  Data members:  long m_length  unsigned long* m_body  Methods:   unsigned long* getBody()   long getLength()  RodModule makes no attempt to understand the reply list, it merely passes it to a user thread for further handling.

17. John Hill ATLAS SCT Week2 October 2002 Text Buffers  Text buffers are more useful for debugging than for data taking.  If a DSP CPU crashes, we cannot execute primitives. But the on-board extended memory interface (EMIF) circuitry still lets us read the SDRAM contents.  Text Buffers are blocks of SDRAM memory to which we can write ASCII records with traceback information, or simple diagnostic info for later analysis. (Recall debugging FORTRAN with print statements.)  Even if DSP is fine, it may be useful to pass ASCII data into a message stream (MRS) in ATLAS.  There are four text buffers defined for the Master DSP  Err: for reporting fatal software error tracebacks  Info: for non-priority information messages  Diag: for diagnostic messages, “print” to it like we used to print out FORTRAN debug messages.  Xfer: A temporary MDSP buffer to relay text buffers from the Slave DSPs

18. John Hill ATLAS SCT Week2 October 2002 ROD DSP software  With sincere apologies to Douglas Fergusen (these few slides don’t begin to do justice to his efforts).  Reminder – each ROD has ONE Master DSP and up to FOUR Slave DSPs  Master DSP ocontrols the activities of the slaves ois main communications route from RCC to ROD internals ois also the route whereby command strings are sent to the F.E. modules  Slaves will typically perform tasks such as event trapping and histogramming (to name a random pair…)

19. John Hill ATLAS SCT Week2 October 2002 State Machines Idle List inserted? (host sets bit) ready to Run? no Executing : one primitive/ loop iteration yes (set busy, executing bits) Pause list? Finished/ Abort list? Abort? Resume? Paused yes no Prepare reply data Acknowledged Host cleared inList bit? yes no yes no Primitive List ExecutionText Buffer Sending Idle Buffer occupied? Waiting Frozen yes no read request? yes no Buffer read? (readReqest reset) no (clear buffer occupied bit) yes Two types: Host List & Inter-DSP List

20. John Hill ATLAS SCT Week2 October 2002 Primitives vs. Tasks Primitives: o Generally are executed once. Primitives can return a value that requests that the primitive be repeated (say, for polling), but only one primitive executes at a time. o Return data via the reply list. Each primitive in the input list contributes a header block to the reply, and optionally adds data, which can either be structured or unstructured. o Generally are simple. The list is a “program”, and primitives are its functions. Tasks: o Execute continuously, and together. Several tasks will happily co-exist on a DSP. Tasks can be paused & aborted, just like primitive lists. Tasks tend to be independent of each other, though there is no need for this. o Return data asynchronously. When a task completes, it can send an info message to the host. Any output data can be transferred via a primitive. sendData will handle large chunks of data processed by tasks & primitives (events, histograms, fits etc.); taskOperation:stop or query return the task’s internal variables. Some tasks (histogramming) also report via registers. o Typically are more complex. Tasks can have several different internal states; the task manager will report the states of any running or completed/halted tasks in the running task register (idata).

21. John Hill ATLAS SCT Week2 October 2002 6701 7) histograms 6201 router FE elec. 1) eventTrapSetup registers Router TE 4) setTrigger (TSD) 5) Event Data 8) sendData (histograms)2) startEventTrapping 6) histogramEvents DMA parameters 3) histogramSetup Data & trigger vme Histogram params Trigger data

22. John Hill ATLAS SCT Week2 October 2002 Common Primitives: Echo Set Err Msg MaskPause ListSet Memory Copy MemoryMemory TestSend Data Module Mask Set Trigger Start Task Task Operation Write Buffer Slave Primitives & Tasks: Start Event TrappingStop Event TrappingHistogram Setup Event TrappingOccupancy Counting Error CountingLink Re-synchronization Set LED Flash LED Transmit Serial DataStart Slave ExecutingR/W Register Field R/W Slave Memory Send Slave List Build Stream Slave List Operation Send Stream Configure SlavePoll Register Field R/W FIFO Start Slave List Master Primitives & Tasks: Trap Request Monitoring Tasks: Event Trap Setup Histogramming Memory MirroringHistogram Control Send Configuration R/W Module Data

23. John Hill ATLAS SCT Week2 October 2002 Other developments  TimModule – see John Lane’s talk.  DAQ-1 – version 0.0.17 appeared in April (2002 “testbeam release”). 14 patches since then, but generally appears stable.  ROD Crate DAQ – Task Force set up and the definition document appeared in May 2002. Since then very little sign of activity (appears to be stalled in the DIG as far as I can tell).  ROD Workshop III – 14/15 November in Annecy. Aimed at integration of the ROD with central T/DAQ, rather than the hardware of the ROD. May provide some pointers to the way that ROD Crate DAQ might develop?

24. John Hill ATLAS SCT Week2 October 2002 Short- and Medium-Term Future  Finish RodModule to first order (3 weeks)  Test slave routines (LED primitives, Memory read/write, DSP reset)  Test text-generator primitive, use to test wrap-around readout  Write prototype status() – in progress  Fix block transfer and HPID++ problems  Test multi-block transfers  Implement threaded version (3 weeks)  Command thread  Reply handling thread  Error handling thread  Background thread  Drive threads from DAQ-1(2 weeks)  Send results to analysis routine(s) (worked on in parallel to above – who?)  Define interface for data  Define transport method (IPC at first, CORBA later)  Write analysis program (root? PAW? Other?)  SCT and/or Pixel calibration program  Hard-wired commands and configuration  Need ROD+BOC+TIM+Module => do at LBNL  Tom proposes a one or two week visit to LBNL of relevant parties in late November to mid January time frame.

25. John Hill ATLAS SCT Week2 October 2002 Phase 1: Calibration Program  1 ROD (=>1 crate)  Posix threads  DAQ-1  Hard-wired, or very simple, configuration  No data base  Simple displays  Analysis on host computer (no CORBA)

26. John Hill ATLAS SCT Week2 October 2002 A Possible Path to Phase 1 Phase 1 “RODfest” @ LBNL RodModule Debugged DAQ-1 pThreads Analysis/ archiving “reply” thread/ IPC ROD+BOC+ TIM+Module Simple Config

27. John Hill ATLAS SCT Week2 October 2002 Longer-term Future  Multi-Rod and multi-crate support  Configure dynamically from data base  Dynamic control of operations  CORBA for command distribution and event publishing  Full implementation of plan from February workshop