1. DAQ and TTC Integration For MicroTCA in CMS
AMC13 Project
DAQ and TTC Integration
For MicroTCA in CMS
Status Report
E. Hazen - Boston University
for the CMS Collaboration
E. Hazen -- Upgrade Meetings

2. MicroTCA DAQ (AMC13 Project)
E. Hazen -- Upgrade Meetings

3. E. Hazen -- Upgrade Meetings
DAQ in AMC13
128 Mbyte DDR3
Data from each
AMC at up to
3.125 Gb/s
(may be combined
With TTC)
CDS
Virtex 6
TTC
SFP? IO
SFP
GTX
GTX
GTX
GTX
(3) SFP for:
DAQ
GbE backup
Other?
At up to 4.25 GB/s
GTX
GTX
Fabric A
To AMC 1..12
SFP
GTX
GTX
GTX
GTX
SFP
GTX
GTX
GTX
Spartan 6
GTP
MMC uC
IPMI
Fabric B
TTC to
AMC 1..12
MCH1 GbE
(“cross-over”)
GTP
E. Hazen -- Upgrade Meetings

4. CMS-MCH Preliminary Design (aka HCAL DTC version 2)
Base configuration has only tongues 1, 2
Base board - “AMC13” with optics and HS links
Clocks board - distributes LHC clock and controls
Mezzanine connector for T3 with IPMI
New custom T1 base board MMC functions (as AMC-13)
TTC optical rx
4x SFP
Cross-over GbE from MCH1 for controls and local DAQ
Optional T3 board Optional (not in std CMS-MCH)
Crosspoint switch for trigger applications
Clocks board Clock / controls fanout T4 T3 T2 T1
E. Hazen -- Upgrade Meetings

5. DAQ options and non-options
Anything in SFP up to 4.25 Gb/s easy to support
Possible upgrade to SFP+ or SNAP-x with 10 Gb/s in the future requires board redesign
No room for mezzanine boards
Most of the “housekeeping” firmware will be on a separate FPGA, so users can design their own firmware for the AMC13 Virtex-6 if required.
E. Hazen -- Upgrade Meetings

6. Legacy DAQ Adapter – Do we need it?
One i.e. 9U VME module handles two uTCA crates, each with 400MBytes/sec out
Crate 1:
SLink (2)
TTS
Carrier PCB (i.e. 9U VME!)
SLink64
SLink64
Power (only)
from VME
Dual RJ-45
Virtex 6
FPGA
Fiber I/O
from 2 uTCA
crates
SFP
SFP
SLink64
Crate 2:
TTS
SLink (2)
SLink64
E. Hazen -- Upgrade Week

7. TTC Receiver without TTCrx
E. Hazen -- Upgrade Week

8. TTC receiver based on AD2814
Clocks Board (T2)
SY89872
Divider
/2 or /4 40
TruLight
DS91M125
M-LVDS repeater
160
Virtex 6
Clock 40
TTC
Opto rx
nRST
uTCA
CLK1
160
ADN2814
Clock/Data Recovery
40Mhz
Lock
TTC Data
Data
TTC
Data
80Mb/s
E. Hazen -- Upgrade Week

9.
E. Hazen -- Upgrade Week

10. TTC Receiver Hardware Truelight TRR-1B43 opto-rx ADN2814 Clock/Data
Recovery IC
Output to
Virtex-6
E. Hazen -- Upgrade Week

11. Decoder waveforms
E. Hazen -- Upgrade Week

12. Decoder Functions (VHDL)
Included in current firmware:
Acquiring lock based on allowed states in stream
Error-checking and re-sync
Decoding L1A and broadcast commands
Phase adjust of L1A and broadcast strobe outputs
Other functions easy to add in FPGA
E. Hazen -- Upgrade Week

13. Status Works to first order
Run for several days with firmware checking for disallowed states... no errors seen
Reset TTCvi/vx repeatedly to check for lock acquisition... works with ~ 100x trial
Used in HCAL test beam setup
To study:
Phase shift / latency
Clock jitter
Alternative opto-rx which is in production (SFP?)
other?
E. Hazen -- Upgrade Week

14. Backup Review of common platform proposal
E. Hazen -- Upgrade Meetings

15. AMC13 Configuration Summary
Dual-star backplane
Standard MCH in MCH1 slot (required)
AMC13 custom module in MCH2 slot (optional)
AMC13
TTC Functions (low-jitter clock, fast controls)
DAQ Functions (data readout, buffer management)
No involvement in services to AMCs (ipmi, Ethernet)
Managed as “13th AMC” by MCH1
REVIEW – As proposed in November 2010 WG and draft document
E. Hazen -- Upgrade Meetings

16. Proposed uTCA Ports Use for CMS
REVIEW – As proposed in November 2010 WG and draft document
E. Hazen -- Upgrade Meetings

17. MicroTCA Fabrics A, B Use
Port 0 is GbE sourced from commercial MCH1
Port 1 – Compatible with Xilinx GTP (<= Gb/s)
DAQ data from AMC to MCH2
May be used also for fixed-latency controls (TTC)
May be used also for buffer status (TTS)
May be used also for auxiliary data (selective readout)
Port 3 – “Low speed” serial LVDS ( Mb/s)
Fixed-latency controls (TTC)
Buffer status (TTS)
Auxiliary data (selective readout)
REVIEW – As proposed in November 2010 WG and draft document
E. Hazen -- Upgrade Meetings

18. MicroTCA Fabrics D and up Use
Use not mandated by MicroTCA WG
Mainly for trigger users
Two options for interconnection
Star connection to tongue 3, 4 of MCH slots
Interconnect passively or with crosspoint switch at MCH
Fixed connections on backplane
Or, a combination:
Ports 4-7 interconnected on backplane
Ports 8-12 routed to crosspoint switch on MCH2
REVIEW – As proposed in November 2010 WG and draft document
E. Hazen -- Upgrade Meetings

19. E. Hazen -- Upgrade Meetings
MicroTCA Crate Layout
12 full-height AMCs
non-redundant power
single-width MCH
i.e. Vadatech VT892
REVIEW – As proposed in November 2010 WG and draft document
E. Hazen -- Upgrade Meetings