Overview of AdvancedTCA, AMC, µTCA xTCA Overview of AdvancedTCA, AMC, µTCA Hope my voice lasts - or maybe better if it doesn’t! Before I begin I would like to say that AdvancedTCA was initally over-hyped - a common problem with new systems. I believe ATCA is now a real commercial product and growing rapidly. Joe Pavlat (chair of PICMG) cringed at the first year talk of a $10’s billions in a year or two. New systems don’t develop that rapidly. AMC has come along as part of ATCA and the basic features set by that spec. Later some of the ATCA developers decided that the AMC module could be used in a small form factor system and be “inexpensive”. Some one was rash enough to talk about the $500 dollar system - some friends of mine in companies that are interested in mTCA would like to choke him for that statement (new hype again). This was a good idea that was not completely thought thru and has caused some indigestion in the committee. Not uncommon when you have an enthusiastic group. That having been said I don’t want to discourage you from mTCA - it definitely has a place in systems that we will build. I just want to put it in perspective and show the evolution of the spec. and why it evolved as it did. 4 Dec 2007 DESY Crate STD. Workshop

General Description of xTCA Very broad overview of ATCA - time! AMC Modules ATCA Carrier µTCA AMC ATCA My emphasis is on how the ATCA-AMC specifications were used to develop mTCA and describe what was necessary to port as many features as possible to mTCA. Also, describe what didn’t make it. I think by understanding the ATCA carrier card the relation to mTCA will be clearer 4 Dec 2007 DESY Crate STD. Workshop

DESY Crate STD. Workshop Advanced TCA Telco driven specification 20 year lifetime - Telco issue High Availability - typically 5 to 6 nine’s Redundancy Hot swap of everything System management fundamental to specification Monitor Control Industrial support Telco shift from internal to external supplied hardware Non-Telco uses of specification Lets start with a quick look at AdvancedTCA. I presume that you already have some familiarity with this piece. The lifetime requirements for TELCO should make us more comfortable since we also need long life systems. SM means that one can have “swivel chair control” (Chuck Byers of Alcatrel-Lucent) - I.e. remote operation of system The technical director of PICMG told me that they have been pleasantly surprised at the amount of non-Telco interest in ATCA. 4 Dec 2007 DESY Crate STD. Workshop

Status of ATCA Hardware Typical start-up curve for new “standard” At knee transition from R&D to production Year Volume 2002 2007 “Hockey Stick” Curve ~$1x109 Common curve that has existed for all standards. Hype at beginning was usual and overly optimistic. In the case of ATCA the knee has occurred at about $1B (less in €) 4 Dec 2007 DESY Crate STD. Workshop

DESY Crate STD. Workshop High Availability This is an issue for large systems that have to “stay up” - not just Telco Research machines Military Access limited critical electronics HA does not ignore Reliability but needs architectural features to implement If an HA failure has occurred then MTTR becomes important operating on Reliability only until repaired. HA especially important for Control Systems necessary to diagnose failures The point here is that operational issues can impact HA. 4 Dec 2007 DESY Crate STD. Workshop

DESY Crate STD. Workshop ATCA Features Redundant shelf controllers - Shelf Manager Redundant Power Redundant links Redundant modules Update channels - module to module direct link Hot swap Mezzanine card - AMC No busses! Soon - JTAC on 4 unused pins in power connector AMC also has HA features No busses should be tempered. One can have a few optional bused or radial lines. The choices need to be evaluated as to the impact on HA. 4 Dec 2007 DESY Crate STD. Workshop

DESY Crate STD. Workshop ATCA Hardware Board Sizes Front - 322.25 mm x 280 mm (8U front panel) Rear - 322.25 mm x 70 mm Pitch - n x 6HP (30.48 mm) Power Front - 200 W Rear - 20 W Temperature Specifications at 55°C air temperature Maybe 2x power at 20°C Air ∆ - 10 °C shelf inlet to outlet Power pins rated for 9 A at 10 °C ∆ If temperature was lowered could increase power - IF proper cooling design if module Power pin ratings allow for increase in power. Some discussion in the PICMG 3. Committee to speck a power option where the temp is lower. 4 Dec 2007 DESY Crate STD. Workshop

DESY Crate STD. Workshop Shelf Management IPMC Shelf Mngr Fan Tray Power Entry IPMB (2x) Key to ATCA specification IP - E-net (2x) 4 Dec 2007 DESY Crate STD. Workshop

DESY Crate STD. Workshop Module Management Need to start talking about AMC’s - a bit of a mix with ATCA but there is no clean way to divide the topics. Radial best for HA (bus vs radial choice) IPMC - Intelligent Platform Management Controller IPMB - Intelligent Platform Management Bus Shelf manager communicates with host thru Ethernet 4 Dec 2007 DESY Crate STD. Workshop

New AMC Size & Terminology Board Heights Extended Full-height  Full-size - 6HP Panel Full-height  Mid-size - 4HP Panel Half-height  Compact - 3HP Panel Board Width Single-wide  Single (or assumed to be Single if not specified) - ~75 mm Double-wide  Double - ~150 mm Board Size Single - ~75 mm x 180 mm Double - ~150 mm x 180 mm Extended Full-height was Full-height and Full-height (Mid-size) did not exist at first Note: in double AMC’s there is no connection in what would be the second connector position! To me the terminology has been confusing. The height seems to me to be the width, e.g. the width of the front panel. The width is the height of the front panel. We are stuck and might as well get used to it. 4 Dec 2007 DESY Crate STD. Workshop

DESY Crate STD. Workshop AMC Electrical Voltage 12 V for payload 3.3 V for management Current 0.75 A per pin - 8 pins Power Power derived from Carrier Module Single Double Compact 24 W 48 W Mid-Size 30 W 60 W Full-Size 80 W Note that current only allows 72 W - spec under review. All AMC power usage has to come out of the 200 W allowed for ATCA Aside: IBM has specified AMC’s in their new server blades. Blade picks up a lot of ATCA, not plug compatible. 4 Dec 2007 DESY Crate STD. Workshop

AMC Port Mapping Strategy Extended Options Region AMC Pin Assignments MMC GbE SAS Sync Clocks JTAG 12V PWR 3.3V MP 85 86 170 Common Options Region TCLKB FCLKA 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 TCLKC/D 17 18 19 20 TCLKA AMC Port Mapping Strategy Fat Pipes Region MISC Extended Options Region PCIe GbE/10GbE PCIe on 4-7 GbE or XAUI on 8-11 Basic Connector RTMs Extended Connector Connector in two versions - 85 single side or 170 double side. Note: ONLY one connector per AMC - in double width the “other” connector does not exist. MMC - Module Management Controller SAS - Serial Attached SCSI XAUI - X = 10gig, AUI - Attachment Unit Interface Note that JTAG has existed but no standard way to connect it to the system Clocks are for Telco but no reason not to use them for research, e.g.synch to beam 4 Dec 2007 DESY Crate STD. Workshop

DESY Crate STD. Workshop AdvancedTCA Carrier ATCA module with 6HP panel Standard ATCA power Standard ATCA management Specified for : 8 Compact Single AMC’s 4 Mid-size Single AMC’s 2 Full-size Double AMC’s Many combinations between all Compact and all Full-size Double 12 and 3.3 Volt converter for AMC power IPMC controls IPMB-L for control of AMC Fabric switch connects AMC’s to Zone 2 Extended options can connect to Zone 3 4 Dec 2007 DESY Crate STD. Workshop

ATCA Carrier Card AMC Links Zone 1 Switch Zone 2 This drawing only addresses links. Note that as many as 8 AMC’s could be here Zone 3 AMC 4 Dec 2007 DESY Crate STD. Workshop

ATCA Carrier Power for AMC’s 12V Current Limit AMC 48V Power Zone 1 MMC 3.3V Zone 2 Module Management Controller Zone 3 Current Limit AMC MMC 4 Dec 2007 DESY Crate STD. Workshop

ATCA Carrier AMC Management Zone 1 IPMC I MMC IPMB Zone 2 IPMB-L MMC = Module Management Controller I = Isolater Could also be radial with protection in IPMC Zone 3 AMC I MMC 4 Dec 2007 DESY Crate STD. Workshop

Status of µTCA Hardware Startup for µTCA if growth is “normal” “Hockey Stick” Curve Volume Where is mTCA in development? Much hype but cooler heads think the above is most realistic. Knee occurs usually at 5 years. Might be a bit sooner since much has been proven in the ATCA/AMC world already knee 2006 2010/11? Year 4 Dec 2007 DESY Crate STD. Workshop

DESY Crate STD. Workshop µTCA Hardware Shelf for AMC modules Single Tier* * = more detail later Two Tier Mixed Width Two Tier Fixed Single Width Back-to-Back Cube* Pico Electrical Signals per AMC Radial power - 3.3 V and 12 V Will show in more detail later Not enough time to do all but these two extremes should let one evaluate the other options. 4 Dec 2007 DESY Crate STD. Workshop

DESY Crate STD. Workshop ATCA relation to µTCA The next slides take the ATCA specification and relate it to the hardware needed in µTCA The ATCA carrier module is the basis to relate the requirements set in ATCA into the µTCA world. First the AMC Module Management functions Next the AMC connectivity as is it implemented in µTCA Carrier power functions for AMC’s that has to be translated to µTCA for hot swap Shelf Management hardware 4 Dec 2007 DESY Crate STD. Workshop

µTCA Module Management SM/IPMC Typically in Hub 4 Dec 2007 DESY Crate STD. Workshop

µTCA Carrier Card AMC Connectivity Zone 1 Switch Zone 2 µTCA Hub Module Zone 3 AMC 4 Dec 2007 DESY Crate STD. Workshop

Radial Power Connections µTCA Carrier Power 12V Current Limit AMC 48V Power Zone 1 MMC Input Power DC or AC µTCA Backplane 3.3V Zone 2 Radial Power Connections µTCA Power Module Zone 3 Current Limit AMC MMC 4 Dec 2007 DESY Crate STD. Workshop

DESY Crate STD. Workshop µTCA Shelf Manager Up to 16 µTCA Carriers per Shelf Manager Shelf Manager is a logical function Interface between Carrier Manager and SM is logical, typically IP based Implemented on any FRU MMC’s (Module Management Controller) in all AMC Modules MCMC’s (µTCA Carrier Management Controller) in µTCA Carrier Hub (MCH) EMMC’s (Enhanced Module Management Controller) in power supplies and cooling units. FRU - Field Replaceable Unit 4 Dec 2007 DESY Crate STD. Workshop

Management of µTCA Shelf 2 max. 4 max. System Manager Carrier FRU Info Device EMMC Cooling Unit EMMC Power Module I2C IPMB-0 IPMB-L PS1#(1:16) Enable (1:16) MCMC MCMC AMC1 MMC AMC12 MMC Carrier Manager MCH MCH Shelf Manager IP 4 Dec 2007 DESY Crate STD. Workshop

µTCA Hardware Examples Backplanes Shelves Industrial Military System Modules Power Module Hub Module Cube Alternate “µTCA” shelf 4 Dec 2007 DESY Crate STD. Workshop

DESY Crate STD. Workshop ELMA µTCA Backplane 16 slot backplane, 26 layers Dual star 10 AMC 2 PM 2 MCH Will look at Hub and Power Module in next slide. 4 Dec 2007 DESY Crate STD. Workshop

CorEdge Networks Hub & Power Module Note connectors on Hub Power feed from front 4 Dec 2007 DESY Crate STD. Workshop

DESY Crate STD. Workshop ELMA µTCA Shelf Uses previous backplane or others 5U chassis Front air intake - rear exhaust 4 Dec 2007 DESY Crate STD. Workshop

Schroff µTCA Shelf (Cube) Front view 4 Dec 2007 DESY Crate STD. Workshop

Schroff µTCA Shelf (Cube) Rear view with power unit open Note that logic has been placed on the backplane - maybe not good but does save money? 4 Dec 2007 DESY Crate STD. Workshop

Schroff 5 Slot Cube Specs MicroTCA Backplane with radial IPMI from MCH in Slot1# to all AMC slots, supporting: 1 MCH Single Full-size slot 1 AMC Single Full-size slot for a CPU Board 2 AMC Single Full-size slot for HDD or PCIe Boards 1 AMC Single Full-size slot for a HDD or CPU or PCIe Board Onboard power management for all AMC slots Active cooling through a 12 V Fan (170 m3/h (100 CFM)) for the MicroTCA Boards and a 12 V Fan (13,5 m3/h (8 CFM)) for the power supply Integrated 150 W AC Power Supply with wide range input 4 Dec 2007 DESY Crate STD. Workshop

Hybricon Military ATR Box Work on-going. Hybricon is a major player here. 4 Dec 2007 DESY Crate STD. Workshop

Hybricon ATR Spec Sheet AC/DC front end power supply supporting 110/220VAC operation with 48VDC output to DC MicroTCA Power Module. Shock Mount MicroTCA card cage inside the outer enclosure. 150MM Front Row 3 Full Size Double Width (150MM) slots for 150MM AMC Modules 2 Full Size Double Width slots with conversion module to convert to (2) Single Width slots 75MM Rear row 2 Full Size Single Width (75MM) Slots for 1 DC or AC MicroTCA Power Module (1) Compact Single Width (75MM) AMC slot (accessible only with DC power supply) 1 Full Size Single Width (75MM) slot for MicroTCA carrier Hub (MCH) 4 Compact Single Width (75MM) AMC Slots Designed to cool 80W per double width full height slot and 40W per single width full height slot at 10,000 feet altitude Simulated 36 CFM average flow rate per full size slot tools 80W per double width (150MM) full size slot, in addition to 40W per single width (75MM) full size slot, and 20W per single width (75MM) compact slot at 10,000 feet altitude. Pressurizing commercial grade fans mounted to card cage at front panel with honeycomb air inlet panel and application specific I/O connectors 900W Embedded AC/DC Power Supply: MIL-STD-461 Line Voltage Filter and EMI Containment. Data sheet for previous slide 4 Dec 2007 DESY Crate STD. Workshop

DESY Crate STD. Workshop µTCA in AdvancedTCA Horizontal ATCA Shelf Use ATCA Carrier Modules Get rear Zone 3 access for AMC’s Can use RTM Shelf Manager can be used 3 Slot Shelf 2 Carriers with 8 compact single AMC slots each (16 total) 1 network module “Simple” backplane This might be an alternate to mTCA. Uses ATCA in a mTCA way. Should be cost effective and give one the rear I/O. 4 Dec 2007 DESY Crate STD. Workshop

DESY Crate STD. Workshop µTCA in AdvancedTCA 2 Slot Horizontal Shelf Front Sorry about the number of slots but this gives the general idea. Rear 4 Dec 2007 DESY Crate STD. Workshop

Standards vs Specifications xTCA Documents are Specifications, not Standards Specifications are sometimes used by industry because of the bureaucracy of the governing bodies - time-to-market Standards are governed by organizations such as ANSI, IEEE, IEC, etc. Rules on changing standards Review every 5 years Gives stability Specifications are governed typically by industrial groups Rules on change are set by group writing specification Review are up to group Stability Not an issue with a large industrial group that has long view Could be an issue with small group without long market view xTCA will be stable because of the constituency of the governing group Legalism’s 4 Dec 2007 DESY Crate STD. Workshop

DESY Crate STD. Workshop xTCA Related Groups PICMG The starting point for xTCA - they “did it” www.picmg.com SAF - Service Availability Forum Formed by some ATCA developers and others High Availability and management software interface specifications www.saforum.org Scope Alliance ATCA specifications are very broad with many options Define a common subset to promote interoperability Recognized that there were choices being made by most suppliers Documents” AdvancedTCA HW Profile, v2.0 10 May 2007 Linux Profiles, latest v1.2 26 Feb 2007 AMC Port Map Gap-Analysis, v1.2, 20May 2007 Others www.scope-alliance.org PICMG - good place to search for products. Click on “Product Listings”. Can also get short form specs (if not a member) from “Specifications” 4 Dec 2007 DESY Crate STD. Workshop

Physics “SCOPE” Document Based on two SCOPE Alliance documents HW Profile AMC Port Gap-Analysis Intent Engage research community in a discussion on a common set of guidelines for building xTCA hardware Try to optimize the exchange of designs, hardware, etc. Content of Document Minimize changes from SCOPE document and track them Add items such and hardware for RTM’s and Zone 3 connectors Propose alternate rack and shelf cooling systems - fluid based Further restrict network protocols to PCI-e and E-net Needs: software section to be expanded on. 4 Dec 2007 DESY Crate STD. Workshop

ATCA for Physics Collaboration Proposed Organization Ray Larsen distributed this recently for comment 4 Dec 2007 DESY Crate STD. Workshop

DESY Crate STD. Workshop Request Please look at “straw man” document Give feed back so a revision 2 can be produced Information on prototypes is of use to the document writers Suggest other topics that could be included Currently it is light on software, too early? Should software be in another document? More information on analog design experience 4 Dec 2007 DESY Crate STD. Workshop

DESY Crate STD. Workshop Conclusions xTCA family offers an excellent next generation platform to build on Industrial support ramping up Long lifetime because of Telco support Telco will be first to drive high volumes, hence lower costs Lots of interest from other sectors of the modular electronic market - more non-Telco hardware Software support for xTCA - something new for new modular electronics in our business - VERY GOOD! Caution, some potential incompatibilities exist in taking AMC’s to µTCA 4 Dec 2007 DESY Crate STD. Workshop

DESY Crate STD. Workshop Thank You Robert Downing SLAC Consultant rwd4@mindspring.com Maybe mention RoHS? BGA’s appear to be a real problem. This is a big reliability issue. Consumer part of industry doesn’t care since they like rapid replacement. 4 Dec 2007 DESY Crate STD. Workshop