Status of the Tracker Outer Barrel J. Incandela University of California Santa Barbara for the TOB Group Tracker General Meeting December 7, 2005 Slides courtesy of: A. Affolder, P. Bhat, C. Campagnari, Y. Gotra, A. Onnela, A. Satpathy, L. Shabalina, L. Spiegel, D. White, and others…

Tracker Outer Barrel- December 7, 2005 – Tracker General Meeting slide 2 US Module Assembly Past Week Cumulative ABCABC% A or B L12pu % L12pd % L12su % L12sd % L34p % L56p % R5N % R5S % R % R % Total % Current inventory of components sufficient for > 5 weeks production 1.3% C grade modules are mostly very near to spec. (~40/49 will be reclassified for use in rods)

Tracker Outer Barrel- December 7, 2005 – Tracker General Meeting slide 3 US Hybrid Testing Status

Tracker Outer Barrel- December 7, 2005 – Tracker General Meeting slide 4 US ARCS Testing Status

Tracker Outer Barrel- December 7, 2005 – Tracker General Meeting slide 5 US ARCS Testing Status 1787 bad channels out of % (UCSB) 0.09% (FNAL)

Tracker Outer Barrel- December 7, 2005 – Tracker General Meeting slide 6 US ARCS Backplane & LT Testing LT Summary 3 failures out of 3729 tested so far Continuing at about 50% sample rate since October LT Summary 3 failures out of 3729 tested so far Continuing at about 50% sample rate since October

Tracker Outer Barrel- December 7, 2005 – Tracker General Meeting slide 7 Module Assembly Planning FNAL: Current rate is TOB modules/day. Finish TOB modules Jan/Feb then TEC R7 UCSB Current default modules/day Allows time to do L5 and L6 retrofits Do mix of TOB and TEC until end of April To do list: Bias connections on some 1,700 modules. Wire bonding and ARCS operations (but not LT).

Tracker Outer Barrel- December 7, 2005 – Tracker General Meeting slide 8 New problem found UCSB 1 rod with three modules with saturated channels and one module with high current 1 rod with one module with a saturated channel Single channels at the edge of wafers Always at the very first test (SRT) FNAL 1 DS rod with one module w/saturated channels 1 rod with three modules w/ saturated channels on a non-production rod 1 rod had a module w/ saturated channels on a non-production rod Groups (~3) of channels in middle of wafer Always on MRT after SRT test OK During tests of several tens of rods, last ~ 2 months

Tracker Outer Barrel- December 7, 2005 – Tracker General Meeting slide 9 Damage on input to APV Saturated channels show damage on APV input Silicon channel itself is OK Check by bonding to a functioning neighbor APV channel UCSB mod 6046 FNAL mod 9103

Tracker Outer Barrel- December 7, 2005 – Tracker General Meeting slide 10 Corresponding Damage on Sensor Damage Location UCSB Module Damage is at the edge of the n+ region at boarder of sensor, (at HV), directly under bias bond, (at ground) Bias bond is lower than channel bonds Nearest signal channel saturated

Tracker Outer Barrel- December 7, 2005 – Tracker General Meeting slide 11 Higher magnification (with bias bond moved aside) No visible damage to APV APV channel 768 dead Bias current not changed No visible damage to APV APV channel 1 dead Bias current not changed

Tracker Outer Barrel- December 7, 2005 – Tracker General Meeting slide 12 FNAL: same story Channel jumper bonds have less clearance than bias bonds Damage again seen under bonds corresponding to broken APV channels Bond shape asymmetric Top picture: 2 nd sensor Where bonds are low Bottom picture: 1 st sensor Bonds higher perhaps passivation flawed here?

Tracker Outer Barrel- December 7, 2005 – Tracker General Meeting slide 13 Hypothesis HV breakdown between metal over n+ and bonds At UCSB, breakdown to bias bond, since it is lowest Nearest channel is then "zapped" through capacitive coupling… At FNAL, breakdown to lowest signal bond Bonds to PA much higher Explains why it is only seen for the jumpers between sensors Raises some obvious questions: Why haven't we seen this before? Could we have missed it? How widespread might this issue be? i.e. How high are our bonds? What can we do about it?

Tracker Outer Barrel- December 7, 2005 – Tracker General Meeting slide 14 How high are the bonds? TOB bond clearance over n+ region (Not loop height) UCSB measurements shown (include some FNAL modules) FNAL measurements ~ agree

Tracker Outer Barrel- December 7, 2005 – Tracker General Meeting slide 15 TEC modules built at UCSB They appear to be a little bit less vulnerable than TOB modules built at UCSB Have not checked carefully that the problems reported for TOB modules do or do not occur for TEC TEC R5 bias 152  m 150  m TEC R6 bias 138  m 143  m TEC R7 bias 138  m 141  m new bias 247  m 289  m

Tracker Outer Barrel- December 7, 2005 – Tracker General Meeting slide 16 Breakdown study Increase HV up to 1000 V on UCSB TOB module. Ramp as fast as possible. No breakdown Plug and unplug HV wire at 1000 V Breakdown after several tries Intentionally lower the bias bond (in steps) and ramp HV for each bond height Breakdown at 40  m height occurred at HV=800 V

Tracker Outer Barrel- December 7, 2005 – Tracker General Meeting slide 17 HV transients Suspect breakdown is caused by HV transients Power supplies are CAEN A132 or A332: (6kV) All "accidents" happened on A132 This supply does not have HV limiting Monitored for HV transients No real success Saw HV go from 0 to 80V in 100  sec once when 1 st establishing SW communication – not reproduced See occasional spurious front panel or SW readings up to 8kV In at least one case we were looking at HV with a DVM when this occurred and did not see anything

Tracker Outer Barrel- December 7, 2005 – Tracker General Meeting slide 18 Initial response New modules Modify bond parameters for max. clearance above n+ region Existing UCSB Modules Remove all sensor-to-sensor bias bonds on and remake them longer and much higher over n+ region. Rod production at UCSB thus temporarily suspended Existing FNAL Modules Inspect sensor-to-sensor bias bonds and pull lowest if less than 200  m above the n+ region Rod Testing All rod test stand HV lines equipped with crowbars and current limits reduced 95%. Continued rod production at FNAL for ~ 4 weeks, with no further accidents

Tracker Outer Barrel- December 7, 2005 – Tracker General Meeting slide 19 Further Information Our worry - local electric fields could be high enough to cause ionization (corona, partial discharge) even if no sparks. What happens? particularly over long periods? Therefore decided to carefully inspect modules which did not have breakdown incidents, and which were never mounted on rods See localized damage !

Tracker Outer Barrel- December 7, 2005 – Tracker General Meeting slide 20 Damage under UCSB bias bonds D amage on 2 nd sensor of all TOB modules! At minimum, dots seen at edges of guard ring and n+ metal directly under bias wire. At maximum, lines of metal(?) deposited in the area between the guard ring and n+ region. Again, directly under bias wires. Beginnings of this trend already appears after 20 min on ARCS test Guard Ring n+ metal

Tracker Outer Barrel- December 7, 2005 – Tracker General Meeting slide 21 Bias Wire Bonds Bias bonds have deteriorated, displaying pock marks, in portion of wire situated just above the outside edge of the bias ring up to halfway between guard ring and n+ metal!

Tracker Outer Barrel- December 7, 2005 – Tracker General Meeting slide 22 Current status and proposed plan Expect problem can br overcome with encapsulation Study underway at UCSB: 4 groups of sensor to sensor bonds High - Low - High – Low One High-Low group encapsulated the other not. Run overnight in LT Pull encapsulant and compare to non-encapsulated region for sensor marks, wire deterioration If positive, will start to encapsulate modules Both PA-Sensor and Sensor-Sensor regions Do enough modules to populate one cooling segment of TOB outer layer Continue additional studies in parallel Thermal cycles of encapsulated modules with final wirebond program Look for corona with pmt Disect and scan discharge burn area with electron microscope

Tracker Outer Barrel- December 7, 2005 – Tracker General Meeting slide 23 Meanwhile: UCSB Rod Status Last week: No rod production. Verified that crowbars work as advertised. Fuse blows at ~640 V Looked for transient voltage spikes from CAEN. 2 blown crowbars, both occurred after CAEN front panel showed voltage reading 0 V  >400 V in ~2 sec. Also not reproduced after ~15 tries. One crowbar modified so that output voltage can be independently monitored (with DVM and/or scope) during test operation. New crowbars now in preparation will also have this feature.

Tracker Outer Barrel- December 7, 2005 – Tracker General Meeting slide 24 FNAL Rod Production: 05-Dec rods assembled 45 passed single rod test 42 rods burned-in 15 shipped All rods will be rebuilt with final modules

Tracker Outer Barrel- December 7, 2005 – Tracker General Meeting slide 25 TOB Integration Installation and test of one cooling segment (19 rods) Learning/training exercise Improved/refined many procedures First look at noise performance

Tracker Outer Barrel- December 7, 2005 – Tracker General Meeting slide 26 Rod insertion and cabling

Tracker Outer Barrel- December 7, 2005 – Tracker General Meeting slide 27 Soldering of cooling lines Unexpected difficulties with the soldering: Brass connection pieces more oxidized and dirty than what we were used to. → Will prepare the surfaces more carefully. 3 weak solder connections found (10 -4 to mbarl/s) - among first solder joints. Connections disconnected, cleaned and resoldered: All connections then leak tight (alarm limit at 1x10 -7 mbarl/s)

Tracker Outer Barrel- December 7, 2005 – Tracker General Meeting slide 28 Electrical tests Low noise. The grounding scheme works! All bad channels found were pre-existing.

Tracker Outer Barrel- December 7, 2005 – Tracker General Meeting slide 29 Rod Tests on Barrel Noise level is acceptable Edge channels are often noisy but this will be fixed with new ICC and associated grounding 10% 0ptical connections were not correct Problem with exterior connections (not at rod) Now have 5 bad out of 342 with 4 from one ribbon Some bad modules/rods – one laser with high CMN, several pinholes etc. Found all known problematic channels

Tracker Outer Barrel- December 7, 2005 – Tracker General Meeting slide 30 Integration exercise Went mostly ‘as planned’. A few discoveries: Need better preparation of solder connections. Solder equipment also improved. Improvements to rod insertion tool Handling cables & connections is … time consuming. Access to rods will be much worse inside the TST. When is the best time to go inside? The 1 st technician team is now trained.

Tracker Outer Barrel- December 7, 2005 – Tracker General Meeting slide 31 To do Dismount pre-assembled 19 rod cooling segment (segment 2.6.2) Start final assembly, again with cooling segment 2.6.2, and train the 2nd technician team Preparation of new clean room for integration work He, dry air, C6F14 lines, tables, etc. Improvements to the tooling Develop a better work platform

Tracker Outer Barrel- December 7, 2005 – Tracker General Meeting slide 32 Rod installation tools  Small improvements to the 1 st rod installation tool  Make a 2 nd rod installation tool (currently in manufacture in FNAL)

Tracker Outer Barrel- December 7, 2005 – Tracker General Meeting slide 33 Tracker Monitoring Workshop The workshop was held on Nov. 15, 2005 (vrvs) Focus was on monitoring tools for tracker I&C at CERN. Speakers from FNAL, CERN, INFN/Pisa, UCL/Louvain Workshop achieved the goals Talks on CDF/DØ experience with Si detector commissioning; lessons learned, useful tools Included some thoughts about what might help CMS Talks and discussions on xDAQ and tools developed for tracker commissioning Rod test data and analysis at CERN Tracker data handling and monitoring framework and tools Remote monitoring/analysis experience at Fermilab for HCAL Procedures, communication, collaborative tools

Tracker Outer Barrel- December 7, 2005 – Tracker General Meeting slide 34 Tracker Monitoring Workshop: Plans Establish communication among interested parties Avoid duplication of effort Identify and help develop appropriate tools Gain experience with the standard DQM software (to be available by spring ’06) and help get it ready for use in I&C Establish a group at Fermilab Remote monitoring and analysis of data from I&C and to provide feed-back  participate in ~real time in data monitoring/analysis and debugging activities Help with building and/or improving diagnostics and other essential tools Follow-up meetings with I&C, xDAQ and DQM groups

Tracker Outer Barrel- December 7, 2005 – Tracker General Meeting slide 35 Summary & Conclusions Quickly diagnosed a very serious problem Will (almost certainly) encapsulate Modified rod stands for safety Rod production lines now robust Integration studies underway Again, many important lessons learned Generally things appear to be in good shape Establishing additional qualified manpower will be critical to schedule and quality These are very critical times Though we are weary, we have to keep doing top quality work

Additional Information

Tracker Outer Barrel- December 7, 2005 – Tracker General Meeting slide 37 FNAL & UCSB Inventory – 11/28/05 HybridsSensorsFrames FNALUCSBFNALUCSB FNA L UCS B STHPKSTHPKITSTHPKSTHPKIT L12sd L12su L12pd L12pu L34pu L56pu R5p R5s R R

Tracker Outer Barrel- December 7, 2005 – Tracker General Meeting slide 38 Damage under UCSB signal bonds Under all channel bonds, passitivation discolored at guard ring, bias ring, DC pad, and bond pad Guard Ring Channel bond location Channel bond location

Tracker Outer Barrel- December 7, 2005 – Tracker General Meeting slide 39 Could we have missed it? Arcs and LT tested 1000's of modules and never saw this. It was not overlooked – no bad channels developed that we did not understand were due to other problems (e.g. pinholes, shorts, opens) This problem showed up on rods We have less rod experience, but we’ve tested rods for over a year Emphasis was on DAQ debug, but we believe we would noticed this

Tracker Outer Barrel- December 7, 2005 – Tracker General Meeting slide 40 He leak testing Three weak solder connections found ( mbarl/s) All were among the first solder joints made. Connections disconnected, cleaned and resoldered: All connections then leak tight (alarm limit at 1x10^-7 mbarl/s)

Tracker Outer Barrel- December 7, 2005 – Tracker General Meeting slide 41 To do: Better work platform

Tracker Outer Barrel- December 7, 2005 – Tracker General Meeting slide 42 A better work platform

Tracker Outer Barrel- December 7, 2005 – Tracker General Meeting slide 43 He Leak check rods No leaks found (alarm limit mbarl/s)