1. Review of CMN Problem/Studies
Ariella requested me to review the current understanding of the CMN problem
What are the symptoms
What studies have been done
What we think the most likely source of the problem is
Also discuss test results of the problem modules test in the current ARCS/LT analysis software
How/will the software flag the CMN problem modules
Will the software be able to find all of these problems prior to installation in rods/CMS

2. Symptoms of CMN Problem
A channel develops an extremely high noise
When the channel’s noise is higher than ADC, the entire chip begins to oscillate
Common mode noise seen, which is not always correctly subtracted
At this point, excess bias current always seen
As low as 0.5 mA excess has been seen to cause the problem
Excess current needed to start an oscillation has varied between mA
Consistent with micro-discharge; dependent on frequency of current spectra of discharge
Problem has (almost) always occurs on first test during voltage ramp up
1 module at FNAL developed problem during module burn-in with no sign of problems on effected channel
Increased bias current (almost) always seen in IV reprobing
1 module at UCSB developed high current after assembly
Rules out module production and assembly as cause of problem
The voltage at which the problem begins has had a wide distribution
Between ADC
No obvious visible damage seen on channels
On three modules probed at Karlsruhe, the bulk of the excess current seen on channel with the increased noise

3. Current CMN Status
15 of 73 (20%) modules produced at UCSB have CMN problem
Once notified of problem, FNAL found 6 of 15 previously tested module also had CMN problem
In addition, 1 module developed problem during module LT testing
1 TEC module module tested at Karlsruhe also found to have problem
2 test beam modules also shown to have CMN problem
See L. Borrello’s talk in sensor meeting
1 of 6 ceramic hybrid modules in test beam
1 of 6 flex hybrid modules in test beam
Both built and test prior to knowledge of problem

4. CMN Turn-on Voltage

5. CMN problem vs. voltage
Once, IV diverges from QTC expections, noise on channel increases rapidly causing CMN at V above the divergence point
IN ALL CASES THERE IS NO INDICATION OF NOISE BELOW THE DIVERGENCE POINT OR UNUSUAL LEAKAGE CURRENTS IN QTC PROBING

6. Are the faults caused by assembly?
Extensive program of sensor re-probing and additional module IV measurement undertaken
Sensors probed prior to assembly in modules
Sensors with >5 mA extra current relative to sensor QTC measurement separated from others
Module then assembled and bias bonded to first sensor
IV measured
Bias is bonded to second sensor
IV re-measured
Module is then fully bonded and tested
During all measurements, environment controlled
Temperature between C
RH <30%

7. IV Correlation with CMN problems
Significant differences from QTC sensor probing have been found
~7% of sensors have current increases >5 mA from QTC prior to module assembly
Roughly consistent with the rate of occurrence of the CMN problem (aka micro-discharge) observed at various production sites
The increased current occurs during ramp up during IV probing
Production Results with IV Pre-Screening
Of the 39 modules produced with sensors whose IV curves in the QTC database matched those obtained in UCSB re-probing, only 2 showed CMN problem (5%)
1 module showed increased currents in some tests and regular currents in others so the problem appears to be intermittent
Another showed CMN problem with only 0.5 mA extra bias current
Of the 5 modules with sensors whose IV curves in the QTC database with 5 extra mA of current from those obtained in UCSB re-probing, 4 had serious CMN problems (80%)
Rules out hypothesis that problems due to mishandling in US
Indicates any change in IV curve relative to original QTC measured a good predictor for sensors that will cause this problem

8. Increased IV In Re-Probing

9. Same Currents in IV Re-Probing

10. IV Re-Probing
Pisa(~75), Perugia(~150), UCSB(~250), FNAL(~350) and UR(~60) all have begun extensive re-probing program
See sensor meeting
Plan on re-probing all sensors not in modules yet
6-8% of sensors re-measured from at all five sites have a 5 mA increase in current
Most modules built with these sensors will have CMN problems
If we had not re-probed, we would have 10-20% modules with this problem now
We have NO understanding of the cause of the change in the bias current
We DO NOT know the time constant/rate for the development of increased
Therefore, we do not know if more sensors would develop higher currents once built into modules, rods, detector
FNAL
PISA

11. FNAL Re-Probing
At a 1.5 mA current increase relative to DB, a significant fraction of ALL sensors fail
Less in 2003 sensors, but still significant
Increase in time or change in processing?
At a 5 mA current increase relative to DB, 2003 sensors look better
But is the fraction of sensors failing increasing with time or change in processing?

12. CMN vs Batch
Sensors which cause CMN are fairly evenly distributed throughout production years
Early indications are that 2003 may be better
Extremely low statistics
Only low bias current sensors used

13. Common Mode Subtracted Noise (Peak Off)
25 ADC
6.5 ADC
Modules with CMN (micro-discharge problem)
Common mode subtracted noise in blue
For majority of modules with problems, the common mode subtraction is imperfect.
7 of 12 have >2.0 ADC noise
3 of 12 have >3.0 ADC noise (Two times regular noise)

14. Common Mode Subtraction Variation
Module 1016
Module 1010
Common mode subtraction results inconsistent
Answer differs mode-to-mode or test-to-test
Would yield varying signal efficiency/noise during data taking
Not clear how this will evolve with time/radiation

15. Study of Common Mode
The common mode point is calculated event-by-event for groupings of 32 channels
The spectra of the common mode is fit for groupings within a chip with CMN problems
Excluding the grouping with high noise channel
Spectra is fit with two Gaussians
Central core plus tail
Fit parameters are:
Fraction of events in tail
Width of central core
Width of tail
Study how parameters vary with current

16. CMN Problem Module After Re-Probing
Last SS6 module built using one sensor with 1.2 mA extra current (450 nA vs 1700 nA) in UCSB re-probing at 450 V.
Well within old selection criteria
No large addition increase in current during module assembly
Old sensors
CMN seen in chip 46 with extremely high noise in channels
Sensor flaw seen between two channels
Not clear if flaw cause of problem
Begins at 400 V where database and measured bias current diverge
~0.5 mA difference

17. CMN Problem Module After Re-Probing
Module tested at slightly elevated voltage to measure effect as function of current
Bias current 3.7 mA, < 2 mA more than expected from database
For first half of chip, CM subtracted noise a factor of ~1.75 higher than typical noise.
A very little amount of micro-discharge can cause the CM subtraction algorithm not to work properly
CM subtraction algorithm used is same as LT, and test beam software

18. CMN Problem Module After Re-Probing
Micro-discharging strip dis-appears/appears randomly
But always with the same 2 IV curves
It IS NOT clear if module which is tested today will be good tommorrow

19. Module 705 before LT (FNAL)
After assembly module was tested (09/08) on ARCS at 400 V and graded “B” (6 faulty channels). No problems observed.

20. Module 705: LT data (FNAL)
Data was taken at 20°C on Sept.19 (10 days after the first test). A group of high noise channels is seen around channel 219 and increased CMN is seen in chip #2

21. Comparison of IV curves (FNAL)
”before” measurement is taken on 09/08 on ARCS before LT
“after” measurement is taken on 09/23 on ARCS after LT
green curve is a measurement done using Keithley on 09/24 with 1 minute interval between steps
No visual defects are observed on the sensors around noisy channel #219
We know it is not a humidity effect: 3 weeks in dry air did not cure it

22. “Fix” For CMN Problem Modules
In most cases, the CMN can be removed from chip by:
Removing bond from effected sensor
Adding Bond between AC pad (AL strip) and bias ring
Uses the coupling capacitor as a high-frequency shunt of the increased current
Thus, neighbors do not see noise
Increases noise on sensor edges due to increased current on bias ring
Does not increase (or decrease) current drawn by module
The long term stability of this fix is not known
Cannot apply this fix once installed on rods, petals, or in detector

23. How Does ARCS React to CMN Problem
25 ADC
6.5 ADC
With standard fault finding, only CMS noise would flag problem
Of 15 modules, 7 would be graded A or B
Since CMN varies with time and mode, grading varies with time
Module’s high current would generally indicate a problem though.
Common mode subtracted noise in blue

24. How To Modify Programs To Increase Sensitivity of CMN
Add grading due to bias current directly into program
Add flag of major problem if noise of any single channel above 20 ADC
Add flag of major problem if average raw noise of a chip above 2 ADC (Peak) and 2.5 ADC (Deconvolution)
The average raw noise already in output can quantify size of CMN
RMS of CMS noise per chip could be used as an indicator of how well the CM subtraction works on module
BUT AS A REMINDER, THERE IS NO GUARANTEE THAT THE PROBLEM WILL BE THERE AT THE TIME OF THE FIRST TEST !!!
1 module at UCSB has the problem coming and going randomly
1 module at FNAL developed the problem during LT test

25. Conclusions We are seeing time evolution of the sensors
These sensors cause CMN noise on a chip with a turn-on distribution between V
The noise is not always subtractable
The noise varies with time significantly
It would be a nightmare to commission/operate a detector of this size with ~5% modules with this effect
With pre-probing, the rate of the problem is reduced
BUT there is NO way to know if the sensors will continue to evolve
The current testing protocol will find the problem if it exists at the time of the test
But there are many good reasons to believe that many modules WILL NOT have the problem at the time of testing, but WILL develop it later

29. 25 ADC
6.5 ADC

30. IV Test Results (UCSB)
Probed UCSB (400 V) – QTC Measurement (400 V)
Sensors
> 2 mA
> 5 mA
>10 mA
>20 mA
>100 mA
< -2 mA
<-5 mA
<-10 mA
OB2 (’01-02)
15% 9% 8% 5% 1% 3%
OB1 (’01-02) 6% 0%
OB2 (’02) 2%
OB2 (’03)
Environmental conditions tightly controlled
Temperature C
RH < 30% at all times
An increase greater than 5 mA can cause CMN
Much better results with newer OB2 sensors (2002)
None of the 20 newest (2003) OB2 sensor show any increase in bias current!!!

31. Could Grounding Cause The Problem?
It is extremely unlikely that grounding could create or enhance the CMN problem
LV and HV supplies floating
Same as the final detector
Clamshell
Module holding plate in clamshell but isolated
> 1cm from metal shell
Grounding achieved with large gauge wire to hybrid-to-utri adaptors
Four grounding schemes studied
Grounding clamshell/module carrier
Grounding used chosen because it minimizes the CM noise and sensitivity to environment
Only changes made relative to standard test stands at time were:
Grounding hybrid-to-utri adaptor to test box instead of module testing plate
Use of a thick, continuous, metal shield
Most centers now have use the same grounding scheme

32. Noise vs. Grounding Raw Noise CMS Noise CMN seen in both ARCS LT
Module 1016 (ARCS)
Module 1016 (LT)
Raw Noise
CMS Noise
CMN seen in both ARCS LT
Answer differs mode-to-mode, test-to-test, test stand-to-test stand
Grounding vastly different
ARCS use floating power supplies and “star” grounding
Only one common point
LT have non-floating power supplies and everything is grounded to everything else
Multiple ground loop

33. Why isn’t problem seen at test beam?
Most of modules pre-screened against the CMN problem prior to shipment to CERN
All of UCSB modules sent either do not have problem or have had enough channels pulled in order to remove problem
Most of FNAL modules sent after they began pre-screening for problem
Only 6 modules made with flex hybrids sent without testing at 400 V
~50% of the time, 0 of 6 modules would not have this problem
Assumes that the rate of problem and distribution of CMN turn-on voltage constant
We know that many sensor effects are severely batch dependent
Many possible reasons why not seen in old prototype modules
Many circuits in prototype ceramic hybrids have been changed
Testing methods have changed
Maybe batch of sensors used in production did not have the problem
Well, it was in test beam. 2 with turn-on voltage of 400 V
See L. Borrello’s talk in sensor meeting

34. Fit Result of Common Mode Point
Fraction of events is flat with bias current (~strip current)
Width of central core increases with bias current (~strip current)
Width of tail increases with bias current (~strip current) and may flattens out at some current

35. Problem: TOB 7 (2003 data) No leaky strips observed on sensors.
A few noisy channels (strips in breakdown appears at very high voltage V
Depletion voltage for TOB 7 is 160 V
→ Working voltage is 240 V
Noise level of APV3 is bad for V>400V.
The CM subtraction algorithm still
works fine, limiting the loss of efficiency
to a few channels.
By lowering the voltage to normal value
the effect disappears.
No effects on performance for the
following runs. If the detectors are not
left in breakdown for days and the total
current is limited, the damage is
reversible.

36. TOB 7 (2003 data): more on APV3
At 400 V APV 3 showed different noise behavior
Other APVs are OK
CM sub. noise

37. TOB 3 – 2002 Data Run at 450V (Vdep @125V)
Raw noise
Noise
Noise vs time
By operating the module at an excessive bias (recommended Vop is 1.5Vdep) a local breakdown may occur. One strip drains tens of microamp and appears as a really leaky strips with a common mode noise affecting the entire chip (current fluctuations that induce significant voltage drop through the polysilicon resistors ( MW))
NEVERTHELESS THE CM SUBTRACTION ALGORITHM STILL WORKS FINE AND ONLY A SMALL NUMBER OF STRIP IS AFFECTED.
WHEN THE OPERATING VOLTAGE IS BROUGHT BACK TO A NORMAL VALUE THE EFFECT DISAPPEARS.