University of California Los Angeles

Slides:



Advertisements
Similar presentations
CPT Week, Nov 2003, B. Paul Padley, Rice University1 CSC Trigger Status, MPC and Sorter B. Paul Padley Rice University November 2003.
Advertisements

Endcap Muon meeting: Florida, January 9-10, 2004 J. Hauser UCLA 1 TMB, RAT, and ALCT Status Report Jay Hauser University of California Los Angeles ALCT/Mezzanine.
TMB and RAT Status Report Endcap Muon OSU April 16, 2004 Yangheng Zheng University of California, Los Angeles  TMB Status  RAT Status.
CSC Muon Trigger September 16, 2003 CMS Annual Review 1 CSC Muon Trigger - Annual Review Jay Hauser, with many slides from Darin Acosta and Stan Durkin.
CSC Muon Trigger September 16, 2003 CMS Annual Review 1 Current Status of CSC Trigger Elements – Quick Summary Jay Hauser, with many slides from Darin.
Martin von der Mey, EMU at CMU, October ALCT and TMB Status Martin von der Mey University of California Los Angeles ALCT production statusALCT production.
CSC Synchronization Procedure and Plans CMS Endcap Muon FNAL October 29, 2004 Jay Hauser* / Martin Von der Mey / Yangheng Zheng University of.
Commissioning of CSCs and Peripheral Crates Task L M. Ignatenko UCLA October
Endcap Muon meeting: CMU, Oct 19, 2003 J. Hauser UCLA 1 CSC Trigger Primitives Test Beam Studies Main Test Beam 2003 Goals: Verify the peripheral crate.
DRAFT version Oct. 15, 2004 Hauser/Mey UCLA 1 Analysis of Oct. 04 Test Beam RPC Data Jay Hauser, Martin von der Mey University of California Los Angeles.
DRAFT version Oct. 15, 2004 Hauser/Mey UCLA 1 Analysis of Oct. 04 Test Beam RPC Data Jay Hauser, Martin von der Mey University of California Los Angeles.
US CMS DOE/NSF Review: June 2002, Darin Acosta, University of Florida1 Muon Track-Finder Trigger Darin Acosta University of Florida June, 2002.
CSC Trigger Report: Results from June 25 ns beam test Production plans Software Darin Acosta University of Florida.
Printed by Topical Workshop on Electronics for Particle Physics TWEPP-08, Naxos, Greece / September 2008 MEZZANINE CARDS FOR.
Status of the CSC Track-Finder Darin Acosta University of Florida.
Track-Finder Trigger at the Beam Test Results and Features Darin Acosta, Rick Cavanaugh, Victor Golovtsov, Lindsey Gray, Khristian Kotov, Alex Madorsky,
CSC Endcap Muon Port Card and Muon Sorter Upgrade Status May 2013.
CPT Week, April 2001Darin Acosta1 Status of the Next Generation CSC Track-Finder D.Acosta University of Florida.
CSC EMU/Track Finder Clock and Control Board (CCB’2004) Status Plans M.Matveev Rice University August 27, 2004.
Upgrade of the CSC Endcap Muon Port Card Mikhail Matveev Rice University 1 November 2011.
CSC Endcap Muon Port Card and Muon Sorter Status Mikhail Matveev Rice University.
CSC ME1/1 Upgrade Status of Electronics Design Mikhail Matveev Rice University March 28, 2012.
T.Y. Ling EMU Meeting CERN, September 20, 2005 Status Summary Off-Chamber Electronics.
US CMS DOE/NSF Review, May Cal. Trig. 4 Gbaud Copper Link Cards & Serial Link Test Card - U. Wisconsin Compact Mezzanine Cards for each Receiver.
US CMS DOE/NSF Review: June 2002, B.Paul Padley, Rice University1 CSC Muon Trigger On Detector Components B. Paul Padley Rice University June, 2002.
Track-Finder Test Beam Results Darin Acosta. Darin Acosta, University of Florida 30 July 2004 Trigger Meeting CSC Beam Test (Muon Slice Test) ME.
US CMS DOE/NSF Review, 20 November Trigger - W. Smith1 WBS Trigger Wesley H. Smith, U. Wisconsin CMS Trigger Project Manager DOE/NSF Status.
US CMS DOE/NSF Review: April 11-13, Trig. - Estimate to Complete.
10 th LECC Workshop, September 2004, Hauser et al.1 Experience with Trigger Electronics for the CSC System of CMS J. Hauser*, E. Boyd, R. Cousins,
Upgrade of the CSC Endcap Muon Port Card with Spartan-6 FPGA Mikhail Matveev Rice University 30 April 2012.
M.Matveev Rice University March 20, 2002 EMU Muon Port Card Project.
W. Smith, U. Wisconsin, US CMS DOE/NSF Review, May, 2004 Trigger Report - 1 CSC on-detector peripheral crate SBS VME Controller Muon Port Card: Output.
CSC Ops/DPG meeting, 05-Oct-2011 Hauser1 ALCT boards for ME4/2 etc.
CMS Week, 3-7 November CSC Trigger Test Beam Report Cast of many.
Update on CSC Endcap Muon Port Card
CSC Synchronization Procedure and Plans
CSC EMU Muon Port Card (MPC)
University of California Los Angeles
CDF Offline Operations
University of California Los Angeles
Muon Track-Finder Trigger
EMU Slice Test Run Plan a working document.
CMS EMU TRIGGER ELECTRONICS
University of California Los Angeles
ALCT Production, Cable Tests, and TMB Status
University of California Los Angeles
University of California Los Angeles
Regional Cal. Trigger Milestone: Major Production Complete
Current Status of CSC Trigger Elements – Quick Summary
CSC Muon Trigger - Annual Review
CSC Trigger Update Specific issues:
Darin Acosta University of Florida
September CSC Beam Test Report
8-layer PC Board, 2 Ball-Grid Array FPGA’s, 718 Components/Board
ALCT, TMB Status, Peripheral Crate Layout, CSC Event Display
TMB, RAT, and ALCT Status Report
Regional Cal. Trigger Milestone: Production & Testing Complete
CSC Trigger Primitives Test Beam Studies
Analysis of Oct. 04 Test Beam RPC Data
CSC Trigger Muon Port Card & Sector Processor in production
New Calorimeter Trigger Receiver Card (U. Wisconsin)
University of California Los Angeles
Changes in Level 1 CSC Trigger in ORCA by Jason Mumford and Slava Valuev University of California Los Angeles June 11,
TMB and RAT Status Report
Effect of an ALCT SEU Much-overlooked good stuff
University of California Los Angeles
Sector Processor Status Report
FED Design and EMU-to-DAQ Test
CSC Muon Sorter Status Tests Plans M.Matveev August 21, 2003.
Plans for the 2004 CSC Beam Test
Presentation transcript:

University of California Los Angeles ALCT Technical Status Martin von der Mey University of California Los Angeles ALCT2001 status Test stand TMB-ALCT-CFEB test Conclusion

Power, computer connectors ALCT2001 Boards Power, computer connectors 80 MHz SCSI outputs (to Trigger Motherboard) Xilinx mezzanine card Main board for 384-ch type Delay/ buffer ASICs, 2:1 bus multiplexors (other side) Input signal connectors Analog section: test pulse generator, AFEB power, ADCs, DACs (other side)

ALCT Functions Inputs discriminated signals from AFEB front-end boards, provides AFEB support: Distributes power, shut-down, test pulse signals. Sets and reads back discriminator thresholds. Monitors board currents, voltages, and temperature. Delay/translator ASIC on input does time alignment with bunch crossings. Searches for muon patterns in anode signals. If found, sends information to trigger motherboard. Records input and output signals at 40 MHz in case of level 1 trigger.

Test stand

Testing Delay ASICs

ALCT Test Program

ALCT Test Program

ALCT Test Program

ALCT-TMB Testing Standalone bench tests Radiation tests Cosmic ray tests ALCT-TMB test

TMB-ALCT-CFEB Test

LVDB-ALCT-CFEB

Trigger Motherboard CFEBs ALCT (Future: will be via transition module) TMB2001 prototypes for use at FAST sites and for system tests Produces cathode patterns from comparator outputs Correlates cathode and anode (from ALCT) patterns Sends chamber-level trigger decision to MPC Raw hits data “spooled” to DMB Interfaces to “everything” CFEBs DMB CCB ALCT MPC VME RPC (later) JTAG CFEBs ALCT (Future: will be via transition module) RPC via transition module

SP2002 (Main Board) 12 Used in CMS System Merged 3 SR2000s Florida Receiver: Florida Optical Transceivers 16 x 1.6 Gbit/s Links VME/CCB FPGA TLK2501 Transceiver Data conversion: Phi Global LUT To/from custom GTLP back-plane Eta Global LUT Phi Local LUT Merged 3 SR2000s Front FPGA

CSC Muon Trigger Scheme TriDAS part: Second generation prototypes EMU part: on-chamber nearing end of production, peripheral crate production > ESR in Nov. ‘03 Muon Portcard (1) Trigger Motherboard (9) Clock Control Board Trigger Timing & Control M P C D B T O N R L E DAQ Motherboard (9) Optical link Peripheral Crate on iron disk (1 of 60) 1 of 5 Muon Sorter (1) Sector Processor (12) CSC CFEB ALCT 1 of 24 1 of 2 LVDB Cathode Front-end Board CSC Track-Finder Crate (1) Anode LCT Board In underground counting room On detector 3-D Track-Finding and Measurement Trigger Primitives Anode Front-end Board

On-chamber CSC Trigger Electronics Comparator ASICs – DONE. Compare pulse heights from adjacent strips to find position of muon to ½-strip 15000 16-channel ASICS on CFEB boards (OSU) ALCT Boards – nearly DONE. Finds tracks among anode hits, stores data for readout 468+spares boards of 3 types (288-, 384-, 672-channel)

CSC Peripheral Crates in UXC55 Crate Controller M P C D B T O N R L E DAQ Motherboard (DMB) TRIG Motherboard (TMB) Muon Port Card (MPC) Clock Control Board (CCB)

Clock and Control Board Common design for both Peripheral and Track-Finder crates 20 Boards exist Have been distributed and used for chamber testing 60+1 required for CMS operation Mezzanine card with PLD TTCrx Mezzanine Card Rice ECL inputs ECL outputs 9U * 400 MM BOARD

Muon Port Card Sorts up to 18 LCTs from 9 chambers and transmits best 3 to Track-Finder crate 6 Boards of second generation have been fabricated and assembled. Board has passed standalone tests, communication tests with TMB, and cosmic ray tests Successfully read data from 2 chambers and sorted correctly Tests with Track-Finder are continuing Tests in time-structured test beam are underway now (for second time this year) 60 required for CMS operation Rice TLK2501 serializers Mezzanine card (same as TMB design) Optomodules (1.6 Gbit/s)

1st Prototype Track-Finder Tests Clock Control Board (Rice) (UCLA) Sector Receiver Muon Port Card (Rice) Sector Processor (Florida) SBS VME Interface Very successful, but overall CSC latency was too long New 2002 design improves latency, reduces # of crates from 6 to 1 Custom ChannelLink Backplane (Florida) Results included in Trigger TDR (2000)

CSC Track-Finder Crate Second generation prototypes Sector Receiver/ Processor Clock and Control Board SR SR SR SR SR SR SR SR SR SR SR SR / CCB / / / / / MS / / / / / / SP SP SP SP SP SP SP SP SP SP SP SP From MPC SBS 620 Controller (chamber 4) Muon Sorter From MPC (chamber 3) From MPC (chamber 2) From MPC (chamber 1B) From MPC (chamber 1A) To DAQ Single Track-Finder Crate Design with 1.6 Gbit/s optical links Custom 6U GTLP backplane for interconnections

SP2002 (Main Board) 12 Used in CMS System Merged 3 SR2000s Florida Receiver: Florida Optical Transceivers 16 x 1.6 Gbit/s Links VME/CCB FPGA TLK2501 Transceiver Data conversion: Phi Global LUT To/from custom GTLP back-plane Eta Global LUT Phi Local LUT Merged 3 SR2000s Front FPGA

2003 Time-structured Beam Test Setup X5A Setup TTC crate Trigger primitives DAQ Data PC Track finder Crate Peripheral Crate 2 DMB, 2 TMB 1 CCB, 1 MPC FED crate 1 DDU TRIDAS beam S1 S2 S3 CSC 1 CSC 2

Typical Muon Event Raw data includes 8 or 16 time bins history

2003 Time-Structured Test Beam Optimal timing found High efficiency (~98-99%) achieved Peripheral crate system basically working as desired Small CLCT efficiency loss at high rates, almost no ALCT efficiency loss 48 bunches 25 ns bunch spacing bunch width 3-5 ns SPS orbit period 1.2 ms 23 ms Scintillation Counters 48 bunches Structure repeats during 2.6 s spill length ALCT BX Number

CLCT Positions Key CLCT half strip from chamber 2 vs.1: On fine scale “staircase” structure indicates good trigger position resolution (note that chamber 1 is vertically higher, thus the offset in position)

CSC Trigger High Rate Tests data consistent with dead-time = 225 ns Chamber #1 CLCT 500 1,000 1,500 2,000 2,500 3,000 Beam Intensity (KHz) CLCT Rate (KHz) Expected LCT rate at LHC < 25 KHz (ME1/1)

CSC Track Finder Test Sector Processor 2 CSCs Successfully passed optical link loopback tests and MPCSP chain tests using 40 MHz crystal oscillator to drive system MPCSP optical link tests failed at the structured beam tests in May 2003 (link errors every few ms) Clock was derived from TTC system (mivivxrx) Combined clock jitter presumably too large to drive optical links PLL was not used to clean clock (i.e. QPLL was not available)

2003 Unstructured Test Beam Results Very high efficiencies achieved Highest trigger efficiency of 99.9% required low rate (few kHz) Improved DAQ throughput allowed readout up to 80k full events per spill. Typical “run” is 1 or 2 spills. Improved scans taken: Logic scope read out on most data HV scan Comparator threshold scan Pattern requirements scan Angle scans

ALCT Production Testing - Example ~510 Boards, ~200000 Channels Semi-automated procedures Using 3 test stations 2 for testing 1 for fixing Crew of up to 14 students testing (3 FTE) Sign-off sheets to track testing failures Test before and after 2-day burn-in 2 students trained for fixing 1 engineer for difficult cases 1 postdoc supervises it all

Preparation for Sept. 2003 Beam Test Cosmic ray test stand in Florida System brought to working order (everything now shipped to CERN) Scintillator Panels HV Supply CSCs TTCvx Periph Crate TF Crate MPC CCB TTCvi SP Dynatem TMB DDU CCB DMB SBS

CSC Muon Trigger Scheme TriDAS part: Second generation prototypes EMU part: on-chamber nearing end of production, peripheral crate production > ESR in Nov. ‘03 Muon Portcard (1) Trigger Motherboard (9) Clock Control Board Trigger Timing & Control M P C D B T O N R L E DAQ Motherboard (9) Optical link Peripheral Crate on iron disk (1 of 60) 1 of 5 Muon Sorter (1) Sector Processor (12) CSC CFEB ALCT 1 of 24 1 of 2 LVDB Cathode Front-end Board CSC Track-Finder Crate (1) Anode LCT Board In underground counting room On detector 3-D Track-Finding and Measurement Trigger Primitives Anode Front-end Board

On-chamber CSC Trigger Electronics Comparator ASICs – DONE. Compare pulse heights from adjacent strips to find position of muon to ½-strip 15000 16-channel ASICS on CFEB boards (OSU) ALCT Boards – nearly DONE. Finds tracks among anode hits, stores data for readout 468+spares boards of 3 types (288-, 384-, 672-channel)

Beam Test Setup PC TRIDAS beam S1 S2 S3 CSC 1 CSC 2 TTC crate Trigger primitives DAQ Data PC Track finder Crate Peripheral Crate 2 DMB, 2 TMB 1 CCB, 1 MPC FED crate 1 DDU TRIDAS beam S1 S2 S3 CSC 1 CSC 2

Beam Test Setup / 2 CSC’s, all on-chamber boards Peripheral crate From front end cards 2 TMBs and DMBs MPC CCB + TTCRx 2 CSC’s, all on-chamber boards Peripheral crate Track Finder CMS readout board Up to 80K events read out in 2.6s spill

2003 Synchronous Beam Structure 48 bunches 25 ns bunch spacing bunch width 3-5 ns SPS orbit period 1.2 ms 23 ms Structure repeats during 2.6 s spill length

Bunch Structure, ALCT Delay Tuning Expect muons in 48 out of 924 bx verified by CLCT bxn from data BX efficiency vs. ALCT delay setting 0-31 ns Chamber 1 Chamber 2

BX Distributions With Optimal Anode Delays Note logarithmic scale Cathodes: Data mostly in 3 bx (no fine time-adjustment possible) Anodes: Data 98.7% in 1 bx (after fine time-adjustment) Chamber 1 Chamber 2

TMB2003A and RAT2003A 4 boards produced and bench-tested Replaces problematic PHOS4 fine delays with commercial DDD devices Faster and larger FPGA (Xilinx Virtex-2 XC2V3000) on mezzanine board ALCT and RPC inputs through RAT (Rpc Alct Transition) board Rad-hard regulators used, 1.5v added for Virtex-2 core voltage

TMB 2003A Detail

ALCT SCSI input connectors RAT2003A Detail To TMB and 3.3v, 1.8v power These Connectors for GND Only Spartan 2E FPGA for RPC RPC connectors ALCT SCSI input connectors

TMB-ALCT Block Diagram AFEB data ~2.2ns/bin ALCT data ALCT latch raw data Master clock Synch. test pulse from TTC command or VME write to CCB Main FPGA OR TMB Master clock Crate Master clock CCB test pulse commands 2ns/bin Test pulse to AFEB amplifier or test strips (select via VME write to TMB to ALCT Slow Control FPGA register) TMB Latch input ALCT data Main FPGA Asynch. test pulse from pass- through Adjust ALCTtx for optimal latching of ALCT output data at TMB ALCT section Adjust ALCTrx for optimal latching TMB output data at ALCT output ALCT ALCT commands AFEB Adjust Delay ASICs for max. probability for ALCTs to come in one BX Internal test pulse via VME command to TMB CSC Test Pulse Strips -RX clock -TX Delay ASICs 2ns/bin

RPCs in Test Beam m- RE2, RE3 behind RE1 on ME1/2 ME3/2 ME2/2 ME1/1 1 RE1/2 (call it RE1) 2 free-standing (call them RE2, RE3) Readout info for experts only: RPC0  RE1 RPC3  RE2 RPC2  RE3 RE2, RE3 behind RE1 on ME1/2 ME3/2 ME2/2 ME1/1 m-

Link Board to RAT Board Setup RAT receives 4 cables, one from each RPC chamber Cables 0, 1, 2 were connected Appears that phasing RAT-TMB incorrect, so that 0  2 and 13 in readout N.B. RE1 seems the healthiest chamber

TMB Bunch Crossings - Data Flat Distribution seen 0-15 (4 bits): N.B. no ALCT or TMB data transmission errors (CRC check) seen in these runs

Internal RPC BXN diff BXN difference for consecutive RPC data arriving at TMB Always incrementing by 1.

No CRC errors for TMB/ALCT ALCT CRC TMB CRC

RPC Bunch Crossings - Data TMB vs RPC see perfect agreement with expectations. At least, ALCT/CLCT bx reset/bx0 protocol = Link board protocol

RPC BX from different RPC Chambers? Nice diagonals, but offsets (2, 7 bx) due either to TTCrx offset or reset timing at Link board

Test clock sent to TMB Inverted RPC clock sent to TMB (Runs=553,554)

Reset with SPS Orbit Better agreement (run=563) Still issues to understand

Beam Spot Size Data triggered by SP covers roughly 30x30 cm (Scintillators are 10x10 cm) Key ½-Strip Key Wire Group Key ½-Strip

RPC configuration Run 562 (Wires 0-11) Run 548 (Wires 9-20)

RE1 Pad Data Quality Compare ME3/2 (rear chamber) CLCT key half-strips (not strips) to RE1 pad (vertical dimension) Good position agreement (one dead pad) Modest RPC efficiency within readout region Fiducial region for RE1 select half-strips 65-100 RPC Pad Key ½-Strip

RE1 Efficiency Within Fiducial Region Unsure exactly where pads are, so plot efficiency versus CSC wire group This RPC chamber fairly efficient Key Wire Group

RE2 Pad Data Quality Same comparison Active part of chamber appears far from beam centerlow statistics Fiducial region: select half-strips 10-50 Key ½-Strip RPC Pad

RE2 Efficiency Within Fiducial Region This RPC chamber not so efficient Key Wire Group

RE3 Pad Data Quality Same comparisons Fiducial region: select half-strips 80-115 Key ½-Strip RPC Pad

RE3 Efficiency Within Fiducial Region RE3 efficiency reaches a high value, but is not flat with wire number Key Wire Group

RPC timing versus TMB RE2 RE3 RE1

RPC versus ALCT Timing RE3 RE2 RE1 First RPC hit per chamber: RE1, RE2 all in one BX RE3 not as good Note RE1 later by 1 BX RE3 RE2 RE1

RPC Afterpulsing? RE2 RE3 RE1 Number of time bins per chamber per event RE1 and RE2 good, RE3 shows considerable after-pulsing: RE2 RE3 RE1

TMB-DMB Block Diagram TMB CFEB CLCT Final logic TMB Master Clock, L1A TTC/CCB Crate Master Clock, L1A TMB Master Clock, L1A Store SCA data command AFF (Active FEB Flags) CFEBs “hit” AFF-L1A Coinc. Starts CFEB digi. & readout CLCT FIFO pre-trigger logic ALCT/ CLCT/ RPC Coincidence Comparators Output FPGA LCT-L1A Coinc. starts TMB readout Readout queue DMB-DDU readout Controller CFEB FIFOs (5) ALCT FIFO From ALCT SCAs, ADCs, Memories From ALCT RPC/ RAT L1A*CLCT DAV Coinc. L1A*ALCT LCTs to MPC 1 ns/bin ALCT-DAV CLCT-DAV -DAV Clock DAV Coinc. Auto set fixed LCT -read delay phase Cable Equal. AFF (external L1A = LHC & Test Beam operation modes)

Power, computer connectors ALCT384 Boards Power, computer connectors 80 MHz SCSI outputs (to Trigger Motherboard) Xilinx Mezz. board 24 Input signal connectors Delay/ buffer ASICs, 2:1 bus multiplexors (other side) Analog section: test pulse generator, AFEB power, ADCs, DACs (other side) Spartan XL

ALCT Production

ALCT Production Testing Done. All ALCT base boards repaired. Also spares. Prepare mezz. Cards spares

RAT2003 Layout VME Backplane Spartan FPGA To TMB RPC connectors ALCT Input connector

Trigger Motherboard (TMB) Input connectors From ALCT Main FPGA (on back) XILINX XCV1000E Mezzanine board From 5 CFEB’s Generates Cathode LCT trigger with input from CFEB (comparator) Matches ALCT and CLCT; sends trigger primitive info via MPC to Lev-1 muon trigger, sends anode and cathode hits to DMB.

New TMB2003A New TMB2003A under test

Beam Test of TMB 2001 See Jay’s talk

Old ALCT radiation results Xilinx vs Altera Radiation results shows small improvements to before… Mean lies at 65.24 Rad compared to 59.2 Rad before… The main improvement (factor 5) comes due to combination of 5 chips (1 concentrator and 4 LCT chips into 1).

Radiation Test results Irradiated XC2V4000 Xilinx FPGA New radiation test gives 141 Rad for proton beam. Compared to 65 Rad before. (~2.5 better) SEU proton fluence 79.8*10 7 cm-2 (M.Huhtinen (1-4)*10 10 cm-2 for 10 LHC years) Also tested 6 GTLP chips up to 5 kRads  no problems.

Adjust Shower Profile Redistribute energies inside cluster 9 strips in one cluster Before After 1 2 3 4 5 6 7 8 9

Ratio Ene3/Ene5 for 1d cluster

Ratio Ene2/Ene5 for 1d cluster

Ratio Ene1/Ene5 for 1d cluster

Energy in U layer divided by 2d energy

1d cluster width for layer 1

Energy in V layer divided by 2d energy

1d cluster width for layer 0

Ratio 5/9 for layer 0

Ratio 5/9 for layer 1

(E2+E8)/E5

(E3+E7)/E5 (E4+E6)/E5

(E1+E9)/E5

Angle between electrons

Cos(q*) =

MZ h

MZ momentum

Z Mass

Z transverse momentum

Results Selected data events: total: 13609 +- 117 c-c : 5172 +- 72 c-p : 8437 +- 91 Cross section: total: 253.0 +- 2.2 pb c-c : 263.8 +- 3.7 pb (260.9+-18.2 pb) c-p : 246.7 +- 2.7 pb (248.4+- 5.1 pb)

CDF Offline Operations Status: Rerun zee validation sample for 5.1.1. No discrepancies found as expected. Checked farm crashes. Reproduced 3 crashes: CdfTrack.cc (in prewrite). if (_siHits.size() > 0) { CdfTrackHits* storedSvxHits; storedSvxHits = new CdfTrackHits; for (SiHitIterator ihit = beginSIHits(); ihit != endSIHits(); ++ihit) { int packed = ((*ihit)->id() & 0x1FFFFFFF) | (((*ihit)->getAmbIndex() & 0x7 ) << 29);  crash (ihit !=0x0) storedSvxHits->accumulate(packed); }  Matt and Chris

Crashes Mark Fischler (needs help with debugging) The location in ELextendedID is basic_string& operator=(const basic_string& str); basic_string& operator=(const charT* s) {return assign( s,traits::length(s) );}  crash basic_string& operator=(charT c) {return assign( size_type(1), c );} and in ErrorObj::clear() is mySerial = 0; myXid.clear();  crash myIdOverflow = ""; Mark Fischler (needs help with debugging) 0x8fa13bd in SiStripCorrectorManager::correctStripSet (this=0xcd5b338,stripSet=0xe392094) at /home/cdfsoft/dist/packages/SvxDaqObjects/V00-00-74/src/SiStripCorrectorManager.cc:62  Matt (fixed)

Valgrind Run valgrind over the other crashes: Other: (Matt & Jason) ==18449== Conditional jump or move depends on uninitialised value(s) ==18449== at 0x420A6879: __mktime_internal (in /lib/i686/libc-2.2.5.so) ==18449== by 0x420A6EBE: timelocal (in /lib/i686/libc-2.2.5.so) ==18449== by 0x9B0D0C1: DateUtil::time_from_string(char const *) (/home/cdfsoft/dist/packages/DBObjects/V00-00-72/src/TimeStamp.cc:264) ==18449== by 0x904C794: ChipStatus::__ct(std::basic_string<char,std::char_traits<char>,std::allocator<char>>, int) (/home/cdfsoft/dist/packages/TrackingObjects/V00-01-73/src/ChipStatus.cc:54) ==18449== by 0x8F94AE5: PedestalUpdator::changed(void) (/home/cdfsoft/dist/packages/SvxDaqObjects/V00-0074/src/PedestalUpdator.cc:226) Other: (Matt & Jason) ==18449== at 0x904EFBB: ChipStatus::putBit(char *, int, int) (/home/cdfsoft/dist/packages/TrackingObjects/V00-01-73/src/ChipStatus.cc:133) ==18449== by 0x904F372: ChipStatus::sortBitString(int, int, char *) (/home/cdfsoft/dist/packages/TrackingObjects/V00-01-73/src/ChipStatus.cc:252) ==18449== by 0x904EC15: ChipStatus::makeMap(int) (/home/cdfsoft/dist/packages/TrackingObjects/V00-01-73/src/ChipStatus.cc:212) ==18449== by 0x904C8CC: ChipStatus::__ct(std::basic_string<char,std::char_traits<char>,std::allocator<char>>, int ) (/home/cdfsoft/dist/packages/TrackingObjects/V00-01-73/src/ChipStatus.cc:67) ==18449== by 0x8F94AE5: PedestalUpdator::changed(void) (/home/cdfsoft/dist/packages/SvxDaqObjects/V00-00-74/src/PedestalUpdator.cc:226)

Valgrind Still there (1X) (Aseet) ==6977== Conditional jump or move depends on uninitialised value(s) ==6977== at 0x914484D: PadSqz::Huffman_T::operator<<( (PadSqz::BitStream_T &)) (/home/cdfsoft/dist/packages/PADSObjects/V00-00-23/src/Huffman.cc:368) ==6977== by 0x9145E4C: PadSqz::PadRawBank::Fluff( (int)) (/home/cdfsoft/dist/packages/PADSObjects/V00-00-23/src/PadRawBank.cc:173) ==6977== by 0x84CF42C: PadRawModule<PadSqz::COTQ>::event(EventRecord *) (/home/cdfsoft/dist/releases/5.1.1/include/PADSMods/PadRawModule.icc:57)

Nodes Check crash rate per node: Node 171 (Take out)

Memory usage

Memory usage per Run Large memory usage

Memory increase

Daily checking New cron job  checks in log files for sever errors: Found yesterday: %ERLOG-s : *Fluffed bank(s) != original(s) PadRawBanks %ERLOG-s CalDataMaker: /home/cdfsoft/dist/packages/Calor/V00-01-52/src/CalDataMaker.cc : 754 unpack HATD bank : more than 8 hits in PHA GlobalLibraryLogger vxfit0() 28-Oct-2003 10:26:23 CST run = 163956 event = 262325 /home/cdfsoft/dist/packages/Calor/V00-01-52/src/CalDataMaker.cc: 745 unpack HATD bank : more than 8 hits in WHA GlobalLibraryLogger chi2wrtVertex() 28-Oct-2003 10:07:22 CST run = 163955 event =191711

fcdflnx3 Problems with disk space Take more scratch space Get a new disk