1. General Purpose Computing with Condor
May 2004
Condor Week 2004
General Purpose Computing with Condor
Brooklin J. Gore
Senior Fellow, Advanced Computing
2003 Micron Technology, Inc. All rights reserved. Information is subject to change without notice. 1 1

2. General Purpose Computing with Condor Agenda
The Condor environment at Micron
Micron’s Grid Computing Best Practices
Parade of Applications (Benefits)
Condor Enhancements
Future Requirements for Condor
4/6/2019
Micron Confidential

3. General Purpose Computing with Condor Condor Environment
4/6/2019
Micron Confidential

4. General Purpose Computing with Condor Condor Environment
Pools
Hosts1
CPUs
TIPS
TFLops
RAM
Disk2
Rank3 9
3967
4254
4.419
1.193
2.752
56.32 87
1. Includes Linux, Solaris and Windows
2. Only includes local disks of Windows systems
3. Illustrative based on Top500 Supercomputer list
4/6/2019
Micron Confidential

5. General Purpose Computing with Condor Condor Environment
4/6/2019
Micron Confidential

6. General Purpose Computing with Condor Best Practices for Grid Computing
Infrastructure
Fast network
Centralized Data
Common system images
Lots of memory
Dual processors
4/6/2019
Micron Confidential

7. General Purpose Computing with Condor Best Practices for Grid Computing
Philosophy
Articulate value proposition
“Low hanging fruit” first
Educate, educate, educate
Integrate with existing practices
4/6/2019
Micron Confidential

8. General Purpose Computing with Condor Parade of Applications (Benefits)
Automated Process Control (scalability, reliability)
DRAM row/column redundancy design (1.5x 24->16 hours)
Engineering extracts (Windows->*nix port in 50 v. 900 hours)
Engineering analysis (6x speedups, “I don’t know how” quote)
Enhanced SW Defect Analysis (our old friend)
Global planning and scheduling (scalability, reliability)
4/6/2019
Micron Confidential

9. General Purpose Computing with Condor Parade of Applications (Benefits)
MSI package creation (6x 12->2 hours)
Network device configuration (6x 24->4 hours)
RDA inspection processing (scalability, reliability)
Matched Routing (capability)
Integrated Data Analysis System (time window)
OOC chart generation (scalability, reliability
4/6/2019
Micron Confidential

10. General Purpose Computing with Condor Parade of Applications (Benefits)
SWR processing (scalability, reliability)
System reporting (15x 55->3.5 minutes)
Web log processing (8x 16->2 hours)
Trend charts (>300k charts in 5 hours)
Data conversion (1mil files, 8 hours)
OS patch management (12x 12->1 hour)
4/6/2019
Micron Confidential

11. Quotes like this make it all worthwhile:
General Purpose Computing with Condor Parade of Applications (Benefits)
Quotes like this make it all worthwhile:
I don’t know how we’d get all of our data extractions done without Condor
-- Melissa Norgard
Savings on order of $ millions
4/6/2019
Micron Confidential

12. General Purpose Computing with Condor Condor Enhancements
Web-based
Host Management Interface
Pool, job, machine status
Pool performance metrics (hosts, idle jobs, queue delay)
Custom Unix/Windows machine ClassAds (via STARTD_CRON)
Admin tools (missing host reports, global capabilities)
Developer tools (msg-enabled job monitor, wait, cleanup)
Need: Stale job notification, cleanup
4/6/2019
Micron Confidential

13. General Purpose Computing with Condor Future Requirements
Queue performance
Need to start a job in < 3 minutes, ideally 1 minute
Handle deep/narrow queues, shallow/wide queues and most importantly a combination of the two
Scale
10,000 hosts per pool
submitters
Web Services protocols
SOAP, UDDI, WSDL, XML
4/6/2019
Micron Confidential

14. Condor Week 2004
Micron and the Micron logo are trademarks and/or service marks of Micron Technology, Inc.
All other trademarks are the property of their respective owners.