Download presentation
Presentation is loading. Please wait.
Published byGeoffrey Craig Modified over 9 years ago
1
The material in this presentation is the property of Fair Isaac Corporation. This material has been provided for the recipient only, and shall not be used, reproduced, copied, disclosed, transmitted, in whole or in part, without the express consent of Fair Isaac Corporation. © 2007 Fair Isaac Corporation. Confidential. Condor at Fair Isaac Improving Analytics with High Throughput Computing Durban Frazer Analytic Scientist, Research & Development Fair Isaac May 1st, 2007
2
2 © 2007 Fair Isaac Corporation. Confidential. We serve clients in 80 countries 2,500+ Direct Clients, 100,000 Through Partners Products Predictive and Decision Models FICO Credit Score Fraud Account Management Decision management and analytics Top banks, Fortune 500 companies, Retailers, Government agencies, etc. 65% of the worlds credit cards 3000 employees > 100 billion decisions a year Publicly traded
3
Production Modeling – Why we installed Condor Research Programs – Other uses of Condor What’s Next? AGENDA © 2007 Fair Isaac Corporation. Confidential.
4
4 Production Modeling: Past Statistics and Modelling Program Developed in-house Repetitive process Design analytic job<- user interaction Execute job<- user waits for job to run Analyze results<- user interaction Data Driven – Virtually every operation requires data 300 terabytes data Analytic processing throughput is bottle neck Client Server Architecture User chooses server at login, stays with server entire time. Choice may last all day or all week Some machines become more utilized than others Increases run time of jobs on that machine Interferes with user interface
5
5 © 2007 Fair Isaac Corporation. Confidential. Production Modeling: Present Create Design and Job Servers. User chooses design server Responsible for User Interface Jobs execute on job servers All servers as homogenous as possible Shared File System Benefits GUI problems decreased Easier choice of servers for users Easier to run concurrent jobs Drawbacks Harder to diagnose problems Problems with a single server can now affect everyone
6
6 © 2007 Fair Isaac Corporation. Confidential. Production Modeling: Future
7
7 © 2007 Fair Isaac Corporation. Confidential. Production Modeling – Why we installed Condor Research Programs – Other uses of Condor What’s Next? AGENDA © 2007 Fair Isaac Corporation. Confidential.
8
8 Additional Benefits Anyone who is on a Condor server can now use Condor Single jobs can be sent to the best server via command line calls Adding simple API for in house programs (Java and Python) Embarrassingly parallel tasks are now parallelized New and existing programs are taking advantage of parallelization Advise 2 – calculate k order interactions between sets of n variables Transmute – variable generation and discovery on large transaction files ART, DataSpiders – genetic algorithms, often with multiple pools Server Administration - Condor can help monitor servers Watching job behavior can help pinpoint server problems Condor monitor provides helpful reports
9
9 © 2007 Fair Isaac Corporation. Confidential. Additional Benefits: Retail Time To Event Predict how likely every customer is to buy each product in the next period 10 million – 1 billion line items;500 – 10,000 products; 1 – 10 million customers Transactions and other data Scalable Analytic Process (RTTE) Probability Matrix Customers Products Odds that Jim will purchase IPod next week
10
10 © 2007 Fair Isaac Corporation. Confidential. Production Modeling – Why we installed Condor Research Programs – Other uses of Condor What’s Next? AGENDA © 2007 Fair Isaac Corporation. Confidential.
11
11 © 2007 Fair Isaac Corporation. Confidential. What’s Next Make Condor easier for basic users Parallelize common routines internally Univariate stats, variable calculations, etc. Add more machines Expand pool to include more heterogeneous machines Create Separate “Low Data” grid Better process control and job monitoring Globus Dagman Java Universe
12
12 © 2007 Fair Isaac Corporation. Confidential. Pain Points Machine failures Condor continues to submit jobs to machines that are failing Condor takes a long time to realize when some machines have failed and so doesn’t restart jobs. Condor management No dedicated monitoring personal or software Easy to use/setup diagnostics would be very helpful Job scheduling policies: How to handle no preemption Commonly have modest number of jobs by individual users Occasional very large batches from single source Can’t block future individual users, but want as many machines as reasonable for any batch of jobs
13
The material in this presentation is the property of Fair Isaac Corporation. This material has been provided for the recipient only, and shall not be used, reproduced, copied, disclosed, transmitted, in whole or in part, without the express consent of Fair Isaac Corporation. © 2007 Fair Isaac Corporation. Confidential. Questions? Answers? Durban Frazer 415-446-6036 DurbanFrazer@FairIsaac.com DurbanFrazer@FairIsaac.com Thank You
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.