1. Delivering “DARPA Hard” Matthew van Adelsberg Chief Data Scientist, CACI Data Tactics

2. Who we are

3. STRATA+HADOOP, San Jose, CA
DARPA hard
“DARPA Hard” is about making the impossible practical
11/30/2018
STRATA+HADOOP, San Jose, CA

4. Lessons Learned
Key insight: to deliver innovation in products, you need corresponding innovation in process
New technological requirements to meet challenges should alter internal workflow for development
Need incentives for collaboration between technical experts

5. Independent Functional Teams
Organizing into independent teams performing functional roles – network engineering, data science, data engineering, software development – can lead to challenges in meeting customer requirements.
Independent Functional Teams

6. Hypothetical Scenario
Data science team develops new capability for batched anomaly detection in streaming Cyber log data
Leadership proposes demo to prospective client
Demo solution briefed to client, whose senior leadership asks for proof of concept at scale
Hypothetical Scenario

7. Issues in Scenario Implementation
Demo not scalable to meet customer requirements
Cannot deploy in client computing environment
Data engineering team does not support ingest into current DevOps environment
Incentives not in place for collaboration among internal team members
Issues in Scenario Implementation

8. Each new challenge should involve the entire technical team
Engagement with domain experts from the beginning
Engineering/hardware challenges augment existing infrastructure
Incentivize collaboration
Avoiding These Issues

9. Integrating Functional Areas
At each step in development, if our current workflow does not support efficient coordination between functional teams, we innovate our process to enable this in future work.
Integrating Functional Areas

10. Key Factors for Integration
Requires commitment from leadership to implement
Needs business development strategy that reflects team structure
Works well with Agile process
Horizontally scalable
Key Factors for Integration

11. Takeaway Points Integrated teams work on problems collectively
Product innovation requires corresponding innovation of process
Leadership must embrace and promote this strategy
Enables relatively small teams to deliver “DARPA Hard” solutions

12. Delivering “DARPA Hard” Matthew van Adelsberg, Chief Data Scientist, CACI Data Tactics
March 31, 2016