Software Verification and Validation 1 Security over Hadoop - Map-Reduce as a Service Team Sofia Neata Sorin Dascalu Tiberiu Popa Tudor Scurtu November 19, 2012

Software Verification and Validation 2 Map Reduce & Hadoop  Map Reduce  “a programming model and an associated implementation for processing and generating large data sets”  Large clusters of commoditiy machines  Highly scalable  Hadoop  An open-source implementation of the MapReduce framework  Top project of the Apache Foundation  Used in companies like: Yahoo, Cloudera etc

Software Verification and Validation 3 Hadoop Problem & Proposed Solution  Problem  No security mechanism when using Hadoop  Solution  A web-server which servers both as a layer of security in accessing Hadoop and also as an interface towards solving Map-Reduce problems.  Customers  medium to large companies that require regular distributed computation and distributed permanent storage capabilities.

Software Verification and Validation 4 Solution Hadoop Request Response

Software Verification and Validation 5 Solution Hadoop Request Response Hadoop Request Response Authentification Server Request Response

Software Verification and Validation 6 Functionality  Create new user and environment  The costumer contacts the system administrator  Delete users and environment  The costumer contacts the system administrator  Login  Logout  Create other users inside an existing enviroment  Delete other users inside an existing enviroment

Software Verification and Validation 7 Functionality(2)  Browse data from an environment  Download data from an environment  Upload data to an environment  Schedule map-reduce jobs in an environment

Software Verification and Validation 8 User Interface Requirements  Underlying storage access  Simple GET and POST HTTP methods  Schedule Map-Reduce jobs  HTTP methods

Software Verification and Validation 9 Security Requirements  Security layer using the web-server  Different access levels for costumers  Forms of security/authentification via HTTP  HTTPS – using TLS/SSL certificates  A signed certificate from a CA  Passwords can be kept in clear text  SHA hashes for password/other sensitive data  Vulnerable to man-in-the-middle attacks  NTLM  Secure challenge/response mechanism  Prevents password capture or replay attacks over HTTP

Software Verification and Validation 10 System Requirements  Hadoop cluster  Minimum five machines  Linux operating system  Hadoop and HTTP web-server with security capabilities  Cluster configuration based on costumer requirements  Memory  Computing performance  Storage capabilities

Software Verification and Validation 11 User Requirements  Large storage space  Cheap – amortized over time  Reliable – no data loss in case of software and hardware incidents  Large processing capabilities  Cheap – amortized over time  Fast – distributed  Failsafe – guarantee that the processing will complete in case of software or hardware failure  Secure access to previously enumerated items  No unauthorized access to data or to metainformation  Isolated from other clients actions

Software Verification and Validation 12 Specification

Software Verification and Validation 13 Conclusion  Authentification web-server over Hadoop  Different access level  HTTP methods  Security/authentification over Hadoop  Disadvantages  Tedious management and administration of a private cluster