Download presentation
Presentation is loading. Please wait.
Published byJustin Ford Modified over 9 years ago
1
NEON TNC2010, May 31, Vilnius Maarten Koopmans for UNINETT Sigma maarten@vrijheid.net
2
2 Who ING Group -2002 SURFnet 2002-6 ICTU (govt) 2006-8 vrijheid.net 2008- qtask.com ibeamsystems.com Uninett Sigma ING Group -2002 SURFnet 2002-6 ICTU (govt) 2006-8 vrijheid.net 2008- qtask.com ibeamsystems.com Uninett Sigma
3
Different mindsets
4
NEON Goals state-of-the art of cloud computing; cost of moving and running non-HPC jobs on a cloud computing environment; how to do this in practice ; a list of identified risks/benefits on a short/long perspective.
5
Areas
6
Shortlists
8
#8: Why deliver? Resource Cost in Medium DC ≈ 1000 servers Cost in Very Large DC ≈ 50,000 servers Ratio Network$95 / Mbps / month$13 / Mbps / month7.1x Storage$2.20 / GB / month$0.40 / GB / month5.7x Administration≈140 servers/admin>1000 servers/admin7.1x
9
#7 Design to fail
10
#6 USP: “root” - really?
11
#5 Can you trust it? “Cloud computing is about gracefully losing control while maintaining accountability even if the operational responsibility falls upon one or more third parties.”
12
#4.5 “Core infra” ComputingComputing STORAGE Queues
13
#4 Public clouds lead Spot instances Elastic load balancing Virtual Private Cloud Elastic map-reduce Cloudfront..... SQS SimpleDB Cloudwatch Autoscaling RDS
14
#3: Management
15
#2 Keep an eye on Apache ZooKeeper
16
#1 The USERS are key
17
or maarten@vrijheid.netmaarten@vrijheid.net
18
Why again?
19
Let’s zoom in on storage
20
Storage: requirements No client to access the data. Transparent versioning. Transparent encryption - both transport and storage AAI integration Allow sharing of resources
21
Current Cloud storage API based - complex for end users
22
No AAI integration at all
23
So...
24
AAI: enrollment AAI Cloud backed storage initial request access granted token returned (rotating?) authenticate user user authenticated
25
“Just WebDAV” Webdav daemon Encryption Resource naming Versioning Storage cloud Continue Map resource to hash code version = 1 Stream data metered through encryption...........into the cloud Metering
26
...Locking... Webdav daemonLocking Resource naming Versioning Return lock Map resource to hash code Get reference to most recent version Acquire lock
27
...downloading... Webdav daemonDecryption Resource naming Versioning Storage cloud Continue Get most recent version = 1 Stream data metered via decryption...........from the cloud Metering
28
Stand on the shoulders of giants WebDAV widely deployed, lots of 3rd party clients. Service on top of Java VM Scala (integration language) Cloud access libraries (often Java based) Apache ZooKeeper (configuration management, locking) Apache Cassandra or HBase (metering) AAI integration components...
29
Lessons learnt so far: WebDAV is a nice start for client-less access to file based resources. CPU intensive due to the encryption per "stream". A language (model) with concurrency support is a big plus. This breaks the trend of asynchronous I/O based network services. Stand on the shoulders of giants: Apache Zookeeper, Bookkeeper, the JVM, Scala language, libraries for cloud access OS X requires DAV level 2 and does a lot of locking. But: from 10.5.x onwards it also does HTTP 1.1 chunked encoding; that broke a lot of servers. Windows works best with digest authentication. Linux seems to be most forgiving and least demanding. all clients support SSL. WebDAV's XML is relatively simple but the usage may differ per client type.
30
Questions so far? or maarten@vrijheid.netmaarten@vrijheid.net
31
Cloud management
32
Computing Torque/PBS on AWS via RightScale OSGi on Eucalyptus MPI R Challenge: Matlab, BLAST etc. - how to deal with licensing?
33
Questions - the final or maarten@vrijheid.netmaarten@vrijheid.net
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.