1. Curation Micro-Services “It’s a Series of Tubes” Curation Micro-Services “It’s a Series of Tubes”

2. The Unix philosophy “Make each program do one thing well” “To do a new job, build afresh rather than complicate old programs by adding new features” “Expect the output of every program to become the input to another, as yet unknown, program” “Design and build software … to be tried early” “Don't hesitate to throw away the clumsy parts and rebuild them” — D. L. McIlroy et al., “Unix time-sharing system forward,” Bell System Technical Journal 57:6, part 2 (1978): 1902

3. The micro-services “philosophy” http://www.flickr.com/photos/elsie/8229790/ CC http://www.flickr.com/photos/oskay/265899811/ CC

4. Curation micro-services MetaphorsAssumptionsPrinciplesPreferencesPractices Pipeline Safety through redundancy Modularity The small and simple over the large and complex Focus on outcomes, not means Lego bricks Meaning through context Granularity The minimally sufficient over the feature laden Complexity through composition, not addition Utility through service Orthogonality The configurable over the prescribed Policy neutral, platform and protocol independent Value through use (and reuse) Emergence The proven over the (merely) novel Approach sufficiency through incrementally necessary steps Stewardship is a relay Evolution Early prototyping, frequent refactoring ParsimonyCode to interfaces

5. Curation micro-services ModeFocusValueServiceValenceVisibility Curation Value Accretion Annotation UI / Access control / Message queuing Interoperation User-facing Visibility Notification Utility Accessibility Access Application Derivation Transformation Selectivity Search Actionability Index Stewardship Ingest Preservation Context Epistemology Characterization Interpretation Provider- facing Ontology Inventory State Reliability Replication Protection Fixity Stability Storage Identity

6. Design goals Principle of least surprise Multiple interface modalities – RESTful HTTP – Command line – Procedural (Java, Perl, Ruby, …) Linked data Stable URL references The file system is the database http://example-store/ State or content Storage node ObjectVersionFile default/1234/3/xyz state/ Storage service

7. “You say micro, I say macro…” Access ANVL ARK BagIt CAN Checkm Dflat ERC EZID GhOST Ingest Inventory LockIt N2T Namaste Noid Pairtree ReDD RUU Storage Access ANVL ARK bagit.plBagIt CAN checkm.plCheckm Dflat ERC EZID GhOST Ingest Inventory LockIt N2T namaste.plNamaste Noid Pairtree ReDD RUU Storage ServiceToolConvention

8. Development roadmap First waveSecond wave Third waveFourth wave Fifth waveSixth wave IdentityInventoryIndexSearchNotificationAnnotation StorageIngest / AccessFixityReplication CharacterizationTransformation IDm / Authn / AuthzMetadata standards Object / collection modelingSemantic interoperability Policy / business model development

9. Ingest process flow Submitting user agent Ingest Inventory Storage Node Identity Submit Create identifier Identifier Add version Get version metadata Version metadata Notification Version metadata Get version metadata Add version

10. Ingest implementation Submitting user agent Submitter Consumer Ingester Storage Queue HTML form Servlet Implicitly multi-threaded Servlet Implicitly multi-threaded Dæmon Explicitly multi-threaded Zookeeper dæmon Job metadata Job payload Submission notification Ingest notification Batch or single object

11. Questions? silverpipes.jog / firstpresmacomb.org

12. More information UC Curation Center (UC3) http://www.cdlib.org/uc3 Micro-service specifications https://confluence.ucop.edu/display/Curation Digital curation group http://groups.google.com/group/digital-curation UC3 Stephen AbramsErik Hetzner Margaret Low Mark Reyes Perry Willett Patricia CruseGreg Janée David Loy Tracy Seneca Scott FisherJohn Kunze Isaac Rabinovitch Marisa Strong