1. SPECIFICATION OF THE FIMS MEDIA SOA FRAMEWORK REPOSITORY SERVICE INTERFACES Loic Barbou & Ashraf Tadros

2. Services Repository Service –Repository Service OperationsRepository Service Operations –Repository Capabilities Registry (RCR) OperationsRepository Capabilities Registry (RCR) Operations Repository Notification Service –Asynchronous Operations NotificationsAsynchronous Operations Notifications –Events SubscriptionsEvents Subscriptions

3. Repository Service - Operations »Lock, Unlock, Clear Lock, Retrieve Open Locks »Generate Unique ID »Add Object »Add Content (Async.) »Update Object (Replace, Update Properties) »Update Status * »Delete and UnDelete Object/Content »Purge Object (Async.) »Purge Content (Async.) »Get BM Content(s) »Retrieve Essence (Async.) »Media Query

4. Repository Service - RCR »Repository State (online, offline, failover) »Storage Capacity (Names, available and total space) »Performance (Performance Indicator %) »Vendor Specific Report (Report HTML) »General Capability (supported functionalities) »Vendor Defined Schema Extensions »Updatable and Searchable Properties »Vendor Defined Query Result Set Definitions »Repository Definitions (name & Description) »Retrieve Administrative Super Lock Token »Supported Object/Content Status

5. Notification Service - Async. Operations »Add Content (Notify and Fault) »Purge Content (Notify and Fault) »Purge Object (Notify and Fault) »Retrieve Essence (Notify and Fault)

6. Notification Service - Events Subscription –System Events »Exceptions »Performance –Asset Events »Status Changed »Object/Content Created »Object Deleted »Content Deleted/Purged »Object/Content Format/Essence Locator Update

7. Clarifications Asynchronous Operations Notifications »Request => ResourceType => AsyncEndpointTypeResourceTypeAsyncEndpointType »Ack: timeStamp + operationId »Notify Result: result + operationId »Notify Fault: RepositoryFaultType + some related info + operationIdRepositoryFaultType Lock Token »RepositoryLockTokenTypeRepositoryLockTokenType »RepositorySuperLockTokenTypeRepositorySuperLockTokenType

8. Clarifications Status »Object - BMContentStatusTypeBMContentStatusType »Content - BMEssenceLocatorStatusTypeBMEssenceLocatorStatusType »BMStatusType - new, online, offline, deleted, purged, invalid, processing. »Vendor specific - BMCustomStatusExtensionType.BMCustomStatusExtensionType »RepositorySupportedStatusTypeRepositorySupportedStatusType

9. Clarifications Object »BMContentTypeBMContentType Content »BMEssenceLocatorType - SimpleFileLocatorType, ListFileLocatorType or FolderLocatorType)BMEssenceLocatorTypeSimpleFileLocatorType ListFileLocatorTypeFolderLocatorType Light Object »BMContentLightTypeBMContentLightType

10. Clarifications Delete vs. Purge »Delete: flags for deletion. »Purge: removes completely from DB and repository storage. »Purge object purges all related contents. Update Object vs. Update Properties »Update object: fully replaces the object. »Update properties: updates the specific passed property only. RepositoryProfileType

11. Repository Service Operations

12. GenerateUniqueID Description »Generates a Unique ID within the scope of a given repository. »Must be obtained before Object or Content Add operations. »Doesn’t have to be generated by this operation and can be created by an external service as long as ID uniqueness is respected. Request »RepositoryProfileTypeRepositoryProfileType Response »ResourceIDTypeResourceIDType

13. AddObject Description »Uses the generated Unique ID. »Submits a new BMContentType to be added to the system.BMContentType Request »AddObjectRequestTypeAddObjectRequestType Response »BMContentTypeBMContentType

14. AddContent Description »Associate a physical essence – BMEssenceLocatorType (SimpleFileLocatorType, ListFileLocatorType or FolderLocatorType) to an existing object – BMContentLightType.BMEssenceLocatorTypeSimpleFileLocatorTypeListFileLocatorType FolderLocatorTypeBMContentLightType »Uses the generated Unique ID. »BMContentFormatType is optional input.BMContentFormatType Request »AddContentRequestTypeAddContentRequestType Ack »AddContentOperationAckTypeAddContentOperationAckType

15. AddContent Notification Notify Result »NotifyAddContentResult Operation => AddContentOperationNotificationType AddContentOperationNotificationType Notify Fault »NotifyAddContentFault Operation => AddContentOperationFaultNotificationType AddContentOperationFaultNotificationType

16. ReplaceObject Description »Updates “metadata” properties of a BMContent object. »Behaves as an object replace not as update properties. »Sets omitted properties in input object to null. »A new version will be created (If versioning supported). Request »ReplaceObjectRequestTypeReplaceObjectRequestType Response »BMContentTypeBMContentType

17. UpdateProperties Description »Updates “metadata” of BMContent object (BMContentType).BMContentType »Applies only on addressable and updatable properties – RepositoryUpdatablePropertyInfoType (defined by the RCR). RepositoryUpdatablePropertyInfoType »PropertyInfosType parameter includes properties Xpath, values and RepositoryActionType (update, new, remove).PropertyInfosType Request »UpdatePropertiesRequestTypeUpdatePropertiesRequestType Response »BMContentTypeBMContentType

18. UpdateStatus ( Remove !) Description »Updates object status only (BMContentStatusType not BMEssenceLocatorStatusType)BMContentStatusType BMEssenceLocatorStatusType »Should the service allow status update from outside the repository system? »Status automatically updated through the object and essence life cycle and other operations (DeleteObject, PurgeContent..) Request »UpdateStatusRequestTypeUpdateStatusRequestType Response »BMContentTypeBMContentType

19. Lock Description »Create a lock on a given resource or add a resource to an existing lock token. »Adds a reference to be locked to the RepositoryLockTokenType.RepositoryLockTokenType »Lock expiration is per RepositoryLockTokenType and not per resource.RepositoryLockTokenType Request »LockRequestTypeLockRequestType Response »RepositoryLockTokenTypeRepositoryLockTokenType

20. UnLock Description »Removes the lock from multiple references. »If no references passed in the request, will unlock all resources associated with the RepositoryLockTokenType.RepositoryLockTokenType Request »UnLockRequestTypeUnLockRequestType Response »RepositoryLockTokenTypeRepositoryLockTokenType

21. ClearLock Description »A mechanism to override a lock. »A RepositorySuperLockTokenType must be obtained from the RCR before calling the operation.RepositorySuperLockTokenType Request »ClearLockRequestTypeClearLockRequestType Response »RepositoryLockTokenTypeRepositoryLockTokenType

22. RetrieveOpenLocks Description »Gets all open locks within the system. »A RepositorySuperLockTokenType must be obtained from the RCR before calling the operation.RepositorySuperLockTokenType Request »RetrieveOpenLocksRequestTypeRetrieveOpenLocksRequestType Response »RepositoryLockTokensTypeRepositoryLockTokensType

23. DeleteContent Description »Sets the BMEssenceLocatorStatusType for an active BMEssenceLocatorType (SimpleFileLocatorType, ListFileLocatorType or FolderLocatorType) to ‘deleted’.BMEssenceLocatorStatusType BMEssenceLocatorTypeSimpleFileLocatorType ListFileLocatorTypeFolderLocatorType »No physical bytes are deleted Request »DeleteContentRequestTypeDeleteContentRequestType Response »BMContentTypeBMContentType

24. UnDeleteContent Description »Sets the BMEssenceLocatorStatusType for the BMEssenceLocatorType (SimpleFileLocatorType, ListFileLocatorType or FolderLocatorType) to an active status (invalid, online, offline..).BMEssenceLocatorStatusType BMEssenceLocatorTypeSimpleFileLocatorType ListFileLocatorTypeFolderLocatorType »Repository system is responsible about tracking and resetting the status in the undelete operation implementation. Request »UnDeleteContentRequestTypeUnDeleteContentRequestType Response »BMContentTypeBMContentType

25. DeleteObject Description »Sets the BMContentStatusType for an active BMContentType to ‘deleted’BMContentStatusTypeBMContentType »Repository system is responsible about setting related essences - BMEssenceLocatorStatusType for the BMEssenceLocatorType (SimpleFileLocatorType, ListFileLocatorType or FolderLocatorType) to ‘deleted’. BMEssenceLocatorStatusType BMEssenceLocatorTypeSimpleFileLocatorTypeListFileLocatorType FolderLocatorType Request »DeleteObjectRequestTypeDeleteObjectRequestType Response »BMContentTypeBMContentType

26. UnDeleteObject Description »Resets the BMContentStatusType for a deleted BMContentType.BMContentStatusTypeBMContentType »Repository system is responsible about tracking and resetting the status in the undelete operation implementation. »Repository system is responsible about resetting related essences - BMEssenceLocatorStatusType for the BMEssenceLocatorType.BMEssenceLocatorStatusType BMEssenceLocatorType Request »UnDeleteObjectRequestTypeUnDeleteObjectRequestType Response »BMContentTypeBMContentType

27. PurgeContent Description »Can only be executed on an Essence in a “deleted” state. »Physically removes essence from repository database & storage. Physical bytes are deleted. »If the essence is the last representation in a BMContentFormatType, the format entry will be removed from the BMContentType. BMContentFormatType BMContentType Request »PurgeContentRequestTypePurgeContentRequestType Ack »PurgeObjectOperationAckTypePurgeObjectOperationAckType

28. PurgeContent Notification Notify Result »NotifyPurgeContentResult Operation => PurgeContentOperationNotificationType PurgeContentOperationNotificationType Notify Fault »NotifyPurgeContentFault Operation => PurgeContentOperationFaultNotificationType PurgeContentOperationFaultNotificationType

29. PurgeObject Description »Can only be executed on BMContentType in a “deleted” state.BMContentType »Physically removes asset information and all related essences from repository database & storage. Request »PurgeObjectRequestTypePurgeObjectRequestType Ack »PurgeObjectOperationAckTypePurgeObjectOperationAckType

30. PurgeObject Notification Notify Result »NotifyPurgeObjectResult Operation => PurgeObjectOperationNotificationType PurgeObjectOperationNotificationType Notify Fault »NotifyPurgeObjectFault Operation => PurgeObjectOperationFaultNotificationType PurgeObjectOperationFaultNotificationType

31. GetBMContent Description »Retrieves a single BMContentType.BMContentType »Ability to specify the structure of the retrieved object by setting: RepositoryMetadataFilterOutType: »Null: All available information »Physical: Metadata describing physical essence information »Editorial: Metadata describing editorial content Request »GetBMContentRequestTypeGetBMContentRequestType Response »BMContentTypeBMContentType

32. GetBMContents Description »Retrieves multiple BMContentType(s).BMContentType »RepositoryMetadataFilterOutType: physical, editorial Request »GetBMContentsRequestTypeGetBMContentsRequestType Response »BMContentsTypeBMContentsType

33. RetrieveBMEssence Description »Copy an essence BMEssenceLocatorType (SimpleFileLocatorType, ListFileLocatorType or FolderLocatorType) to a known supported destination by the repository – RepositoryKnownLocationTypeBMEssenceLocatorTypeSimpleFileLocatorTypeListFileLocatorType FolderLocatorTypeRepositoryKnownLocationType Request »RetrieveBMEssenceRequestTypeRetrieveBMEssenceRequestType Ack »RetrieveBMEssenceOperationAckTypeRetrieveBMEssenceOperationAckType

34. RetrieveBMEssence Notification Notify Result »NotifyRetrieveBMEssenceResult Operation => RetrieveBMEssenceOperationNotificationType RetrieveBMEssenceOperationNotificationType Notify Fault »NotifyRetrieveBMEssenceFault Operation => RetrieveBMEssenceOperationFaultNotificationType RetrieveBMEssenceOperationFaultNotificationType

35. Query Description » Consumer is an orchestration engine. »A search parameter must match a property defined in the asset schema BMContentType or RepositorySchemaExtensionType and defined in the RCR - RepositorySearchablePropertyInfoType.BMContentTypeRepositorySchemaExtensionTypeRepositorySearchablePropertyInfoType »The max number of supported search parameter across all query operations is defined in RCR. Operation Structure Query Payload Query Definition Output Definition Parameters Operands Group Amount of Data Data Structure

36. Query Request »MediaQueryRequestTypeMediaQueryRequestType »QueryProfileType - RCRQueryProfileType »Configures number of total items returned, number of total items returned per page and timeout value for query operations. »QueryInputDefinitionTypeQueryInputDefinitionType »Multiple simple and compound search criteria parameters - QueryExpressionType (QueryParameterType, QueryGroupType). QueryGroupOperandType (and, or) – QueryParameterOperandType (equal, greater, less) and Page number.simplecompound QueryExpressionTypeQueryParameterTypeQueryGroupType »QueryOutputDefinitionTypeQueryOutputDefinitionType »Defines the QueryResultInfoType: noItems, full, light, metadata, physical, specific (Service specific result set – must be defined in the RCR). And specify ordering within query operation – QueryResultSortDirectionType (ascending, descending). Response »MediaQueryResponseTypeMediaQueryResponseType

37. Query QueryExpressionType : Simple Query: Title = NYQueryExpressionType  QueryParameterType QueryParameterType name = Title value = NY valueType = string operand = equal negated = false  00000000-0000-0000-0000-000000000001 00000000-0000-0000-0000-000000000002 Title NY string equal false

38. Query QueryGroupType (Compound Query.xml)QueryGroupTypeCompound Query.xml –operand = OR –parameters »name = Title, value = NY, valueType = string, operand = equal, negated = false »name = Title, value = Boston, valueType = string, operand = equal, negated = false –groups –operand = AND –parameters »name = DateModified, value = 2010, valueType = YYYY, operand = greater, negated = false »name = DateModified, value = 2012, valueType = YYYY, operand = less, negated = false »name = DateCreated, value = 2011, valueType = YYYY, operand = equal, negated = true Title = NY OR Title = Boston OR (DateModified > 2010 AND DateModified < 2012 AND Not (DateCreated=2011)) QueryExpressionType: Compound Query:QueryExpressionType Title = NY OR Title = Boston OR (DateModified > 2010 AND DateModified < 2012 AND Not (DateCreated=2011))

39. Repository Capabilities Registry RCR

40. RCR ? Definition »Datastore containing a list of properties describing the implementation behavior of the repository interface. »Provide the ability for a vendor to implement a FIMS repository interface on top of their product based on its capabilities »Service interoperability is achieved by having a common interface with a preset list of behaviors and settings (Support Lock, Extension Registration, Queryable Properties,… ) »External service can query the RCR to retrieve settings and behaviors

41. RCR Clarifications RCR - Repository Capabilities Registry –Option 1: Define an operation for each type: »Example: GetSearchableProperties, GetUpdatableProperties.. »Pros: »Prevents RCR changes from affecting implementation »Easily supports fast failures »Cons: »Adds complexity to interface –Option 2: Define a single operation for all RCR data: »Example: GetRCRSettings() »Pros: »Simple vendor implementation »Cons: »Potentially affect implementation when changes are made to RCR 

42. GetRepositoryState Description »Retrieve configuration parameters indicating the repository service status: online = active, offline = inactive or not available and failover = instance of the service is not in use, but another instance is active. Request »RepositoryProfileTypeRepositoryProfileType Response »RepositoryStateType: online, offline, failover

43. GetStorageAvailableCapacity Description »Retrieves the repository storage components and their available and total space. Request »RepositoryProfileTypeRepositoryProfileType Response »StorageComponentsTypeStorageComponentsType

44. GetPerformanceIndicator Description »Retrieves a generic overall indicator on the performance or health of the repository system. »The health is represented by a percentage number »100%: System is running at 100% of its capacity (CPU, Storage,…) »0%: Nothing is using the system ! Request »RepositoryProfileTypeRepositoryProfileType Response »PerformanceIndicatorTypePerformanceIndicatorType

45. GetRepositoryReport Description »Retrieves a vendor specific health or general condition report. Request »RepositoryProfileTypeRepositoryProfileType Response »RepositoryReportTypeRepositoryReportType

46. GetRepositoryGeneralCapability Description »Retrieves general supported functionality. »RepositoryEventSubType: assetStatusChange, assetObjectCreated, assetContentCreated, assetContentDeleted, assetContentPurged, assetObjectUpdated, assetContentFormatUpdated, assetContentEssenceLocatorUpdated, assetObjectDeleted, repositorySystemException, performanceEvent. »FormatProfileMetadataProcessingType: mustBeProvided, autoGenerated, autoGeneratedWhenNotProvided »RepositoryEventListenerEndpointTypes: soap, rest Request »RepositoryProfileTypeRepositoryProfileType Response »RepositoryGeneralCapabilityTypeRepositoryGeneralCapabilityType

47. GetSchemaExtensions Description »Retrieve vendor defined metadata schema extension. Request »RepositoryProfileTypeRepositoryProfileType Response »RepositorySchemaExtensionsTypeRepositorySchemaExtensionsType

48. GetUpdatableProperties Description »Retrieves the list of properties the repository system allows to be updated. »RepositoryActionType: update, new, remove. Request »RepositoryProfileTypeRepositoryProfileType Response »RepositoryUpdatablePropertyInfosTypeRepositoryUpdatablePropertyInfosType

49. GetQueryResultsetDefinitions Description »Gets a set of vendor specific defined result sets retuned by the Query operation. Request »RepositoryProfileTypeRepositoryProfileType Response »QueryResultsetDefinitionsTypeQueryResultsetDefinitionsType

50. GetSearchableProperties Description »Retrieves list of properties exposed by the system to be used in the QueryParameterType.QueryParameterType Request »RepositoryProfileTypeRepositoryProfileType Response »RepositorySearchablePropertyInfosTypeRepositorySearchablePropertyInfosType

51. GetRepositoryDefinitions Description »Retrieves the internal repository system volumes. Request »RepositoryProfileTypeRepositoryProfileType Response »RepositoryDefinitionsTypeRepositoryDefinitionsType

52. GetSuperLockToken Description »Retrieves a super token to override existing RepositoryLockTokenType to unlock specific resources. RepositoryLockTokenType Request »RepositoryProfileTypeRepositoryProfileType Response »RepositorySuperLockTokenTypeRepositorySuperLockTokenType

53. GetSupportedStatus Description »Retrieves the system supported statuses: BMContentStatusType, BMEssenceLocatorStatusType, and vendor specific BMCustomStatusExtensionType for the content and object.BMContentStatusType BMEssenceLocatorStatusType BMCustomStatusExtensionType Request »RepositoryProfileTypeRepositoryProfileType Response »RepositorySupportedStatusTypeRepositorySupportedStatusType

54. Repository Notification Service

55. RepositoryEvent Description »Single event per message. »SOAP and REST »Ordered delivery: events are sent in the same order they are generated »One time delivery: no duplicate events are sent »Push to a single end point with no throttling for V1.0. Pull is out of scope for v1.0. »Client event Listener Endpoint (SOAP or REST) and supported vendor event types defined in the RCR – RepositoryGeneralCapabilityType.RepositoryGeneralCapabilityType »Customer can disable in RCR supported vendor event types »RepositoryEventType: system, asset »RepositoryEventSubType: assetStatusChange, assetObjectCreated, assetContentCreated, assetContentDeleted, assetContentPurged, assetObjectUpdated, assetContentFormatUpdated, assetContentEssenceLocatorUpdated, assetObjectDeleted, repositorySystemException, performanceEvent. »RepositoryEventClassificationType: error, warning, information. »RepositoryOperationType: ingest, archive, delete

56. RepositoryEvent Message –RepositoryEventEnvelopeTypeRepositoryEventEnvelopeType RepositoryEventPayloadType –SystemRepositoryEventTypeSystemRepositoryEventType »RepositorySystemExceptionEventTypeRepositorySystemExceptionEventType »RepositorySystemPerformanceEventTypeRepositorySystemPerformanceEventType –AssetRepositoryEventTypeAssetRepositoryEventType »AssetStatusChangeEventTypeAssetStatusChangeEventType »AssetObjectCreatedEventTypeAssetObjectCreatedEventType »AssetContentCreatedEventTypeAssetContentCreatedEventType »AssetContentDeletedEventTypeAssetContentDeletedEventType »AssetContentPurgedEventTypeAssetContentPurgedEventType »AssetObjectUpdatedEventTypeAssetObjectUpdatedEventType »AssetContentFormatUpdatedEventTypeAssetContentFormatUpdatedEventType »AssetContentEssenceLocatorUpdatedEventTypeAssetContentEssenceLocatorUpdatedEventType »AssetObjectDeletedEventTypeAssetObjectDeletedEventType

57. Questions?

58. Do we need BMContentLightType?BMContentLightType BMContentType inheritance from BMContentLightType and move BMContentLightType to baseMediaService.xsd.BMContentTypeBMContentLightType Changing baseMediaService effect on previous fims versions implementation and rules for fims XSD(s) versioning. RepositoryProfileType: rename to something more generic and descriptive, like CredentialsProfile or AuthorizationProfile. Its scope can exceed the Repository and be used in other services. Perhaps, it should even live in the baseMediaService.xsd.RepositoryProfileType Do we need UpdateStatus operation?

59. Questions? Will the Lock operation on an object - BMContentType locks the associated contents – BMEssenceLocatorType?BMContentTypeBMEssenceLocatorType ResourceIDType as a parameter – REST. Many classes inheriting from bms:Resource – REST. I understand that if my repository exposes a custom searchable and updateable property it should be returned in GetSchemaExtensions, GetUpdatableProperties and GetSearchableProperties. Is this correct?GetSchemaExtensions GetUpdatablePropertiesGetSearchableProperties

60. Questions? What about "FIMS standard" supported properties (the ones defined in description-V1_0_7.xsd)? Should we also return them in GetSchemaExtensions, GetUpdatableProperties and GetSearchableProperties? GetSchemaExtensionsGetUpdatableProperties GetSearchableProperties Could we also expose standard FIMS technical metadata to be used in the searches? (For example, "title = NY and aspect_ratio = 16/9")

61. Questions? Locking is an optional feature, the token parameter should be optional in ReplaceObjectRequest, UpdatePropertiesRequest and DeleteContentRequest - DONE. Should we add a lock token to DeleteObjectRequestType?DeleteObjectRequestType Naming can be confusing at first sight. The functions and structures that deal with "BMContent" use "Object" and the ones that deal with "EssenceLocator" use "Content", which could be thought as BMContent. For coherence, the GetBMContent(s) functions should be GetObject(s) and RetrieveBMEssence() should be RetrieveContent()

62. Questions? QueryParameteType.valueType and RepositorySearchablePropertyInfoType.dataType, I am not sure if specifying datatypes by name using a string can create interoperatibility problems. Perhaps we should define and use a Resource "DataType" and provide a list of well-known fims data types, which could be extended by inheritance.QueryParameteType RepositorySearchablePropertyInfoType Do we plan to have a way of extending QueryParameterOperand with custom operands? It would be desirable. enumeration vs. object (resource) in xsd.

63. Questions? Purge Operations – Rename Delete operations - Rename AddObject – Rename AddContent – Rename (RegisterContent) Responses that contain BMContentType can we have an option in the content to return BMContentLightType instead. The use of container object with “choice” would this be a good represntation.BMContentTypeBMContentLightType RCR to define max. lock expiration duration (seconds, minutes, hours?) allowed by the system, null means no restriction.

64. Questions? RetrieveOpenLocks operation – RenameRetrieveOpenLocks RCR to define if dirty read allowed When a parent is locked will this lock the children? Do we need GetBMContentor GetBMContents should be enough?GetBMContentGetBMContents