Upload
dwain-palmer
View
220
Download
2
Tags:
Embed Size (px)
Citation preview
SPECIFICATION OF THEFIMS
MEDIA SOA FRAMEWORK
REPOSITORY
SERVICE INTERFACESLoic Barbou
&
Ashraf Tadros
Services• Repository Service
– Repository Service Operations– Repository Capabilities Registry (RCR) Operations
• Repository Notification Service– Asynchronous Operations Notifications– Events Subscriptions
Repository Service - Operations» Lock, Unlock, Clear Lock, Retrieve Open Locks» Generate Unique ID» Add Object» Add Content (Async.)» Update Object (Replace, Update Properties)» Update Status *» Delete and UnDelete Object/Content» Purge Object (Async.)» Purge Content (Async.)» Get BM Content(s)» Retrieve Essence (Async.)» Media Query
Repository Service - RCR» Repository State (online, offline, failover)» Storage Capacity (Names, available and total space)» Performance (Performance Indicator %)» Vendor Specific Report (Report HTML)» General Capability (supported functionalities)» Vendor Defined Schema Extensions» Updatable and Searchable Properties» Vendor Defined Query Result Set Definitions» Repository Definitions (name & Description)» Retrieve Administrative Super Lock Token» Supported Object/Content Status
Notification Service - Async. Operations» Add Content (Notify and Fault)
» Purge Content (Notify and Fault)
» Purge Object (Notify and Fault)
» Retrieve Essence (Notify and Fault)
Notification Service - Events Subscription
– System Events» Exceptions» Performance
– Asset Events» Status Changed» Object/Content Created» Object Deleted» Content Deleted/Purged» Object/Content Format/Essence Locator Update
Clarifications• Asynchronous Operations Notifications
» Request => ResourceType => AsyncEndpointType» Ack: timeStamp + operationId» Notify Result: result + operationId» Notify Fault: RepositoryFaultType + some related info +
operationId
• Lock Token » RepositoryLockTokenType» RepositorySuperLockTokenType
Clarifications• Status
» Object - BMContentStatusType
» Content - BMEssenceLocatorStatusType
» BMStatusType - new, online, offline, deleted, purged, invalid, processing.
» Vendor specific - BMCustomStatusExtensionType.
» RepositorySupportedStatusType
Essence Locator StatusBMContentType Status
Online Offline Deleted Purged Invalid New ProcessingOnline x x x x x x xOffline x x x x x xDeleted x x xPurged x xInvalid x x x x x x xNewProcessing x x x
Clarifications• Object
» BMContentType
• Content» BMEssenceLocatorType - SimpleFileLocatorType,
ListFileLocatorType or FolderLocatorType)
• Light Object» BMContentLightType
Clarifications• Delete vs. Purge
» Delete: flags for deletion.» Purge: removes completely from DB and repository storage.» Purge object purges all related contents.
• Update Object vs. Update Properties» Update object: fully replaces the object.» Update properties: updates the specific passed property only.
• RepositoryProfileType
Repository Service Operations
GenerateUniqueID• Description
» Generates a Unique ID within the scope of a given repository.» Must be obtained before Object or Content Add operations.» Doesn’t have to be generated by this operation and can be
created by an external service as long as ID uniqueness is respected.
• Request» RepositoryProfileType
• Response» ResourceIDType
AddObject• Description
» Uses the generated Unique ID.» Submits a new BMContentType to be added to the system.
• Request» AddObjectRequestType
• Response» BMContentType
AddContent• Description
» Associate a physical essence – BMEssenceLocatorType (SimpleFileLocatorType, ListFileLocatorType or FolderLocatorType) to an existing object – BMContentLightType.
» Uses the generated Unique ID.» BMContentFormatType is optional input.
• Request» AddContentRequestType
• Ack» AddContentOperationAckType
AddContent Notification• Notify Result
» NotifyAddContentResult Operation => AddContentOperationNotificationType
• Notify Fault» NotifyAddContentFault Operation =>
AddContentOperationFaultNotificationType
ReplaceObject• Description
» Updates “metadata” properties of a BMContent object.» Behaves as an object replace not as update properties.» Sets omitted properties in input object to null. » A new version will be created (If versioning supported).
• Request» ReplaceObjectRequestType
• Response» BMContentType
UpdateProperties• Description
» Updates “metadata” of BMContent object (BMContentType).» Applies only on addressable and updatable properties –
RepositoryUpdatablePropertyInfoType (defined by the RCR).» PropertyInfosType parameter includes properties Xpath, values
and RepositoryActionType (update, new, remove).
• Request» UpdatePropertiesRequestType
• Response» BMContentType
UpdateStatus ( Remove !)
• Description» Updates object status only (BMContentStatusType not
BMEssenceLocatorStatusType)» Should the service allow status update from outside the repository
system?» Status automatically updated through the object and essence life
cycle and other operations (DeleteObject, PurgeContent ..)
• Request» UpdateStatusRequestType
• Response» BMContentType
Lock• Description
» Create a lock on a given resource or add a resource to an existing lock token.
» Adds a reference to be locked to the RepositoryLockTokenType .» Lock expiration is per RepositoryLockTokenType and not per
resource.
• Request» LockRequestType
• Response» RepositoryLockTokenType
UnLock• Description
» Removes the lock from multiple references.» If no references passed in the request, will unlock all resources
associated with the RepositoryLockTokenType .
• Request» UnLockRequestType
• Response» RepositoryLockTokenType
ClearLock• Description
» A mechanism to override a lock.» A RepositorySuperLockTokenType must be obtained from the
RCR before calling the operation.
• Request» ClearLockRequestType
• Response» RepositoryLockTokenType
RetrieveOpenLocks• Description
» Gets all open locks within the system.» A RepositorySuperLockTokenType must be obtained from the
RCR before calling the operation.
• Request» RetrieveOpenLocksRequestType
• Response» RepositoryLockTokensType
DeleteContent• Description
» Sets the BMEssenceLocatorStatusType for an active BMEssenceLocatorType (SimpleFileLocatorType, ListFileLocatorType or FolderLocatorType) to ‘deleted’.
» No physical bytes are deleted
• Request» DeleteContentRequestType
• Response» BMContentType
UnDeleteContent• Description
» Sets the BMEssenceLocatorStatusType for the BMEssenceLocatorType (SimpleFileLocatorType, ListFileLocatorType or FolderLocatorType) to an active status (invalid, online, offline ..).
» Repository system is responsible about tracking and resetting the status in the undelete operation implementation.
• Request» UnDeleteContentRequestType
• Response» BMContentType
DeleteObject• Description
» Sets the BMContentStatusType for an active BMContentType to ‘deleted’
» Repository system is responsible about setting related essences - BMEssenceLocatorStatusType for the BMEssenceLocatorType (SimpleFileLocatorType, ListFileLocatorType or FolderLocatorType) to ‘deleted’.
• Request» DeleteObjectRequestType
• Response» BMContentType
UnDeleteObject• Description
» Resets the BMContentStatusType for a deleted BMContentType.» Repository system is responsible about tracking and resetting the
status in the undelete operation implementation.» Repository system is responsible about resetting related essences
- BMEssenceLocatorStatusType for the BMEssenceLocatorType.
• Request» UnDeleteObjectRequestType
• Response» BMContentType
PurgeContent• Description
» Can only be executed on an Essence in a “deleted” state.» Physically removes essence from repository database & storage.
Physical bytes are deleted.» If the essence is the last representation in a
BMContentFormatType, the format entry will be removed from the BMContentType.
• Request» PurgeContentRequestType
• Ack» PurgeObjectOperationAckType
PurgeContent Notification• Notify Result
» NotifyPurgeContentResult Operation => PurgeContentOperationNotificationType
• Notify Fault» NotifyPurgeContentFault Operation =>
PurgeContentOperationFaultNotificationType
PurgeObject• Description
» Can only be executed on BMContentType in a “deleted” state.» Physically removes asset information and all related essences
from repository database & storage.
• Request» PurgeObjectRequestType
• Ack» PurgeObjectOperationAckType
PurgeObject Notification• Notify Result
» NotifyPurgeObjectResult Operation => PurgeObjectOperationNotificationType
• Notify Fault» NotifyPurgeObjectFault Operation =>
PurgeObjectOperationFaultNotificationType
GetBMContent
GetBMContents• Description
» Retrieves multiple BMContentType(s).» RepositoryMetadataFilterOutType: physical, editorial
• Request» GetBMContentsRequestType
• Response» BMContentsType
RetrieveBMEssence• Description
» Copy an essence BMEssenceLocatorType (SimpleFileLocatorType, ListFileLocatorType or FolderLocatorType) to a known supported destination by the repository – RepositoryKnownLocationType
• Request» RetrieveBMEssenceRequestType
• Ack» RetrieveBMEssenceOperationAckType
RetrieveBMEssence Notification• Notify Result
» NotifyRetrieveBMEssenceResult Operation => RetrieveBMEssenceOperationNotificationType
• Notify Fault» NotifyRetrieveBMEssenceFault Operation =>
RetrieveBMEssenceOperationFaultNotificationType
Query• Description
» Consumer is an orchestration engine.» A search parameter must match a property defined in the asset
schema BMContentType or RepositorySchemaExtensionType and defined in the RCR - RepositorySearchablePropertyInfoType.
» The max number of supported search parameter across all query operations is defined in RCR.
• Operation Structure
Query Payload
Query DefinitionQuery Definition
Output DefinitionOutput Definition
ParametersParameters
OperandsOperands
GroupGroup
Amount of DataAmount of Data
Data Structure Data Structure
Query
Query• QueryExpressionType : Simple Query: Title = NY
QueryParameterType• name = Title• value = NY• valueType = string• operand = equal• negated = false <queryExpression >
<resourceID>00000000-0000-0000-0000-000000000001</resourceID> <parameter> <resourceID>00000000-0000-0000-0000-000000000002</resourceID> <name>Title</name> <value>NY</value> <valueType>string</valueType> <Operand>equal</Operand> <negated>false</negated> </parameter></queryExpression>
Query
• Title = NY• OR• Title = Boston• OR
• (DateModified > 2010• AND• DateModified < 2012• AND• Not (DateCreated=2011))
• QueryExpressionType: Compound Query:
Title = NY OR Title = Boston OR (DateModified > 2010 AND DateModified < 2012 AND Not (DateCreated=2011))
Repository Capabilities RegistryRCR
RCR ?• Definition
» Datastore containing a list of properties describing the implementation behavior of the repository interface.
» Provide the ability for a vendor to implement a FIMS repository interface on top of their product based on its capabilities
» Service interoperability is achieved by having a common interface with a preset list of behaviors and settings (Support Lock, Extension Registration, Queryable Properties,… )
» External service can query the RCR to retrieve settings and behaviors
RCR Clarifications
GetRepositoryState• Description
» Retrieve configuration parameters indicating the repository service status: online = active, offline = inactive or not available and failover = instance of the service is not in use, but another instance is active.
• Request» RepositoryProfileType
• Response» RepositoryStateType: online, offline, failover
GetStorageAvailableCapacity• Description
» Retrieves the repository storage components and their available and total space.
• Request» RepositoryProfileType
• Response» StorageComponentsType
GetPerformanceIndicator
GetRepositoryReport• Description
» Retrieves a vendor specific health or general condition report.
• Request» RepositoryProfileType
• Response» RepositoryReportType
GetRepositoryGeneralCapability• Description
» Retrieves general supported functionality.
» RepositoryEventSubType: assetStatusChange, assetObjectCreated, assetContentCreated, assetContentDeleted, assetContentPurged, assetObjectUpdated, assetContentFormatUpdated, assetContentEssenceLocatorUpdated, assetObjectDeleted, repositorySystemException, performanceEvent.
» FormatProfileMetadataProcessingType: mustBeProvided, autoGenerated, autoGeneratedWhenNotProvided
» RepositoryEventListenerEndpointTypes: soap, rest
• Request» RepositoryProfileType
• Response» RepositoryGeneralCapabilityType
GetSchemaExtensions• Description
» Retrieve vendor defined metadata schema extension.
• Request» RepositoryProfileType
• Response» RepositorySchemaExtensionsType
GetUpdatableProperties• Description
» Retrieves the list of properties the repository system allows to be updated.
» RepositoryActionType: update, new, remove.
• Request» RepositoryProfileType
• Response» RepositoryUpdatablePropertyInfosType
GetQueryResultsetDefinitions• Description
» Gets a set of vendor specific defined result sets retuned by the Query operation.
• Request» RepositoryProfileType
• Response» QueryResultsetDefinitionsType
GetSearchableProperties• Description
» Retrieves list of properties exposed by the system to be used in the QueryParameterType.
• Request» RepositoryProfileType
• Response» RepositorySearchablePropertyInfosType
GetRepositoryDefinitions• Description
» Retrieves the internal repository system volumes.
• Request» RepositoryProfileType
• Response» RepositoryDefinitionsType
GetSuperLockToken• Description
» Retrieves a super token to override existing RepositoryLockTokenType to unlock specific resources.
• Request» RepositoryProfileType
• Response» RepositorySuperLockTokenType
GetSupportedStatus• Description
» Retrieves the system supported statuses: BMContentStatusType, BMEssenceLocatorStatusType, and vendor specific BMCustomStatusExtensionType for the content and object.
• Request» RepositoryProfileType
• Response» RepositorySupportedStatusType
Repository Notification Service
RepositoryEvent• Description
» Single event per message.» SOAP and REST» Ordered delivery: events are sent in the same order they are generated» One time delivery: no duplicate events are sent» Push to a single end point with no throttling for V1.0. Pull is out of scope for v1.0.» Client event Listener Endpoint (SOAP or REST) and supported vendor event types defined in
the RCR – RepositoryGeneralCapabilityType.» Customer can disable in RCR supported vendor event types» RepositoryEventType: system, asset» RepositoryEventSubType: assetStatusChange, assetObjectCreated, assetContentCreated,
assetContentDeleted, assetContentPurged, assetObjectUpdated, assetContentFormatUpdated, assetContentEssenceLocatorUpdated, assetObjectDeleted, repositorySystemException, performanceEvent.
» RepositoryEventClassificationType: error, warning, information.» RepositoryOperationType: ingest, archive, delete
RepositoryEvent• Message
– RepositoryEventEnvelopeType• RepositoryEventPayloadType
– SystemRepositoryEventType» RepositorySystemExceptionEventType» RepositorySystemPerformanceEventType
– AssetRepositoryEventType» AssetStatusChangeEventType» AssetObjectCreatedEventType» AssetContentCreatedEventType» AssetContentDeletedEventType» AssetContentPurgedEventType» AssetObjectUpdatedEventType» AssetContentFormatUpdatedEventType» AssetContentEssenceLocatorUpdatedEventType» AssetObjectDeletedEventType
Questions?
Questions?• Do we need BMContentLightType?• BMContentType inheritance from BMContentLightType and move
BMContentLightType to baseMediaService.xsd.• Changing baseMediaService effect on previous fims versions
implementation and rules for fims XSD(s) versioning.• RepositoryProfileType: rename to something more generic and
descriptive, like CredentialsProfile or AuthorizationProfile. Its scope can exceed the Repository and be used in other services. Perhaps, it should even live in the baseMediaService.xsd.
• Do we need UpdateStatus operation?
Questions?• Will the Lock operation on an object - BMContentType locks the
associated contents – BMEssenceLocatorType?• ResourceIDType as a parameter – REST.• Many classes inheriting from bms:Resource – REST.• I understand that if my repository exposes a custom searchable and
updateable property it should be returned in GetSchemaExtensions, GetUpdatableProperties and GetSearchableProperties. Is this correct?
Questions?• What about "FIMS standard" supported properties (the ones defined
in description-V1_0_7.xsd)? Should we also return them in GetSchemaExtensions, GetUpdatableProperties and GetSearchableProperties?
• Could we also expose standard FIMS technical metadata to be used in the searches? (For example, "title = NY and aspect_ratio = 16/9")
Questions?• Locking is an optional feature, the token parameter should be
optional in ReplaceObjectRequest, UpdatePropertiesRequest and DeleteContentRequest - DONE.
• Should we add a lock token to DeleteObjectRequestType?• Naming can be confusing at first sight. The functions and structures
that deal with "BMContent" use "Object" and the ones that deal with "EssenceLocator" use "Content", which could be thought as BMContent. For coherence, the GetBMContent(s) functions should be GetObject(s) and RetrieveBMEssence() should be RetrieveContent()
Questions?• QueryParameteType.valueType and
RepositorySearchablePropertyInfoType.dataType, I am not sure if specifying datatypes by name using a string can create interoperatibility problems. Perhaps we should define and use a Resource "DataType" and provide a list of well-known fims data types, which could be extended by inheritance.
• Do we plan to have a way of extending QueryParameterOperand with custom operands? It would be desirable. enumeration vs. object (resource) in xsd.
Questions?• Purge Operations – Rename• Delete operations - Rename• AddObject – Rename• AddContent – Rename (RegisterContent)• Responses that contain BMContentType can we have an option in
the content to return BMContentLightType instead. The use of container object with “choice” would this be a good represntation.
• RCR to define max. lock expiration duration (seconds, minutes, hours?) allowed by the system, null means no restriction.
Questions?• RetrieveOpenLocks operation – Rename• RCR to define if dirty read allowed• When a parent is locked will this lock the children?• Do we need GetBMContentor GetBMContents should be enough?