clearMDM - API Guide v1.0€¦ · clearMDM – API Guide v0.1 (product version 1.20.x) Date: October 2016 Work in Progress Draft Product support can be contacted directly ([email protected])

clearMDM®isaregisteredtrademarkofAudit9LtdacompanyregisteredintheUnitedKingdomwithcompanynumber08088394. 1

clearMDM–APIGuidev0.1(productversion1.20.x)Date:October2016WorkinProgressDraftProductsupportcanbecontacteddirectly([email protected])forspecificquestionsnotaddressedbycontentofthisdocumentorforadditionalinformationorfeedback.

TableofContents

clearMDM–APIGuidev0.1(productversion1.20.x).........................................................1ProductIntroduction..................................................................................................................2KeyConcepts..............................................................................................................................3APIDesignPrinciples..................................................................................................................4

RESTAPI.........................................................................................................................................4CustomOperations........................................................................................................................4

KeyUseCases............................................................................................................................5MDMJobInvocation.....................................................................................................................5DistributedMDM-Point-of-entryProtection...............................................................................5DistributedMDM-Transactional..................................................................................................5

TechnicalSpecification...............................................................................................................6Authentication...............................................................................................................................6StatusCodes..................................................................................................................................6APIOperations–Low-LevelMDMResources...............................................................................6APIOperations–High-LevelMDMProcesses.............................................................................15

AppendixA–FlowDesigner–ExampleFlowAction.................................................................16


ProductIntroductionclearMDMisanativeSalesforceapplication(installedfromtheAppExchange®)thatdeliversamasterdatamanagementsolutionfortheSalesforce®platform.Therearenooff-platformcomponentstothesolution,customerdataisnotmovedoff-platformforprocessingbyexternalservicesandthereisnomobileapplication.TheclearMDMproductsupportstheMDMoperationsoutlinedbelowintheirlogicalsequence.(ABT)Normalisation.StandardisationoffieldvaluesusinglistCustomSettinglookups(Countrycodes,Root FirstNames etc.) or predefined transformation formats (email domain,URL, Title case etc.).NormalisationgeneratesBlockingKeys(ormatchkeys)usedasthefirstpassinrecordmatching.(ABTU)Matching.Cross-objectmatchingofrecordsusingkey,deterministicandprobabilistic(fuzzy)matchingrules.(B)InternalMatching.Singleobjectlargedatavolumematchingoperation.(ABU)Merge. Custommerge functionality compatiblewithStandardandCustomobjectsandalsoportal enabled accounts, person accounts and contacts. There is no dependency on the standardplatformmergecapability.(AB) Synchronisation. Master Records are updated directly to reflect fieldmodifications on theirSource Record. Synchronisation adds efficiency as the 2-step match and merge cycle is avoided;existingSourceRecordtoMasterRecordrelationshipsareutilised.(AB)Conversion.MasterRecordcreationonaone-to-onebasisforunmatchedrecordsorexternaldatawherematching isnotrequired.OncepairedMasterRecordsupdateddirectly toreflect fieldmodificationsontheirSourceRecords.(B)Re-parenting.ChildRecordsrelatedtoaSourceRecordcanbere-parentedtotherelatedMasterRecordvialookuprelationship.(B)CustomRollups.Numeric fieldsonChildRecordscanbeaggregated to theMasterRecord (vialookuprelationship).Prefixesindicateentrypoints:B=Batch(ScheduledJoborCustomAction–RESTAPI/ProcessBuilder)T=ApexTriggerU=UserInterface/ManualA=RESTAPI


KeyConceptsTargetObject.A compatibleobject that isdefinedas the target forNormalisationoperationsandwhereMasterRecordswillbecreatedorupdated.Data Source. A compatible object that provides data toMDM operations. Partition data sourcesenableasingleobjecttosupportmultipledatasourceseachwithdistinctconfigurations.AbroadrangeofStandardObjects(includingPersonAccounts)aresupportedasbothTargetObjectsandDataSources.CustomObjectsarealsosupported.MasterRecord.TheMasterRecordistheoutputoftheConversionandMergeMDMoperationsandis optionally related to the underlying SourceRecords via relationship field or simply through theconcatenationofrecordidentifiersintoafieldontheMasterRecord.SourceRecord.SourceRecordsare the input toMDMoperationsandmayberelatedtoaMasterRecord.SourceRecordscanberetainedordeleted.


APIDesignPrinciplesRESTAPIThe clearMDM API is a lightweight Web Service interface to the MDM operationsimplementedwithintheclearMDMmanagedpackage.TheAPIisimplementedasacustomsetofresourcesthatextendthestandardForce.comRESTAPI.ThedocumentationfortheForce.comRESTAPIcanbefoundatthelocationbelow.https://developer.salesforce.com/docs/atlas.en-us.api_rest.meta/api_rest/intro_what_is_rest_api.htmCustomOperationsThe clearMDMproductdesignphilosophy is todeliver a robust setof independentMDMoperations that can be assembled in a flexible manner to deliver the required datamanagement processes. Within the Salesforce application MDM operations can beorchestrated via Batch Job scheduling, manual invocation or through standard processautomationfeaturessuchasProcessBuilder.TheclearMDMAPIisconsistentwiththisdesignapproach. MDM operations (Matching, Merge etc.) are exposed as independent APIresourcesthatcanbeinvokeddirectlywhereexternalorchestrationisrequired.IntechnicaltermstheAPIresourcesareimplementedasApexActionsandcanbeinvokedwhereverApexActionsaresupportedontheSalesforceplatform.TheForce.comRESTAPIexposesApexActionsascallableresourcesviatheactionsendpoint(examplepathbelow)./v38.0/actions/custom/apex/clearmdm__RecordMatchingAction Force.comFlow(orVisualWorkflow)canbeimplementedtoorchestrateMDMoperationsintohigher-levelMDMprocessesthatblendsingleormultipleMDMoperationswithcustombusiness logic.FlowsareexposedasCustomFlowActionsvia theForce.comRESTAPI.Assuch,high-levelMDMprocessesimplementedasflowscanbeexposedasdistinctAPIcallableresources–examplepathbelow./v38.0/actions/custom/flow/clearmdm__MatchAndMergeSourceRecord TheclearMDMpackageincludesacompletesetoflow-levelAPIresources(ApexActions)andasingleexamplehigh-levelMDMprocess(FlowAction).AppendixAprovidesascreenshotofhowtheexampleflowappearswithintheFlowDesignEnvironment.


KeyUseCasesThe tablebelowprovidessummaryof thekeyuses that theclearMDMAPI is intended toaddress.Note,thisisnotexhaustive–andasoutlinedintheprecedingsection,customAPIresources are straightforward to assemble using Force.com Flow and the low-level ApexActionsthatexposeMDMfunctionality.MDMJobInvocationJobsthatrepresentbulkrecordMDMoperationscanbeinvokedviatheAPI.Thisallowsdataintegrationtools(ETL,Middlewareetc.)toinvokeMDMjobsasapost-processingstep.Thiscapability increases the efficiency of the end-to-end data processing cycle and removesdependenciesonscheduledtasks.DistributedMDM-Point-of-entryProtectionWhereclearMDMisimplementedinaMDMhubarchitecture,theAPIallowsexternal(spoke)systemstoefficientlychecktheexistenceofnewrecordsinthemasterdatasetwithinpoint-of-entry functions.Validationof this typecanpreventthecreationofduplicaterecordsatsource.DistributedMDM-TransactionalThe clearMDM API has been designed to support message-based or transactional dataintegrationflowswherenew/updatedsourcerecordsarepresentedinnear-real-timeviaanESB (Enterprise Service Bus) or equivalent technology. A message-oriented architecturetypicallyrequiresthatend-to-endrecordprocessingoccursonreceiptofthemessagenotviasubsequentbatch-styledataprocessingtasks.Forexample,itshouldbepossibletoreturnthemasterrecordidforanewsourcerecordwithinasingleAPIinteraction.


TechnicalSpecificationThefollowingsub-sectionsoutlinethekeytechnicalaspectsoftheclearMDMAPI.AstheAPIis implementedasanextension to theForce.comRESTAPI inmanyplaces referencesareprovidedtotheappropriatestandardAPIdocumentation.PrimaryForce.comRESTAPIResources:https://developer.salesforce.com/page/REST_APIhttps://developer.salesforce.com/docs/atlas.en-us.api_rest.meta/api_rest/Forclarity,theclearMDMisaRESTfulwebserviceinterfacethatexposesRESTresourcesforthepurposesofmanipulatingMDMoperationsforbulkrecordortransactionalrecordprocessing.TheAPIsupportsbothXMLandJSONformatmessages;JSONisthedefault.AuthenticationAuthenticationtotheForce.comRESTAPIisviathestandardOAuth2.0protocol.Acompletedescriptionoftheauthenticationmodelcanbefoundatthelinkbelow.https://developer.salesforce.com/docs/atlas.en-us.api_rest.meta/api_rest/StatusCodesTheprimaryindicatorofthesuccessorfailureofanAPIrequestisindicatedbytheHTTPStatusCode,thelistofapplicablevaluesisfoundatthelinkbelow.https://developer.salesforce.com/docs/atlas.en-us.api_rest.meta/api_rest/errorcodes.htmInfailurecasesitislikelythattheresponsebodywillcontainadditionalinformationinrelationtothecauseoftheerror.APIOperations–Low-LevelMDMResourcesThefulllistofApexActionsandFlowActionscanbequeriedasbelow.GET https://<instance>.salesforce.com/services/data/<version>/actions/custom/apex GET https://<instance>.salesforce.com/services/data/<version>/actions/custom/flow

Thedetailsofaspecificresourcecanbequeriedasbelow–theresponseincludesinputandoutputparameterdefinitions.GET https://<instance>.salesforce.com/services/data/<version>/actions/custom/apex/clearmdm__BatchJobRunAction


1. BatchJobRunAction

Purpose:InvokethespecifiedMDMJob(bulk-record)forimmediateprocessing.Path:/services/data/v38.0/actions/custom/apex/clearmdm__BatchJobRunActionInput DataType Required? Example DescriptiontargetObject Text Yes Account Account,PersonAccountetc.batchJobType Text Yes Matching Normalisation,Matching,

Mergeetc.dataSources Text No “LegacyAccounts” Commaseparatedlistof

DataSourcenames.processCount Integer No 5 1to10assupportedbythe

specificoperation.Output DataType Example Descriptionoutput Boolean True Jobsubmissionsuccessflag.

ExampleRequest: { "inputs": [ { "batchJobType": "Normalisation", "processCount": 1, "targetObject": "Account" }, { "batchJobType": "Normalisation", "processCount": 1, "targetObject": "Contact" } ] }

AstheJSONaboveshows,multiplejobscanbeinvokedfromasingleAPIcallastherequesttakesaJSONarrayastheinput.ExampleResponse: [ { "actionName": "clearmdm__BatchJobRunAction", "errors": null, "isSuccess": true, "outputValues": { "output": true } }, { "actionName": "clearmdm__BatchJobRunAction", "errors": null, "isSuccess": true, "outputValues": { "output": true } } ]

The highlighted section of the example response shows that both job requests weresuccessfullyvalidatedandsubmitted.


2. RecordNormalisationAction

Purpose:NormaliseoneormoreSourceRecords.Path:/services/data/v38.0/actions/custom/apex/clearmdm__RecordNormalisationActionInput DataType Required? Example DescriptionrecordId Text Yes 0010Y0000091mql SalesforceRecordId Output DataType Example Descriptionoutput Boolean True Operationcompletionflag. Pre-requisites : Source Records must exist within a Salesforce object that is correctlyconfigured(TargetObjectplusDataSource).ExternalrecordsmustbeinsertedorupsertedviaaseparateAPIcallbeforecallingrequiredMDMresources.ExampleRequest: { "inputs": [ { "recordId": "0010X0000051mqk" }, { "recordId": "0010Y0000051mql" } ] }

AstheJSONaboveshows,multiplerecordscanbeprocessedfromasingleAPIcallastherequesttakesaJSONarrayastheinput.ExampleResponse: [ { "actionName": "clearmdm__RecordNormalisationAction", "errors": null, "isSuccess": true, "outputValues": { "output": true } }, { "actionName": "clearmdm__RecordNormalisationAction", "errors": null, "isSuccess": true, "outputValues": { "output": true } } ]

Thehighlightedsectionoftheexampleresponseshowsthatbothoperationrequestsweresuccessfullyvalidatedandcompleted.


3. RecordMatchingAction

Purpose:Performmatchingforagivensetofrecordsandreturnmatchingresults.Path:/services/data/v38.0/actions/custom/apex/clearmdm__RecordMatchingActionInput DataType Required? Example DescriptionrecordId Text Yes 0010Y0000091mql SalesforceRecordId Output DataType Example DescriptionrecordId Text 0010Y0000091mql InputrecordId.matches Text 1ExistingRecordMatches[Record

Name:AcmeInc,DataSource:Account,Score:70.00,MatchType:Fuzzy]

Id,Namepairsformatchingrecords.

Pre-requisites : Source Records must exist within a Salesforce object that is correctlyconfigured(TargetObjectplusDataSource).ExternalrecordsmustbeinsertedorupsertedviaaseparateAPIcallbeforecallingrequiredMDMresources.ExampleRequest: { "inputs": [ { "recordId": "0010X0000051mqk" }, { "recordId": "0010Y0000051mql" } ] }

AstheJSONaboveshows,multiplerecordscanbeprocessedfromasingleAPIcallastherequesttakesaJSONarrayastheinput.ExampleResponse: [ { "actionName": "clearmdm__RecordMatchingAction", "errors": null, "isSuccess": true, "outputValues": { "matches": "1 Existing Record Matches [Record Name:BALANCE POINT LTD, Data Source:Account, Score:90.00, Match Type:Fuzzy]", "recordId": "0010X0000051mqkQAA" } }, { "actionName": "clearmdm__RecordMatchingAction", "errors": null, "isSuccess": true, "outputValues": { "matches": "1 Existing Record Matches [Record Name:BALANCE OUT LTD, Data Source:Account, Score:90.00, Match Type:Fuzzy]", "recordId": "0010Y0000051mqlQAA" } } ]


Thehighlightedsectionoftheexampleresponseshowsthatbothoperationrequestsweresuccessfullyvalidatedandcompleted.Thereturnedmatchingresultsprovidesthedetailsofanymatchingrecordsincludingrecordname,matchscore%andmatchtype(KeyorFuzzy).Note,theMatchingAPIresourcecreatesMatchedRecordPairrecordsforidentifiedmatches;thisapproachallowsasubsequentMergeAPIresourcecalltomergethematchesefficiently.AlaterversionoftheclearMDMAPIwillintroducetwochangestotheMatchingAPIresource;firstly the dependency that Source Records must be persisted before matching will beremoved, secondly the creation of Matched Record Pair data will become optional andcontrolledviaanadditionalinputparameter.


4. RecordMergeAction

Purpose:PerformmergeforagivensetofrecordsthathavepreviouslybeenmatchedandreturnthemasterrecordIdpersourcerecord.Path:/services/data/v38.0/actions/custom/apex/clearmdm__RecordMergeActionInput DataType Required? Example DescriptionrecordId Text Yes 0010Y0000091mqk SalesforceRecordId Output DataType Example DescriptionrecordId Text 0010X0000091mqk InputrecordId.masterRecordId Text 0010Y0000061mql RelatedmasterrecordId. Pre-requisites : Source Records must exist within a Salesforce object that is correctlyconfigured(TargetObjectplusDataSource).ExternalrecordsmustbeinsertedorupsertedviaaseparateAPIcallbeforecallingrequiredMDMresources.ExampleRequest: { "inputs": [ { "recordId": "0010X0000051mqk" }, { "recordId": "0010Y0000051mql" } ] }

AstheJSONaboveshows,multiplerecordscanbeprocessedfromasingleAPIcallastherequesttakesaJSONarrayastheinput.ExampleResponse: [ { "actionName": "clearmdm__RecordMergeAction", "errors": null, "isSuccess": true, "outputValues": { "masterRecordId": "0010X0000051mqqQAA", "recordId": "0010Y0000051mqlQAA" } }, { "actionName": "clearmdm__RecordMergeAction", "errors": null, "isSuccess": true, "outputValues": { "masterRecordId": "0010Y0000051mqlQAA", "recordId": "0010Y0000051mqlQAA" } } ]


Thehighlightedsectionoftheexampleresponseshowsthatbothoperationrequestsweresuccessfullyvalidatedandcompleted.Note,wherethesourcerecordisthemasterrecordforthematchedrecordgroupthereturnedmasterrecordIdwillbethesameastheinputrecordid.


5. RecordConversionActionPurpose:Performconversionforagivensetofrecordsandreturnresults.Note,conversionoperationsrelatetoExternalDataSourcesonly(TargetObject&SourceObjectaredistinct).Path:/services/data/v38.0/actions/custom/apex/clearmdm__RecordConversionActionInput DataType Required? Example DescriptionrecordId Text Yes 0010Y0000091mql SalesforceRecordId Output DataType Example Descriptionoutput Boolean True Operationcompletionflag. Pre-requisites : Source Records must exist within a Salesforce object that is correctlyconfigured(TargetObjectplusDataSource).ExternalrecordsmustbeinsertedorupsertedviaaseparateAPIcallbeforecallingrequiredMDMresources.ExampleRequest: { "inputs": [ { "recordId": "0010X0000051mqk" }, { "recordId": "0010Y0000051mql" } ] }

AstheJSONaboveshows,multiplerecordscanbeprocessedfromasingleAPIcallastherequesttakesaJSONarrayastheinput.ExampleResponse: [ { "actionName": "clearmdm__RecordConversionAction", "errors": null, "isSuccess": true, "outputValues": { "output": true } }, { "actionName": "clearmdm__RecordConversionAction", "errors": null, "isSuccess": true, "outputValues": { "output": true } } ]



6. RecordSynchronisationActionPurpose:Performsynchronisationforagivensetofrecordsandreturnresults.Path:/services/data/v38.0/actions/custom/apex/clearmdm__RecordSynchronisationActionInput DataType Required? Example DescriptionrecordId Text Yes 0010Y0000091mql SalesforceRecordId Output DataType Example Descriptionoutput Boolean True Operationcompletionflag. Pre-requisites : Source Records must exist within a Salesforce object that is correctlyconfigured(TargetObjectplusDataSource).ExternalrecordsmustbeinsertedorupsertedviaaseparateAPIcallbeforecallingrequiredMDMresources.ExampleRequest: { "inputs": [ { "recordId": "0010X0000051mqk" }, { "recordId": "0010Y0000051mql" } ] }

AstheJSONaboveshows,multiplerecordscanbeprocessedfromasingleAPIcallastherequesttakesaJSONarrayastheinput.ExampleResponse: [ { "actionName": "clearmdm__RecordSynchronisationAction", "errors": null, "isSuccess": true, "outputValues": { "output": true } }, { "actionName": "clearmdm__RecordSynchronisationAction ", "errors": null, "isSuccess": true, "outputValues": { "output": true } } ]



APIOperations–High-LevelMDMProcesses

1. MatchandMergeProcessPurpose:PerformMatchandMergeoperationsforagivensetofrecordsandreturntheMasterRecordId.Path:/services/data/v38.0/actions/custom/flow/clearmdm__Match_and_Merge_Source_Record

Input DataType Required? Example DescriptionrecordId Text Yes 0010Y0000091mql SalesforceRecordId Output DataType Example DescriptionFlow__InterviewStatus Text Created Oneof{Created,Started,

Finished,Error,Waiting}MasterRecordId Text 0010Y0000081mqq MasterRecordId Pre-requisites : Source Records must exist within a Salesforce object that is correctlyconfigured(TargetObjectplusDataSource).ExternalrecordsmustbeinsertedorupsertedviaaseparateAPIcallbeforecallingrequiredMDMresources.ExampleRequest: { "inputs": [ { "SourceRecordId": "0010X0000051mqk" } ] }

AstheJSONaboveshows,asinglesourcerecordcanbeprocessedbyeachAPIcall.ExampleResponse: [ { "actionName": "clearmdm__Match_and_Merge_Source_Record", "errors": null, "isSuccess": true, "outputValues": { "MasterRecordId": "0010Y0000051mqqQAA", "Flow__InterviewStatus": "Finished" } } ]

The highlighted section of the example response shows that the process request wassuccessfullyvalidatedandcompleted.


AppendixA–FlowDesigner–ExampleFlowAction

Documents

clearMDM - API Guide v1.0€¦ · clearMDM – API Guide v0.1 (product version 1.20.x) Date: October 2016 Work in Progress Draft Product support can be contacted directly ([email protected])