Global Keys: A Unified KeyMapping Architecture

grationhaving

By Eric Copenhaver

K ey mapping, the linking ofunique identifiers from mult i -ple systems, is d widely used(t'chniqui' to address data inle-

challenges brought about bycommon information scattered

<icross muitiplf tipplitafions. The typicalfnterprise uses key mopping today fordata integration because databasesemploy loci l unique idenlifiers as Ihemost common ntivigation path to apiece of data and because most enter-prises have the unfortunate need to storeidentical, similar or related facts in manyplaces. Some optimisls claim thnt thisscattering of data will subside as mono-lithic enterprise applications deliverrequired business functionality. However,most realists underHt.ind that Ihe scatter-ing md)' subside but wil l never disap-pear Others fantasize that the panacea ofmaster data management (MDM) willbring us to the more enlightened state ofone fact in one place, Greal stuff, hulduring this slow and painful journt-y toparadise, we mere mortals must continueto rely on key mapping to rationalizenative identifiers of common legaty data.

The key mapping o[»portunity existsfor many data subjects, including cus-tomer, product, person, place, machine,state and even .simple domains such asdctivit)' status and unit of measure.Regardless of subject, it turns out to bedie very same logical problem. We needto know that this unique idrntifier overhere corresponds to that unique identifierover there to meet the everyday recpjire-ment of integrating this common dataappearing in multiple applications,

VVe all understand the underlyingbusiness need to consolidate, integrate orsynchronize common data f rom mul t i -ple applications. After all, this is what IT

does. Overall, integration could be themost common and expensive require-ment in all of IT. I am sure that examplesof data integration opportunities aboundin ever)' organization and need not helisted here. It probably would not be toorisky to assert that 80 percent of all ITpain comes from this business require-ment alone- At the same time, data inte-gration may be what provides 80 percentof the business value, and it may also bethe source of 80 percent of business dis-satisfaction with IT, Sure, it's painful, but

16 January 2008 I DM Review

we gotta have it,l lowfvtT, integration is not really

the problem addressed here. Sure, it is anopportunity, but one that has alwaysexisted, and IT continues to provide dataconsolidation, integration and synchro-nization as best i t can. Key mappingdesign is not necessarily the problem,either It is pretty straightforward - cer-tainly not rocket science - although thrrcis often some complexity arising fromsemantic dissonance and from schemareconciliation required for subtly different

www.dmreview.cam

implementatlotts of a common concept,such as physical address. There are manyways to solve key mapping, and mostsolurion architects have found a way toaddress it one way or another.

The much larger business problem isthe more fundrimmtal issue that IT isw.isting important resources (time,money, talent) solving this same challengerepeatedly in different ways and with l im-ited scope. Not surprisingly, this results induplicatrd eftort, unnecessary cotnplexity,inocccssibie translations, redundant andconflicting mappings and maintenancesprtvid across multiple areas of expertise.The sucking sound of importantresources going down thr dniin should

^ Data integration may bewhat provides 80 percent ofthe business value, and itmay also be the source of 80percent of business dissatis-faction with IT.

be doalening, but no one hears it becauseihe key mapping design effort is repeatedquietly within every project that has inte-gration requirements (all, wouldn't thatbe t'Vfr\' project ever?)- IT is applyingsmall amounts of time, monq' and tolcntnil key mapping design so frequently thatIt adds up to Itirge amounts of time,money dnd talent. Now that's a problem.

Previous OptionsTo address this ubiquitous key mappingnppnrtiinity, ihe typical enterprise has thelull gamut ol solutions, developed inde-pendently over the years, each with itsscope limited to a particular type of data.Each implementation may be detlitated toa particular data How, application, dala-ba.sc, subject area, development platform oriiitcgration technology. The scope is alwayslimited. Consf-qucntly, multiple independ-ent solutions exist lor essentially the sameproblem, sometimes right next to eachother at the same data integration point.

Perhaps the key mapping is hardcoded in a data interlace using if-thcn-else or extract, transform and load (El l )transformation logic (IF source.status_key= "A". THEN target.statusJD = "2"). Ormaybe a column has ba=n added to a localdatabase table to store another system's

key (SAP_VENDOR_ID), Anotherapproach is t:o dedicate a local table to aparticular mapping (VENDOR_XREF) ora generic table for all local mappings(XRHF table with a XREiLTYPE column).Perhaps mappings are managed withinintegration middleware. Each of thesesolutions bites od a larger piecT handlinga broader scope, but tJieR' i.s the potentialfor overlap of solutions and key mappingdiscrepancies when many solutions coexist.

Global Keys SolutionFuture-thinking architects are gravitatingto a universal key mapping solutionknown as global keys, designed generi-cally for multiple data subjects, imple-

mented once,wrapped in Webservices and ut i -lized broadly acrossthe enterprise.

The premise otthe globfil keysapproach is thatcentralizing keymapping as meta-data and isolating itfrom any particularapplication is themost effective way

to Increase its,value as a sharable enter-prise information asset. We can think ofglobal keys as a master repository' of keys.perhaps implemented as an extension toan existing metadata repository', manag-ing information about technology com-ponents such as applications, databases,tables, columns, servers, workstations, etcWhen we view key mappings as metadata,we realize ihat they should not be ownedby a particular application, nol even MDM,Cloistering key mappings within an appli-cation diminishes their value by limitingexposure, access and reu.sc across theenterprise, Tliis is happening everywhereand every day.

The global keys architecture isolateske)' mappings from data conlent. Globalkeys contain only native keys and theirassociations. We know that these keys areimmutable (lefs hope so - keys that(hango are their own special problem),and we suspect that their associations arestable, but the data content itself is likelyto be more volatile. In global keys, wecapture the ke)'s and support Ihe man-agement of their associations. Where andhow to manage tlie actual date contentcan he addressed independently on acase-by-case basis. In the system ofrecord? Distributed across multiple appli-

cations? For each subject area, this deci-sion will depend on many factors, and itwill likely be a volatile mix ol these datamaintenance platforms for years to come,1 lowever. key mapping does not need tobe addressed independently and does notneed to be a volatile mix of platforms, A.safe and sensible decision can be made todeploy tlie global keys architecture cen-trally and stabilize the key m.ippinj;aspect to build an enduring and v.iluahlcenterprise asset,

'ITie global keys architecture also iso-lates the management of key mappingsfor a data subject from the actual matdidetermination made with human inter-vention or using domain-specific toolssuch as those for customer or productdata management or for MDM, In asense, we pull the management of persist-ent mappings out of M D M and put thatmanagement into a set of services focusedentirely on registering keys and updatingtheir associations. Decoupling the datamapping determination from the resultsenables various mapping tools and tech-niqiie.s to handle the unique matchingchallenges of their special domain whileusing a common key mapping dataresource. The consolidated global keysdata resource can then be exposedbroadly (on the enterprise service bus,perhaps) and accessed widely under asingle security framework. At last, in lieuof one fact in one place, we can deliverand manage one key fact in one place.

Once established, the global keysasset can be consumed broadly for inte-gration opportunities suth as applicatinriintegration, data consolidation and dalaintegration commonly present in {imenterprise. Each of these types of integra-tion has a need for key translation.

Eliminate Cost RedundancyAny lime multiple solutions or patternsemerge for the same problem, as theyhave for key mapping cxjst-redudionoppoiiunities exist- When you think of allthe assets that need to be inaniiged tosupport any solution lifecyde, tlie costsbegin to add up. Outlays for technology,licensing, design, documentation, aj)plica-tion c:ode, training, tools and upgrades arerec^iired for every type of implementa-tion- We know that cost containment hasbeen a top issue for IT over Ihe years, )'etwe continue to reinvent the wheel timeand time again. It follows Ihat consolidat-ing the various key mapping solution.-iunder a unified global keys archiledure

COPENHAVER continued on page 34

18 lanuary 2008 I OM Review www.dmreviewcom

FEW continued from page 15here?" which you can immediately inves-tigate by interacting with the data, such asby zooming in to take a closer look andtillering out what's irrelevant.

A few good products that demonslratethe strengths of intovis are now establish-ing a presence in the business place, indud-ing Tableau from Tableau Software, SpotfireDedsionSite and Spottire DXP Irom TIBCO,IMP from SAS and Advizor Analyst fromAdvizor Solutions, Linlike these goodexamples, however, many so-called visuali-zation products are poorly designed or missthe mark entirely They misunderstand howinformation visualization works - howvisualizations must be designed to utilizeIhe strengths of visual perception and toaugment cognition - sometimes re.sulting inprodLicts that no! only work poorly, theydon't work at all. To put this into perspec-tive, just think back lo the early days of theInterne! and remember how poorly mostWeb sites were designed.

Business intelligence (BIl vendors arenow paying attention to infovis. They arebeginning to see that traditional methodsof data analysis involving tabular text-based displays arc extremely limited. Theymust now take the time to sliidy infovisand learn from iht' experts belort- devel-oping products tliat claim to support it.As software consumers begin to recognize!hf difference between superficially cutevisualizations and (hose that actuallyenligliten, vendors will no longer be ableto feed their customers cheap candy inplace of the nourishment they actuallyneed. As vendors embrace infovis in aninformed manner. Ihe intelligence that BIhas always promised might finally bewithin reach.

Don't be fooled by dazzle over rflec-tiveness or entertainment over usefulness.Take tlie time to step back and ask. "Doesi! reaiiy work and does it really matter?"Use yoLir eyes. They're more powerfulthan most nl us realize, 0

jStiurl K. t,<ird .iiid ktrk D. M<ifkiiil;iy, Raiiliiuis inIti/iirmmioN Vi»uiliziilwn- Usimj V'tiitiii Jo ThinkMarj an Kaufrnann Publishers; I'.'W.Colin VVjrc. liilonmtm VifualiZiilion: Perception lor

f'(5iijri, Mor.g.in K.itilhicinn I'litilishcrs; 2004.

l-m. /(fiiiiijiiil ii/ Ilir ivmuHiiiU} l'mq>Uiittwi worked l\ir J,i ffiirs as im il iummitor. con-iimt eiluciiUir. Vuri mil Ifnni more abotil i-fw's

COPENHAVER continued from page 18

will Iu-I[> reduce reduiKiani costs and cutIT spending for the enterprise.

Single Scalable SolutionProvitiing a single scalable solution ior acommon opportiinity is a worthy goal forany information or solution ardiitect.Designing once and utilizing many timesdelivers benefits beyond providing theenterpri.se with an eflkient utilization oldesign resources and typical co.st savings.The narrow functional scope and broadapplicability' of global keys promotes aflexible, extensible design yielding a stableand durable implementation and provid-ing value for the long term. The globalkeys architecture provides a focused, fle.v-ible, simple soiution to one of the mostwidespread integration needs in all of IT- key mapping to support applicationintegration, data consolitiation and dataintegration across all data subjects.

One Fact in One PlaceOne key feet in one place wrapped in acompreliensive set of services provides auniversal platform to be accessed widelyas a truly reusable information resource.With a global keys archiietiure. the entiretechnical enterprise knows where to go totap into this valuable knowledge asset.Any new data consolidation opportunity,such as a new aggregation or cross-appli-cation integration, may easily leverageexisting key mappings. Once tlie globalkeys infrastructure is established, a newsource or nev\' domain can be defined !oimmediately register and persist a neces-sary' cross-application key mapping withonly a small investment in Web servicecalls or messages. Global keys provides acrucial component for successful dataintegration.

ImplementationA global keys implementation maps appli-catit)n data stores such as tables to globalentities. The existence of global entitiessuggests an enterprise logical data modelthat defines tiie hub of the global keysarchitecture. The first step of a global keysimplementation is the development of aglobal or enterprise data model definingtile common gbbal entities having keys towhidi kxal ke\'S are mapped. Of course,we know tliat an enterprise logical datamodel exists for every enterprise (indud-ing yours). In some enterprises, it Is

explicit; in others, implicit. As in mostendeavors, explicit outperforms implicit. Iwould be preaching to the choir f or thosewho understand this and wasting mybreath on those who do not.

Because global keys are essentiallymetadata, decisions are required aroundintegration with existing metadata. Alongwith keys, the global keys architecturedescribes data repositories, data storesand logical entities that may already bemanaged as metadata. Perhaps the globalkeys metadata of keys can simply extendthe existing metadata repository Or per-haps a new home for global keys meta-data is established with metadata itselfbeing the first domain lo map. Scary,

Establishing atiministrative Webservices or user interlaces (or both) tomanage definitional metadata is a prereq-uisite to capturing keys. Once again, beingexplidt about roles, prc)cesses, approvalsand the governance of global keys and itsassociated metadata is conducive to asuccessful implementation.

Comprehensive services enabling ,theregi.stration and assignment of keys pro-vide the interfaces required to build theglobal keys information resource,Transactional services are packaged iosupport bulk-loading requirements,Ejcposing the.se services using precise andunambiguous descriptions, training, con-sultation and ongoing support alsi,i fosterssuccess.

Implementations of small independ-ent domain.s can help build momentumand gain repeat customers when succe.ss-fltl. Starting with relatively static map-pings that are commonly referenced helpsdeliver confidence in the global keys plat-form. Building key mapping assets thaihave greater complexity, such as one-to-many mappings (i,e,, one local table tomany global entities), can be implementedwitli confidence over time.

Deploying a global keys architectureto address the commonplace key map-ping opportunity as a unified, exten.sibleutility is a cost-effective solution tha! condeliver a popular enterpri,se .service that isnarrow in scope, broad in utility, simplein design, quick to implement and cen-trally supported, #

lirii Copeiibuvn is a ilala iinbai'il mui /Mihitum

Iraimwork priuliluinrr consuttiiuj in Stvak. Jlr ttmi'

dc miiJirrl lJiri>iii;li liU htnij i rt tjtiA'iit kry-i <a

34 January 2008 t Dt Review www,d mreview.com