Atrium Integration Engine: What Is It? How Do You Use It?

AgendaAtrium Integration Engine (AIE) OverviewWhat is it?Overview of latest features

Tell me how to do itUnder the hood details of creating CMDB Integration using AIE

Q&A

CMDB-Enabled Atrium Integration EngineCMDB Enabled AIE with Adapters

Mapping TechnologySynchronizationImport/ExportData TransformationMulti threaded

BMC ConfigurationDiscoveryBMC Foundation &Topology DiscoveryOracleDatabaseXML FileSQLDatabaseDelimitedFileDB2DatabaseAPI

BMC Atrium Integration EngineWhat is it?A mechanism to manage data transfers between external data sources and CMDB (also AR Forms)

Bi-directional and multi-threaded data transfer engine

An Enterprise Application Integration tool with data transformation capability

It supports event-based or scheduled information flow

Ideal for bulk, incremental, data synchronization transfers

Provides interfaces to external data source and the CMDB

Provides an intuitive graphical interface for field mapping configuration

BMC Atrium Integration EngineHow is it used?Transform data to the CMDB data modelEfficiently load data in bulkCan create relationships between instances as they are inserted in the CMDB.AdvantagesDesigned for complex integrationsEasily configurableCan be used against a variety of applicationsExtensibleWhen Is AIE a Fit?Complex transfers requiring data transformation Handling large amount of data transfersMultiple integration points

BMC Atrium Integration Engine 7.1New Name and Repackaging for EIEReleased in late August 2007Broad Enhancement Areas Performance and ScalabilityImproved performance of Relationship exchangesBetter memory management for all update operationsUsability ImprovementsLot of usability improvements (please see the release notes for details on new features)PackagingShips with Atrium CMDB 2.1 to integrate with Atrium CMDBOne product, multiple adaptersOracle SQL DB2 New!XMLNew!XML adapter only supports class based relationships (not vendor)Flat fileImproved!

How to Create AIE Based Integrations to CMDB

The Ingredients To SuccessData Structure KnowledgeExternal Data from Discovery/InventoryAtrium Integration EngineAtrium CMDBReconciliation RulesWorkflow Filters if Needed

AIE TerminologyData Exchange Application (DEA)AR System based application at the heart of AIEAIE ServiceDEA passes the service instructions on how to interact with the adaptor Data MappingCMDB Class/AR Form mapped to External TableCMDB Attributes (or fields of AR Form) mapped to Columns of External TableValues mapped to External Table ColumnsExchangeCollection of similar MappingsDefinition of when to runPipe that manages the flowAdapterThe Plug at the end of the wireDataExchangeApplicationData MappingsData ExchangesAdapterAIEService

Configuration SQL ExampleDiscovery EnvironmentRemedy EnvironmentSQL DatabaseDiscoveryAR System

(can be on either server)DataExchangeApplicationDatabase for AR SystemCMDBDiscovery DatasetData Cleansing WorkflowCMDBBMC.ASSET DatasetRemedyAssetManagementCMDBReconciliationEngineCMDBReconciliationRulesData MappingsData ExchangesAdapterAIEService

External Data SourceDatabase dataXML dataFlat file dataSource Relationships can be stored as foreign keyrelationship tables

Atrium CMDBConfiguration Management DatabaseStore multiple datasetsReconcile datasetsStore RelationshipsBase data for OOTB ApplicationsData SetsData value in a field... but so much moreData Sets you should useBMC.ASSET The Gold StandardBMC.SANDBOX Temp location for GUI changes to AssetsVENDOR.PURPOSE.PRODUCT custom datasetsExamples: BMC.IMPORT.MARIMBA, MS.IMPORT.SMS, ACME.IMPORT.LEGACY

AIE Structure SimpleData ExchangeData Mapping1-NRelationship ExchangeRelationship Mapping1-NOn Complete, Trigger Next

AIE Structure ComplexData MappingComputerData ExchangeRelationship ExchangeRelationship MappingComputer O/SData MappingComputerData MappingComputerData ExchangeData MappingOperating SystemData ExchangeData MappingDrivesData ExchangeData MappingMonitorData ExchangeData MappingSoftwareTrigger NextRelationship ExchangeRelationship MappingComputer - DrivesRelationship ExchangeRelationship MappingComputer - MonitorRelationship ExchangeRelationship MappingComputer - SoftwareTrigger NextTrigger NextTrigger NextTrigger Next

AIE Main Console

Configuration > Database Field MenusPopulates the vendor table and column menusRun before you start any mapping work

1. Load2. Add Desired Tables3. Create Menus

How To Data MappingMapping Name = name for new mappingDirection = External Data into CMDBNamespace = BMC.CORE in most casesClass = Destination for data, pick from the listExternal Data Store = SqlServer, Flatfile or OracleTable Name = Source of data (pick from list) if not in the list, run rulehelper.exe to create the menu items.Row Element = (XML only) the element that represents a rowClick Create to save

How To Data MappingIf the Table Name does not show in the drop down list, it is because you have not configured the Database Field Menus from the main console

How To Data MappingUsed to map source and destination side primary/foreign keysKeys must be unique on both sidesIf multiple keys, stored as AND to define Unique

How To Data MappingMap appropriate CDMB Class.Attribute to a Data Store AttributeMake sure to specify CMDB mandatory fields such as Fields use in Key mappingsData Set ID (automatic if Use Dataset is checked)Short Description Submitter Name = anything that identifies the entryAssetLifeCycleStatus = 3 (if Asset Management is installed)

How To Data MappingRules Works like Function key in Remedy User and Administrator ToolsSelect by highlighting the row and click AddRule is addedRefer to the Rules box for hints on how to complete the ruleAvailable Rules

How To Data MappingOnly works if you can write to the Vendor / Source DatabaseCan be any valueUsually is completion status0 = transfer with no errorN = error # - corresponds to ARERROR

How To Data MappingIf you Update Row to Indicate Delete i.e. Soft DeleteSet MarkAsDeleted to 1 (Yes)Other attributes may be set as desiredIf you choose to Permanently Delete Hard DeleteNo field to set. AIE will do trigger the deletion

How To Data MappingUse to limit amount of data in a given transfer. Important if you are experiencing performance or memory issuesIf you specified Restrict to Dataset on the Main Tab, you dont need to include it here.

How To Data MappingIf set to all data that can be transferred will be. This could result in partial data records.Transfer as NULL cleans up any data that might be mistaken for null in a fieldIf trailing blanks are possible in your key values, choose to considerStores change history in a diary field you designate. Attribute must exist in CMDB.If Yes, updates are only made when a change is detected in the source dataYou must select a field that can store a negative integer to store the checksumAllows you to retrieve new or change data since last transferYou must select a date/time field to store last transfer time

1234

56

How To - Data ExchangeExchange Name Name you create to identify the exchangeInstance Name Specify the AIE thread to useData Exchange Type Specify the type of mapping contained in the exchangeExternal Data Store Must match the mapping(s)Execute Exchange AsSchedule OnlyUse Schedule tabEvent Driven Onlyrun eiexfer.exe from command line, filter or active link

How To - Data ExchangeSelect the mapping(s) to include in the Exchange container.Multiple MappingsCuts down on queries for source or targetMain MappingNeed to designate if multiple mappings includedKey values only come from Main Mapping

How To - Data ExchangeIntervalWorks like the interval function in escalation workflowOn Completion SectionWhen this exchange is done, you can configure another exchange to run

How To - Data ExchangeTimeWorks like the time function in escalation workflowSet day and time to run

How To - Data ExchangeTo Be TriggeredUse this option when you want to exchange to run on completion of some other exchangeMust list the exchange in the On Completion section on the other exchange

How To - Data ExchangeYou can either set the external connection information on each exchange or on the Connections link from the Data Exchange ConsoleAlternate MethodUse if the AR Server involved in the exchange is NOT the server that DEA is installed on set to No and fill in the info.

Advantage 1 license/DEA can service Dev, Test & Production

How To - Data ExchangeWhile TestingLimit records so you have less to undoIn Production Set Enable Debug to No

How To Relationship MappingData MappingData MappingVendor Relationship MappingMap using External Data Store Mapping.Trigger on Complete of Data Exchange.V_ComputerV_DiskDriveVendor Relationship TableBMC_ComputerSystemBMC_DiskDriveBMC_HostedSystemComponent

How To Relationship MappingData MappingClass Relationship MappingMap using CI Class Relationship MappingPut in an Event Driven Only Exchange*

*In this case, the relationship data exchange only runs after the data transfer for the two configuration item classes has completed.V_ComputerV_DiskDriveBMC_ComputerSystemBMC_DiskDriveBMC_HostedSystemComponentData Mapping

How To - Relationship MappingThis is where you define which 2 classes to relate and which relationshipMust Select Relationship Source on CreateExternal Data into CMDB is the only valid DirectionIf you choose Relationship Source of External Data Storemust specify DB2, FlatFile, Oracle, SqlServerTable NameIt is best to Restrict to DatasetID and specify

How To - Relationship MappingDefine mapping criteria that defines the relationshipWorks just like creating Relationship in CMDBSource on Left, Target on Right

How To - Relationship MappingUsed to define values for a relationship instanceMandatory attributes shown above must be mappedTo get a list of Attributes of each CI Class choose class from drop down. Attributes list will then appear

How To - Relationship MappingUsed to define values for a relationship instanceMandatory attributes shown above must be mappedTo get a list of Attributes of each CI Class choose class from drop down. Attributes list will then appear

How To - Relationship MappingIf you Update Row to Indicate Delete i.e. Soft DeleteSet MarkAsDeleted to 1 (Yes)Other attributes may be set as desiredIf you choose to Permanently Delete Hard DeleteNo field to set. AIE will do trigger the deletion

How To - Relationship MappingUse if you are doing class based mappingShould match query used on class mappingIf you specified Restrict to Dataset on the Main Tab, you dont need to include it here.

How To - Relationship MappingUse if you are doing class based mappingShould match query used on class mappingIf you specified Restrict to Dataset on the Main Tab, you dont need to include it here.

How To - Relationship MappingUse if you are doing class based mappingShould match query used on class mappingIf you specified Restrict to Dataset on the Main Tab, you dont need to include it here.

How To - Relationship MappingUsed only if a External Data Store mappingThis is used to limit the external records returned

How To - Relationship ExchangeWorks the same as a normal Data ExchangeData Exchange Type = Relationship Class

General AIE HintsWhen testing an exchange Test one exchange at a time (when done chain them to be triggered)Use Run Now button to trigger an exchange immediatelySet Limit = 1 (so that you only run once)Be sure you know your debug pathWhen writing a Vendor Query Test it with Direct SQL If it doesnt work there, it wont work hereUse Indexed fields if possibleThis wont work MacAddress != NULLThis will MacAddress IS NOT NULL

General AIE HintsWhen setting an AR System field in a mapping Doesnt work like a setfield or pushfield operationMust Use Rules to convert functions, keywords, etc.

Example of ConcatenationThis wont work $Manufacturer0$+-+$MediaType0$This will function|concat($Manufacturer0$,-,$MediaType0$)Example of an Integer FieldThis wont work 0This will Constant|0

Atrium CMDBReconciliation RulesIdentify Look for potential duplicatesCompareMergeCleanup / PurgeCan delete entire data set from all tablesCan delete selected itemsConfiguration Item Set field MarkedForDelete = YesCreate Reconciliation Rule to Purge if field is setDefine how to handle related itemsRun as a Reconciliation JobCMDB 3rd PartyDatasetCMDB Reconciliation EngineCMDB Reconciliation RulesCMDB MasterDataset

How Atrium Integration Engine is Acquired

BMC AIE is bundled withBMC Service Management Products

AIE comes bundled at no additional cost with BMC applications which ship BMC Atrium CMDB 2.0.01 onwards, such as:Service DeskChange ManagementAsset ManagementService Request Management (SRM)Configuration Management *Service Impact Management *Topology Discovery, Foundation Discovery, and Identity Discovery * * AR with 3 user licenses are also bundled (paper license) limited to support the CMDB (and DSL) Existing ITSM customers can upgrade to v7.0.02 and later versions to get AIE at no cost to integrate with BMC Atrium CMDB onlyCustomers need to purchase AIE if it is used to load data into AR Forms directly instead of BMC Atrium CMDBAR User licenses would be needed to use AIE Application

BMC AIE LicensingIf AIE did not ship with a BMC product they can download it through EPD

For customers using AIE to load data into BMC Atrium CMBD (2.0.01 onwards) there is no chargeFor customers using AIE to load data into any AR System form directly, they would need to purchase AIE

AIE is our recommended approach for mapping 3rd party data sources or discovery tools to the BMC Atrium CMDB common data model. BMC Atrium CMDB customers can use our open API interfaces and/or AIE at no charge to integrate applicationsOther APIs dont include data transformation and syncing technology that is available through AIE.

How to Deal with AIE and adapter legacy customersIf customer bought Enterprise Integration Engine (EIE) They now own AIE. They can upgrade to AIE 7.1 seamlessly. If customer bought Remedy Link for SQL or Remedy Link for Oracle Remedy Link for SQL and Oracle has been withdrawn with the release of AIE 7.1For use with CMDB integration they are entitled to AIE at no extra charge (upgrading to latest ITSM/SIM/TD might be needed)No product migration is being offered for these link productsIf a purchased copy of AIE is needed (to load data into AR System Forms directly)Customers would need to purchase AIE and pay the differenceMaintenance cost would be based on AIEs base priceIf customer bought Remedy Discovery Services Adapter for SMSRemedy Discovery Service for SMS has been withdrawn with the release of AIE 7.1For use with CMDB integration they are entitled to AIE at no extra chargeThey can use SMS2CMDB Open Source Integration for the mappings.If a purchased copy of AIE is needed (to load data into AR System Forms directly)Customers would need to purchase AIE and pay the differenceMaintenance cost would be based on AIEs base price

What is it?The provider registry is a critical element of efficient federationDefines a source of data (provider) and the mechanisms to interface with that data source.

How is it used?Defines the types of data providersContains the methods to interface with the data providersEnables efficient link following in real-timeReduces the amount of customization required to connect an external data source.Touch on lightly, suggest that they attend other sessions on CMDBDiscuss all settings quick overviewUse Multiple Mappings if - Multiple source files to one destination or Multiple Destination from one source.Key values will only come from MainExplain Interval fields and how can set to trigger on completionExplain Time Can still trigger on completion but have to set to Interval or To be Triggered to unlock the fields belowExplain To Be Triggered be sure to mention that more than one can be triggered on completion, starting multi threading ...