Upload
buidung
View
277
Download
0
Embed Size (px)
Citation preview
© Copyright 6/20/2013 BMC Software, Inc 1
Atrium Webinar Series – Understanding Atrium Integrator
Presentation Materials will be posted to the following location: https://communities.bmc.com/communities/docs/DOC-22363
Int'l Toll and US Cell Phone: 913-312-0414 US/CAN Toll free: 877-702-9054
LOCAL: Australia, Brisbane: +61 (0) 7 3123 0047 Australia, Canberra: +61 (0) 2 6111 2007 Australia, Melbourne: +61 (0) 3 9034 3217 Australia, Sydney: +61 (0) 2 9126 3023 Austria, Vienna: +43 (0) 1 9945 6051 Bahrain, Manama: +973 1619 9040 Belgium, Brussels: +32 (0) 2 303 2273 Brazil, Sao Paolo: +55 11 3163 0461 Bulgaria, Sofia: +359 (0) 2 491 6411 Canada, Montreal: +1 514 669 6112 Canada, Toronto: +1 647 426 9209 China: +86 400 120 2693 China: +86 400 120 26 93 Czech Republic, Prague: +420 234 147 003 Denmark, Copenhagen: +45 78 78 79 63 Estonia, Tallinn: +372 622 5742 Finland, Helsinki: +358 (0) 9 7479 0101 France, Lille: +33 (0) 359 69 03 42 France, Lyon: +33 (0) 426 10 30 24 France, Paris: +33 (0) 1 70 71 29 53 Germany, Berlin: +49 (0) 30 2555 5430 Germany, Frankfurt: +49 (0) 69 1200 9864 Germany, Munich: +49 (0) 89 1436 7911 Hong Kong, Hong Kong: +852 3008 0383 Hungary, Budapest: +36 1 577 9957 India, Bangalore: +91 (0) 80 6127 5134 India, Mumbai: +91 (0) 22 6150 2334
Ireland, Dublin: +353 (0) 1 437 0560 Israel, Tel Aviv: +972 (0) 3 721 9373 Italy, Milan: +39 02 9978 1800 Italy, Rome: +39 06 8743 4377 Japan, Tokyo: +81 (0) 3 4455 1996 Latvia, Riga: +371 6601 3678 Lithuania, Vilnius: +370 5205 5590 Luxembourg, Luxembourg: +352 2786 0224 Malaysia, Kuala Lumpur: +60 (0) 3 7724 0847 Mexico, Mexico City: +52 55 4777 2663 Netherlands, Amsterdam: +31 (0) 20 262 0137 New Zealand, Auckland: +64 (0) 9 929 1884 Norway, Oslo: +47 21 95 32 33 Poland, Warsaw: +48 (0) 22 295 36 31 Portugal, Lisbon: +351 21 120 9698 Romania, Bucharest: +40 (0) 21 529 1340 Russian Federation, Moscow: +7 495 620 9818 Singapore, Singapore: +65 6416 9957 Slovakia (Slovak Republic), Bratislava: +421 (0) 2 3278 6632 Slovenia, Ljublijana: +386 (0) 1 888 8397 South Africa, Johannesberg: +27 11 589 8382 Spain, Madrid: +34 91 080 0153 Sweden, Stockholm: +46 (0) 8 4030 4953 Switzerland, Geneva: +41 (0) 22 555 0258 Switzerland, Zurich: +41 (0) 44 556 8481 Taiwan, Taipei: +886 (0) 2 2650 7292 United Kingdom, London: +44 (0) 20 8150 0796
TOLL FREE: Argentina: 0800 666 2571 Australia: 800 635 764 Austria: 0800 295 994 Bahamas: 800 205 6295 Belgium: 0 800 72 785 Brazil: 0800 891 8445 Bulgaria: 00 800 115 1110 Chile: 123 0020 9601 China, Southern Region: 10 800 140 1378 Colombia: 01 800 518 0504 Czech Republic: 800 142 277 Denmark: 8088-6980 Dominican Republic: 1 888 751 4437 France: 0 800 914 669 Germany: 0 800 182 4414 Greece: 00 800 161 2205 1068 Hong Kong: 800 908 710 Hungary: 06 800 184 72 India: 000 800 100 7363 Indonesia: 001 803 011 2660 Ireland: 1 800 760 204 Israel: 1 80 945 2068 Italy: 800 873 593 Japan: 00531 12 0058 Korea, Republic Of: 00798 14 800 4574 Latvia: 800 03 656 Lithuania: 88 003 0200 Luxembourg: 800 25 164 Malaysia: 1 800 812 644 Mexico: 001 800 514 1058
Monaco: 800 93 304 Netherlands: 0 800 024 9645 New Zealand: 0 800 440 611 Norway: 800 191 83 Panama: 00 800 226 7179 Philippines: 1 800 111 00400 Poland: 00 800 112 41 37 Portugal: 800 819 382 Russian Federation: 810 800 2544 1012 Singapore, Singapore: 800 101 1738 Slovenia: 0 800 80815 South Africa: 0 800 999 562 South Korea, Korea, Republic Of: 003 0813 1654 Spain: 900 941 991 Sweden: 02 079 3671 Switzerland: 0 800 896 853 Taiwan: 00 801 126 826 Thailand: 001 800 12 066 3284 Trinidad and Tobago: 800 205 7179 United Arab Emirates: 800 017 7175 United Kingdom: 0 800 051 7166 Uruguay: 000413 598 2551 Venezuela: 0 800 100 2525
Participant passcode: 378410.
Participant passcode: 378410.
Understanding Atrium Integrator
Vaibhav Kulkarni
© Copyright 6/20/2013 BMC Software, Inc 3
What will be covered?
AI Overview The Basics 8.X Architecture Overview Deployment / Installation AI Jobs & Job Scheduling Server Group Changes & Enhancements in 8.X Demo - Job Creation using the
Wizard
Best Practices Upgrade from 7604 to 8.X Demo – How to Edit jobs in Spoon Modes in CMDBOutput Step Carte Server details
AIE to AI Migration How AI is different from AIE AIE to AI migration tool Best Practices
Troubleshooting Log file locations
Troubleshooting tips
References
Q&A
Atrium Integrator Overview
© Copyright 6/20/2013 BMC Software, Inc 5
Atrium Integrator Overview The Basics
What is it?
Why is it useful?
When do you use it?
© Copyright 6/20/2013 BMC Software, Inc 6
Situation:
The Atrium CMDB Administrator needs to load data into CMDB from database tables of a discovery source. The discovery source provides both CI and Relationship data.
Steps:
• Launch the AI Console and create a new
connection in the AI Data Store
• Start the job creation wizard and identify
classes/relationships to be populated
• For each class/relationship, map fields in the
source database tables to CMDB attributes
• Add a schedule for the job
Atrium Integrator Overview The Basics
© Copyright 6/20/2013 BMC Software, Inc 7
ETL Capabilities - Read input from a source (Extract) - Apply transformations on data (Transform) - Write to the target (Load)
Pentaho Data Integration Story - Industry leading ETL capabilities - Custom jobs using spoon - More flexibility to transform
Runtime job monitoring
Atrium Integrator Overview The Basics
© Copyright 6/20/2013 BMC Software, Inc 8
Integration Job Builder
Wizard
Job Scheduler
Job Monitoring
Manage Data Store
Create job Schedule job Monitor job Create data store
connections
AR Systems Database
BMC Atrium Integrator
Carte Server
CMDB Plug in - CMDB Input - CMDB Output - CMDB Lookup
AR System Plug in - AR Input - AR Output - ARX Input
(Desktop Designer Tool)
Atrium Integrator Console (Accessible via Mid tier/Browser)
Remedy AR System Server
CMDB
Atrium Integrator
Spoon
Core Data Integration Engine
Atrium Integrator Overview Architecture Overview (v8.x)
© Copyright 6/20/2013 BMC Software, Inc 9
Step1: AR Server Installer installs BMC Atrium Integrator - Spoon - Carte Server - AR Steps, e.g. ARInput / AROutput
Step2: Atrium Integrator Server 8.x - Deploys the UI - Flex-based guided workflow - CMDB Steps Input, Output, and Lookup in the already Installed Spoon
directory on the AR server
Step3 (Optional): Client Installer - Allows remote connection to AR System DB to view / design Pentaho JOBS - Requires installing AR Client components during AR Server Installation - Install the Atrium Integrator Client after the AR client installation
Atrium Integrator Overview Deployment/Installation
© Copyright 6/20/2013 BMC Software, Inc 10
Job Sub-Job
Transformation(Steps)
Job
Sub Job
Sub Job
Transformation
Transformation
Transformation
Transformation
Atrium Integrator Overview Jobs, Sub job & Transformation
© Copyright 6/20/2013 BMC Software, Inc 11
Entry created in UDM:Execution
Form
Carte server reads the entry
Job executes in carte server
Result/Logs are saved
Start/Schedule Job
Atrium Integrator Overview Job Execution Lifecycle
© Copyright 6/20/2013 BMC Software, Inc 12
Login as AIUser/AIAdmin, select a JOB from the JOB LIST, and click on the Schedule icon to create a Schedule
Every JOB has its own independent Schedule
Multiple Schedules per JOB is allowed
Atrium Integrator Overview Job Scheduling
© Copyright 6/20/2013 BMC Software, Inc 13
In a Server Group installation, multiple servers will appear in the Carte Server drop down list
Select the specific Carte server so that a JOB will always run on the configured Carte server
Note: No failover will occur in a Server Group setup
Atrium Integrator Overview AI in Server Group
© Copyright 6/20/2013 BMC Software, Inc 14
Design - In 8.x, all Pentaho repository tables are converted to AR regular forms
- Carte server is now monitored by ARServer
- Start/Stop/Pause/Resume of job is now controlled through an AR Plug-in
Implementation - AR Forms with the prefix “UDM:Repository” were tables in AINTRepository
in 7.6.04.
- The UDM:Execution form controls Carte server job operations like Start/Stop/Resume
Atrium Integrator Overview Architecture Changes in v8.0
© Copyright 6/20/2013 BMC Software, Inc 15
Pentaho upgrade from 3.2.4 to 4.1
New AISAMPLE_MachineOS _DELTA sample job
CSV and XML data store connection refresh
X-Path is displayed for XML jobs
Delete option in the Integration Job Builder wizard
Job scheduling enhancements
New access point to launch Atrium Integrator
Atrium Integrator Overview New Features in 8.0/8.1
© Copyright 6/20/2013 BMC Software, Inc 16
Job Creation using the Wizard
Atrium Integrator Overview Demo
© Copyright 6/20/2013 BMC Software, Inc 17
Few points to remember about AI Wizard
Wizard is available for transferring data to key CMDB classes. - It doesn’t include feature for designing jobs that involves data transfer to
AR forms
You need to use AI spoon on Windows for modifying any job that you created using wizard
Use AI wizard as starting point for designing your integration job - It provides you with a structure that you can refine using AI Spoon later
You can define job schedule as part of the wizard. - Schedule can be added/modified from AI console any time
You should select ‘Delta’ and ‘Response’ options during job design in wizard. - Else, modifying jobs or transformations to handle ‘Delta’ or ‘Response’ in
Spoon at later stage will be significant manual effort
Best Practices
© Copyright 6/20/2013 BMC Software, Inc 19
AI Best Practices 7.6.04 to 8.X Upgrade
© Copyright 6/20/2013 BMC Software, Inc 20
Edit Job using the Spoon
AI Best Practices Editing Job in Spoon
© Copyright 6/20/2013 BMC Software, Inc 21
Only Insert New CIs
Identical to the AIE option ‘Do not update’ and ‘Create only new CIs’
Always Insert CIs
Insert CIs into CMDB without checking for duplicates
AI Best Practices Options in ‘CMDB Output’ Step
© Copyright 6/20/2013 BMC Software, Inc 22
Use Cache
Should always be ON to reduce CMDB lookups and improve performance
Cache files are created for every class mapping
Use CheckSum
Equivalent to Checksum option in AIE.
Very useful when the delta option is not used in AI
AI Best Practices Options in ‘CMDB Output’ Step
© Copyright 6/20/2013 BMC Software, Inc 23
Carte Server Behavior
Atrium Integrator Carte server is a web server used to execute
transformations and jobs remotely
- All the jobs scheduled via Atrium UI run on the Carte server.
Accessing the Carte server
- http://<ARserver_host>:20000
20000 is the default port
- Use the Remedy Application Service userid / password
Changing the Carte server port
- Carte server port is specified in UDM:Config form and armonitor.cfg file
- After changing the port, restart the BMC Remedy AR System server
- If it is a Server Group enviornment, restart all other AR servers in the
server group
AIE to AI Migration Tool
© Copyright 6/20/2013 BMC Software, Inc 25
AIE
• Limited adapters available out-of-the-box
• Limited set of transforming capability
• No support for hierarchical XML
AI
• Exhaustive infrastructure connectors (JDBC, complex XML, web services, JMS etc)
• Support for JavaScript with many transformation functions
• Out-of-the-box templates to simplify most new CMDB integrations
• Wizard-based UI for creating integrations
AIE to AI Migration Tool Overview
© Copyright 6/20/2013 BMC Software, Inc 26
BMC Atrium Integration Engine term - Data exchange - Data mapping
Corresponding Atrium Integrator term - Job - Transformation
Data Exchange
Data Mapping Relationship Exchange
Relationship Mapping
Job
Sub Job
Sub Job
Transformation
Transformation
Transformation
Transformation
AIE to AI Migration Tool Terminology
© Copyright 6/20/2013 BMC Software, Inc 27
Uses the AR API to connect to the AIE source
Uses the same API library used for generating JOBS from the AI UI
Internal flow for migrating the AIE exchanges to AI - Read all exchanges from the AIE source - ALL Schedule exchanges are migrated as Parent JOBS on the target server
AI Migration Tool
Target AR Server with
AI Server Installed
(Carte server)
Source AR Server with
AIE Exchanges
AR API
Block Diagram
AIE to AI Migration Tool Design
© Copyright 6/20/2013 BMC Software, Inc 28
AIE Server Details Server where the AIE Exchanges are
present. All the exchanges from the server are displayed in the list.
Important: You have to know the Parent Exchange to Start the Migration.
Target Server Details Server where the AI JOBS / Transformations converted from the AIE exchanges will reside.
Important: You have to know the Application Service password of the target AR server where the JOBS are getting migrated. File Name/Table Name column is editable for all file type
exchanges. In case if the paths for these files is going to be different on the target then one can type in the correct path from target server.
AIE to AI Migration Tool UI Overview
© Copyright 6/20/2013 BMC Software, Inc 29
In this example, we will use the Sample ITSM exchanges which are shipped with AIE - The source for all these exchanges is a CSV file
Select the Parent exchange in the Migration tool
Because the underlying technology is different, after the exchanges have been migrated to AI they will be slightly different than they were in AIE
AIE to AI Migration Tool View Migrated Jobs in Spoon
© Copyright 6/20/2013 BMC Software, Inc 30
Configuration - Before migrating Oracle/DB2 exchanges, you have to insert the SID and
Host Name of the AIE server into the DB.properties file on the AI server
Error & Logging messages - The Migration tool displays high level messages on the message console - You can also look in the log4j file in the log folder for detailed messaging
Best Practices - See section Migrating existing data exchanges in the Atrium Integrator
User Guide
AIE to AI Migration Tool Additional Information
© Copyright 6/20/2013 BMC Software, Inc 31
Exchanges with multiple mappings will be migrated only with the main mapping. To create the transformations for child mappings refer to section Creating multiple mappings in the AI User Guide
No Support for .dat type file exchanges, but richer support in AI for text files
Independent Exchanges with Type = Relationship will not be migrated
AIE to AI Migration Tool Known Limitations
Troubleshooting
© Copyright 6/20/2013 BMC Software, Inc 33
Logging: Importance of various Log files
Log File Name Log File Location Purpose of the Log file
arfileout .txt <AR Server Installed
Directory>\diserver\data-
integration\ngie\bin\log
This log file will contain info about file
metadata and execute Run Process call.
arcarte-stderr-2012-09-
04.log/arcarte-stdout-2012-
09-04.log/arcarte.log
<AR Server Installed
Directory>\Arserver\Db
This log file will contain info about
Start/ Stop and Job/Transformation
Execution for Carte Server
arjavaplugin.log <AR Server Installed
Directory>\Arserver\Db
Related to AR ARDBC Pentaho Plug-in
“ARSYS.ARDBC.PENTAHO”
AI cache file is generated
with the below format
[ServernamePort-
NameSpace-ClassName-
DatasetId-Keys
]
It will be located at any of
the following locations based
on environment variable set
•ATRIUMINTEGRATOR_HO
ME
•ATRIUMCORE_HOME
•BMC_AR_SYSTEM_HOME
The cache files are always generated
and filled by CMDBOuput Plug-in.
Mid-tier Log/Web Server Log(Tomcat)
<Mid-tier Installed Location>\Logs\
AI activities from AUI console i.e.
Start/Stop/Create/Delete can be
monitored here.
© Copyright 6/20/2013 BMC Software, Inc 34
Common Troubleshooting Steps
- Class /Relationship list not appearing: Check Midtier stdout logs to find out the API error while fetching data from DatamodelRefernceCIMapping forms. CSV/XML Connection fails: Capture AR server logs to check for ARExecute Process failure which in turns give a call to script file to populate form NGIE:FileMetadata. Data Store Connection fails: Check for Midtier stdout logs and verify if the DB server is not behind the fire wall. Create job fails: Check Midtier stdout logs
© Copyright 6/20/2013 BMC Software, Inc 35
Common Troubleshooting Steps
Cannot Start a job: Check Midtier stdout + AR Plug-in logs Job Schedule: Check in AR System Job/System Pending Job Queue form for the next schedule Plugin server log Cannot Delete AICache files: Stop AR Server and Close Spoon. Run a JOB on specific Carte server: Check if the Carte server is up and running and job is associated to Carte Server in UDM:Permissioninfo form.
© Copyright 6/20/2013 BMC Software, Inc 36
References
- Understanding Atrium Integrator
- Data store connections in AI
- AI Job Monitoring
© Copyright 6/20/2013 BMC Software, Inc 37
Q & A
© Copyright 6/20/2013 BMC Software, Inc 38
Feedback
Any follow up questions – post on BMC DN with prefix “Webinar –” in subject line 1. http://communities.bmc.com/communities/community/bmcdn/bmc_atrium_and_foun
dation_technologies/bmc_atrium_cmdb/bmc 2. https://communities.bmc.com/communities/community/bmcdn/bmc_atrium_and_fou
ndation_technologies/bmc_atrium_cmdb
Thank you!