Efficiently reuse within your Enterprise

© 2009 IBM Corporation Innovation for a smarter planet

Quality Driven Software DeliveryAchieving greater value and performance from software & systems investments

Speaker Name

Speaker Title

City, Date

IBM Rational Technology Events 2009

Quality Driven Software Delivery 2

Manage business transformations

Enable business flexibility

The successful businesses of the future will be those that use software to a competitive advantage

Differentiate products

Delivery real customer innovation

React to market shifts

Manage regulatory mandates

Exploit globalization

Businesses are facing an unparalleled rate of change



Growing focus on business outcomes

30% of project costs are due to rework and poor execution of requirements”

“50% of outsourced projects are expected to under perform”

“34% of software projects are deemed successful costing $300B annually”

“CIOs are under pressure to enable their teams to become catalysts for change. However, change is outpacing their ability to deliver. They are reorienting their teams to focus on ROI & quantified business outcomes and to mitigate risk and reduce costs”

– IBM CIO Survey, 2008

“Only 22% of executives felt that their IT and business strategy were tightly integrated”

“49% of budgets suffer overruns and 62% fail to meet their scheduled”

Challenges to the effective delivery of software A history of cost overruns, schedule slips and quality issues



30% of project costs are due to rework and poor execution of requirements”

“50% of outsourced projects are expected to under perform”

“34% of software projects are deemed successful costing $300B annually”

“Only 22% of executives felt that their IT and business strategy were tightly integrated”

“49% of budgets suffer overruns and 62% fail to meet their scheduled”

Challenges to the effective delivery of software A history of cost overruns, schedule slips and quality issues

Silos of people, process, and projects

Organizational BarriersGeographic Barriers

Poor communication

Language, culture, time

Process gaps resulting in rework

Infrastructure Barriers

Lack of meaningful collaboration

Weak project governance

Poor LOB oversight

Security of IP when outsourcing

Incompatible tools

Unreliable access

Lengthy on-boarding

Inflexible integration



How can I get there from where I am today?

“How do I further automate software delivery within my organization?”

“How can I unobtrusively gather measurements to ensure progress towards desired business outcomes?”

“How can I enable collaboration throughout the software delivery process?”

“How do I make incremental, iterative progress towards more effective software delivery?”



Drive organizational consensus on priorities and improve workforce

productivity

CollaborateContinuously improve by

measuring progress against desired business outcomes

ReportLower costs and improve

quality by automating workflow based on

real-time information

Automate

The business process of software delivery aligned with ever-evolving business priorities and stakeholder constituencies

Presenter

Presentation Notes

Enable IT to mitigate risk and deliver quality solutions at lower cost. The Business Process of Software & Systems Delivery: Policies, procedures, decision rights, workflow, etc.


Quality Driven Software Delivery

Analyst

Challenge: Poor collaboration between Business and IT Stakeholders Organizations often struggle to achieve cooperation

Current ConcernsStakeholder interaction is inconsistent and disparate (email / docs)

Stakeholders overlooked or rely too heavily on delegates

Stakeholders see needs as unique, unable to understand and negotiate

No synergy, 41% of projects fail to achieve investment return

No Meaningful dialogue or alignment to strategy

Analyst: “People don’t seem to be on the same page”

Stakeholders not engaged Stakeholder:" I do not have time, and no resources, so we

will have to catch up later”

User: “Let me ask my manager if I can get more involved

Low user involvement

No cross-functional collaboration or understanding

Stakeholder:" Here are my needs, in my models, and my documents. Call me today.”

Sidebar conversations

Stakeholder: “My process drives 90% of revenue, tell me when to test my solution

Unrealistic Unyielding

Expectations



Business Objectives

Business Processes

Use Cases

Storyboards

Glossaries

Documents

Sketches

Enable business & technology experts to build business value Introducing IBM Rational solutions for requirements definition and management

Collaborate in context across stakeholders and delivery teams

Capture, connect and organize the web of requirements information

Guide teams to execution through a controllable, repeatable, and measurable approach

Prototypes

Visual Validation

Rational Requirements

Composer Requirements

DefinitionNEW

Lifecycle Traceability

Impact & Coverage Analysis

Baselines and Scope

Management

Rational RequisitePro Requirements Management

ENHANCED

http://www.ravenflow.com/index.php



Introducing the first wave of innovative Jazz offerings

Elicit, capture, elaborate, discuss and review requirements

Requirements Composer

JAZZ TEAM SERVER

Best Practice Processes

Rational Requirements

Composer

Quality ManagerCoordinate quality assurance

plans, processes and resources

Team Concert"Think and work" in unison and provide real-time project heath

Rational Quality

ManagerRational

Team Concert

offeringoffering offering

Business Partner Jazz

Offerings

ClearQuest

ClearCaseBuild Forge

Asset ManagerRequisite

Pro

Presenter

Presentation Notes

Our first wave of Jazz-based offerings is built from the ground up from Jazz technology. Rational Team Concert, now available, is designed for the core software delivery team. It helps developers, project managers and extended team members "think and work in unison" across geographic and functional silos. Rational Requirements Composer is designed for the business analyst who seeks a more comprehensive, multi-dimensional approach to collaborating with business experts to capture business requirements to design more innovative, groundbreaking solutions. And Rational Quality Manager is designed to help testers, performance engineers and other quality assurance professionals improve software quality by providing superior collaboration and automation of test plans, processes and resources. In addition to these offerings -- built from ground up with Jazz technology -- we are continuing to infuse our flagship offerings like ClearCase, ClearQuest and BuildForge with Jazz technologies and integrations that enable rich participation in Jazz-managed processes. Latest availability: IBM Rational Quality Manager (Beta June 2008; GA September 2008 IBM Rational Requirements Composer (Beta June 2008; GA Q4 2008

http://www.surgient.com/

http://www.irise.com/

http://www.ravenflow.com/index.php

http://www.cm-logic.com/



Centralized test management hub allowing full lifecycle support across all types of testing and platforms

JAZZ TEAM SERVER

ManageTest Lab

CreatePlan

BuildTests

ReportResults

IBM Collaborative Application Lifecycle Management

FunctionalTesting Performance

TestingWeb Service

QualityCode

Quality

Security andCompliance

Test Management and Execution

Rational Quality ManagerQuality Dashboard

Open Lifecycle Service Integrations

DefectManagement

RequirementsManagement

Best Practice Processes

homegrown

Open Platform

JavaSystem z, iSAP

.NET

Presenter

Presentation Notes

Summarize the components for the benefit of the engineers in the room.

http://images.google.com/imgres?imgurl=http://www.bppmsummit.com/bppmsummit07/imgmgr/compuware_large_430x250.jpg&imgrefurl=http://www.bppmsummit.com/bppmsummit07/compuware.html&h=250&w=430&sz=48&hl=en&start=1&um=1&usg=__laYqw6Z4o0G46gBZNHGrddKiDcU=&tbnid=RWNlAXnnjL7UOM:&tbnh=73&tbnw=126&prev=/images%3Fq%3Dcompuware%26um%3D1%26hl%3Den%26rlz%3D1T4GGIC_enUS270US270%26sa%3DN

http://images.google.com/imgres?imgurl=http://www.laptopgram.com/wp-content/uploads/2007/05/hp-logo.png&imgrefurl=http://infimate.wordpress.com/tag/software/&h=386&w=475&sz=45&hl=en&start=1&um=1&usg=__UX57DqWB6x_CR3-h_pHac67bp1s=&tbnid=5UrKyzNj7QXw5M:&tbnh=105&tbnw=129&prev=/images%3Fq%3DHP%26um%3D1%26hl%3Den%26rlz%3D1T4GGIC_enUS270US270%26sa%3DN

http://images.google.com/imgres?imgurl=http://blog.protectwebform.com/images/microsoft_logo.jpg&imgrefurl=http://blog.protectwebform.com/p/category/ocr&h=360&w=450&sz=35&hl=en&start=1&um=1&usg=__Vv6WIvsd1d9hhYEhhvEL2txuPr8=&tbnid=MiRo0V9oQGxE8M:&tbnh=102&tbnw=127&prev=/images%3Fq%3Dmicrosoft%26um%3D1%26hl%3Den%26rlz%3D1T4GGIC_enUS270US270



Mitigate business risk in an environment of constant change

Rational Quality Manager

What is needed: Comprehensive quality impact analysis beyond walls of QA with lifecycle collaboration and process orchestration

Out of the box requirements management and integration

Advanced work item analysis prevents duplicates and duplicate efforts

Review and approvals of tasks across projects

Asset versioning with dynamic updates Prevent scope creep

New and Updated

Requirements

Requirement update notification

Duplicate Requirement Prevention

Requirement impact analysis

Dynamic Test Plan

Review & Approve

Presenter

Presentation Notes

So let’s see how this works. The most important task to mitigating risk in a changing environment is to manage requirements and prevent scope creep. RQM offers two levels of requirements management – either via integration to tools such as ReqPro and DOORS, or through native requirements. One interesting aspect about the native requirements management is that there’s an ability to search for duplicate requirements. The concept here is to keep your data clean – duplicate requirements cause waste and rework to clean them up and consolidate them down. With RQM – you can search for potential duplicates upon entry and prevent future waste. Regardless of the source, requirements are visible in the test plan. They’re visible to the whole team – who can be formally assigned as reviewers and approvers. This is really helpful with distributed teams, to ensure full cross team visibility and agreement on the foundation of the test plan. And of course, with version control, at any milestone you can take a snapshot to capture a given state of requirements definition and coverage in the plan.



Improve operational efficiencyWhat is needed: Know what others are doing, know what is expected of you.

Requirements Analyst

Individual Task List

Team event logRational Quality Manager

Stay on schedule and under budget

Enforceable process discipline

Team and individual task workflow

Easy to visualize commitments

Monitor asset ownership and events

Web 2.0 style access

Developer

Customer

Quality Assurance Team

Presenter

Presentation Notes

Once requirements are defined, the real work starts. Creating test cases, building tests and executing are all tasks to be assigned. Again, for distributed teams, it’s essential that each individual understand what is expected of them and what they can expect from others. On each members dashboard there are two lists: an individual task list and a team event log that do just that. Every time a task is assigned, the assignee gets a notification on their list. A entry is also made on the team event log – to keep the team up to date on all project activity. Visualizing commitments is key to collaboration.



Make confident decisionsWhat is needed: Moment-by-moment project delivery metrics tailored by role for the right

stage coupled with trends to accelerate decision making.

Rational Quality Manager

Convert Data into Information

On demand reporting for vital project information customizable by individual

Process and asset based predictive analytics

Multiple export and sharing options

Test Team

Development Team

Management Team

Quality Assurance Team

Predictive Reports

Live Dashboard

Presenter

Presentation Notes

Once requirements are defined, the real work starts. Creating test cases, building tests and executing are all tasks to be assigned. Again, for distributed teams, it’s essential that each individual understand what is expected of them and what they can expect from others. On each members dashboard there are two lists: an individual task list and a team event log that do just that. Every time a task is assigned, the assignee gets a notification on their list. A entry is also made on the team event log – to keep the team up to date on all project activity. Visualizing commitments is key to collaboration.



Comprehensive rich test plan A quality contract for the entire software delivery team

Collect and track all test dataCentral location for business objectives requirements, resources, platform and exit criteria to name a few

Defined ResponsibilitiesIndividual sections are assigned to team members to clearly establish ownership

Goal OrientedFormalized and documented exit criteria

ExtensibleAdd sections, import custom data

Keep track of changesSnapshot version control to track plan history throughout the life of the project

Security Officer

Project Manager

Tester

Business Stakeholder

Test Cases

Skill Availability

Project Logs

Use Cases

Requirements

Security Mandates

Defect Logs

Business Objectives

Quality Asset Infrastructure

Central hub captures

everything that matters

for quality releases

14

QA Manager

Presenter

Presentation Notes

BBryson: At the core of RQM is the test plan. It is a dynamic document that captures everything that matters for quality releases. Some key areas (read slide points)



Improve operational efficiency

Step by step capture and execution of manual tests

Assisted data entry and validation for efficient execution

Keyword support for integrated manual and automated testing

Rich defect capture during execution, including screenshot and attachments

Manual Test Execution

Track execution results and defects from manual test efforts

What is needed: Integrated manual test authoring and execution

Presenter

Presentation Notes

BBryson: Probably the most common thread uniting all test teams is manual testing. Not every team can or is leveraging test automation, but virtually every test team performs manual testing. Because it’s so integral to test efforts, Manual testing was built right into RQM. In it’s most basic form, RQM”s manual testing capability enables users to author and execute manual tests. This means they can capture manual test steps in the tool, and then at execution time, have the tool echo those steps back as they run through the test, while providing users with the ability to note test results and captures screenshots and defects. A unique feature of the manual test capability is the support for assisted data entry and validation. With assisted data entry, as part of the test steps you include any necessary test data. This can be individual items, or captured as a datapool for cases where multiple values are required. When it comes to test execution time, the tool will populate the clipboard with this test data, so you don’t have to re-enter it into the system. This saves time and prevents transcription errors. The assisted validation works in a similar manner. With assisted validation, you enter expected results for a given test step. Once the action is performed during the test, you can copy the results from the application to the clipboard, and RQM will compare the onscreen results with the baseline captured during execution. Again, there’s a time savings here, and effort – especially when the need is to validate complex returned results. Finally – RQM supports keywords. Keywords allow you to group common steps between tests. What you’ll typically find doing manual tests is that some steps are repeated consistently across multiple tests. A common example here is the steps required to login to the application. Instead of copying these steps across multiple tests – you can enter them once, tag them as a keyword, and then reference that keyword across any test that requires those steps. The benefit is that you avoid test step duplication, and reduce test maintenance when it comes time to change those steps. Further, these keywords can be automated. So instead of having RQM simply list the steps to perform, through Rational Functional Tester you can automate the tests, so that at test execution, RFT can perform the login then return control to tester to complete the remaining manual portion of the test. It’s hybrid testing – and a great way for teams not using automation to begin to experience some of the key benefits of automation.



Manage

Verify that I have the resources required to fulfill my test plan

Deploy

Deliver the configurations my teams require for test

Optimize

Analyze patterns to minimize cost and maximize utilization

Improve operational efficiencyWhat is needed: Test Lab Management

Work smarter, save on test lab overhead, infrastructure and duration costs

Presenter

Presentation Notes

BBryson: Finally, often to address all the configuration possibilities, a test lab is required. Test labs come in many shapes and forms – from a couple extra old PCs hanging around the office, to bank upon bank of virtual PCs running across multiple physical machines. Managing this test lab – to ensure the right resources exist and scheduling execution (to ensure availability of a required configuration) can be a real challenge for testers. Accordingly, within Rational Quality Manager, there is an integrated lab management facility which lets you inventory your lab, and schedule execution. With the addition of Rational test Lab Manager, a plug in for RQM, users gain further automation to assist with the automated inventory of their lab as well as additional lab reporting which analyzes usage patters for lab assets to ensure optimal utilization.



What You’ll See:Performance Testing with IBM Rational Performance Tester

Test creation, modification and execution

Root Cause Analysis diagnosing the cause of performance problems

What You Won’t See:Programming or Code



Ongoing analysis & process improvement

Proactive risk management

Greater predictability

1818

Improve operational

efficiency with automation

Mitigate business risk with

collaboration

Make confident decisions with

actionable reporting

Team coordination of test planning

Enforceable process workflow

Upstream and downstream quality

Lab efficiency and asset utilization

Test coverage optimization

Environment and lifecycle coverage

Rational Quality Manager Central hub for business-driven software quality delivery

“… IT organisations now have more options… They are not forced to use HP … And with the good integration with version control system it is my opinion that they [Rational] have a better integrated testing offer than the competitors right now.” –- Peer Eland

“Easy to use and comprehensive.” -- Massimiliano Russi

“Customers will see an immediate return on investment.– Russell Stanley

Presenter

Presentation Notes

(Provide some detail of what RQM is and it’s differentiated capabilities to set the stage for the scenario discussion) Rational Quality Manager is a centralized test management for business-driven quality software that addresses the 3 main objectives listed here. Based on the Jazz collaboration framework, RQM delivers the following capabilities: Mitigating business risk is accomplished through: Distributed project team collaboration on a dynamic, customizable test plan over a Web 2.0 interface Process enforcement through user-defined roles/customized process workflows and work item assignments Lifecycle quality enforcements through coding and deployment standards Operational efficiency is accomplished through automation. For example: Reducing testing time by 30-40% through capabilities to help users track, schedule and execute tests on physical and virtual lab assets Optimized test environment coverage Accelerating manual testing processes Confident release decisions are supported by: Process improvements with version histories and trend analysis Proactive risk management with automated, filtered and prioritize reporting Customizable dashboards that map pattern analysis to KPIs for greater predictability (Michael Franzen - Sweden) Also one thing that touches on reporting and that we need to stress is the RQM Dashboard, QC doesn't have anything like that (QC Dashboard is for people not working in QC). For example "My Tasks" is something that QC customers has been asking for, for many years as they don't know what they need to work on when they log in. When I show RQM for QC customers here in Sweden, they immediately love the dashboard and especially "My Tasks". Open extensible platform to interface with Rational and 3rd party testing tools and add-ons Let’s have a look at a few scenarios around the 3 objectives to see how RQM addresses each one. 47 WINS IN 2008 !!! Your colleagues tell us why they won... . “ HP licensing was cost prohibitive and restrictive.. client needed an alternative” “ web client” “client liked Jazz” “good customer relationship, trusted BP who show cased solution” “like the simplicity.. this is out of the box” “Proven business value and integration with other products such as RTC, RFT and RPT

http://www.zurich.com/main/



IBM Rational Quality Management Strategy

Enterprise Quality Management

Functional Regression Testing

Break

Service Testing

Performance Validation

Wrap Up

IBM Rational Quality Management Quality Driven Software Delivery



Manual testing can’t keep pace!

Rerun the first tests and...

Time

Tests

...take more time

...the new

tests...

Build 1 Build 2 Build 3, 4, 5 …

Challenge of Manual Testing with Short Test Cycles

Manual testing alone can’t keep pace with the rapid development of applications

Quality will be sacrificed to keep on schedule

Presenter

Presentation Notes

Before we go on to discuss an automation use case, let’s discuss first why we need automation. This graph depicts the relationship between the number of tests and the time it takes to run them. In a manual environment, if you run all your tests on build 1 it takes a certain amount of time. <click> Now, in build 2, you need to rerun all the tests you did in build 1, and that takes the same amount of time it did before, and... <click> ...the new tests you’ve added for the new functionality in Build 2 require even more time. It’s easy to see where the continuation of this process eventually takes us.... <click> Manual testing can’t keep pace with an iterative environment. The difficulty lies in the amount of time each test cycle takes vs. the speed at which Development can deliver new builds. In this example, the QA group starts a test cycle on build 3, but before they are finished Development delivers build #4...and 5 and 6... and so on. Ironically, the problem arises from the fact that iterative development works! Modern GUI development technology is very advanced. It offers the developer rapid development power. This means they can change the application very quickly and deliver new versions to testing. In fact, many companies deliver a new build every day. It is physically impossible for any number of people manually testing to keep up with this rapid development.



“Our testers are not experienced in

building automated tests.”

“Application UI changes are always

breaking our test scripts.”

Challenges of Functional TestingChallenges of Functional Testing

Control cost of test automationInexperienced testers face steep learning curve for toolsTest maintenance grows with project


Presenter

Presentation Notes

The main point on this slide is that test maintenance becomes a significant cost for test teams as they grow their testing projects. Test maintenance is the death of automation projects. The text in the bubbles illustrates sample comments/complaints that your customers may have heard from their teams.



Challenges of Functional TestingChallenges of Functional Testing“How do I keep my team in connected

with the project team?”

“We’re geographically distributed and we’re worried about parallel script development .”


Test Team IntegrationManaging script changes with parallel developmentSharing test assets with the larger software team



Presenter

Presentation Notes

The main point on this slide is that configuration management is necessary to enable parallel and distributed development. The text in the bubbles illustrates sample comments/complaints that your customers may have heard from their teams.



Challenges of Functional TestingChallenges of Functional Testing

“I need one tool for both my expert and

novice testers.”

“Our script language limits our ability to

overcome some automation obstacles.”



Testing Tools Lack FlexibilityPower users hands are tied by tool limitations



Testing Tools Lack FlexibilityPower users hands are tied by tool limitations

Presenter

Presentation Notes

The main point on this slide is that in other test tools (like Mercury’s QuickTestPro) the product’s efforts to simplify testing make it difficult for power users to overcome inevitable challenges of automation. The text in the bubbles illustrates sample comments/complaints that your customers may have heard from their teams.



Quality in Action: Functional Testing Automate regression testing to keep pace with development

Manual testing cannot keep up with rapid pace of modern development

Automated regression testing enables rapid validation of core functionality

Minimize test maintenance with scripts resilient to application changes

Wizard enhanced automation to speed test creation for the new user

Powerful scripting language and IDE for the professional tester



ChallengeOvercome the complexity of test automation to ensure teams spend more time on high value quality assessment rather than on low return test maintenance.

ChallengeOvercome the complexity of test automation to ensure teams spend more time on high value quality assessment rather than on low return test maintenance.

ResolutionProvide technologies to enable novice testers to craft complex tests with no programming

Data driven wizardDynamic data verification

Reduce script maintenanceWizard driven global updates to centralized object map

ResolutionProvide technologies to enable novice testers to craft complex tests with no programming

Data driven wizardDynamic data verification

Reduce script maintenanceWizard driven global updates to centralized object map

Challenge 1: Controlling the cost of AutomationChallenge 1: Controlling the cost of Automation

Presenter

Presentation Notes

To resolve this challenge RFT offers wizards for common tasks such as datapooling (datapooling means not hard coding data into scripts, instead it means having the script access data in a text file. This way the same test script can run hundreds of tests based on different data). RFT also has a object map update utility which reduces script maintenance by allowing users to globally update the object map used by all scripts. Typical example here is when a test is recorded against one web server, but played back against another. The URL which is used to identify objects will be different based on the two servers. With the utility you can quickly update all URLs.



Challenge 2: Test Team IntegrationChallenge 2: Test Team Integration

ChallengeAccommodate the requirements of both dedicated, independent QA teams as well as integrated project teams to ensure high operational integrity.

ChallengeAccommodate the requirements of both dedicated, independent QA teams as well as integrated project teams to ensure high operational integrity.

ResolutionProvide version control support in the box with ClearCase LTProvide integrations for standard version control features

Check In/OutCompare withPrevious VersionsShow change history

ResolutionProvide version control support in the box with ClearCase LTProvide integrations for standard version control features

Check In/OutCompare withPrevious VersionsShow change history

Presenter

Presentation Notes

To resolve this challenge RFT includes ClearCase LT in the box, and has menu options for all standard CM activities such as check-in and check-out.



ChallengeFulfill both standard test needs as well as expert requirements to ensure usage limited only by the imagination of the project team.

ChallengeFulfill both standard test needs as well as expert requirements to ensure usage limited only by the imagination of the project team.

ResolutionProvide two test script development languages

JavaVisual Basic .NET

Give professional testers the flexibility to overcome testing challenges in the language of their choice

ResolutionProvide two test script development languages

JavaVisual Basic .NET

Give professional testers the flexibility to overcome testing challenges in the language of their choice

Challenge 3: Tools Lack FlexibilityChallenge 3: Tools Lack Flexibility

Presenter

Presentation Notes

To resolve this challenge RFT offers two scripting languages, Java and VB.NET. Both are “pure” languages. We do not use any specialized versions of these languages, we use the regular off the shelf Java and .NET. This means power users can do anything in our tool that they can do in Java or VB.net. For testers, these are also good languages to learn for career development.














Break

Service Testing


Wrap Up




SOA: Service Oriented Architecture Definitions

To the IT Executive

Flexible applications built upon re-usable building blocks that are easily connected

Presenter

Presentation Notes

BBryson: SOA is an overloaded term that means different things to different people/roles. For the IT executive, SOA means business flexibility – applications that are not monolithic, but instead are made of components which can be easily updated, and re-organized to quickly address business needs




To the IT Executive


To the Developers and Testers

Web Services.

Period.

Presenter

Presentation Notes

BBryson: For developers, SOA means Web services – the technology at the foundation of SOA, the technology which is the technical foundation of SOA applications




To the IT Executive


To the Software Architect

An IT architectural style which assembles loosely coupled distributed services to implement a business process


Web Services.

Presenter

Presentation Notes

BBryson: Somewhere in between those two definitions is the recognition that yes it addresses flexibility, yes it’s composed of individual web services, and it requires an infrastructure upon which to run



SOA: Implications for Quality Management

To the IT Executive


Validate Business Process

Challenges

• Identifying test cases• Managing Data Complexity

• Requirements, Test Cases, Defects• Ensuring optimal test & configuration coverage

Presenter

Presentation Notes

So – what are the quality implications of all these perspectives? For the IT executive perspective – the primary concern is the business process – so from a QM (Quality Mgmt) perspective, that means the concern is validating that the business process is fit for use. This is primaily what we were talking about this morning in terms of test management and functional validation.





Web Services. Validate Web Services

• No user accessible interface• Multiple test case data cases per test• Service interaction testing

Challenges

Presenter

Presentation Notes

So – what are the quality implications of all these perspectives? For the IT executive perspective – the primary concern is the business process – so from a QM (Quality Mgmt) perspective, that means the concern is validating that the business process is fit for use. This is primarily what we were talking about this morning in terms of test management and functional validation.










To the Software Architect

An IT architectural style which assembles loosely coupled distributed services to implement a business process

Validate Infrastructure

• Ensuring service operability post deployment• Service upgrade & interoperability management• Service Performance

Challenges

Presenter

Presentation Notes

BBryson: Finally, from the architectural perspective, the QM implication is that we need to validate the infrastructure upon which the system is built – and also ensure continual operation of that system.



Health Monitoring of SOA componentsNEW! Application Management dashboards for IT and business reporting supports Web Services, ESB and Process Server

Performance Monitoring and Transaction Management

Populate WSRR with live service performance dataNEW! Response Time dashboards

Change and Configuration managementDataPower configuration support Reroute services by making changes and upgrades while staying in full production mode Auto-discover services for populating WSRR with services information

Monitor Deployed Services and Service Quality Tivoli Composite Application Management (ITCAM)

“Show me the flow details in my service development tool!”

ITCAM supports production and development needs with a unified console!

Presenter

Presentation Notes

BBryson: Once your service is deployed, it’s important to ensure the continued operation of the application into production. from http://www-306.ibm.com/software/tivoli/products/composite-application-mgr-soa/ IBM Tivoli Composite Application Manager for SOA monitors, manages and controls the Web services layer of IT architectures. Identify the source of bottlenecks or failures and pinpoint services that use the most time or resources with this composite application management solution designed for SOA. Recognizes and quickly isolates Web service performance problems, alerts you when Web service performance is degraded and reports results against committed service levels Provides an integrated, easy-to-use console that helps you visualize the flows of Web services in their entirety Helps maintain service performance without operator intervention Monitors your services where you want them with heterogeneous platform support Is a core component of the IBM SOA Foundation Management Essentials, an integrated and open set of software, best practices, patterns and skills resources to get you started with service-oriented architectures Includes Web Services Navigator for deeper understanding of service flows, patterns and relationships Now provides support for SAP NetWeaver, IBM WebSphere CE and SCA infrastructure, JBoss and DataPower proxy and web browser launch Operating systems supported: AIX, Linux, Sun Unix, Windows, z/OS



Security Validation Detect Vulnerabilities Before Release

BuildCoding QA Security Production

Provides Developers and Testers with expertise

on detection and remediation ability

Enables Chief Security Officers to drive remediation back into

development

Address vulnerabilities before application goes

live






Break

Service Testing


Wrap Up




Challenges of Performance TestingChallenges of Performance Testing

“We don’t have a lot of experience building

good load tests.”

“Most load test tools are much more

complicated than we require.”

No performance test experienceTool complexity and lack of experience intimidates many first time users


Presenter

Presentation Notes

BBryson: Emphasize this point to sell our ease of use benefit. Visual editor is the feature we’re getting at. The though bubbles are common symptoms that audience may have heard and resonate with.




“Our system is too complex – even we don’t understand it.”

“Getting a test to actually work takes

forever.”


Complexity of system under testComplexity of system under test prohibits simple record and playback



Presenter

Presentation Notes

BBryson: Emphasize this point to sell benefit of low memory footprint of tool




“We don’t have a good understanding of what is causing the problem”

“I need to know more than simply where a

problem is occurring.”



Testing tools lack insightTool can find the problem, but not diagnose the root cause



Testing tools lack insightTool can find the problem, but not diagnose the root cause

Presenter

Presentation Notes

BBryson: Emphasize this point to sell benefit of root cause analysis which goes beyond problem identification, and provides information for problem diagnosis



Performance test automationBuilt for Day 1 Productivity

Mask complexity to get the job doneAdvanced Data Access & Manipulation

Automated data variation and synchronization

Root Cause AnalysisIdentifies location and root cause of performance problem in hardware and software

Quality in Action: Functional Testing Performance test automation for the novice and the professional

Presenter

Presentation Notes

http://www.stockxpert.com/browse.phtml?f=view&id=4433931



Challenge 1: No in-house experienceChallenge 1: No in-house experience

ChallengeTool complexity and lack of experience intimidates many first time users

ChallengeTool complexity and lack of experience intimidates many first time users

ResolutionRepresent tests as a tree view of sequential flow through application

Simplify test editing with wizards LoopingConditional eventsData validation

Integrate Java code to handle unique performance challenges

ResolutionRepresent tests as a tree view of sequential flow through application

Simplify test editing with wizards LoopingConditional eventsData validation

Integrate Java code to handle unique performance challenges

Presenter

Presentation Notes

BBryson: You’ll now re-address each of the challenges and discuss what features we have to address them.



46

Challenge 2: Complexity of System Under TestChallenge 2: Complexity of System Under Test

ResolutionIntegrate data pooling technology to ensure each unique data for each user

Identify data relationships to dynamically reference server generated data during playback

Utilize unique TCP/IP addresses for each user to ensure realistic load

ResolutionIntegrate data pooling technology to ensure each unique data for each user

Identify data relationships to dynamically reference server generated data during playback

Utilize unique TCP/IP addresses for each user to ensure realistic load

ChallengeComplexity of system under test prohibits simple record and playback

ChallengeComplexity of system under test prohibits simple record and playback

Data Referencing Data value used during playback will be dynamically linked to previous server response

Data Pooling Data value used during playback will be unique value for each user read from datapool



Challenge 2: Tools Lack InsightChallenge 2: Tools Lack Insight

ResolutionRoot Cause Analysis features provide additional insight to diagnose the cause of a bottleneck

Resource Monitoring data monitors hardware during test

Response Time Breakdown report breaks down response times into

ResolutionRoot Cause Analysis features provide additional insight to diagnose the cause of a bottleneck

Resource Monitoring data monitors hardware during test

Response Time Breakdown report breaks down response times into

ChallengeTool can find the problem, but not diagnose the root cause

ChallengeTool can find the problem, but not diagnose the root cause

Presenter

Presentation Notes

BBryson: Key point is that root cause analysis goes beyond traditional benefit of finding performance problems. The first question one asks after finding a performance problem is Why? Root Cause Analysis answers that question by providing information on hardware (resource utilization) and software (code execution time statistics) to diagnose the cause of performance problems.












Break

Service Testing


Wrap Up




Enabling business innovation and agility requires a significant investment in software

Software is increasingly being managed as a strategic business asset, key enabling sustained business differentiation and flexible operations

Businesses everywhere are deploying increasingly intelligent, interconnected and instrumented software & products

Enabling innovation, lowering costs and managing change is dependent on effective software delivery

$600B spent annually on Software & Information Technology

Presenter

Presentation Notes

Accelerating innovation relies on our ability to manage effective software delivery



Next Steps – Learn and Explore

ibm.com/rationalFull Product InformationProduct demonstrationsProduct datasheets

ibm.com/developerWorksDownload Evaluation SoftwareUser ForumsProduct tutorialsTech Notes, etc…

Presenter

Presentation Notes

Success Stories http://www-306.ibm.com/software/success/cssdb.nsf/topstoriesFM?OpenForm&Site=rational PowerPack Evaluation Program http://vrmnaweb.ibm.com/@^@DP00161107000000080707247/bin/devworks/entry.cgi?cl=USEN&pg=../../docs/devworks/lp1.html&id=48

http://www-01.ibm.com/software/awdtools/tester/performance/

http://www.ibm.com/developerworks/rational/products/performancetester/



Visit www.ibm.com/rational/rsdc

for updates and more information

1. Over 300 sessions in 12 tracks 2. Interact with over 4,000 of your industry peers 3. Keynotes with industry-leading experts 4. IBM Expo featuring key IBM Business Partners

and the IBM Solution Center 5. IBM Rational Labs (see possible future capabilities in our products!)

6. 3 and 5 hour Technical Workshops 7. NEW! Access to 275+ sessions at the IBM Telelogic User Group

Conference being co-located 8. Executive Summit 2009 9. Interactive Birds-of-a-Feather Sessions 10. Unlimited networking opportunities

Top 10 reasons not to miss this year’s IBM Rational Software Development Conference!

IBM Rational Software Development Conference 2009Save the Date! May 31 – June 4, 2009 – Orlando, FL

http://www.ibm.com/rational/rsdc

http://www.ibm.com/rational/rsdc



Download a PDF of this technical presentation from

www.ibm.com/developerworks/offers/techbriefings



We appreciate your feedback in order to improve this educational event.

Please fill out the survey form.