Embed Size (px)
Company Confidential
Reporting in ALM 11.0 & Assure TotalView Presenter: Shir Goldberg
Assure
Company Confidential
Assure - Release Quality on Time
We know HP ALM & Quality Center
Since 2006, offices in USA, Finland and Israel
HP AllianceOne Technology Partner of the year 2011
We provide innovative products, solutions and services
for HP ALM & Quality Center customers
With Assure’s offering, customers are able to streamline
processes, boost efficiency and decrease overall QA and
administration costs
Assure works with some of the world’s largest and most
demanding HP ALM & Quality Center customers
Company Confidential
New Reporting Possibilities in ALM 11.0
Enhanced monitoring of release performance vs. plan
Track planned release scope and objectives and their
expected progress throughout release duration
Graphs are available for publishing on web pages as
HTML (SharePoint, Wiki, etc)
Company Confidential
New Reporting Features in ALM 11.0
Project Reports
Single configuration for mutiple output formats (doc, docx, html)
Rich style and layout options using MS Word templates
Centralized template management
Release Analysis
Milestones are placed on the release timeline to define a
lifecycle of a product
Key Performance Indicator (KPI)
• Measure your release objectives and analyze bottlenecks
• Set on Milestones to track performance
Company Confidential
New Reporting Features in ALM 11.0
In Release Scorecard you can easily follow the
performance of a release based on KPI`s
KPI Analysis Graphs gives possibility to drill down to find
root cause of possible problems on release
Company Confidential
New ALM11 Features: Project Reports
Only in ALM 11
Rich style and layout options using MS Word templates
Reports can be created with custom layout and for example
company logo
Single configuration for mutiple output formats (doc,
docx, html)
Centralized template management
Enhanced performance
Project reports can be created based on Baseline
Company Confidential
New ALM11 Features: Release Scope
Deliverables for your release can be defined by creating
Scope Items
Scope Items can be for example
New feature
Change for existing feature
New theme
Scope Items will be linked to Requirements, Tests, Test
Sets and Defects
Company Confidential
Define milestones
Define KPI’s for milestone
Define releases
Project Planning & Tracking
Define & Preview Thresholds
Define Scope
Company Confidential
New ALM11 Features: Milestones
Milestone is a point in release timeline that signifies the
completion of a deliverable
It enables users to track and validate the progress of the
release
Milestones can be for example
Integration between subsystems
Code freeze
Release to production
Scope Items will be associated to Milestones to follow
release deliverables
Company Confidential
New ALM11 Features: Key Point Indicator (KPI)
KPI`s will give release stakeholders a clear picture of
Where are we now?
Are we reaching the goals set for the Release?
Predefined KPI`s cover for example
Passed Test %
Test Instances Executed %
Severe Defects
Measured entities can be filtered by UDF`s and TDF`s
Company Confidential
New ALM11 Features: KPI Thresholds
Thresholds will give clear indication of what is expected
on release performance for current milestone
Thresholds curve is set per KPI to follow the releases progress
KPI thresholds reflect set objectives. • A threshold specifies an acceptable value for a KPI measurument
on specific time
• For example Requirement Coverage KPI is set close to 100% and
Severe Defects KPI is set close to 0%
Company Confidential
New ALM11 Features: Release Scorecard
Release Scorecard view provides the release status at
one glance
How KPI Tresholds are met will be indicated on different
colours: green (OK), yellow (warning) or red (critical)
Company Confidential
New ALM11 Features: Release Analysis Graphs
Individual Scorecard cells can be opened and it will show
drill down of a chosen KPI over time
KPI Type Analysis will give a clear picture of release
performance
Company Confidential
Best Practices of using PP&T
Use Release Templates in the Template project to
standardize Releases across projects!
Decide on terminology, attributes, KPI’s that will be used
throughout the organization
Limit the number of Scope Items in order to keep
tracking the release manageable for all stakeholders
Maximum of 30 Scope Items is recommended
Maximum amount can be set with a Site Admin parameter
When rescheduling Milestones, remember to re-adjust
KPI thresholds as they are not automatically adjusted!
Resources:
PP&T Best Practices book from HP
Company Confidential
Known Limitations of new ALM 11 Reporting Features
KPIs can not be copied between projects (can be shared
from Templates)
KPIs are taken once/twice per day
Historical PPT data will be purged and purged data
cannot be restored
PPT data calculations effects performance
KPIs are calculated once they are defined – no past data
Company Confidential
Administrative Aspects of new ALM 11 Features
Scheduling of Project Planning and Tracking calculations
should be done in site`s off-peak hours
Calculation duration can be limited but Release
calculations that have yet to start are considered failed
Calculations can be performed every 24 or 12 hours.
Business demands, database strength and number
milestones and KPI`s should be considered
”Run Now” –button allows PPT calculations to be
triggered manually
Company Confidential
ALM/QC Reporting in General
ALM/ Quality Center offers four tools for reporting purposes
Dashboard module
• For creating textual reports or graphs based on field values
• With ’Graph Wizard’ you can create an own graph
• Excel-reports (SQL queries to QC database)
Document Generator for generating project level Word documents
• You can get all project data into a single document
• Can contain pre-defined Graphs and Attachments
– Snapshot of the current situation
Analysis menu in each module
• Standard Reports and Graphs that can be customized and saved to Dashboard module for future use
Live Analysis in Test Plan and Test Lab
• Two configurable graphs side by side showing statistics of the selected folder
Company Confidential
Features of the Dashboard module
Fully integrated Dashboard module to centralize HP
Quality Center reporting tools including standard reports,
graphs, Excel reports and dashboard
Standardized cross-project or cross-module reporting
and drill-down graphs
Customizable pages
Including two tabs:
Analysis View for generating graphs / reports and viewing them
one at the time
Dashboard View for viewing several graphs at the same time
Possible to create private and public graphs / reports
Access rights to Dashboard are granted based on the
usergroups, everybody can view, but selected groups can create
graphs and reports
Company Confidential
Known Limitations of Dashboard Module
Different results might be shown in Analysis items and
Dashboard pages in public folders depending on the
data hiding definitions for the user group
By default it is possible to perform any kind of SQL
statements with the Query Builder of the Excel reports
(also deleting whole table in database)
Restrict the allowed SQL commands in the Site Admin!
Company Confidential
Benefits of the Dashboard Module
Simplified installation, configuration and maintenance
Dashboard Page lets you compare several reports side
by side in one view
Project Reports:
Rich style and layout option using MS Word templates
Enhanced performance
Single configuration for multiple output formats (doc, docx, html)
Centralized template management (only in ALM 11)
Company Confidential
Live Analysis
Provides a dynamic quick visual overview of all the tests
in a folder or test sets in a test set folder
Live Analysis tab is visible in tree view when folder is
selected
Graphs are updated on a fly when different folder is
selected
Company Confidential
Document Generator
A Microsoft Word based solution for reporting that
enables you to create a hard copy of the data contained
in a Quality Center project
Possibility to export data from following modules • Requirements
• Test Plan
• Test Lab
• Defects
Useful for off-line work
Created document is always a snap shot of the current
situation
Company Confidential
Excel Reports
Data from ALM can be exported to Excel with Excel
Reporting
This allows users to analyze the data with Excel
capabilities
Set of data will defined by SQL queries on the project
database
Excel reports can be exported from one project at a time
Company Confidential
Best Practises: Excel Reports
It’s recommended to allow creation of new Excel reports only to selected user groups
Complex or bad SQL queries can affect QC performance
Remove the creation rights from most people
Review new Excel reports before they are deployed to production environment
Company Confidential
Best Practices: Project Planning
Define all the needed reports already in the early phase of the QC Deployment
Needed reports affects to the data model of the QC Project
Needed fields
Mandatory fields
Definition and enforcement of list values
• Verify value
• Use one lookup list for same field in different entities (e.g. Product , Feature etc)
Company Confidential
Best Practices: Product Maturity Reports
Requirement point of view:
How much a certain Requirement is tested?
How many Defects has been found?
Number of critical Defects found?
Requirement types point of view?
Functionality maturity
Performance maturity
Usability maturity
Release point of view:
How many tests of the release has been executed?
• Pass / Fail ratio of those tests
Company Confidential
Reporting related Site Admin Parameters
DASHBOARD_PAGE_ITEM_LIMIT Maximum amount of graphs on a dashboard page
Default value is 4
SQL_QUERY_VALIDATION_ENABLED If set as ”N” QC doesn’t check that SQL query of an Excel report is valid
SQL_QUERY_VALIDATION_BLACK_LIST To make sure that certain SQL commands (DELETE, DROP, UPDATE etc) are not allowed
Ensures that you don’t harm QC database
REPORT_QUERY_RECORDS_LIMIT Maximum number of records that can be retreived from database for an Excel report
If value is set to ”-1” the number is unlimited
Company Confidential
Summary
ALM 11 is a flexible tool for all projects reporting
Different reporting cababilities give detailed information
about project performance for various stakeholders
Multi-Project analytics and Reporting for HP Quality Center and ALM Shir Goldberg
Company Confidential
INNOVATIVE PRODUCTS FOR HP ALM & QUALITY CENTER
ASSURE PROJECT HARMONIZER
ASSURE TOTALVIEW
ASSURE ADMIN CONSOLE
ASSURE TRASHBIN
ASSURE BUGZILLA ADAPTER
ASSURE TEST DATA INTEGRITY
Company Confidential
Next generation reporting for HP Quality Center & HP ALM
ASSURE TOTALVIEW
Company Confidential
Your Vision
A complete picture of your QC/ALM data
Unlimited access to QC/ALM data
Status of requirement coverage and product maturity
Reporting without performance impact on the production
environment
Visualize trends to improve efficiency and output
Multi-Project analysis and reporting
A reliable and accurate reporting solution
A product that is user friendly and quick to deploy
A solution that supports business processes
Company Confidential
Our vision
Provide a Business Intelligence solution...
Where data consumers can access HP Quality Center
data without boundaries or limitations
That provides trend and history analysis
That supplies compehensive multi-project analytics
Which is a user friendly data source for reporting and
data analysis interfaces
That empowers business decision makers
A Data Mart is the intelligent data solution for QC reporting!
Company Confidential
Why BI?
“You can't beat a BI solution for reporting. The source
system will never be able to compete. It is a proven fact
by leading companies all over the world.”
“If a company wants to beat competition it must be one
step ahead-BI is the key”
“All leading software providers supply a separate
platform for BI that is not part of any legacy system”
“The source system will never be able to
incorporate BI best practices as it is always limited in its
resources and capabilities”
Company Confidential
Assure TotalView
Company Confidential
What is TotalView?
TotalView is a BI solution for HP Quality Center & HP
ALM
Revolutionary approach to QC/ALM reporting and data analysis
Supports decision makers and business processes
Leverages BI Best Practices
Product Components
Data Mart
Management Console
ETL (Extract-Transfer-Load) processes
Ready to use dashboard (using QlikView)
Report library
Company Confidential
Key Features
Gives access to all HP Quality Center & HP ALM data
Multi-Project analytics
Reliable historic data and trend analysis
QC/ALM version upgrades are supported
A complete picture of QC/ALM data
A ”ready to go” dashboard
Management Console
Define project groups for reporting
Select the User Defined (custom) Fields required for reporting
Manage the Data Mart
Company Confidential
Benefits of a Data Mart
A separate database dedicated to data analysis
Minimal impact on the production database
Intuitive data structure and easily accessible
Accurate and trustworthy data
Recording of data changes enables trend analysis
Based on BI methodology and best practices
Company Confidential
The make up of a Data Mart
Company Confidential
A ”Star Schema”
A star schema consists of a Fact table and multiple Dimension
tables and represents a single business subject for analysis.
For example – “Execution Progress”
Company Confidential
A data model
A logical representation of the Data Mart and all its star
schemas.
Execution Progress
Test Readiness
Defect Detection
Priority Bug Status
Severity
Plan Status
REQ
TEST
Release Cycle
Release
Test Set
Person
Time Exec
Status
Test Instance
Company Confidential
TotalView Management Console
Company Confidential
Why TotalView?
Without TotalView
Complex table structure
Limited Depth
No QC/ALM version support
Complex maintenance
Short-term solution
Analysis on Prod. DB
Complex data management
Complex report build
Limited “Cross-Project”
With TotalView
Intuitive table structure
Trend and Progress
QC/ALM version support
Minimal maintenance
Long-term solution
Dedicated analysis DB
Management console
Simple report build
Unlimited “Multi Project Analytics”
Company Confidential
Other Considerations
Multi-Project Analytics require a uniform data structure
QC10 Premier / ALM11
Assure Project Harmonizer
Strict Policies for project creation and changes
Multi-Project Analytics require a uniform data values
QC10 Premier / ALM11
Assure Admin Console
Assure Data Scrub
Strict Policies for project creation and changes
Company Confidential
Contact Assure
Web: www.assure.net
Email: [email protected]
Twitter: http://twitter.com/assureqc
