QTP 10 Review

8/8/2019 QTP 10 Review

1/20

Page 1

All rights reserved Solmar Knowledge Networks

Welcome toAdvancedQTPsreview of Quicktest Professional 10 (QTP Atlantis).

QTP 10 has recently come out (early February), and is now available for evaluation from HPs

main site (you can find ithere). QTP 10, like QTP 9.5, is also delivered in a complete DVD that

contains all the add-ins and SDK packages, so brace yourself for a lengthy download (1.3GB

zipped, 1.8GB extracted).

How this review is structured: The review is built as a free-style walk-through across QTP. If you

are interested in a concise feature breakdown, scroll down to the end of the review for a

feature-by-feature summary.

Disclaimer: Many of the killer feature of QTP 10 only work when its coupled with Quality Center

10. As I dont have a working Quality Center just sitting there, waiting to be upgraded, I was

unable to cover those features. You can read more about my impressions of them in ourQTP 10

live demonstration breakdown. I will just say that they are very impressive, and bring real added

value to the QTP-QC dual.

The Install Process

Installing QTP 10 is pretty straightforward and is practically identical to installing QTP 9.5.

However, unlike QTP 9.5, you can upgrade your previous installation, instead of removing it

completely (Personally, I think removing the old one is for the best no pesky leftovers to

contaminate your system). If you have any add-ins installed, they would also be automatically

upgraded.
http://www.advancedqtp.com/http://www.advancedqtp.com/http://www.advancedqtp.com/https://h10078.www1.hp.com/cda/hpms/display/main/hpms_content.jsp?zn=bto&cp=1-11-127-24%5e1352_4000_100__https://h10078.www1.hp.com/cda/hpms/display/main/hpms_content.jsp?zn=bto&cp=1-11-127-24%5e1352_4000_100__https://h10078.www1.hp.com/cda/hpms/display/main/hpms_content.jsp?zn=bto&cp=1-11-127-24%5e1352_4000_100__http://www.advancedqtp.com/knowledge-base/reviews/qtp-id8/qtp-10-live-demonstration-breakdown/http://www.advancedqtp.com/knowledge-base/reviews/qtp-id8/qtp-10-live-demonstration-breakdown/http://www.advancedqtp.com/knowledge-base/reviews/qtp-id8/qtp-10-live-demonstration-breakdown/http://www.advancedqtp.com/knowledge-base/reviews/qtp-id8/qtp-10-live-demonstration-breakdown/http://www.advancedqtp.com/knowledge-base/reviews/qtp-id8/qtp-10-live-demonstration-breakdown/http://www.advancedqtp.com/knowledge-base/reviews/qtp-id8/qtp-10-live-demonstration-breakdown/http://www.advancedqtp.com/knowledge-base/reviews/qtp-id8/qtp-10-live-demonstration-breakdown/https://h10078.www1.hp.com/cda/hpms/display/main/hpms_content.jsp?zn=bto&cp=1-11-127-24%5e1352_4000_100__http://www.advancedqtp.com/


2/20

Page 2


Figure 1 - The Main Install Screen

Figure 2 - The Upgrade Dialog


3/20

Page 3


Figure 3 - Add-Ins Selection Page

I ran the upgrade from a wireless network drive, through my MacBook Pro, to an XP virtual

machine, and it only took approximately 10 minutes. I guess this is a much elaborated way for

saying the install process is very fast.

After the installation is complete, you can expect to find all the familiar icons at their familiar

places, with one significant addition : an out of the box installation of extensibility help files for

.Net, Web, Java and Delphi. I shall address the matter of extensibility later on.

Figure 4 - Extensibility Help


4/20

Page 4


Overall Look and Feel

QTP 9.0+ users will feel right at home. Most of the user interface is unchanged. At first glance,

the IDE seems to be completely identical, from the default panes to the start page (the start-

page help links have been updated, of course).

The menus are now presented as trees rather than tabs, giving a more updated look. This is

undermined, though, by the old-looking ridged buttons and panels, creating an unpleasant

overall feeling. It is immediately noticeable when compared to the option tree in Visual Studio,

which is more UI consistent and soft. But maybe it is just that I am too demanding.

Figure 5 - QTP option tree as compared with VS

(on the left QTP: ridged and unpleasant; on the right VS: soft and consistent).

The open / save dialogs were also updated to display a much friendlier big-button style:

Figure 6 - Save / Open Dialog


5/20

Page 5


Even with all the extra text-parsing features, QTP seems to run more smoothly (which quite

surprised me), and opens / saves large tests slightly faster than QTP 9.5; at least for me. The

overall feeling is very responsive, and as an extra bonus, you can define a global Stop-Test

hotkey in case QTP hangs on an endless loop, and the window is inaccessible.

Figure 7 The (same old) IDE

Custom Toolbars

A nice new feature presents the ability to customize the toolbars and button layouts. Any menu

command can be planted in any of the toolbars (though you cannot add new toolbars of your

own). The process is pretty intuitive, and basically only requires you to locate the relevant

command (the All Commands category helps in this regard), and drag it to the toolbar you want

it in.


6/20

Page 6


Figure 8 - Selecting a command to add

Another nice touch is the ability to add external commands to the tools menu Just select an

external tool / file, and it will be available though the tools menu.

Figure 9 - Adding an external command (left), and how it shows in the tools menu (right)

Version Control and Save to go

The file menu already hints some of the new killer-features of QTP 10: Version control (onlyworks with QC), and saving a test with all its resource (which I like to think of as Save to go).


7/20

Page 7


Figure 10 - Source control, and "Save to-go"

As I could not work with the version control features, I will only refer you to ourQTP 10 live

demonstration breakdown, where I reviewed them, and was much excited by it. However, I was

able to play with the Save To-Go feature, and I must say its just what the doctor ordered.

We were all frustrated at one point or another, by the impossible task of transporting our test to

a different directory / computer. It seems that no matter how much we tried to keep all our

resources organized, one of them would always slip through, leaving us with a broken tests and

a weary heart. Well, No more!

Now we can simply choose File->Save test with resource which will display the tests resource

tree; you can also choose either to save it to a new location, or to export it and all its resources

to a compressed ZIP file (smart shows HP has kept the user, not only the feature, in mind).

Once you hit the Save button, QTP will save the test to the location you specified, copying all the

function libraries, object repositories and recovery scenarios to dedicated folders, and re-linking

them to the main test (via their relative paths). In addition, youll be presented with an HTML

summary of the procedure.

Figure 11 - "Save to-go"
http://www.advancedqtp.com/knowledge-base/reviews/qtp-id8/qtp-10-live-demonstration-breakdown/http://www.advancedqtp.com/knowledge-base/reviews/qtp-id8/qtp-10-live-demonstration-breakdown/http://www.advancedqtp.com/knowledge-base/reviews/qtp-id8/qtp-10-live-demonstration-breakdown/http://www.advancedqtp.com/knowledge-base/reviews/qtp-id8/qtp-10-live-demonstration-breakdown/http://www.advancedqtp.com/knowledge-base/reviews/qtp-id8/qtp-10-live-demonstration-breakdown/http://www.advancedqtp.com/knowledge-base/reviews/qtp-id8/qtp-10-live-demonstration-breakdown/http://www.advancedqtp.com/knowledge-base/reviews/qtp-id8/qtp-10-live-demonstration-breakdown/


8/20

Page 8


I messed a bit with the entire mechanism, and I am happy to say it works perfectly. It is a

straightforward solution to a very nagging problem. Well done!

Tasks and Comments

Another area that may appear unchanged at first glance is the IDE panes. You still have all theold ones (DataTable, Missing Resources, Debug, Active Screen, Available Keywords, Resources,

and Test Flow), and hiding among them is the new To-Do pane.

The To-Do pane contains two tabs Tasks and Comments. Tasks are a structured way to create

and manage ongoing project tasks, and comments do pretty much the same thing in a more ad-

hoc, on-the-fly manner.

Whenever you write a comment starting with To Do, todo, to-do, or TODO, the editor

will automatically pick that up and show it in the TODO comment box. Double clicking on the

comment will jump to the relevant position in the code, and usually, thats pretty much all you

need a concise reminder of the issue at hand, and an anchor to the relevant position.

Figure 12 - The To-Do Pane - Comment Tab

The comment parser usually only operates on the tests internal actions, but you can make it

also parse the external actions and open function libraries that are associated with the test. I

guess its not the default behavior due to performance issues, although even in extremely large

tests, the parser didnt put much of a strain on my system only a split second overhead

(impressive, considering the fact it runs on every cursor move, and not just on file save).

While the comment feature was perfectly implemented, and is very useful indeed, the Tasksfeature left me puzzled and unsatisfied. When adding / editing a task, you can specify certain

details such as its subject, priority etc. Personally, I dont see any added value here, compared to

writing that meta-data as a simple TODO comment.


9/20

Page 9


Figure 13 - Task details

If I ever need added control and manageability for my tasks, Id use a dedicated task application

(such as the tasks in outlook, etc.); otherwise, I would much rather just quickly write a TODO

comment, and be done with it. The fact that I cannot link a task to a specific code-line creates

the absurd situation where ad-hoc comments are more powerful (in this sense) than the

structured tasks.

To sum up, I liked the Comment feature, and once I could use that, I see no point to the Tasks

feature. If youd like, you can export both the TODO comments and the tasks to an XML / XLS /

CSV file.

Report Improvements

Well, comments and tasks might help us to organize our workflow, but dont really extend QTPs

automation features and possibilities. To begin our exploration of features that are directly

linked to automation, lets take a closer look at the results and the reporter.

The most significant new feature to the results and reporter is the Local System Monitor. You

have the ability to monitor several resources on your computer throughout the run-session, and

define custom thresholds for them. Values that exceed these thresholds will fail the test.

Defining the counter is done via the File->Settings menu, and its a quick and easy process. There

are many different counters, and each one is accompanied by the relevant explanation, so you

wont have to guess what Virtual Bytes (for example) is.


10/20

Page 10


Figure 14 - Defining system monitors

The system monitor feature is fused seamlessly with the report it has a dedicated tab next to

the Screen Recorder, and each result node you select will focus on the relevant spot on the

resource graph. Similarly, if you select a point on the graph, the closest result node will be

automatically selected in the test steps tree.

The graph itself allows us to zoom, hide / display specific monitors, and clearly displays the

monitor limit weve defined (if any). The useful UI, combined with the Step-Graph Sync makes

this feature extremely powerful and fun to use (yes, work can also be fun). I expect this ability to

develop into a sort of an impoverished load test, although the ability to monitor only one

process may be too limiting in that regard.

You can also access the monitors throughout the test-run via the SystemMonitor.GetValue

command, allowing you to write your own custom reports, decision structures and checkpoints

based on the monitors data.


11/20

Page 11


Figure 15-The graph and the steps tree are in perfect Sync

Not only do the System Monitor Graph and the Screen Recorder sync with the step tree, but the

results now offer the ultimate step sync right click any node in the test-step tree, and select

Jump to step in QuickTest. QTP will focus on the relevant script line within the relevant action.

I was really excited about this feature, until I realized that if youre using a custom report

function (and lets be honest we all do), QTP will always jump to that one function, making the

whole thing completely useless.

Another long-awaited feature is the ability to export the results to more formats. You can exportthe Screen Recorder movie (as before), the system monitor data, or the report itself either to

HTML, Doc or PDF. The export works very well, and its quite customizable you can still choose

a short or long format, or point to your own custom XLS file, which will define the final data,

look and feel of your report. The only thing missing is an API call to do this from behind the

scenes, or automatically at the end of each run session.


12/20

Page 12


Figure 16 - The Export Report window

One last new feature to the reporter is the out-of-the-box ability to attach images to report

nodes. Forget about all your elaborate workarounds for attaching images to the report in QTP

10 just need to add the picture file path to the report command. Ive tried it out, and it works

well enough.

Intellisense and Auto-complete

Besides the QC-integration features, the biggest and most significant change to QTP has been in

QTPs internal intellisense and auto-complete mechanisms. For the first time, we have bothvariable auto-complete, as well as COM intellisense up to any hierarchy and level wed wanted.

Needless to say, these features got me very excited, and that may have been part of the reason I

was so disappointed by them.

Dont get me wrong things are much better with these features than without them, but when I

think ofwhat they couldve been, I cant help but feel that a great deal of potential remains

untapped.

But first, lets acknowledge what does work fine even great about these features. The

variable auto-complete works just like the functions auto-complete just type in the first letter

or letters of a variable, hit Control-Space, and QTP will auto complete it, or at least present a list

of viable options. It works both on your current action, as well as on every associated function

library.


13/20

Page 13


Figure 17 - Variable auto complete

On a basic level, the COM auto complete feature also works fine Insert any object into avariable, and you will have intellisense for all its inner methods and properties, up to as many

level s as youd like. The process is immediate and transparent to the user. Any other variable

that will receive this reference will present the same intellisense, as will any function that

returns it.

Figure 18 - Intellisense on an Excel object. Notice the auto complete list is shown for a child property

Figure 19 - Intellisense is passed between functions and variables

The intellisense mechanism works on more than script created COM objects. You can use it on

test objects in your AUT. Just add the relevant object to the object-repository (will not work with

DP), use the . Object property to punch through to the AUTs runtime environment, and


14/20

Page 14


navigate on through the intellisense. This is really wonderful in rich environments such as .Net,

Java and Web, where the objects have an unending variety of runtime properties, function and

inner objects.

Figure 20 -Inserting a Test-Object into a variable, and using the inner intellisense to spy on nested properties and

methods

This seemed to me like the ultimate solution to the Object-Spy inability to drill down throughcomplex runtime properties. And indeed its a valuable exploratory tool, and it greatly improves

the nonexistent intellisense in prior QTP releases.

A very nice touch is the inclusion ofTarun Lalwani's Intellisense Resizerin QTP 10. You can now

drag the edge of the auto complete window to widen / narrow the display, which comes in

handy when youre dealing with long property / method names.

The Debug pane has also undergone some improvements, though these are more UI related

than meaningful advancements. You can now see the type of the variable youre monitoring,

and familiar objects now have a little plus sign for expanding object properties.

Figure 21- The new debug pane look. Notice the variable type column

So, if everything is so great, why am I so disappointed? Well, critical little bugs and overlooks still

prevent this from becoming a significant enhancement for advanced programming in QTP.

So now, lets turn to the bad stuff:


15/20

Page 15


First of all, neither the auto complete nor the intellisense work for VB Script classes. As one who

uses them extensively throughout his code, this deeply saddened me (even though I knew

beforehand this would be that case)

As for the auto complete Variables are not recognized when declared with the Public or

Private keywords (only with Dim statements). Also, variables defined with brackets (e.g. [Dim])show up on the list of possibilities, but are never automatically completed. And while these are

mere annoyances, the following present real difficulties.

There is no intellisense for functions defined in associated library files, only for those defined

within the actions themselves (Variables directly defined in associated libraries are parsed, but

not functions). Ill say it again There is not intellisense for functions stored in associated library

files. Personally I think this is a no-go issue. But in any case, the functions result is parsed at

the lowest possible code-line, so for example, in this function:

Figure 22 - The intellisense parser is blind to decision trees

The lowest return value is nothing (appears in line 6), so the function will always have the

intellisense of Nothing. In this case you can just switch the two cases, but what of a function that

returns 3 possible values? Or have multiple exit gates?

Actually, things are even worse functions inner variables leak out and contaminate the entire

action scope. Objects put into an inner function variables will show up on the intellisense of

outer variables with the same name, as shown here:


16/20

Page 16


Figure 23 - Notice how the global variable is affected by changes made to the function's inner variable

But, at least we have the Test-Object intellisense, right? Well, things arent perfect in this area

either. It seems that the intellisense chain is broken whenever a collection is involved. Forexample, no further intellisense is provided for this line, since weve passed through a

collection:

Since many of the interesting properties in .Net, Web and Java are stored in collections, this

presents a real problem. Its not that this feature is entirely without merit, but just a few final

touches from being amazing. Update: Im happy to say that HP also acknowledges the

importance of this issue, and they plan a patch that will allow intellisense to work though

collections. Its supposed to be released shortly, and we will update you when it does.

Another related major issue is breaking thePDM.DLL debug hack, which allowed for some sort

of limited intellisense in the debug pane. In QTP 10, registering PDM.DLL version 9+ will cause

the debug pane not to show any object variables whatsoever not in the Watch tab, nor in the

Variables tab. This was quite a crippling blow to me personally, as Ive grown to relay on it,

though I guess its not fair to cry over a discontinued backdoor hack that was unsupported to

begin with.

So, in conclusion, these are all worthwhile features, but they are all deeply flawed. While theyll

make your life easier, they do not bring the revolution I expected of them.

Another almost feature is highlighted code regions. Unlike most modern editors, which

include folding code regions (i.e. regions you can collapse and expand using + and 0), QTP 10

only highlights the current code block with a blue box. While this provides some visual aid, its

severely outdated for a 2009 product (or a 2000 product for that matter).
http://www.advancedqtp.com/knowledge-base/articles/qtp-tricks4/qtp-hacks/vs2008-pdmdll-dramatically-improves-qtps-debug-engine/http://www.advancedqtp.com/knowledge-base/articles/qtp-tricks4/qtp-hacks/vs2008-pdmdll-dramatically-improves-qtps-debug-engine/http://www.advancedqtp.com/knowledge-base/articles/qtp-tricks4/qtp-hacks/vs2008-pdmdll-dramatically-improves-qtps-debug-engine/http://www.advancedqtp.com/knowledge-base/articles/qtp-tricks4/qtp-hacks/vs2008-pdmdll-dramatically-improves-qtps-debug-engine/


17/20

Page 17


Figure 24 - Highlighted code regions (QTP - left), compared to collapsible code regions (VS - right)

Improvements to Actions and Checkpoints

QTP 10 includes several improvements to the actions and checkpoints mechanisms. The most

significant improvement is the new ability to dynamically call an action, without inserting it to

the test beforehand. This can truly revolutionize the interoperability of cross project tests and

actions, and Im very happy this feature found its way into the version.

All the run-time debug abilities will be available for these dynamically added actions, so youre

not giving anything up by using this feature. Just add this command in your script, and it will

execute when the test run:

LoadAndRunAction(TestPath, ActionName, [Iteration], [Parameters])

As for checkpoints Bitmap checkpoints has been given quite a buff in addition to the ability

to view the difference between the expected and actual results, you can now build your own

custom comparison DLL, and integrate it with the native checkpoint mechanism of QTP. There

are lengthy reference documents and tutorials explaining exactly how this can be done.

In addition, QTPs automation API has been updated, and among its many changes, those

dealing with actions really stand out. You can now read an actions script, edit it or overwrite it

entirely before starting the run session. You can also add a new action to the beginning or the

ending of the current test. There is plenty of potential in these new API interfaces, and I cant

wait to find new and exciting uses for them (parse a script to automatically add class

inheritance?).

Environment and Extensibility Improvements

Beside the constant expansion of the supported environments and applications, QTP 10 also

includes several improvements to its extensibility packs.

Web extensibility now includes an out-of-the-box limited support for standard AJAX controls; aglobal Window object to be used within JavaScript to access the IE namespace; Extensibility

code can now be debugged by the Script debugger or Visual Studio debugger; and there are

minor improvements to reporting capabilities.


18/20

Page 18


.Net Add-In and Extensibility now includes VS2008 project templates; Attaching to property

grids (via the new SwfPropertyGrid object); Added the TO property and method for reading

tooltip error text (via ErrorProviderText) ; and support for custom table checkpoints;.

Terminal Emulator Add-In now has a Validate option in the configuration page, allowing you to

validate your emulator configuration before the actual run session.

Delphi is now also supported, with a SDK for building your own custom Delphi extensibility.

The following lists the new environments that QTP 10 now supports:

Windows 2008 Server 32-bit Edition

Windows 2008 Server 64-bit Edition

Windows Vista, Service Pack 1, 32-bit Edition

Windows Vista, Service Pack 1, 64-bit Edition

Windows XP Professional 32-bit EditionService Pack 3

Citrix Server 4.5Internet Explorer 8, Beta 2

Firefox 3.0.x

Delphi: IDE, versions 6, 7, and 2007 (for controls based on the Win32 VCL l ibrary)

SAP: CRM 2007 (For controls that support test mode enhancements. Requires SAP

notes: 1147166, 1066565, and 1002944. Later SAP notes related to test mode

enhancements are not supported.)

Java: IBM 32-bit JDK 1.5.x, SWT toolkit version 3.4

Java Extensibility: Eclipse IDE 3.4

.NET: .NET Framework 3.5Service Pack 1

Text Recognition has also been given an upgrade, and now you can select specific languages to

be processed, and also the OCR mode (single / multiple block).


19/20

Page 19


Figure 25 - The new text recognition option screen

Summary and Conclusion

QTP 10 brings many improvements to the users central workflows, and adds some real value

with these. The two main killer-features show great potential: The QC integration features

(Version Control and Resource Management) make versioning a viable option, for the first time;

and the IDE and intellisense improvements will definitely improve your productivity and

efficiency. Many of the other features are significant and long awaited improvements to some of

QTPs central mechanisms, and they are mostly well executed.

However, weighing QTP 10 potential (at least for the IDE improvements) against the actual

features and changes, I am left with a strong sensation of a missed opportunity. This is not tosay that QTP 10 is not a worthwhile version, but that in my view it couldve been great.


20/20

Page 20

ll h d l l d k

Heres a concise bottom line review:

Area Feature Conclusion

QC integration Version Control and comparisons

(based on live demo).

Amazing. Completely transforms the CM

for the automation team.

Portability Save test with associated

resources

Excellent solution to a very bugging

problem. Well executed.

IDE To-Do comments Excellent collaborative and tasks

management tool.

To-Do tasks Nice idea, but without the ability to link a

task to a specific line, Id rather work with

Comments.

Variable auto complete and inner

object intellisense

Nice and will certainly come in handy. The

bad implementation makes the whole

thing useless when its mostly needed (not

parsing library files or VB Script classes,

straightforwardly parsing functions, no

support for collections, etc.)

Customize menus Works fine, and does exactly what its

supposed to.

Code block highlighting Nice, but pales in comparison to other,

older, and cheaper IDEs.

Report Export to Doc, PDF, HTML Excellent. Still lacking an automated API for

exporting / saving / sending the results.

System Monitor Great idea, Wonderfully executed. Will

probably become a must for almost every

project. Missing the ability to monitor

more than one process.

Natively add images to reportevents

A true necessity, great implementation

Jump to script line from report

step

Sounds cool, but meaningless if youre

using a report function

Actions Dynamically add actions without

pre-attaching them to the test

Yes, Yes! YES! Works just as I expected

(which is good).

Add, edit and overwrite actions

through the automation API

Excellent. Cant way to start experimenting

with it. Power users will love this.

Bitmap

Checkpoints

See the actual-expected

difference bitmap

Greatly improves the usefulness of the

checkpoint report.

Write custom bitmap DLLs Seems unnecessary serves power users

which I doubt use checkpoints. However,very well documented, and easy enough to

implement. Well executed.

Misc. Overall look and feel Runs quite fast, despite the added

features. Mostly looks updated and

professional.

Documents

QTP 10 Review