Embed Size (px)
Citation preview
Testing And Tools ForPrint And Imaging Drivers
OutlineOutline
Basic testing philosophy
Where Microsoft finds bugs
What testing does Microsoft currently do
Microsoft provided test tools
Importance of Debugging
Summary
Basic Print and ImagingTesting Philosophy
Test Device DriversTest Device Drivers
Right amount of QA before you shipFewer bugs for users to find
Lower support costs
Quality not just the responsibility of QA teamNeed good tools for developers
Enable developers to run unit tests
Microsoft InvestmentMicrosoft Investment
Microsoft is interested in the test environment to improve the overall driver ecosystem
Microsoft QA approachInbox driver program
Logo Program
Provide tools for drivers we never see
Driver education on tools and methods
Where Microsoft Finds Bugs and How They’re Triaged
Bug BucketsBug Buckets
Main areas bugs are found (highest to lowest)Through code analysis (PREfast)
Printable regions
Basic printing features
Stress testing
Font handling
Advanced features
Setup
Communications and I/O
Microsoft supplies tools that target each bucket
What Happens if Microsoft Finds a Bug?What Happens if Microsoft Finds a Bug?
Reporting the bugWhat did you see?
What were you expecting?
Exact steps to reproduce
TriageIs it in Microsoft code?
Assign to a developer for investigation
FixDeveloper builds private for test to verify
Ship team decides whether to accept the change to the product
Check in
ResolveFix is verified against live build by testTester who opened original bug is the only person allowed close the issue
Microsoft Provided Test Tools
Microsoft Testing of Printing and Imaging Microsoft Testing of Printing and Imaging Device DriversDevice Drivers
Tool DistributionsWinparty 2005
Used for device and driver testing today
Used for inbox driver check in
Included in upcoming Windows Driver Kit (WDK)
WDKUses WTT Test Shell
Makes it very simple for anyone to run tests and collect results
What Architectures Are Supported?What Architectures Are Supported?
Tools work on a variety of architecturesx86
x64
Itanium
Inbox check-in requires testing on all three architectures by the IHV and Microsoft
To test a 64bit operating system (OS)Run 64-bit tools
Run 32-bit tools
Get 64-bit machines todayRequired for WOW64 testing
Microsoft Provided Imaging Test ToolsMicrosoft Provided Imaging Test Tools
Data transfer tool
Error Handler test tool
Filters test tool
Properties and Item Tree test tool
Imaging Driver Robustness tool
Twain Spec Compliance tool
Power Management Stress test tool
Twain Datasource Reliability/Robustness tool
WIA Navigation and Property tool
WIA Trace Viewer
Microsoft Provided Print Test ToolsMicrosoft Provided Print Test Tools
Print Device Capabilities
Print Area
Photo Print test
Print Gremlin
Office Tests
Inbox App tests
DevMode Corruptor
Print Driver Stress
Font Test
Importance of Debugging
Increases stability of product
Increases customer satisfaction
Increases tester output
Increases developer output
Lowers cost of development
Lowers cost of testing
Lowers support costs
How Does a Debugging Environment Help?How Does a Debugging Environment Help?
Invest in Your QA ProcessInvest in Your QA Process
Example of testing without debugging:Tester notices applet crashes every few minutes, disregards break
Example with debuggingTester notices applet crashes, creates debugger break
Passed on to developer, developer finds heap corruption ( developer finds code bug )
Fixed before driver is releasedAvoids support costs
Increases stability of product
Avoids customer dissatisfaction
Every single debugger break is a bug to investigate
Kernel vs. Local DebuggingKernel vs. Local Debugging
Easy to set up:Two approaches
User mode debuggingEasy to set up
Lots of cmd windows on screen
Can’t catch kernel-mode bugs
Kernel mode debuggerHarder to set up (separate computer needed)
No visible testing machine effects
Catches kernel-mode bugs
Debugging user mode drivers can be a probleme.g. Spooler context and loading symbols
Debugging ToolsDebugging Tools
PREfast for driversPerformed mostly by developers
You need to have the commitment to fix all the errors
Zero bug count required for inbox check-in
Configurable through filters
Application Verifier, Driver Verifier, clean test systems, Pageheap, NTSD etc.
Expands scope of testing
Performed by developers and testers
Very important for quality testing
AppVerif vs. VerifierAppVerif vs. Verifier
Application Verifier = appverif.exe
Driver Verifier = verifier.exe
Verifier is a kernel-mode driver debugging toolVerifer was and continues to be widely adopted to find and fix kernel mode problems
IHVs have found and fixed lots of bugs because of Verifer since Windows 2000
AppVerif is a user-mode driver and application debugging tool
Used when testing user mode components
Good Test Debug SetupGood Test Debug Setup
Development resources to debug a break
User mode or kernel mode debugger
“Clean” test machineMake use of imaging software
Latest debug tools on debugger and test machineWe have a new tool to make debugging set-up easier!
Winparty 2005
Good Test Debug Setup (con’t)Good Test Debug Setup (con’t)
Application Verifier (user mode debugger)SpoolerExplorerWIAIHV spawned processAll test tools
Driver Verifier All installed kernel-mode components
NTSD is the Microsoft standard user-mode debugger
Enable NTSD on spoolerEnable NTSD on WIA serviceEnable NTSD on all IHV-spawned processes
Debugging is for test and development
PREfast OverviewPREfast Overview
Static code analysis toolRun against source code
Automated tool to find code defectsNot limited by test cases
Identify location of bug (easy to fix)
Run on source code, not binaries
Analyzes code for bugs before expensive testing begins
PREPREffast Overview (con’t)ast Overview (con’t)
Checks for common coding bugsBounds (array bounds violations)
Precedence & stack hogs
HRESULT (abuses of HRESULT type)
Divide by zero errors
Null pointers/invalid pointers It’s just a tool, not a magic bullet
Complements other tools such as Application Verifier, Pageheap, etc.
Does this tool find all the defects? No!
PREPREffast Buffer Over-run Exampleast Buffer Over-run Example
extern TCHAR g_szName[MAX_PATH + 1];
static TCHAR c_szServerName[] = "SERVER_NAME";
DWORD dwSize = sizeof(g_szName);
TCHAR szAnsiName[MAX_NAME_LENGTH + 1];
pECB->GetVariable (pECB->ConnID,
c_szServerName, szAnsiName, &dwSize);
Security bulletin (and patch) MS01-023
Detected by PREfast 4.0: Fixed in Windows XP
SummarySummary
Microsoft provides many printing and imaging test tools
Proper debugging saves time and money and increases customer satisfaction
Testing is not just for testers
Invest in your QA org
Call to ActionCall to Action
Use Microsoft tools and methods to improve the efficiency of your QA org
Setup proper debugging to find bugs
Work with Microsoft to improve the stability of your device drivers
Additional ResourcesAdditional ResourcesCommunity Sites
http://www.microsoft.com/communities/default.mspx MSDN Developer Community Chats:
Windows Printer Driver Development http://msdn.microsoft.com/chats/transcripts/windows/windows_092104.aspx Windows Drivers: Printer Drivers http://msdn.microsoft.com/chats/windows/windows_101602.aspWindows Drivers Printing and Networking http://msdn.microsoft.com/chats/windows/windows_022002.asp
Web resourcesWinFXhttp://msdn.microsoft.com/downloads/ WS-Devices Profile
http://msdn.microsoft.com/webservices/understanding/specs/default.aspx?pull=/library/en-us/dnglobspec/html/devprof.asp Printer Working Grouphttp://www.pwg.org WHDC Printing home page: http://www.microsoft.com/whdc/device/print/default.mspxWHDC Still Imaging / WIA home page: http://www.microsoft.com/whdc/device/stillimage/default.mspx
Additional ResourcesAdditional Resources
E-mailFor developer print questions: prninfo @ microsoft.com
For developer scan questions: wiainfo @ microsoft.com
For developer color questions: mscolor @ microsoft.com
© 2005 Microsoft Corporation. All rights reserved.This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.
