How to Improve Driver Quality With Winqual / WHQL

Scott HerrboldtDirectorWindows Hardware Online Services (WHOS)scotther @ microsoft.comMicrosoft Corporation

Session OutlineSession Outline

Windows Hardware Online Services

State of Driver Quality Today

Participating in the Designed for Windows (DFW) Logo program

Recent improvements

Windows codenamed “Longhorn” Partner Logo Program

Using Windows Error ReportsResponding to customer issues

Updated tools now available

Using Windows UpdateCustomer feedback update

The right driver for the right machine

Planned expansion of Windows Update Content for Longhorn

Windows Hardware Online ServicesWindows Hardware Online Services

The Windows Hardware Online Services (WHOS) team focuses on helping IHV and OEM partners create, support and deliver high quality products running Windows

Key Services offered:Windows Hardware Quality Labs (WHQL)

Winqual.Microsoft.comProduct testing submissions

Windows Error Reports

Driver Distribution Center for Windows Update

Windows Catalog & Marketplace Listings

State of Driver Quality - TrendsState of Driver Quality - Trends

March ‘04 March ‘05

 # Category # Category

1 Display 17.9% 1 Display ▬ 17.9%

2 Pool/Memory Corruption 13.6% 2 Pool/Memory Corruption ▲ 15.9%

3 OS Core 13.5% 3 OS Core ▲ 14.2%

4 Networking 12.6% 4 Networking ▼ 10.8%

5 Multimedia 9.51% 5 Multimedia ▼ 7.68%

6 Application Drivers 6.57% 6 Application Drivers ▲ 7.51%

7 Anti-virus 5.54% 7 Hardware Failures ▲ 7.00%

8 CD-Burning 5.39% 8 Anti-virus ▼ 5.42%

9 Hardware Failures 4.99% 9 Storage ▲ 4.94%

10 Storage 4.60% 10 CD-Burning ▼ 4.28%

Customers are Not Getting Needed AssistanceCustomers are Not Getting Needed Assistance

Customer responses exist for only 48% of all crashes

Only 58% of those responses offer the user a solution

20% identify what driver or hardware caused the problem, but no solution is available

We need to work togetherShare symbols with Microsoft so we can better analyze customer crashes

Security & Privacy of symbols covered by legal agreements

Use Winqual to provide assistance to for our mutual customers

Customers Expect Better ReliabilityCustomers Expect Better Reliability

Freq % Cust % V-Sat % D-Sat

0 0% - -

1 33% 70% 3%

2 19% 53% 6%

3 12% 39% 9%

4 8% 40% 10%

5 6% 35% 5%

>5 23% 26% 21%

On average, how many times per month does Windows crash or experience serious bugs?

Driver reliability issues are costing you money

January 2003 survey of 754 homes and 503 small businesses

$0

$5,000

$10,000

$15,000

$20,000

$25,000

$30,000

$35,000

$40,000

$45,000

100 2000 4000 6000 8000

Derived Crash to PSS Cost Ratio (@ $30/call)

Crashes

A large OEM did a study of blue screen related calls

Result: OCA predicted blue screen-related calls on a 6:1 ratio

In a Microsoft survey, crash frequency was found to be highly correlated with decreases in customer satisfaction

Customers Expect Ease of UseCustomers Expect Ease of Use

Expectation

When I report a crash, I will receive help

When I plug in a new device, it works automatically

When a solution exists for a problem I’m having, Windows should automatically update me

Reality

52% of crashes still need solutions

Many product installation experiences require loading software manually

Only 19% of improved drivers that have been submitted to WHQL are available to users through Windows Update

Microsoft Service Investments in Driver QualityMicrosoft Service Investments in Driver Quality

Need

An understanding of what your top customer satisfaction issues are

Mechanisms to validate quality in your products

The ability to reach your customers when an update is needed

Solution

Windows Error Report data analysis for your products

Logo Programs, HCT & WDK, WHQL assistance

Driver Distribution Center for Windows Update

What tools does Microsoft provide to help meet increasing customer expectations?

Participating in the DFW Logo program

WHQL Response & Supportability ImprovementsWHQL Response & Supportability Improvements

Dramatic simplification of tech support

Reduced complexity in contacting WHQL support

2-tier support model

Reducing turnaround and improving response quality

Latest HCT releases address most critical errata

Planning regular service pack cycle focused on errata reduction

Top 36 Errata fixed

WHQL Program Improvements & ExpansionWHQL Program Improvements & Expansion

Open for x64 driver businessHCT 12.1.01 released for testing x64 drivers

WHQL is now accepting x64 submissions

x64 submissions free of charge at this time

Improved Universal Test ProcessFor devices where formal logo program does not exist

Test process is changing from full-test to self-test effective May 1, 2005

Submission fees reduced to normal self-test fees

Turnaround time is now 7 vs. 30 days

RequirementsRun & pass the universal section of HCT 12.1.01

Sign the Windows Error Reports Terms of Use agreement

Provide symbols

Introducing the “Premium" Logo Program Introducing the “Premium" Logo Program

A new qualification level for the Longhorn hardware logo program

The “Premium" program will recognize products that deliver the premium Longhorn experience

The “Standard” program equates to today’s DFW program

New level created in response to partner feedback and Microsoft improvement efforts

Need to differentiate products

Relevance of requirements to an evolving industry

Harmonize the different Microsoft logo programs

A Foundation for Future Driver Quality ProgramsA Foundation for Future Driver Quality Programs

Microsoft is producing a set of device class independent tests and tools* to verify driver quality

Drivers that successfully pass these tests will be digitally signed, regardless of logo program participation

““Premium”Premium”

Req

uire

men

ts b

uild

on

each

oth

er

““Standard”Standard”

““Quality”Quality”

Introducing Driver Quality Rating (DQR)Introducing Driver Quality Rating (DQR)

Method of measuring driver quality in the fieldDerived from mathematical probability that a given driver caused a system crash

Data is normalized to treat low and high volume drivers equally

The DQR will be a score from 1 to 9Values 1 to 3 (Red) indicates a high level of problems

Values 4 to 6 (Yellow) indicates a medium level of problems

Values 7 to 9 (Green) indicates a low level of problems

DQR for your products will be available to you on Winqual

Uses of DQRPrioritize driver maintenance work

Help OEMs update preloads for optimal user experience

Gold logo requirement

Clarify Windows Update content policy

Industry DQR Distribution TodayIndustry DQR Distribution Today

Most drivers on today’s scale fall into the green category

91% Green

5% Yellow

5% Red

Providing symbols is critical for an accurate DQR

Providing symbols will likely increase your DQR (higher is better)

Rating Percent of Drivers*

1 2.43%

2 0.75%

3 1.31%

4 1.08%

5 1.83%

6 1.53%

7 5.35%

8 12.19%

9 73.53%

*Most popular drivers from Feb 2005

Utilizing Windows Error Reports

Using Windows Error ReportsUsing Windows Error Reports

Kernel mode crash data is updated hourly on Winqual

Example of Realized ValueRelease of driver to web increased crashes by 10x in 3 days

In this example it took a number of days to catch problem

This company now uses Winqual to monitor new releases of drivers in real time

Using Windows Error Reports to AssistUsing Windows Error Reports to Assist

Through Winqual you can provide customized responses to customer who are experiencing crashing problems

Using Windows Error Reports to AssistUsing Windows Error Reports to Assist

Microsoft regularly surveys customers and will work with you to provide the best possible customer experience

Announcing Windows Error Reports 4.0Announcing Windows Error Reports 4.0

Improved UI including navigation and shrink & grow sections

New chart modeComparison against industry average

Various trending views

New result set filtering with over 20 new filtering attributes

Now search by bucket, driver, device and PnP ID

New data drill-downs by vendor, driver, and device

Improved bucket details

New CAB details with call stacks, in memory processes and system uptime

Filter CAB files prior to downloading

Available NOW on http://Winqual.Microsoft.com

Using Windows Update

Windows Update Operational FactsWindows Update Operational Facts

Total Investment $60+ million per year (HW, FTE’s, egress)

$23 million FY’04 egress bandwidth

Data Centers,

Download Servers and Egress

3 Data Centers

436 Servers

2000+ Caching Servers

24 Languages

100Gb/s peak egress capacity

WUv5 Site AvailabilityMicrosoft Domain Availability

99.85% (Keynote report 2004)

Top most available site/service (Keynote) when compared with AOL, Google

Cost Effective Distribution Service

Technological advantages

Economics of scale

Inexpensive in-house hosting

Relationships with CDNs, ISPs

Average Monthly Driver Downloads(Nov04-Mar05)

20,545,964

Customers Requesting Hardware Updates on WUCustomers Requesting Hardware Updates on WU

Consumers rank most important to update:1. Windows

2. Anti-virus

3. Hardware

4. Anti-spam

5. Office

Corporations rank most important to update:1. Outlook/Exchange

2. ISA Server

3. MBSA

4. .NET Framework

5. SQL

Request6/03Rank

12/04Rank

More hardware updates 2 1

Better information 3 2

More update types 1 3

Speed up the process 4 4

Fewer reboots 6 5

Filter updates 5 6

Misc. smaller updates, CD) 7 7

Effectiveness of Windows UpdateEffectiveness of Windows Update

Huge impact on driver quality when we partner for the fix and get the right driver on Windows Update

Audio Driver ExampleFix available for everyone in February 2004

Worked with a key OEM to put fix on WU as a critical update

0%

20%

40%

60%

80%

100%

120%

2003Nov

2003Dec

2004Jan

2004Feb

2004Mar

2004Apr

2004May

2004Jun

2004Jul

2004Aug

2004Sep

2004Oct

2004Nov

2004Dec

2005Jan

All Others OEM with WU Fix

WU Critical Fix released

Drivers on Windows Update TodayDrivers on Windows Update Today

Increased reach with WU v5 release at Windows XP SP2

Hosting 1000’s of driver packages from 100’s of companies

44 active driver packages being tracked for OCA reduction

Drivers can be posted to WU through Driver Distribution Center on Winqual

Critical Driver UpdatesPush critical OCA fixes down to customers

Currently 18 critical update packages posted for drivers

Contact whqlwu @ microsoft.com to participate

Rich Targeting on Windows UpdateRich Targeting on Windows Update

ChallengesOEM/IHV wants custom drivers on generic hardware that supports custom value-add software

Driver is specific to a premium sku of Windows (MCE, Tablet)

Driver should be critical for customers who are experiencing problems, recommended for everyone else

GoalsAllow driver packages to specify granular applicability criteria beyond PnP ID

Provide flexibility in detection mechanism without adding arbitrary code

Make it simple for driver package developers

Rich Targeting Rich Targeting

Technology basisWindows Management Instrumentation (WMI)

A component of Windows that provides management information and control in an enterprise environmentExposes Windows component properties through a consistent object-oriented interface

WMI queries are similar to (SQL) queriesAvailable fields

Win32_ComputerSystem.ManufacturerWin32_ComputerSystem.ModelWin32_BIOS.SMBIOSBIOSVersionWin32_BIOS.Name

Available operators=<> or !=LIKE

Acceptable query valuesVendor provided text strings

Rich Targeting – How It WorksRich Targeting – How It Works

USB\VID_045E&PID_008COEM: Unknown

USB\VID_045E&PID_008COEM: ComputerCo

Standard Retail driver forUSB\VID_045E&PID_008C

Custom OEM driver forUSB\VID_045E&PID_008C

Windows Windows UpdateUpdate

Announcing the Online Provisioning InitiativeAnnouncing the Online Provisioning Initiative

Microsoft plans to expand Windows Update to support all device-related software in Longhorn

What this means:Your device software product can be distributed through Windows Update, not just the device driver portion

Goals:“It Just Works” - deliver an integrated end user experience that makes devices fully functional with Windows automatically

Adjust content policies to balance customer, partner & Microsoft needs

What you need to do:Include your full product software in your WHQL submissions

Testing requirements remain unchanged, focus on the driver

Move to DIFx based setup packages

Planned Improvements to Longhorn Driver SearchPlanned Improvements to Longhorn Driver Search

Windows Update treated as an extension of inbox driver support

Online Provisioning of device related software ensures full functionality after install

If support not found customer can be routed to 3rd party website

Call To ActionCall To Action

Improve Driver Quality using Winqual / WHQLUnderstand what your top driver quality issues are

Provide symbols so we can better analyze your crashes and provide you with an accurate DQR

Distribute fixes to customers via Windows Update

Move forward with your x64 driver plans, demand is growing

Provide feedback on the "Gold" logo requirements

Will be available in the draft 0.6 version of the Windows Logo Program 3.0 requirements document

Include your full product software in your WHQL submissions

Community ResourcesCommunity Resources

Windows Hardware & Driver Central (WHDC)www.microsoft.com/whdc/default.mspx

Technical Communitieswww.microsoft.com/communities/products/default.mspx

Non-Microsoft Community Siteswww.microsoft.com/communities/related/default.mspx

Microsoft Public Newsgroupswww.microsoft.com/communities/newsgroups

Technical Chats and Webcastswww.microsoft.com/communities/chats/default.mspx

www.microsoft.com/webcasts

Microsoft Blogswww.microsoft.com/communities/blogs

Additional ResourcesAdditional Resources

Email: winqual @ microsoft.com

Web Resources:Tools: http://Winqual.Microsoft.com

Information: http://www.Microsoft.com/WHDC/

Related SessionsBusiness Value of the Windows Device Software Life Cycle

The Next Generation Logo Program An Introduction

Windows Driver Quality Signature

The WDK for Engineering Managers and Product Planners: An Introduction

DMI: Driver Install Frameworks Tools and Strategies

© 2005 Microsoft Corporation. All rights reserved.This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.