Verification Strategy for PCI-Express

Presenter: Pradip Thaker July 4th, 2008

2

Outline

PCI-Express Protocol Overview

Verification Paradigm

Design-for-Verification (Well-aligned implementation and verification architectures)

A key ingredient for a timely verification closure

3

PCI to PCI Express

Limitations of PCINot enough bandwidth

32-bit/33 MHz (132 MB/s)64-bit/66 MHz (528 MB/s)

Shared bus bandwidthNo support for Isochronous applications (TDM or Synchronous Traffic application)Cost of hardware for parallel busses

Evolution PathGrowing faster is the only possibility (not wider)Point-to-point communication (Shared bus connectivity impossible above 100/150 MHz)CDR architecture (Speed limitation of a synchronous bus above few hundred MHz) Backward compatibility – a must

Fast forward to future – PCI Express (PCIe)Packet-level data-units over high-speed SERDES based connectivityLayered architecture – much like networking protocols

Mechanical, Physical, Data-link, Transaction, Software and System LayersCompatible with existing PCI software infrastructureWeird wedding of two distinct architectural and business practices – Networking and Computer Creation of nightmarish scenario for chip verification (Details on later slides)

4

PCI-Express Protocol Overview - TerminologyDual Simplex – a related set of two differential pairs (Tx and Rx)Lane – “Dual Simplex” when PCI-Express compliantPort – A group of Txs and Rxs within a single device that represent a single connection to PCI-Express fabricLink – Two ports and the collection of lanes that interconnect themx1, x4, x8, xN – Number of lanes within a port or a link

Upstream – Flow of traffic towards the CPU or a port that establishes link in that direction within the hierarchyDownstream – Flow of traffic away from the CPU or a port that establishes a link in that direction within the hierarchyIngress Port – the portion of a PCIe port that receives the incoming trafficEgress Port – the portion of a PCIe port that transmits outgoing traffic

Root Complex – The combination of a PCIe host bridge and one or more downstream portsEndpoint – A device that terminates a path within the hierarchyBridge – A device that physically and electrically connects PCIe to another protocolSwitch – A device that provides a physical connection between two or more PCIe ports

5

PCI-Express Hierarchy

6

PCI-Express Protocol Overview : PhysicalLogical Functions

8B/10B Encoding and DecodingScramblingReset, initialization, multi-lane de-skewLane mappingAdjustments of bit-transmission order for various throughput options (x1 through x32)Logical idle behavior and transition to active state as per protocolTLP and DLLP transmission and reception: Insertion and Processing of Special Symbols per protocol conditionsLink initialization (recovery from link errors, transition from low power states)Link negotiations

WidthData-rateLane reversalPolarity inversion

Link synchronizationBit-wise per laneSymbol-wise per laneLane-to-lane de-skew

Ordered (TS and Skip) set handling and processingFast training sequenceLink power managementDelay insertions as per protocol……………………more that could not fit here

Electrical Functions Link within 600 ppm at all timesSpread spectrum clocking AC couplingInterconnect parasitic capacitance adherenceReceiver DC commong mode voltage of 0 VTransmitter DC common mode established during “Detect”Receiver Detect under various scenariosTotal jitterMaximum loss budgetDe-emphasisMaximum BERBeacon………………………………more that could not fit here

7

PCI-Express Protocol Overview : Data-link LayerLink management

DL_UP, DL_Down, DL_Inactive, DL_Active, DL_Init state transitionsSlot power limit handlingPropagation of link-reset downstream

Point-to-point reliable data exchangeError detection, re-try as well as Error Logging and ReportingPower Management message decoding, state transitions for activation and de-activationTLP sequence number generation and trackingLCRC computation and decodingDLLP integrity encoding and decodingACK/NAK generation and processingACK time-out notification and handlingFlow control computation, tracking and processing – Credit based flow-controlData poisoningCompletion Time-outRe-transmission of packetsPackage storage for re-try/replayDLLP generation, processing and actuation based on current status

ACK DLLPNAK DLLPInitiFC1InitFC2UpdateFCPower ManagementVendor specific

Cut-through routingTLP/DLLP ordering permutations per protocolTLP integrity check insertion and processingACK/NAK latency timer rules processing a limit-triggered response………………….more that could not fit here

8

PCI-Express Protocol Overview : Transaction LayerFlow control management

TL manages, DL executesPoint-to-point, not end-to-endIndependent for each VC IDMechanism presumes “Ideal” conditionsCredit types – PH, PD, NPH, NPD, CPLH, CPLD

Data transactionsTLP storage and processing for transmission or consumptionTLP generation: Header, Payload and DigestTLP generation and handling of various lengths (4 Bytes to 4096 Bytes)Transaction types

Memory (32-bit and 64-bite addressing) I/OConfigurationMessage

INTxPMEERRUnlockSlot PowerHot PlugVendor-defined

Transaction CompletionReads and non-posted writesCompletion routing is by IDProvide completion status

Transaction OrderingRouting rulesArbitration

Port arbitrationVC arbitration

Virtual channelsTraffic classesLocked transactions supportIsochronous supportAdvance error processing and reporting………………………….………more that could not fit here

9

PCI-Express Protocol Overview: SummaryOpen standard containing over 500 pagesMany more pages of supporting literature

Each line of each page in the standards document is a cryptic edict dictating a specific behavior for each condition

and not a detailed explanation about behavior or implementation

Much space for protocol detail misinterpretation resulting into mal-function or non-compliance

Hundreds of configuration bits – each controlling a complex behavior within the chip with strict adherence to standard dictate to guarantee backward software compatibility

No wiggle room to claim bug as a feature!!!

10

Verification Paradigm

Chips based on Open-Standard – Pressure PointsTechnology/Feature differentiator – Marginal or Non-existing

Commodity product – Power, Performance and PriceTime-to-market – Very Critical

First product – To Establish Credible PresenceSub-sequent products with various flavors – To Capture Market Share

Bridges: PCI-to-PCIe, SATA-to-PCIe, 1394-to-PCIe, USB-to-PCIe etc.Switches: 4-port x1 throughput, 4-port x4 throughput, 8-port x4 throughput, etc.Root Complex: x1 throughput, x4 throughput, etc.

Quality of First Silicon – Critical

Verification Plays A Major Role in Success of Chips based on Open-StandardAddresses Two Key Aspects: TTM and Quality of Silicon

Verification Execution: Focal Points FunctionalityPerformanceInteroperability (Compliance and Compatibility)

Verification Platform Architecture and Methodology: Focal PointsRe-usabilityScalability (Modularity)Comprehensiveness (with leveraging of automation)

11

Verification Strategy: A Broader Definition

Verification – A vehicle to deliver chips with “Zero Bugs(!)”, Compliance and Superior performance

Performance Modeling (C/C++/SystemC)Architecture and Micro-architecture of Key Data and Control Paths

RTL VerificationFPGA-based Emulation

Compliance and Compatibility testingPCI-SIG certification to be on Integrator’s ListPerformance verification

3rd party Compliance Checkers and VectorsMixed-signal Simulations

12

Functional Verification: Four PillarsCoverage-driven constrained-random testing with reference models (HVLs)

Reference Model (RFM)Temporal CheckersProtocol MonitorsSequence GeneratorsConstraintsFunctional CoverageTest-plan

Assertion-based verification for key building blocksDetects design errors at the source – increases observability and decreases debug-timeCan identify subtle bugs that may be hard to reach with SBVBlack-box assertions – Protocol orientedEffective for size/complexity to an extent (memory-size and run-time limitations)

Suitable for block-level deployment rather than end-to-end chip-level stand-alone verification methodComplex properties are verified through bounded-proof (neither proven nor falsified)

Effective for control-path oriented logic (state space exploration rather than data-path logic) verification Assertions when written by engineer other than designer can help detect specification (interpretation) class of errors

Asynchronous clock-domain simulations

Power-domain simulations – Power Management Compliance Check-listImproper Buffer Insertion, Missing Level Shifters, Missing Power Good, Power Sequencing Tests

13

Functional Verification: CDV (Re-usability and Scalability)

14

Functional Verification: Golden Rules for RFM

Reference Model shall be independent of the DUT implementationReference Model to be created by engineer other than designer of the blockReference Model created in high-level language and hence it does not have any low-level mechanics analogous to RTL implementation to realize functionality

Reference Model shall support co-simulation with the DUT in order to predict and verify run-time behavior

Reference Model for each block shall be created such that it can be integrated into chip-level verification environment seamlessly

Hybrid ModelingControl paths: Cycle-accurate modelingData paths: Packet-accurate or Data-unit-accurate modelingFully cycle-accurate model is maintenance nightmare as well as a cumbersome task without significant value-add to verification quality

Comprehensiveness (with leveraging of automation)CDV is only as powerful as comprehensiveness of automated checking features of reference model and monitorsCan run millions of RTG cycles with comprehensive reference model and monitors without much manual overhead

15

Performance VerificationPerformance Parameters (to be supported with variable sized packets across mixed-traffic types, across all traffic patterns, mixed VCs and mixed-packet sizes)

Aggregate ThroughputLatency (to be balanced against power dissipation)Jitter in LatencyAvailability/Blocking – Internal back-pressureN+1 Performance limitation (small TLPs back-to-back)Flow-control creditsLoad distribution and balancing (peer-to-peer as well as vertical traffic flows with mixed of traffic types, VCs and packet sizes)Link utilization – No bubbles within or between TLPs (really challenging for cut-through mode)Zero tolerance for packet lossZero tolerance for wrong packet routing

20% overhead lost in 8B/10B codingSmall TLPs with header as well as DL layer overhead impacting transaction layer efficiency even with 100% link utilization Traffic-aware flow-control credit updates (large and small TLPs)

Performance Modeling (C/C++/SystemC)Architecture and Micro-architecture of Key Data and Control Paths

FPGA-based EmulationRTL Verification – Not an adequate method for performance testing for PCIe development

16

Compliance VerificationElectrical Compliance Check-list

Signal Quality AnalysisEye pattern, jitter and BER analysisSignaling for upstream and downstream

Jitter Analysis DLLClock recoveryInterpolationTransition/non-transition eye points

Data-Link Layer Compliance Check-listReserved Fields testingNAK ResponseReplay TimerReplay CountLink RetrainReplay TLP OrderBad CRCUndefined PacketBad Sequence NumberDuplicate TLP

Transaction Layer Compliance Check-listCompletion request, completion time-out, read-dataMessaging – Legacy interrupts, Native power management, Hot-plug, Error SignalingFlow Control – Initialization, Transmit and Receive States, Negotiated Link WidthVirtual Channel

System Architecture/Platform-configuration Check-listCapability registers testingDefault valuesStress testSlot reportingHot plug event reporting

17

Compliance Verification

Separate compliance check-list with some overlap for RC, Endpoints and Switches

Integrated PHY in the silicon FPGA platforms with discrete PHY and digital logic

FPGA-based emulation (Native or 3rd Party)Compliance testing with Agilent PTC and PCI-SIG Golden SuiteCompatibility testing with over 80% of the systems during PlugFestPCI-SIG certification to be on Integrator’s List

Native protocol checkers – static and temporal3rd party Compliance Checkers and Vectors

Synopsys, Denali, nSys and others

18

Design-for-Verification

Cafeteria Architecture: Modular and Scalable For rapid deployment of various flavors of bridges and switches based on flagship platform partSpeed of Capturing market-share as critical as first product deployment to establish credible presence

Modular architecture to enable thorough block-level or sub-system level simulations

Functional partitioning to reduce scope of chip-level verification effort and complexity

Push v/s Pull Inter-block Data-threadsDistributed v/s Centralized Control Processing

Standardized block interfaceReduce scope of “Error of Specification” and “Error of Omission”

Promote verification component re-use (BFMs, Sequences, etc.)Minimum number as well as flavors of physical interconnects between blocks (may use in-band signaling where applicable)

Emphasis on correct-by-construction practices during design-creation phaseOtherwise TTM Window will be missed due to prolonged verification or multiple re-spins (PCIe non-forgiving of bugs that hamper compliance or compatibility)

19

Thank You!