vrm_user

Verification Run Manager User Guide

Software Version 6.6d

2010 Mentor Graphics CorporationAll rights reserved.

This document contains information that is proprietary to Mentor Graphics Corporation. The original recipient of thisdocument may duplicate this document in whole or in part for internal business purposes only, provided that this entirenotice appears in all copies. In duplicating any part of this document, the recipient agrees to make every reasonableeffort to prevent the unauthorized use and distribution of the proprietary information.

This document is for information and instruction purposes. Mentor Graphics reserves the right to makechanges in specifications and other information contained in this publication without prior notice, and thereader should, in all cases, consult Mentor Graphics to determine whether any changes have beenmade.

The terms and conditions governing the sale and licensing of Mentor Graphics products are set forth inwritten agreements between Mentor Graphics and its customers. No representation or other affirmationof fact contained in this publication shall be deemed to be a warranty or give rise to any liability of MentorGraphics whatsoever.

MENTOR GRAPHICS MAKES NO WARRANTY OF ANY KIND WITH REGARD TO THIS MATERIALINCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY ANDFITNESS FOR A PARTICULAR PURPOSE.

MENTOR GRAPHICS SHALL NOT BE LIABLE FOR ANY INCIDENTAL, INDIRECT, SPECIAL, ORCONSEQUENTIAL DAMAGES WHATSOEVER (INCLUDING BUT NOT LIMITED TO LOST PROFITS)ARISING OUT OF OR RELATED TO THIS PUBLICATION OR THE INFORMATION CONTAINED IN IT,EVEN IF MENTOR GRAPHICS CORPORATION HAS BEEN ADVISED OF THE POSSIBILITY OFSUCH DAMAGES.

RESTRICTED RIGHTS LEGEND 03/97

U.S. Government Restricted Rights. The SOFTWARE and documentation have been developed entirelyat private expense and are commercial computer software provided with restricted rights. Use,duplication or disclosure by the U.S. Government or a U.S. Government subcontractor is subject to therestrictions set forth in the license agreement provided with the software pursuant to DFARS 227.7202-3(a) or as set forth in subparagraph (c)(1) and (2) of the Commercial Computer Software - RestrictedRights clause at FAR 52.227-19, as applicable.

Contractor/manufacturer is:Mentor Graphics Corporation

8005 S.W. Boeckman Road, Wilsonville, Oregon 97070-7777.

TRADEMARKS: The trademarks, logos and service marks ("Marks") used herein are the property ofMentor Graphics Corporation or other third parties. No one is permitted to use these Marks without theprior written consent of Mentor Graphics or the respective third-party owner. The use herein of a third-party Mark is not an attempt to indicate Mentor Graphics as a source of a product, but is intended toindicate a product from, or associated with, a particular third party. A current list of Mentor Graphicstrademarks may be viewed at: www.mentor.com/terms_conditions/trademarks.cfm.

Verification Run Manager User Guide, v6.6d 3

Table of Contents

Chapter 1Verification Run Manager Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

Verification Run Manager Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17Use Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

General Terms. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20XML Terms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22Typographical Conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

Chapter 2Basic Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

Invoking VRM. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25Modes of Operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

Execution Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26Control/Abort Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26Status Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26Notification Mode. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

vrun Command. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27vrun Syntax. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27Execution Mode Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28Status Mode Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34Control/Abort Mode Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36Notification Mode Options. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36Help Options. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38Verbose Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

Message Verbosity and Debugging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39RMDB Configuration Database File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

Comments in the RMDB File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40Embedding Options in the RMDB File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41Dry Run of the RMDB Configuration Database. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42Locating the RMDB Configuration Database. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43Validation of the RMDB Configuration Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43Split RMDB Configuration Database Among Several Files . . . . . . . . . . . . . . . . . . . . . . . 43Set the Search Path for Shell Scripts. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45Locating Script and Log Files. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

Status Reporting. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47Status Report for Latest Regression Run . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

Status Column Values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49Predefined Status Report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50Custom Status Report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50Follow Status Events as They Occur . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

HTML Status Report. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

Table of Contents

4 Verification Run Manager User Guide, v6.6d

Enabling HTML Reporting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51Determining What to Report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52HTML Report Content . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53Controlling the Format and Content of the HTML Report . . . . . . . . . . . . . . . . . . . . . . . 57

Managing the SCRATCH Directory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58Manage a Questa work Directory and HDL Sources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59Implement a Macro by Calling VRM Recursively . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

Limit Concurrently Running Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62Override Repeat Counts on Runnable Nodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63Integration with Compute Grid Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

Launch an Action Onto a Grid System. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64Interaction with a Job Once Launched . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

Finding the JobId . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65Reporting on Grid-job Status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65Killing a Queued Job . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

Specify Selected Runnables. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66Showing which Runnables are Selected . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68Selecting Instances of Repeating Runnables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68Wildcards and Repeating Runnables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69Selecting Runnables Based on Results of Previous Run . . . . . . . . . . . . . . . . . . . . . . . . . . 70Context Chains . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

Including and Excluding Context Subtrees . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73Rooted Context Chains . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74Globbing in Context Chains . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75Using Regular Expressions in Context Chains . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75Adhoc Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

Batch Mode Usability Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77View RMDB Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77

Show Actions that will Run . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78Build a Runlist of Runnables to Execute . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79Executing the Selected Runnables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80Selecting Values for Embedded Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81

String Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83Numeric Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83Enumerated Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84Interaction with Parameter Prompts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

Use Model Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86Example of front-end Runnable Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86Combined Selection and Prompting Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

Override Parameter Values from Command Line . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88Limiting Parameter Overrides to a Specific Context . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89Context Partial-Match Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92

Access RMDB API via the vrun Command Line . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94

Chapter 3Graphical User Interface (GUI) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95

GUI Environment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95VRM to/from GUI User. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105

Table of Contents


GUI Mode, vrun Launched from GUI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105GUI Mode, GUI Launched from vrun . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107Attach a Separately Launched GUI to a Running vrun Process . . . . . . . . . . . . . . . . . . . . . . 107

Format of Messages from vrun Core to Runtime GUI . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108Format of Messages from Runtime GUI to vrun Core . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112

Create a RMDB File from Existing UCDB File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113Action Scripts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113

Compilation and Post-processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115

Chapter 4RMDB Configuration Database Topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117

RMDB Configuration Database Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117Database Content and Naming Conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118Database Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119

Runnables and the Regression Suite Hierarchy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120Execution Method Elements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121User-defined (usertcl) TCL Code Fragments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121

Execution Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123Actions and Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124Repeating a Task or a Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125

Controlling the Repeat Index (foreach Functionality) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126Dynamically Repeating Runnables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129

Use Model. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129Activating the Feature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129Controlling the Feature. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131Lexical Requirements for foreach Attribute Values and Tokens . . . . . . . . . . . . . . . . . . . 132GetNextIteration Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133Concurrency and Multiple Repeating Runnable Instances . . . . . . . . . . . . . . . . . . . . . . . 135

Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136Selective Membership in a Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139Basic Regression Execution Algorithm. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140

Run (and Rerun) Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142Notification of Regression Start and/or Completion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142Failed Test Rerun . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142

Job Progress Monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142

Chapter 5Execution Graph and Advanced Topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145

Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145Execution Graph Expansion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145

Execution Graph Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147Restrictions on Group Membership . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148

Handling errors from Action scripts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148

Chapter 6Parameters and Parameter Expansion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151

Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151Predefined Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154

Table of Contents


Including TCL Code in a Parameter Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156Referring to Source Files Via a Parameter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157Nested Parameter Expansion and TCL Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158

Inheritance, Overrides, and Search Order . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159Parameter Inheritance Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166Runnable Types and Inheritance. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168Search Order When Expanding a Parameter Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . 168

Inheritance Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168Parameter Expansion Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169

Parameterizing the Regression Suite for Debug . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170Exporting Parameters to Action Scripts. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171

Chapter 7Working Directory and Script Execution. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173

Working Directory Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173Local File Generation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174

Code Examples. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177Disposition of Test Collateral . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178File Safety . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178

Script Execution and Timeouts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178Action and Wrapper Script Generation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179

vsim Mode Wrapper . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180exec Mode Wrapper . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182

Using Existing Scripts with VRM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183Passing Values Via Position-dependent Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183Passing Values Via Environment Variables. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184Copying the Script File to the Working Directory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184Running Scripts in Existing Directory Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185

Script Return Values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186Modifying vsim Mode User Scripts to Return Status . . . . . . . . . . . . . . . . . . . . . . . . . . . 187Modifying exec Mode User Scripts to Return Status. . . . . . . . . . . . . . . . . . . . . . . . . . . . 188Execution Mode Simulations Hang . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189

Pipe-based Command Launching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189Sending User-defined Messages from an Action Script . . . . . . . . . . . . . . . . . . . . . . . . . . . 190Timeouts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191

Execution Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193Selecting an Execution Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195Conditional Execution Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195Named Execution Queues. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199Methods and Base Inheritance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201

Recommended Execution Method Configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204Execution Method Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208

Timing the Execution of Each Action . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208Queuing Actions to Run on a Server Grid . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208

Methods and Timeouts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210Clubbing Multiple Actions in a Single Grid Job . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210

Multi-Action Clubbing Fixed-list Mode. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211Multi-Action Grid Jobs and Concurrency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213

Table of Contents


Multi-Action Grid Jobs and Local Rerun. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213Multi-Action Grid Jobs and maxrunning Execution Limits. . . . . . . . . . . . . . . . . . . . . . . 213Multi-Action Grid Jobs and Job Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213

Limiting the Execution Time of a Given Named Queue . . . . . . . . . . . . . . . . . . . . . . . . . . 214Failure Rerun Functionality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215

Local (Immediate) Rerun . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215Action Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216Default Behavior . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216OkToRerun Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217Possible Use Models. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218

Global (Command-line Based) Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219Logging Failed Actions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219Replaying Failed Actions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220

Global (Parameter-Based) Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221Semi-automatic (Two-command) Mode. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222

Automating Failure Rerun With a Macro Runnable . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223Fully Automatic (One-command) Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224

Reuse of Random Seeds in Automated Rerun . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225

Chapter 8Post-execution Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227

Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227Pass/Fail Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229

Failure Propagation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230Determining Pass/Fail for a Given Action . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232ActionCompleted . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234OkToMerge. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234MergeOneUcdb. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235OkToDelete. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235StopRunning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237StopPropagating . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237

Coverage Merge Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 238Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 238Automated Merging Example. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239Post-execution Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240

Coverage Merging Use Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241Do-It-Yourself (DIY) Merging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241

Assumptions and Detailed Behavior . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241DIY Merge Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242

List-Based Merging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245Assumptions and Detailed Behavior . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245List-Based Merge Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246

Incremental Merging. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247Assumptions and Detailed Behavior . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247Incremental Merge Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 248

Queued Merging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249Assumptions and Detailed Behavior . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249

Queued Merging Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251

Table of Contents


Multi-level Merging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252Pass/Fail Analysis and Auto-Merging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252

Auto-Merge Tips and Traps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252Auto-Merging and Delete-on-Pass . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252Sorting Coverage Information Based on Test Status . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252

Automatic Deletion of Merge/Triage Files. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 254Testplan Auto-Import . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255Enabling and/or Disabling Coverage Merge Flows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256Automated Results Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258UCDB File "clubbing" in Automated Merge and Automated Triage. . . . . . . . . . . . . . . . . 260

Multi-level Auto-Merge Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261Auto-Merge Example Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 262Auto-Merge Example RMDB Configuration Database . . . . . . . . . . . . . . . . . . . . . . . . . . . 262First-Level Merge in Detail . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264Avoiding Problems with Inheritance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265Second-Level Merge in Detail . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267Complete Multi-level Auto-merge Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267

User-definable Procedures and Loading of Arbitrary TCL Code . . . . . . . . . . . . . . . . . . . . . 270Modifying VRM Behavior with User-definable Procedures . . . . . . . . . . . . . . . . . . . . . . . 271Loading User-Supplied TCL Code Using usertcl into VRM . . . . . . . . . . . . . . . . . . . . . . . 273

Loading Conditional User-defined TCL Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277Collating Passed/Failed UCDB Files for Merge and/or Triage . . . . . . . . . . . . . . . . . . . . . . . 278

Basic Regression Suite Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279Fully Automated Merge and Triage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281Using the Event Log to Gather UCDB Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282Override OkToMerge and Rely on Auto-merge. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283Maintain Separate Lists of UCDBs Internal to VRM. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 284

Chapter 9Communication Channels, Event Log, and Status Report . . . . . . . . . . . . . . . . . . . . . . . . . 287

Action Script to VRM. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287Unattended Batch Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287Communication Channels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 288

Basic Communication Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289Format of Messages from Action Script to vrun Core . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289Error Response . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 290

Event Logging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 290Attach to a Running vrun Process via Event Log . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292Send Status Message from Wrapper/User Script. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 294

Chapter 10Controlling Grid Jobs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 295

Specifying the Type of Grid System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 296Specifying the Job Control Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297

Locally Launched Jobs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 298Predefined User-definable Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299

Controlling Jobs Remotely . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299Command Line Control from a Second vrun Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301

Table of Contents


Adjust Timeout Values to Account for Time Suspended . . . . . . . . . . . . . . . . . . . . . . . . . . 302Unsupported Environments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302

Running with JobSpy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303

Chapter 11Environment Variables and Built-in Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305

Environment Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305Environment Issues. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 306

Built-in Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307Parameters Recognized/Defined by VRM. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 308Parameterized Element Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 309Relative File Rules. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 310

Chapter 12Catalog of User-Definable and Utility Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313

Catalog Format. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313User Definable Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 314

Common Arguments Passed to All User Definable Procedures. . . . . . . . . . . . . . . . . . . . . 314Global Status Change Procedures. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 314

RegressionStarting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315RegressionCompleted. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315ProcessIdleTasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 316

Action Status Change Procedures. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 316ActionEligible. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 317ActionLaunched . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 317ActionStarted . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 318ActionCompleted . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 319

Post-execution Processing Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 320AnalyzePassFail . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 321OkToRerun . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 322OkToMerge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323MergeOneUcdb . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 324OkToTriage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 326TriageOneUcdb . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 327OkToDelete . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 328

Regression Control Procedures. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 330SelectRunnables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 330StopPropagating . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 332StopRunning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333

Grid Management Procedures. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333LsfGetJobId . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 334LsfGetStderr . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335LsfKillJob . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 336SgeGetJobId . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 336SgeGetStderr. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 337SgeKillJob . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 338

Utility Procedure Catalog . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 339System-level Utilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 339

Table of Contents


VerbosityLevel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 339RightNow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 340

Path-conversion Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 340PlatformPath . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 340PathRelativeToVrun . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 341PathRelativeToRmdb . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 341PathRelativeToQuesta . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 342PathRelativeToScratch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 342RelativePath . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 343ActionToLogFile . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 343ActionToLogFiles. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 344

File and Directory Manipulation Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 344VerifySafeTarget . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 345MakeDirectory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 346DeleteDirectoryContents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 346DeleteDirectory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 347WriteToFile . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 347AppendToFile. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 348CopyFile . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 348LinkToFile . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 349GetFileLines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 349GetFileContents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 350DoNotDelete. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 350

Parameter Expansion Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 351OverrideRmdbParameter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 352ExpandRmdbParameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 352

UCDB Access and Analysis Procedures. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353GetUcdbTestAttribute . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353GetUcdbSummaryAttribute . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 354GetUcdbSeed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 354GetUcdbStatus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 355ConvertUcdbStatus. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 355TranslateUcdbStatus. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 356

Event Log Access Procedures. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 356FetchEventRecords. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 357FetchEventActions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 358FetchEventUcdbs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 358

Appendix AVRM Configuration Database (RMDB) API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 359

RMDB API Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 359Types of Elements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 359Data and Meta-data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361Definitions of Variables Used in Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361Command Shortcuts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 363

Special Case for method Access. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 363Attributes Per Element Type. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 364

Document (root) Elements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 364

Table of Contents


runnable Elements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 364parameter Elements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 365

action Elements (preScript, execScript, and postScript). . . . . . . . . . . . . . . . . . . . . . . . . . . 365localfile Elements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 366method Elements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 366usertcl Element . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 367

Element Attribute Default Values. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 367Parameter and Script Inheritance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 368Document Type Definition and Validation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 368Including Nested XML Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 369

Global API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 370Create, Open, Save, and Close Database Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 370Information Regarding Database Status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 371Control of Per-Command Debug Tracing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 372Information Regarding Enumerated Attribute Choices and Defaults. . . . . . . . . . . . . . . . . 372

Read access API. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 373Database and/or document Element . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 373runnable Elements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 373parameter Elements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 374action Script Elements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 374localfile Elements Defined Under runnable Elements . . . . . . . . . . . . . . . . . . . . . . . . . . . . 375method Elements Defined under runnable Elements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 375method Elements Defined under the document (rmdb) Element . . . . . . . . . . . . . . . . . . . . 375usertcl Elements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 375

Write Access API. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 376Modify the document Element . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 376Add and/or Modify Runnables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 377Add and/or Modify Parameters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 377Add and/or Modify Actions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 378Add and/or Modify Top-level method Elements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 379Add and/or Modify localfile Elements within Runnables . . . . . . . . . . . . . . . . . . . . . . . . . 379Add and/or Modify method Elements within Runnables . . . . . . . . . . . . . . . . . . . . . . . . . . 380Add and/or Modify usertcl Elements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 381

Cached Access API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 381Opening and Closing Cached RMDB Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 381Additional Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 382Enhanced Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 383

End-User License Agreement


List of Examples

Example 2-1. RMDB File Example. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40Example 2-2. XInclude Allows Database Contents Divided Among Multiple Files . . . . . . 44Example 2-3. Predefined SCRATCH and CONFDIR Directories in a Database . . . . . . . . . 60Example 2-4. Enum and Foreach Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85Example 8-1. Automated Merging Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239Example 8-2. DIY Merge Invoking vcover merge After Each Simulation . . . . . . . . . . . . . . 242Example 8-3. DIY Merge Adding postScript to Top-Level Group Runnable. . . . . . . . . . . . 243Example 8-4. DIY Merge User-Defined TCL Override Procedure. . . . . . . . . . . . . . . . . . . . 244Example 8-5. List-Based Merge postScript Action at end of Simulation . . . . . . . . . . . . . . . 246Example 8-6. Incremental Merge with mergefile Parameter. . . . . . . . . . . . . . . . . . . . . . . . . 248Example 8-7. Multi-level Auto-merge. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267Example 8-8. Cancel Regression Run Code Fragment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272Example 8-9. Delete-om-pass Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272Example 8-10. Basic Regression Suite . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279Example 9-1. Event Log File Sample . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291Example 9-2. Incomplete Event Log File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292Example 10-1. LSF Grid Management System Regression Sample . . . . . . . . . . . . . . . . . . . 296Example 11-1. Hard-code Questa or 3rd Party Tool Release Path in Script . . . . . . . . . . . . . 307


List of Figures

Figure 1-1. Verification Run Manager Flow Diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18Figure 2-1. VRM HTML Status Report. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54Figure 2-2. VRM HTML Status Report Linked Script Log . . . . . . . . . . . . . . . . . . . . . . . . . 56Figure 2-3. VRM HTML Status Report Linked Script UCDB Test Data Record. . . . . . . . . 57Figure 2-4. Scratch Directory Tree Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58Figure 3-1. Invoke Run Manager Using QuestaSim GUI . . . . . . . . . . . . . . . . . . . . . . . . . . . 95Figure 3-2. Verification Run Manager Window . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96Figure 3-3. Run Manager. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97Figure 3-4. Create New Regression Window . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97Figure 3-5. Run Manager Database Files Window . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98Figure 3-6. .rmdb File in the Create New Regression Window. . . . . . . . . . . . . . . . . . . . . . . 98Figure 3-7. Runnable "nightly" . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99Figure 3-8. Selecting Groups to be Run. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100Figure 3-9. Set Display Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100Figure 3-10. Vrun Run Regression Window . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101Figure 3-11. Verification Run Manager Pane with Regression Results . . . . . . . . . . . . . . . . 101Figure 3-12. Transcript Pane Regression Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102Figure 3-13. View Log Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103Figure 3-14. execScript.log File. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104Figure 3-15. UCDB File in Browser . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104Figure 3-16. vrun Launched from Existing GUI Window. . . . . . . . . . . . . . . . . . . . . . . . . . . 106Figure 3-17. GUI Launched from Within vrun . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107Figure 3-18. Connect to Existing vrun from the GUI Window . . . . . . . . . . . . . . . . . . . . . . . 108Figure 4-1. VRM Execution Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123Figure 5-1. Grouping of Tasks and Groups Diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147Figure 6-1. Parameter Inheritance Diagram. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167Figure 8-1. Post-execution Analysis Flow. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228Figure 8-2. Results Analysis (Triage) Flow. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258Figure 8-3. Topology of the Example RMDB Configuration Database File. . . . . . . . . . . . . 263Figure 9-1. Batch Regression Run. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 288


List of Tables

Table 2-1. Modes of Operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25Table 2-2. Status Report Output Columns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48Table 2-3. Status Column Possible Values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49Table 2-4. Status Report Predefined Columns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50Table 2-5. Status Keywords . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71Table 2-6. accept Attribute Values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82Table 2-7. Numeric Parameter Limit Strings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83Table 3-1. Keywords Describing Event . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108Table 3-2. Contents of Status Field . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110Table 3-3. Complete List of Status Event Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111Table 3-4. Messages Sent by RunTime GUI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112Table 3-5. Generate Vrun Config Menu Picks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113Table 4-1. XML Internal Entities for Meta-characters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118Table 4-2. GetNextIteration Input Arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134Table 6-1. Predefined Parameters for Expansion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154Table 6-2. Parameters for Expansion of if, unless, repeat, foreach . . . . . . . . . . . . . . . . . . . 154Table 6-3. Additional Parameters (Action script) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155Table 6-4. Execution Method Parameter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155Table 6-5. Configuration Database Data Item Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166Table 7-1. Convenience Attributes for the method Element . . . . . . . . . . . . . . . . . . . . . . . . 196Table 7-2. Status Test Keyword Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222Table 7-3. Composite Keywords Recognized . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222Table 8-1. Possible UCDB Return Values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233Table 8-2. Coverage Merge Flow Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256Table 8-3. Automated Results Analysis (Triage) Modes . . . . . . . . . . . . . . . . . . . . . . . . . . . 259Table 8-4. Merges for Avoiding Inheritance Problems in Example . . . . . . . . . . . . . . . . . . . 266Table 8-5. Methods of Collating Passed/Failed UCDB Files . . . . . . . . . . . . . . . . . . . . . . . . 279Table 9-1. Event Field Strings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289Table 10-1. Post-launch Control Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 295Table 10-2. User-definable Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297Table 10-3. Signals Sent Via the Kill Command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 298Table 10-4. Predefined User-definable Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299Table 10-5. vrun Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299Table 10-6. Regression Run Control Command Options . . . . . . . . . . . . . . . . . . . . . . . . . . . 301Table 11-1. VRM Environment Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305Table 11-2. Built-in Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307Table 11-3. Parameters with Special Treatment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 308Table 11-4. Parameterized Element Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 309Table 11-5. File/Pathname Specification Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 310Table 12-1. Common Key/Value Pairs Arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 314

List of Tables


Table 12-2. ActionEligible Input Arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 317Table 12-3. ActionLaunched Input Arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 317Table 12-4. ActionStarted Input Arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 318Table 12-5. ActionCompleted Input Arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 319Table 12-6. passfail Keywords . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 320Table 12-7. AnalyzePassFail Input Arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 321Table 12-8. OkToRerun Input Arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 322Table 12-9. OkToMerge Input Arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323Table 12-10. MergeOneUcdb Input Arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 324Table 12-11. Arguments Passed in as Data Elements set by OkToMerge . . . . . . . . . . . . . . 325Table 12-12. OkToTriage Input Arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 326Table 12-13. TriageOneUcdb Input Arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 327Table 12-14. Arguments Passed as Data Elements set by OkToTriage . . . . . . . . . . . . . . . . 327Table 12-15. OkToDelete Input Arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 328Table 12-16. Keywords for passfail Element . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 329Table 12-17. Element Members for SelectRunnables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 330Table 12-18. SelectRunnables Input Arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 331Table 12-19. StopPropagating Input Arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 332Table 12-20. StopRunning Input Arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333Table 12-21. Postfix String and Behavior . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 334Table 12-22. LsfGetJobId Input Arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 334Table 12-23. LsfGetStderr Input Arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335Table 12-24. LsfKillJob Input Arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 336Table 12-25. SgeGetJobId Input Arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 336Table 12-26. SgeGetStderr Input Arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 337Table 12-27. SgeKillJob Input Arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 338Table 12-28. VerbosityLevel Arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 339Table 12-29. PlatformPath Input Arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 341Table 12-30. PathRelativeToVrun Input Arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 341Table 12-31. PathRelativeToRmdb Input Arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 341Table 12-32. PathRelativeToQuesta Input Arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 342Table 12-33. PathRelativeToScratch Input Arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . 342Table 12-34. RelativePath Input Arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 343Table 12-35. ActionToLogFile Input Arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 343Table 12-36. ActionToLogFiles Input Arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 344Table 12-37. VerifySafeTarget Input Arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 345Table 12-38. MakeDirectory Input Arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 346Table 12-39. DeleteDirectoryContents Input Arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . 346Table 12-40. DeleteDirectory Input Arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 347Table 12-41. WriteToFile Input Arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 347Table 12-42. AppendToFile Input Arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 348Table 12-43. CopyFile Input Arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 348Table 12-44. LinkToFile Input Arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 349Table 12-45. GetFileLines Input Arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 349Table 12-46. GetFileContents Input Arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 350

List of Tables


Table 12-47. DoNotDelete Input Arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 351Table 12-48. OverrideRmdbParameter Input Arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . 352Table 12-49. ExpandRmdbParameters Input Arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . 352Table 12-50. GetUcdbTestAttribute Input Arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353Table 12-51. GetUcdbTestAttribute Input Arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 354Table 12-52. GetUcdbSeed Input Arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 354Table 12-53. GetUcdbStatus Input Arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 355Table 12-54. Strings Returned for GetUcdbStatus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 355Table 12-55. ConvertUcdbStatus Input Arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 356Table 12-56. TranslateUcdbStatus Input Arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 356Table 12-57. FetchEventRecords Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 357Table 12-58. FetchEventRecords -return Selections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 357Table A-1. Variables Used in Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361Table A-2. Keys for rmdb method Command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 363Table A-3. Document Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 364Table A-4. runnable Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 364Table A-5. Parameter Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 365Table A-6. action Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 365Table A-7. localfile Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 366Table A-8. method Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 366Table A-9. usertcl Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 367Table A-10. Enumerated Type Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 368Table A-11. Options for dbName Argument . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 382


Chapter 1Verification Run Manager Overview

Verification Run Manager IntroductionVerification Run Manager (VRM) provides a mechanism for the user to define an arbitrarynumber of regression tasks, execute the entire set of tasks, or any subset of tasks with a singlecommand. VRM serves as the front-end to a more extensive "Regression Management"functionality within Questa, including Results Analysis (triage) and Trend Analysis tools toanalyze the results of the user's regression suite over time.

VRM features enables the user to do the following:

Organize, optimize, and manage regressions.

Automate the regression process.

Manage the simulation regression results efficiently, which enhances debugproductivity.

Transform and sort regression results in a way that aids in fast bug identification.

Manage different kinds of compute job. For example, 0-In Formal runs, QuestaSimsimulation runs, or runs of other executables.

VRM can be invoked from the command line, from the Questa GUI, or by an automated scriptas part of a scheduled regression run. Once invoked, VRM consults a configuration databaseand initiates one or more "jobs" (also called "Actions"). These Actions can be handled asfollows:

Execute as background tasks on the local machine.

Exported to a specific server.

Queued for execution on a grid system.The resulting jobs are monitored in order to satisfy any inter-dependence requirements. When ajob is completed, notification can be passed to one or more downstream activities (that is, triage,trending, loading results into a running test browser, etc.).

VRM contains many built-in short-cut Questa hooks to address the conflict between generalityand ease-of-use for Questa users. For example, UCDB files can be automatically collected andmerged without the user needing to explicitly specify complex merge scripts, based on thepresence of a specially recognized parameter.

Verification Run Manager User Guide, v6.6d18

Verification Run Manager OverviewUse Models

There are conventions for Questa users that make it possible for Questa to perform automatedpost-execution tasks. (See Overview on page 227).

Figure 1-1 illustrates the Verification Run Manager flow.

Figure 1-1. Verification Run Manager Flow Diagram

Use ModelsFollowing are some use models for VRM:

Nightly regressions

A project-wide configuration file defines a top-level test suite, possibly consisting ofsmaller subgroups of tests. The run is kicked off in batch mode, unattended. Jobs arefarmed out to a grid manager.

Verification Run Manager OverviewDefinitions


Desktop run of user's tests

A user-maintained configuration file consists of individual tests grouped into one ormore suites. The user kicks off test runs manually. The tests run in batch mode(foreground or background). The user generally waits for the results.

Rerun of failing tests after the source code fix

A group of tests are select from a list of failed tests, often from a GUI browser. Testdefinitions are drawn from the project-wide configuration file. A temporary suite isgenerated to define the list of tests to run. The tests run interactively. The user waits forthe results.

Automated rerun in debug mode

The failing tests from a nightly run can be automatically rerun in debug mode in order togather additional debug information (for example, WLF file with logging of manysignals enabled) that can help trace the failures.

Smoke test prior to checkin

A suite of tests or short test suites are preselected (from the project-wide configurationfile) for maximum short/broad coverage and added to a short-broad suite. These sametests generally fall into larger suites within the project-wide configuration file. Theshort-broad run is kicked off by the user manually. Each test runs in the background (orfarmed out to a grid manager). The user is notified when all the tests are done.

Search for high-coverage random seeds

A random test can be run a fixed number of times (possibly overnight). At the end of therun, coverage from each test can be ranked and the most promising seeds added to thedatabase for regular execution.

DefinitionsThe following terms are used throughout this manual to refer to specific objects with the VRMenvironment.

ActionA single atomic job to be executed. There are three types of Action: preScript, execScript, andpostScript.

GroupA nonleaf node in the regression suite representing a group of Task or nested Group nodes.

MacroA macro consists of the name of a parameter surrounded by parentheses and percent signs. Forexample, (%name%).



Merge FileA UCDB file that contains the merged results of one or more regression tests and possibly oneor more test plans.

ParameterA name/value pair owned by a Runnable and defined in the RMDB configuration database oron the command line. A parameter is nothing more than a name/value pair that allows multipleTasks to share the same execution script (or multiple Groups to share the same Task) byproviding unique per-Group or per-Task values to be used in the scripts.

Parameter ReferenceA unique combination of characters (in this case, "(%somename%)") used to parameterize ascript command or a parameter value.

Regression TestOne atomic simulation run represented by a Task Runnable in the configuration database andresulting in, at most, one UCDB coverage file.

RunnableA generic term used to refer to nodes in the regression hierarchy tree. A runnable element in theconfiguration database that can represent a Task or a Group.

TaskA leaf-level node in the regression suite, typically associated with a simulation run.

Target Merge FileA merge file that has been designated as the UCDB file into which the coverage results of agiven regression test are to be merged.

Test-level UCDB FileThis term refers to a UCDB file generated from a single regression test. It contains a singletestdata record for that test and no test plan.

General TermsCalling Chain

The nesting history leading to the execution of a given Action (see Context Chains on page 71for more details).



Configuration DatabaseOne or more files containing all the information needed to execute a complete regression suite.The RMDB file is the VRM configuration database.

The database is implemented as a single XML file, although this XML file can refer to otheruser-supplied files. The content of the XML database is described in the VRM ConfigurationDatabase (RMDB) API Appendix, starting on page 359.

DTDDocument Type Definition.

Results AnalysisResults Analysis is a Mentor Graphics product. The command and the database that is createdare called triage. This means that when referring to the process, Results Analysis is used; whenusing commands, it refers to creating a triage database for results analysis (see AutomatedResults Analysis on page 258).

Merge FileA UCDB file that contains the merged results of one or more regression tests and possibly oneor more test plans.

Regression TestOne atomic simulation run represented by a Task Runnable in the configuration database andresulting in, at most, one UCDB coverage file.

RMDBVerification Run Manager DataBase (RMDB) file (see Example 2-1 on page 40). The RMDBfile is the VRM configuration database.

Scratch DirectoryA directory tree created and maintained by VRM in which the various Actions that make up aregression suite are executed. The scratch directory consists of a tree of working directoriesarranged in a manner identical to the hierarchical grouping of the Runnable items in theconfiguration database (see Figure 2-4 on page 58).

Target Merge FileA merge file that has been designated as the UCDB file into which the coverage results of agiven regression test are to be merged.

Test-level UCDB FileThis term refers to a UCDB file generated from a single regression test. It contains a single testdata record for that test and no test plan.



TriageTriage is the command name and database that is created for results analysis (see ResultsAnalysis on page 21 and Automated Results Analysis on page 258 for additionalinformation).

UCDBUnified Coverage DataBase. UCDB API is an application programming interface for theUnified Coverage Database included in the Questa SV/AFV and ModelSim SE products.TheUCDB and its API are completely independent of Questa and ModelSim, however UCDBs areeasily created with these tools.

Working DirectoryA directory created and maintained by VRM and corresponding to a particular Task or Group ofTasks in a regression run. All Actions are executed in a working directory in order to preventcollateral files from multiple VRM Actions (or multiple regression runs) from colliding.

XML TermsThe following terms are used to reference parts of the XML database format. The terms andtheir definitions are inherited from the XML specification.

AttributeA piece of meta-data attached to an element to describe more about the element and/or itsbehavior.

ElementA semantic object in the configuration database. An element can contain other nested elements,user data, or meta-data (information about the element itself). Each element has a unique name,based on the tag used to mark the boundaries of that element in the database file format.

TagA string representing the start or the end (in some cases both) of a semantic object (called anelement) in an XML file.

Typographical ConventionsCertain typographical conventions are used to indicate when a word is being used for its VRMspecific definition or simply as an English word. In general, the following rules apply:



When referring to a node in the configuration database or in the execution graph that theVRM uses to schedule jobs (that is, Runnable, Group, Task, Action, etc.), the word usedis capitalized.

Words used as attribute values, parameter names, or parameter values (for example, the"fred" Group) are enclosed in double quotes (" ").


Chapter 2Basic Operations

Invoking VRMVerification Run Manager (VRM) can be invoked stand-alone (from a command shell) or fromwithin vsim using the vrun command (see vrun Command on page 27).

When run stand-alone, vish is used as a shell to execute VRMs TCL code. The general form ofthe command is as follows:

vrun []

The are context chains (in execution mode) or completed Action paths (innotification mode). These objects are supplied on the command line as bareword arguments(that is, not preceded by an -option flag). The objects can also be referred to as non-optionarguments.

In addition, VRM has a GUI that can be invoked from the Graphical Interface as follows:

View -> Verification Management -> Run Manager

Refer to Graphical User Interface (GUI) Chapter starting on page 95 for information.

Modes of OperationVRM has several major modes of operation: execution, control/abort, status, and notification(see Table 2-1). The mode is determined by the presence of command-line options. Thesemodes are described in the subsections below.

Table 2-1. Modes of OperationMode Main Option DescriptionExecution -run (default) Execute the Runnable items (tests/groups)

specified on the command line.Control/Abort -kill Abort a currently running vrun session.Status -status Generate status report on a previous or current

vrun session.Notification -send Inform VRM daemon of the completion of a

scheduled Action/job.


Basic OperationsModes of Operation

Execution ModeExecution mode is used to launch one or more regression Actions, as define in the VerificationRun Manager DataBase (RMDB) file. The RMDB file is the VRM configuration database. TheRMDB file generally contains data for one or more regression test suites arranged in ahierarchical tree topology, which allows the user to group tests and other tasks in various ways.

Execution mode is the default mode unless the -send option (notification mode, see page 26) isspecified.

Refer to the vrun Command on page 27 for information on the execution mode options.

Control/Abort ModeThe control/abort mode allows the user to abort a currently running vrun process. The -killoption causes vrun to attempt to connect to the running process via its TCP/IP control port. Ifsuccessful, it sends the listening process the kill command, which causes the regression run toterminate.

The port@host setting is derived from the event log of the running process; therefore, theSCRATCH directory setting for the sending and listening process must be the same.

Refer to vrun Command on page 27 for information on the control/abort mode options.

Status ModeStatus mode causes vrun to emit a status report on a previously completed or currently runningregression run. The -status option causes vrun to read one or more event log(s), previouslysaved to the logs subdirectory of the designated SCRATCH directory, and format a status reporton stdout summarizing the status of each Action executed in that SCRATCH directory.

By default, only the status of the immediately previous (or current) run is shown, although acumulative report of all activity in the SCRATCH directory is also available.

Refer to the vrun Command on page 27 for information on the status mode options.

Notification ModeNotification mode is used to communicate changes in job status to a master VRM process, andnotify an existing VRM instance that a specific Action has completed. The user generally doesnot need to deal with this mode, as the notification mode command is called from the samewrapper script used to launch the job in question.Notification mode is enabled by the -send command-line option that must be supplied one of theevent values: start, user, done.

Status Modevrun Command


Refer to the vrun Command on page 27 for information on the notification mode.Status Mode

vrun CommandThe general rule for processing command-line options is that all options are read and recordedbefore processing begins. In the case of options with the same semantic (that is, -loadtcl, -G,etc.), the semantic of the option can impose an order-based priority on the individual optionvalues. Between options of different types and/or bareword arguments, there is no inherentorder-dependency.

VRM command-line options can be abbreviated as long as enough of the option name isspecified to unambiguously identify a single option keyword.

vrun Syntaxvrun [-run] [] [ [...]]

Execute named runnables (context chains)[-actions] [-adhoc] [-all] [-apicmd ] [-ask][-clean] [-clear] [-config ][-dump] [-dumpgraph] [-dumpgraphviz][-exclude ][-faillog ][-g=] [-G=][-html] [-htmldir ][-include ][-j ][-loadtcl ] [-loadtcl @][-mintimeout ][-noautoclean] [-noautodelete] [-noautomerge] [-noautomergedelete][-noautotriage] [-noautotriageclean] [-nodtdvalidate] [-noexec][-noqueuemerge] [-noqueuetriage] [-noreuseseeds] [-notimeout][-nolocalrerun][-pipelaunch][-R=] [-regex][-rerun ] [-run] [-runlist ][-select ] [-scratch ][-show] [-showcmds] [-showparams][-tasklist ]

vrun -status []Display cumulative status of regressions executed in SCRATCH directory[-all][-columns ][-full]



[-hierarchical] [-host] [-html] [-htmldir ][-notruncate][-scratch ][-tail] [-times]

vrun (-pause | -suspend | -resume | -kill | -exit) [ [ [...]]]Control ongoing (remote) regression run[-dest ][-exit][-kill []][-pause][-resume []][-scratch ][-suspend []]

vrun -send [] [ [...]]Send predefined event (start, user, or done) to a listening vrun process[-code ][-dest ][-message ]

Help options:Emit the designated information and immediately exit[-help] [-sample]

Verbosity options (common to all modes):[-debug][-quiet][-verbose]

ArgumentsNote that all arguments are optional unless specified as required.

Execution Mode Optionsvrun [-run] [] [ [...]]

Execute named runnables (context chains). -actions

Shows Actions instead of Runnables (requires -show). See Show Actions that will Run onpage 78 for additional information.



-adhocBypasses member checking in context chains. See Adhoc Mode on page 76 for additionalinformation.

-allShows all selected Runnables (requires -show). See View RMDB Contents on page 77 foradditional information.

apicmd Access RMDB data (use API commands), then exit. The -apicmd option consumes allremaining options on the command line and sends the tokens as a command to the RMDBAPI, printing the result and exiting. See Access RMDB API via the vrun Command Lineon page 94 for additional information.

-askIn -ask mode, the user is prompted for values of key parameters, based on attributes set inthe configuration database (RMDB) file. See Selecting Values for Embedded Options onpage 81 for additional information.

-cleanDelete the SCRATCH directory prior to the regression run.

-clearClear existing runlist contents (requires -runlist). Note also that if the -clear option is notincluded on the command line, then any contents already in the runlist is also a part of theregression run. See Build a Runlist of Runnables to Execute on

Documents

vrm_user