XOS 9.5.4.0 Xos Configuration Guide

Part Number 03392PMarch 2011

XOS Configuration Guide

Version #: XOS 9.5.1

Copyright and Trademark InformationCopyright © 2011 by Crossbeam Systems® Boxborough, MA, USA

All Rights Reserved

The products, specifications, and other technical information regarding the products contained in this document are subject to change without notice. All information in this document is believed to be accurate and reliable, but is presented without warranty of any kind, expressed or implied, and users must take full responsibility for their application of any products specified in this document. Crossbeam Systems disclaims responsibility for errors that may appear in this document, and it reserves the right, in its sole discretion and without notice, to make substitutions and modifications in the products and practices described in this document.

This material is protected by the copyright and trade secret laws of the United States and other countries. It may not be reproduced, distributed, or altered in any fashion by any entity (either internal or external to Crossbeam Systems), except in accordance with applicable agreements, contracts, or licensing, without the express written consent of Crossbeam Systems.

For permission to reproduce or distribute please contact your Crossbeam Systems account executive.

This product includes software developed by the Apache Software Foundation: http://www.apache.org.

Crossbeam, Crossbeam Systems, X-Series, XOS, X20, X30, X45, X60, X80, X80-S, and any logos associated therewith are trademarks or registered trademarks of Crossbeam Systems, Inc. in the U.S. Patent and Trademark Office, and several international jurisdictions.

All other product names mentioned in this manual may be trademarks or registered trademarks of their respective companies.

ContentsAbout This Guide

Intended Audience . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11Related Documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11Conventions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

Typographical Conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12Cautions, Warnings, and Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13IPv6 Address Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

Customer Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

Chapter 1: IntroductionHardware Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

Network Processor Module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18Control Processor Module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18Application Processor Module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18Module Link Status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

Virtual Application Processor (VAP) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19Traffic Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19Networking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20High Availability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20Application Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21Greenlight Element Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21Automated Workflow System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22Management Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

Greenlight Element Manager (GEM) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23Command Line Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23Element Management System. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

Chapter 2: Initial ConfigurationConfiguration Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25Before You Begin. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

Network Topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26Application Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

Using the Interview Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28Working with Configuration Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

Validating the Running Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29Saving the Running Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

Changing Default Usernames and Passwords . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30Changing Default SSH Keys . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

Chapter 3: Configuring System SettingsConfiguring Basic System Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

Defining the Host Name. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33Defining the IP Domain Name . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34Defining the Control Network IP Address and Subnet Mask . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34Displaying the Current Control Network and Subnet Mask . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34Defining the System Identifier . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35Defining the System Time Zone. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35Setting the System Calendar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35Enabling the Greenlight Element Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

Configure Facility Alarms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36Access Facility Alarm Settings in the CLI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

Configuring and Displaying the System Operational Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37Configuring an NTP Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37Configuring a DNS Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

XOS Configuration Guide 3

Chapter 4: Configuring Administration SettingsManaging User Accounts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

Creating a User Account . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39Displaying User Accounts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

Changing the CPM Root Password and Expiration Interval . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41Changing the Root Expiration Interval . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41Changing the Root Password . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41Recovering from an Expired CPM Root Interval . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

Changing the Web Server SSL Certificate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43Installing the Certificate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

Enabling SSH Access to VAPs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

Chapter 5: Configuring VAP GroupsOverview of VAP Groups in XOS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

Number of VAPs in a Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46Master VAP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46Master VAP Reselection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46VAPs and Serialization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46VAPs and APMs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46Standby VAP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47IPv6 Traffic. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47Load Balancing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47Jumbo Frame Support and VAP Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47Scatter Gather and Fragmented Ethernet Frame Support. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47Flow Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47RAID Capability and Settings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47VAP Operating Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48VAP Backup and Restore . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48Application Monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49Configure VAP Group for Ethernet Jumbo Frame Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49Mapping a Management Circuit to a VAP Group. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

Overview of IPv6 Traffic. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51Load Balancing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51IPv6 Circuit Considerations and Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51Supported IPv4 to IPv6 Tunnels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

Configuring a VAP Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52VAP Group Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55Virtual Environment VAP Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

Additional Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58Configure a Virtual Environment VAP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

Displaying VAP Group Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59Displaying VAP Group Mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

Chapter 6: Configuring InterfacesData Interface Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

Physical Interfaces. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62Logical Interfaces. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62Interface Internal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62Circuits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62Virtual Network Device (VND) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63VLANs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63Domains. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65Group Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65Redundant Interfaces. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65VND Tap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

Configuring Interfaces on the NPM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66Configuring Physical and Logical Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66Configuring Circuits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

Contents 4

Packet Mirroring on the NPM Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73Access Control Lists (ACL Interface) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73configure acl-interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73Configuring Access Control Lists . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75

Configuring Chassis Resource Protection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78Resource Protection Workflow. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81Disabling Flow Table Limit Action per Interface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82

Configuring Redundant Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83Configuring a Multi-Link Group Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

Chapter 7: Configuring Management InterfacesPhysical Management Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89Configure Management Physical Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90Configure Access Lists . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91Define an Access List to a Management Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94Display Management Access Lists . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94

Chapter 8: Configuring Flow ProvisioningFlow Rule Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95

VAP IP Flow Rules. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95VAP Non-IP Flow Rules. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97Default VAP Flow Rules. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97System IP Flow Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98Interaction Between System and VAP IP Flow Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99Multi-VAP Group IP Flow Rule Match . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99System Non-IP Flow Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100Priorities for IP Flow Rules. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100

Configuring IP Flow Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101Configuring Non-IP Flow Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103Configuring System IP Flow Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103

Troubleshooting IP Flows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104Configuring System Non-IP Flow Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106

Chapter 9: Configuring ProtocolsConfiguring Static IP Routes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107Configuring ARP Entries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109Displaying ARP Entries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109Configuring Neighbor Discovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109Displaying Neighbor Discovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110Configuring a RADIUS Server Host . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110Configuring an LDAP Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111Defining the LDAP Version Number . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111Configuring RSA SecurID to Authenticate Administrative Users . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112Configuring RSA SecurID to Authenticate Remote Users . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113

Integrating an ACE/Server with VPN-1/FireWall-1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113Configuring ACE/Server in a VPN-1/Firewall-1 Clustered Environment . . . . . . . . . . . . . . . . . . . . . 113Configuring an ACE Server to Work with a High Availability Cluster . . . . . . . . . . . . . . . . . . . . . . . . 114

Chapter 10: Configuring Bridge ModeConfiguring Bridge-Mode Bridges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115Additional Configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117

Circuit Restrictions for Bridges. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117Basic Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117VLAN Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118Configuration Using Multi-Link Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119


Chapter 11: Configuring CPM RedundancyOverview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121

CP Redundancy Software Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122CP Redundancy Hardware Requirements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122CP Redundancy and RAID Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122CP Redundancy Runtime States . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123

Installing CPMs and Configuring Redundancy in a New System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124Installing a Second CPM into an Existing System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125Managing CP Redundancy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125

Displaying the CP Redundancy State . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125Configuring the CP Redundancy Administrative State . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126

Troubleshooting CP Redundancy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126Using the CBS PSI Tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127Configuring a Redundant CPM Management Virtual IP Address . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128

Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128Redundant CPM Management Virtual IP Address Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . 128Virtual IP Address Interaction during In-Service Upgrade (ISU) . . . . . . . . . . . . . . . . . . . . . . . . . . . 128

Chapter 12: Configuring Multi-System High AvailabilityVRRP Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129

Failover Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129Virtual Router (VR). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130VRRP Priority. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130OSPF Failover in Conjunction with VRRP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131Control Link Port (HA Port) and Management Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131VRRP Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131

VRRP Configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133Configuring Control Link Ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134Creating an Active-Standby Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136Manually Force a VRRP Failover. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141View VRRP Status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142

Example Dual Box High Availability Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143Example VRRP Configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144

Simple Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144Complex Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145

Configuring VRRP and Automatic ARP for NAT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148

Chapter 13: Configuring Secure Flow ProcessingWhat Is Secure Flow Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151Serial Traffic and Circuits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151

Configuring a Serial Circuit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152Managing Serialized VAPs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153Secure Flow Processing on Layer 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154Parallel Traffic Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156

Chapter 14: Configuring RMON and SNMP TrapsConfiguring RMON . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159

System Owned RMON Alarms and Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159Configuring RMON Events. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160Configuring RMON Alarms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160Displaying RMON Alarms, Events, and Logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161Displaying Disk Monitoring Event and Alarm Entries. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162

Configuring Simple Network Management Protocol (SNMP) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163Configuring an SNMP Server. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163Configuring SNMP Traps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164SNMP MIB Dependencies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166Allowing SNMPv3 User Access . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167

Contents 6

Chapter 15: Using UNIX Commands on VAP Groups and Upgrading ApplicationsExecuting UNIX Commands on a Designated VAP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169Executing UNIX Commands on All VAPs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170

Copying a File from the CPM to all the VAPs in a VAP Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170

Chapter 16: Viewing Performance, Statistics, and StatusUsing GEM to View Module Information and Status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173Viewing Module Information and Status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174Viewing Disk Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175Viewing Heartbeat Status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175Viewing Module Utilization and Load Averages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176Viewing Interface Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177Viewing Switch Data-Path (SDP) Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178View VRRP Status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178

show vrrp status. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178show vrrp detail-status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179Show Remote Box . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179

Viewing Flow Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180show flow active. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180show flow distribution. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181show flow-path active. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182

Chapter 17: Viewing Alarms, Events, and LogsViewing Alarms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185

Using GEM. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186Using the CLI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187

Configuring a Remote Syslog Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190Viewing Message Logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190

Viewing Alarm Details from the Message Log . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191Viewing Audit Logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192

Chapter 18: Upgrading XOS Software and FirmwareXOS Automated Workflow System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195XOS shar Upgrade . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196

Copying and Extracting the shar File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196Upgrading with the Extracted XOS Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197Files Generated During the Software Upgrade Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199

Upgrading the XOS Firmware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200Manually Upgrading Firmware on an NPM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200Manually Upgrading Firmware on an APM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202Manually Upgrading Firmware On a CPM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202

XOS Migration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204Applications Supported for Migration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205Workflows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205Upgrade Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208Log Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209

Chapter 19: Configuring Routing ProtocolsRouting Software Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211

Converting Static IP Routes From a Previous RSW Version. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212Installing Routing Protocols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213Installing a Routing Protocol on a VAP Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214Managing Routing Configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214

Accessing NSM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215Configuring FIB Retain. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215

Redistributing Routes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216IPv6 Tunneling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217


6to4 Tunnel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217ISATAP Tunnel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219IPv6IP Tunnel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221GRE Tunnel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222

Chapter 20: Backups, Upgrades, and ReinstallationVAP Backup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225Preparing for Safe Upgrade . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226

Manual Safe Upgrade Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227Upgrading to CPM-8600 or APM-8600 Modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229

Upgrade a CPM-8400 to a CPM-8600. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229Upgrade an APM-8400 to an APM-8600 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229

Creating a New Configuration Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230Reinstallation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230

Changing the Disk Partitioning Scheme . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231

Appendix A: In-Service UpgradeIn-Service Upgrade (ISU) Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233

Prerequisites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234

In-Service Upgrade Process Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234Modes of In-Service Upgrade . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234Understanding Optimal Batch Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235

Sample Configuration Analyzer Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235Creating In-Service Upgrade Batches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235

Executing a Sample In-Service Upgrade . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236Executing a Non-Interactive In-Service Upgrade. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240

Aborting a Non-Interactive In-Service Upgrade . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240

Appendix B: RAID-Related Hard Drive Configuration and RepairRAID Feature Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242Additional Documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242Prerequisites for RAID . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242Important RAID Information. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243

Disk Drive and RAID Information for CPM-8600s, APM-8600s, and APM-8650s . . . . . . . . . . . . . . 243Disk Drive and RAID Information for CPM-9600s and APM-9600s . . . . . . . . . . . . . . . . . . . . . . . . . 243RAID Information for CPM-8600s Only . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243RAID Information for CPM-9600s Only . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243Important RAID Information for APM-8600s, APM-8650s, and APM-9600s . . . . . . . . . . . . . . . . . . 244

Obtain Hard Drive Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244Using mdstat to Determine RAID-Related Drive Information. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244

Error Situations and Error Messages. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247Configure RAID on an Existing Non-RAID System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 248

Adding a Second Drive to a CPM-8600 and Enabling RAID . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 248Adding Hard Drives to an APM-8600, APM-8650, or APM-9600 and Configuring RAID. . . . . . . . . 249

Change the Existing RAID Setting on a CPM-8600 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 250Change the Existing RAID Setting on the APM-8600s, APM-8650s, or APM-9600s in a VAP Group . 251Configuring Non-RAID on a CPM-8600 with Existing RAID 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251Detect and Replace Failed Drives on an Existing RAID System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253

Replace Failed Drive on a CPM-8600 or CPM-9600 Configured for RAID 1. . . . . . . . . . . . . . . . . . 253Replace Failed Drive on an APM-8600 with RAID 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 254Replace Failed Drive on an APM-9600 with RAID. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 254

Identifying Drives in an APM-8600, APM-8650, or APM-9600 with Non-RAID Configured . . . . . . . . . 255

Appendix C: Swatch Dynamic Display ToolOverview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257Understanding the Swatch Tool. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258Performance Monitoring with the Swatch Tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258

Contents 8


Starting and Stopping the Swatch Tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259Using Single Key Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259Swatch Tool Configuration Guidelines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 260Creating a Swatch Tool Configuration File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 260

Using the swacquire Command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 260Using the swread Command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261Using the swcalc Command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261Using the swaggr Command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 262Using the swscreen Command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 262Using the swdisplay Command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 262Logging Screen Data to a File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263Using Predefined TCL Variables in a Swatch Tool Configuration File. . . . . . . . . . . . . . . . . . . . . . . 263Using Swatch Tool Variables in All Commands. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263

Using Existing Swatch Tool Configuration Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264Performance Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264

Swatch Tool Child Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264Single Shared-Memory Region and Semaphore . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265

Troubleshooting the Swatch Tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265Data Not Being Displayed or Updated . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265Data is Not Displayed in the Correct Location . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265Swatch Tool is Affecting CPM Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265Swatch Tool is Affecting Performance of Remote APMs or NPMs . . . . . . . . . . . . . . . . . . . . . . . . . 265

Appendix D: Crossbeam SNMP MIB ReferenceOverview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267Using a MIB browser . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 268Using HP OpenView Network Node Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 268New Crossbeam MIB Entries in XOS V9.5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 268Greenlight Element Manager (GEM) MIB Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269XOS Alarms and SNMP Traps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275CROSSBEAM-SYSTEMS-MIB Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 280

Identifying the CROSSBEAM-SYSTEMS-MIB OIDs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 280CROSSBEAM-SYSTEMS-MIB product entries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 280

CBS-HARDWARE-MIB Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281Identifying the CBS-HARDWARE-MIB OIDs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282CBS-HARDWARE-MIB entries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282CBS-HARDWARE-MIB Trap Entries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285

CBS-MODULE-RESOURCES-MIB Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 288Identifying the CBS-MODULE-RESOURCES-MIB OIDs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 288CBS-MODULE-RESOURCES-MIB entries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 288CBS-MODULE-RESOURCES-MIB Trap Entries. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292

CBS-VAPGROUP-MIB Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293Identifying the CBS-VAPGROUP-MIB OIDs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293CBS-VAPGROUP-MIB entries. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293CBS-VAPGROUP-MIB Trap Entries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 294

CBS-VRRP-MIB Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 294Identifying the CBS-VRRP-MIB OIDs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 294CBS-VRRP-MIB entries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 294CBS-VRRP-MIB Trap Entries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 296

Standard MIBs on the Crossbeam X-Series Platform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 296

Appendix E: Maintenance ModeUsing Maintentance Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297

Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299

Contents 10

About This Guide

This guide provides step-by-step instructions for configuring and troubleshooting the X-Series Platform running XOS Version 9.5.x or later.

For a complete software version compatibility matrix, refer to the XOS Release Notes for the specific XOS version that you are using.

IMPORTANT: For the latest updates and revisions to X-Series Platform documentation, log into the Crossbeam Customer Support Portal at http://www.crossbeam.com/support/online-support/.

Intended AudienceThis guide is intended for qualified service personnel responsible for installing, configuring, and managing software on Crossbeam X-Series Platforms.

Related DocumentationThe following documents are provided on the Crossbeam Systems USB Installer (USBI) or are available on the Crossbeam Systems Customer Support Portal located at http://www.crossbeam.com/support/online-support/.

APM, CPM, and NPM Installation Notice

X80-S Platform Hardware Installation Guide

X60 Platform Hardware Installation Guide

X20 and X30 Platform Hardware Installation Guide

XOS Command Reference Guide

Multi-Application Serialization Configuration Guide

Multi-System High Availability Configuration Guide

Install Server User Guide

USB Installer User Guide

RSW Installation Guide (available with the RSW kit, purchased separately)

XOS V9.5.1 Release Notes

Configuration Guide 11

http://www.crossbeam.com/support/online-support/


Conventions

Typographical ConventionsFor paragraph text conventions, see Table 1 on page 12.

For command-line text conventions, see Table 2 on page 13.

Table 1. Typographical Conventions Used in Paragraph Text

Typographical Convention Types of Information Usage Examples

Bold Elements on the graphical user interface.

In the IP Address field, type the IP address of the first VAP in the group.

Click OK to close the dialog.

Select the Print to File check box.

Courier Keys on the keyboard.

File names, folder names, and command names.

Any information that you must type exactly as shown.

Program output text.

Press Esc to return to the main menu.

Save the user.txt file in the user_install directory.

Use the start command to start the application.

In the Username field, type Administrator.

The XOS CLI show calendar command displays the system calendar:

Fri Feb 25 13:32:03 2011

Courier Italic

File names, folder names, command names, or other information that you must supply.

In the Version Number field, type 8.5.patch_number.

> A sequence of commands from the task bar or menu bar.

From the taskbar, choose Start > Run.

From the main menu, choose File > Save As...

Right-click on the desktop and choose Arrange Icons By > Name from the pop-up menu.

About This Guide 12About This Guide

Table 2. Typographical Conventions Used in Command-Line Text

Cautions, Warnings, and Notes

IMPORTANT: Lists important steps that you must perform properly or important information that you must take into consideration to avoid performing unnecessary work.

NOTE: Provides special information or tips that help you properly understand or carry out a task.

Typographical Convention Types of Information Usage Examples

Courier User prompts and program output text.

CBS# show calendar

Fri Feb 25 13:32:03 2011

Courier Bold Information that you must type in exactly as shown.

[root@xxxxx]# md crossbeam

<Courier Italic>

Angle brackets surrounding Courier italic text indicate file names, folder names, command names, or other information that you must supply.

[root@xxxxx]# md <your_folder_name>

[ ] Square brackets contain optional information that may be supplied with a command.

CBS# configure dns server <IP_address> [vap-group <VAP_group_name>]

| Separates two or more mutually exclusive options.

CBS# cp-unknown-state {cp1|cp2}

{ } Braces contain two or more mutually exclusive options from which you must choose one.

CBS# configure vap-group <vap_group_name>

CBS(config-vap-grp)# raid {0|1}

Caution: Lists precautions that you must take to avoid temporary data loss or data unavailability.

Warning: Lists precautions that you must take to avoid personal injury, permanent data loss, or equipment damage.


IPv6 Address NotationWithin this guide, two notation types are used to represent IPv6 addresses.

Standard Notation

Compressed Notation

NOTE: XOS V9.5.x does not support mixed notation IPv6 addresses.

Standard notation

The standard notation for IPv6 addresses is eight 16-bit hexadecimal words separated by colons. For example:

2001:BA95:AC10:0000:CF6A:000D:2145:3713

Specifying leading zeros is not required, provided that there is at least one numeric value in each field of the address. The above address can also be represented as:

2001:BA95:AC10:0:CF6A:D:2145:3713

Compressed notation

Many IPv6 addresses contain multiple fields of zeros. Use the double colon (::) notation to represent a single contiguous group of zero fields within an IPv6 address. For example:

1458:0:0:0:0:A03:5:AC17AF06:0:0:0:BC:0:0:40:0:0:0:0:0:0:10:0:0:0:0:0:0:0

These can be represented as:

1458::A03:5:AC17AF06::BC:0:0:4::1::

NOTE: The AF06:0:0:0:BC:0:0:4 example is represented as AF06::BC:0:0:4 because only one “::'' is allowed in an address.

In this guide, the example IPv6 addresses are taken from:

The “Unique Local Unicast” address range (FC00::/7) because these addresses are not propagated over the Internet

The 2002: address range as part of the prefix for IPv6 6to4 tunnels


NOTE: In accordance with the IANA IPv6 address space allocations, an XOS VAP of type xslinux_v3 ("V3"), xslinux_v5 ("V5"), or xslinux_v5_64 ("V5_64") will treat some address ranges as reserved.

At this writing, the IANA range assignments are as follows (excerpted from http://www.iana.org/assignments/ipv6-address-space/):

With regard to reserved address blocks, some kernel versions make address-type determinations that are different from those made by other kernel versions. It should be noted that, in particular, V3 VAPs treat the following ranges as strictly reserved (that is, these addresses are reserved and are not treated as Global Unicast addresses), whereas V5 and V5_64 VAPs treat these ranges as reserved Global Unicast addresses:

0000::/8

0100::/8

0200::/7

0400::/6

For purposes of XOS circuit/interface configuration, it is recommended that the above "Reserved by IETF" ranges NOT be used unless warranted by special circumstances. In the general case, user-assigned XOS IPv6 unicast circuit/interface addresses should be allocated from the 2000::/3 (Global Unicast) and/or the FC00::/7 (Unique Local Unicast) address block(s).

IPv6 Prefix Allocation Reference

0000::/8 Reserved by IETF [RFC4291]






2000::/3 Global Unicast [RFC4291]




A000::/3 Reserved by IETF [RFC4291]

C000::/3 Reserved by IETF [RFC4291]

E000::/4 Reserved by IETF [RFC4291]

F000::/5 Reserved by IETF [RFC4291]

F800::/6 Reserved by IETF [RFC4291]

FC00::/7 Unique Local Unicast [RFC4193]

FE00::/9 Reserved by IETF [RFC4291]

FE80::/10 Link Local Unicast [RFC4291]

FEC0::/10 Reserved by IETF [RFC3879]

FF00::/8 Multicast [RFC4291]


Customer SupportCrossbeam Systems offers a variety of service plans designed to meet your specific technical support requirements. For information on purchasing a service plan for your organization, please contact your account representative or refer to http://www.crossbeam.com/support/technical-support/.

If you have purchased a Crossbeam Systems product service plan and need technical assistance, you can report issues by telephone:

United States: +1 800-331-1338 OR +1 978-318-7595

EMEA: + 33 4 8986 0400

Asia Pacific: +1 978-318-7595

Latin America: +1 978-318-7595

You can also report issues via e-mail to [email protected].

In addition, all of our service plans include access to the Crossbeam Customer Support Portal located at http://www.crossbeam.com/support/online-support/.

The Crossbeam Customer Support Portal provides you with access to a variety of resources, including Customer Support Knowledgebase articles, technical bulletins, product documentation, and release notes. You can also access our real-time problem reporting application, which lets you submit new technical support requests and view all your open requests.

Crossbeam Systems also offers extensive customer training on all of its products. For current course offerings and schedules, please refer to the Crossbeam Education Services Web pages located at http://www.crossbeam.com/support/training-services/.


http://www.crossbeam.com/support/technical-support/

mailto:[email protected]


http://www.crossbeam.com/support/training-services/

1Introduction

The X-Series Platform running the XOS software is an open networked application server designed to deliver enhanced application services while providing high-performance and availability. The modular design of the X-Series Platform allows it to run multiple applications, while providing multi-gigabit throughput performance. The XOS software, using secure standard interfaces with multiple levels of controlled access, enables cost effective integration of the X-Series Platform into the overall operation of your network.

This chapter provides an overview of the various features and functions of the X-Series Platform.

Hardware Overview on page 17

Virtual Application Processor (VAP) on page 19

Traffic Flow on page 19

Networking on page 20

High Availability on page 20

Application Overview on page 21

Greenlight Element Manager on page 21

Automated Workflow System on page 22

Management Options on page 23

Hardware OverviewThere are various XOS hardware platforms for different business needs. Each hardware platform contains three types of modules:

Network Processor Module (NPM) for packet reception, classification and load balancing.

Control Processor Module (CPM) maintaining overall system configuration, management and integrity.

Application Processor Module (APM) for hosting applications and network services that process packets belonging to individual flows.

These modules contain standard network interfaces including 10/100/1000 Ethernet, and RS-232 ports.

When populated with two CPMs, two to four NPMs, and two to ten APMs, the X-Series Platform operates without a single point of failure. The X-Series Platform also supports multiple, redundant power supplies (1200W or 2700W). For more information about power supply compatibility, refer to the X-Series AC Power Supply Installation and Replacement Guide.


Network Processor ModuleThe NPM provides network connectivity, and handles the flow of traffic into and out of the system. The X-Series Platform supports up to four NPMs. The NPM uses both a network processor and a microprocessor. The NPM-8650 and the NPM-8650-BG support Gigabit Ethernet and 10 Gigabit Ethernet interfaces. The NPM-8620 supports only Gigabit Ethernet interfaces. The ports are located on the front panel of the module.

The NPM makes load balancing decisions based on the following feedback loop:

Each APM runs a flow agent (cbsflowagentd) that collects OS performance information from the “/proc” file system, such as /proc/uptime (how long the system has been up since the last reboot) and /proc/net/dev (network information). The load agent communicates with the load calculator on the CPM.

The CPM flow calculator (cbsflowcalcd) uses the information from the load agent to compute the scheduling vector (a table that assigns weights to each APM). The CPM sends the scheduling vector to the NPM once every second. The NPM uses this scheduling vector to determine how to load balance the traffic to each VAP.

Control Processor ModuleThe CPM provides all the generic system-wide functions, including:

Switched Ethernet control network for all slots in the chassis.

10/100/1000 external Ethernet management port, logging port, console port, and chassis interconnect port (for dual-box high availability).

Management of the system boot process.

Management of insertion or removal of new modules into the X-Series Platform.

Software upgrade.

Service provisioning.

CP redundancy.

RAID data protection or hard drive maximization.

Management of alarms.

The X-Series Platform supports up to two CPMs, with both CPM slots providing identical control plane connectivity. The CPM designated as the Primary CPM can manage and support all NPMs and APMs in chassis. When two CPMs are present, one module remains in a standby (secondary) state until needed, which provides CP redundancy.

The CPM communicates to the other modules through a dual-redundant, private, switched control plane. The switching elements for the control plane are housed on the CPM, with all the point-to-point connections from each of the other slots in the chassis arriving at the CPM through the backplane connector.

Application Processor ModuleThe APM completes the application level processing of traffic flows. All traffic is load balanced between the APMs by the NPM. An APM processes the incoming network traffic and sends all outgoing traffic to the NPM.

The APM contains a high-performance processor and the associated hardware needed for network flow processing. Depending on the hardware model, the X-Series Platform supports up to ten APMs and can provide APM redundancy when two or more modules are present. For APMs with two hard drives installed, the RAID options support data duplication or enhanced disk drive efficiency.

When an APM is inserted into the chassis, it automatically boots and sends DHCP queries to get an IP address. Once this is received from the CPM, the system retrieves a kernel using TFTPBoot. The APM boots this kernel then mounts its file system using NFS. Optionally, the APM can have a hard disk drive for IDS and A/V engines.

Introduction 18

Module Link StatusEach module in the chassis maintains a link with the other modules. The state of this unidirectional link is based on the number of heartbeat messages received over time. The CPM maintains the state of all connections in the system. The APM and NPM keep track of connectivity from remote modules to themselves.

Heartbeats are sent four times a second. If no heartbeats are received within one second, the link is declared down. Otherwise, the link state number (0 to 100%) represents how many heartbeats on average have been received within a 4-second interval. A link state of 0 indicates the link is down; 100% indicates the link is fully operational.

Heartbeats have their own Ethernet protocol ID; they are not sent as IP packets. There are two types of heartbeat packets:

Hellos: Broadcast tracking states of links interconnecting modules.

Election: Unicast between two CPMs to negotiate which CPM is to become the primary.

Use the show heartbeat command to view the heartbeat status.

Virtual Application Processor (VAP)A Virtual Application Processor (VAP) is an application operating environment that is run on an APM. A VAP consists of the operating system, the system software, and an application.

A VAP group is a collection of VAPs configured with identical policies running the same application, grouped together to provide redundancy and increased throughput. The VAP group modules act in concert with one another as a virtual processing engine. The NPM load balances packet flows across the group based on a feedback loop from the APMs and CPMs.

The assignment of VAPs to physical APMs is controlled using several parameters. The number of VAPs in the group is set using the vap-count parameter. Additional capacity is achieved by adding APMs and increasing the VAP count.

The CPM tracks which VAPs are currently loaded, compared to the VAP count. The operating system and file structure for each VAP is stored on the CPM. To run an application, the APM loads the appropriate image to its memory. This allows for faster reload and reboot when preemption is enabled. The CPM also works in conjunction with the VAPs to provide load balancing information to the NPMs.

Traffic FlowThe X-Series Platform dynamically creates internal flow paths for each serviced flow. On the arrival of the first packet of a new flow, a new path is created based on the flow rule associating user traffic with provisioned application services. All subsequent packets of the flow follow the same path through the system and are processed by the same set of applications.

The X-Series Platform can parallelize or serialize traffic. As shown in Figure 1, parallelization is copying a traffic flow to one or more passive applications. For example, traffic can be passed to a Firewall and IDS simultaneously. This eliminates the need for external taps or SPAN ports. You configure parallelization by mapping a circuit to two VAP groups.

Also shown in Figure 1, serialization is defined as moving traffic through two or more active applications. For example, passing a flow first to an intrusion protection application then to a Firewall. Serialization is configured with the use of an internal circuit.

Flow rules determine how a new traffic flow is processed when it arrives on a logical interface. A traffic flow is identified by a set of parameters in the flow rule, configured by the user.


Figure 1. Example of Traffic Flow

NetworkingThe X-Series Platform supports these networking features:

Bridge mode for firewall and IDS/IPS

Virtual Local Area Network (VLAN)

IP static routes

Equal Cost-MultiPath (ECMP) support for static routes

Dynamic routing protocols, BGP, OSPF, OSPFv3, RIP, RIPNG, PIM-SM, and IGMP; however, these protocols require the optional Routing Software (RSW) kit (purchased separately)

High AvailabilityThe X-Series Platform provides a number of features to ensure that the system remains operational. The features include:

CP Redundancy. Each chassis contains two CPMs, where one is Active and the other is Standby.

Interface Redundancy. A physical interface can be configured as a backup to another physical interface. Failover occurs quickly and is transparent to the application.

LACP (Link Aggregation Control Protocol). Multiple physical interfaces can be combined into one interface. A failure of one physical link does not prevent traffic flow.

VAP Load Balancing. Each VAP corresponds to an APM. In a VAP group, each VAP runs an instance of the application, and traffic is load balanced among the VAPs. Should an APM fail, the system automatically redistributes the traffic flow among the remaining VAPs.

VAP Load Balancing with Synchronization. State synchronization may be necessary when switching a flow from one VAP to another while stateful packet processing is being performed. Using Check Point FireWall-1 synchronization, when a flow is reassigned to another VAP within the group, that VAP will have the proper FireWall-1 connection state to process the arriving packets. There may be a delay in synchronizing the new connection state to other VAPs so some packets may be dropped by the Firewall on the new VAP until the connection state is created.

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

Introduction 20

Standby VAP. An unused VAP is configured as Standby. Should a VAP in any VAP group fail, the standby VAP automatically joins that group.

APM Preemption Mode. VAP groups are assigned a priority. Should a higher priority VAP group have a VAP failure, a VAP from a lower priority application is taken and reloaded into the high priority group.

Multi-System High Availability. Using VRRP (Virtual Router Redundancy Protocol), two or more X-Series Platforms can be configured to provide High Availability (HA).

Application OverviewThe X-Series Platform supports these basic application configurations:

Multiple instances of the same application on different VAP groups. For example, you can have many distributed firewalls and have distinct policies in each instance.

One VAP group can span multiple Application Processor Modules (APMs). The flow classification and distribution allow for even load balancing across all VAPs, thereby increasing the performance of a single application.

Multiple applications can be configured to run concurrently. The multi-application configurations can use the Check Point OPSEC APIs to allow configurations of Check Point and one or more OPSEC-compliant applications. For example, Check Point FW-1/VPN-1 can be configured to use the Squid and Websense URL filtering applications using the Check Point OPSEC APIs. These applications would run in separate VAP groups. Other non-OPSEC compliant multi-application scenarios are supported as well.

Multiple applications can be configured to run in series. Multiple, different applications can be linked through an interface-internal, allowing traffic to be inspected by each application. Traffic is passed from one application to the next, allowing for multi-layered, in-depth inspection by best-of-breed security applications within a single X-Series Platform.

Security applications are provided separately.

Greenlight Element ManagerThe Greenlight Element Manager (GEM) provides a view into the components and health of your X-Series Platform. GEM is a Web application that displays system and component information in an easy to reference format. GEM provides system health monitoring capabilites only; future releases will incorporate additional tools and capabilities.

Individual views provide module status, load averages, flow rates, VAP group information, application data, system alarm information, DBHA and failover group status, and more, in a single glance. The tabbed interface can be customized to show specific module information or system historical data.

Detailed user assistance provides additional information about the views and steps to add or customize the display for your specific needs.

The Greenlight Element Manager application has the following client system requirements:

2 Ghz, dual-core processor, with 2GB RAM

Minimum screen resolution of 1024x768

Microsoft Windows 7, Vista, or XP

with Internet Explorer 7 or 8

with Firefox 3.0 or 3.5

Apple Mac 10.5 with Firefox 3.0 or 3.5

Linux with Firefox 3.0 or 3.5


To access the Greenlight Element Manager, configure the Web server during the XOS software installation, or from the command line after install. Use a secure connection (https://) to enter the system IP address or name in a Web browser.

https://<ip_address>

Third Party Pop-ups on GEMThird party pop-up windows may appear while using GEM. The appearance of pop-ups can be controlled on individual browsers, and are based on user preferences.

Users running GEM on a Microsoft Windows system using Internet Explorer may see content related to Microsoft's Windows Live toolbar add-on. Specifically, when attempting to create an APM view, a "Get an instant stock quote" button may display. Use the following workaround to prevent the appearance of these pop-ups.

Disable the Windows Live add-on as follows:

1. In Internet Explorer, select Tools --> Manage Add-ons. The Manage Add-ons dialog box appears.

2. For each of the Show views, select all items beginning with "Windows Live" and then click "Disable".

3. Click OK to close the dialog box.

4. Restart Internet Explorer.

Automated Workflow SystemThe Automated Workflow System (AWS) is a menu driven, command line interface tool, designed to simplify the process of updating XOS versions, performing firmware upgrades, and viewing the current status of your X-Series Platform.

To use the AWS, start the Automated Workflow System from the command line:

CBS# automated-workflow-menu

Welcome to the X-Series Platform Automated Workflow System!Version: 1.1.0-x

1. Configure XOS...2. Upgrade XOS software and firmware...3. View system configuration and status...4. Applications...5. Custom...

Select a submenu to view available automated workflows.Enter x to exit or ? for help.

Please Enter Selection:

Use the question mark to access the help on any of the menu or submenu commands.

Introduction 22

Management OptionsXOS provides the following system management options:

Greenlight Element Manager

Command Line Interface (CLI)

Element Management System (EMS)

A user account has CLI, GEM, AWS, and EMS access. However, the level of access for CLI and EMS is configured differently.

XOS supports Access Control Lists (ACLs) for the Ethernet management port.

Greenlight Element Manager (GEM)GEM is a Web application that displays system and component information in an easy to reference format. To access the Greenlight Element Manager, configure the Web server during the XOS software installation, or from the command line after install. Use a secure connection (https://) to enter the system IP address or name in a Web browser.


Command Line InterfaceThe Command Line Interface (CLI) is accessed locally using an RS-232 serial connection on the CPM. The CLI can be accessed remotely by Telnet or SSH through the Ethernet management port on the CPM.

You can log into the system as a system administrator, which accesses the system commands, or as a UNIX root user, as shown in Figure 2. Although some UNIX commands are used, generally, configuration changes using the UNIX environment are not recommended or supported.

You can assign a user a CLI access of 0 to 15, where 15 is the highest level of access. The access levels are used to restrict users to specific sets of commands. Users at a particular access level can access any commands whose level is at or lower than their user level. For example a user at level 6 can access any commands whose level is 0 to 6.

Each CLI command is also assigned an access level. All CLI commands are set at either level 0 or 15 by default. Most of the configuration commands are set at level 15 while most show commands are at level 0. For a description on using the CLI, refer to the XOS Command Reference Guide.

Figure 2. Login as Admin or Root

��

��

��

��

��

!��"#$%��&��'��(

!��"#)%��&��'��(

!��"#*%��&��'��(

�+��"#) �+��"#$ �+��"#*


Element Management SystemThe Element Management System (EMS) is a Java based application that offers a browser-based interface for configuring and monitoring the X-Series Platform. The XOS software and server reside on the CPM and are accessed through the CPM management interface. All communications are XML-based, and secured through SSL. Using EMS, you can define, configure, and monitor X-Series Platforms, applications, and users throughout your network. Using the EMS client, you can:

Configure and monitor the NPM, APM, and CPM

Configure and monitor circuits

View information about modules

Maintain user accounts and access levels

View performance statistics

EMS requires Java Runtime Environment (JRE) 1.6.x or higher on the client system. If you are using Internet Explorer as your browser, you will be automatically taken to Sun's web site to download JRE the first time you start EMS. Otherwise, you can download it manually from http://java.sun.com.

NOTE: EMS cannot be used to install applications. This can only be done using the CLI. Also, EMS cannot configure offline modules; this must also be done using the CLI.

EMS access levels differ from the CLI access levels. Each user account is assigned a specific role, restricting the user to view and configure only certain windows within EMS.

EMS is displayed in a browser window that directly connects to a web server running on the X-Series Platform. Multiple X-Series Platforms can be managed from a workstation by using separate browser windows. Communication between the workstation and each individual X-Series Platform uses an HTTPS-based private interface and exchange XML-formatted data. To access EMS, configure the Web server during the XOS software installation, or from the command line after install.

NOTE: The log on process to EMS has changed. Enter the IP address of the X-Series Platform followed by /ems (i.e. https://<ip_address>/ems) into your browser’s address field. The user name and password are case sensitive.

The connection between the X-Series Platform and a dedicated workstation is made secure by the Secure Socket Layer (SSL). SSL provides authentication, integrity, and security between a browser and the XOS web server.

The EMS window contains three sections:

The menus in the top frame provide access to administrative functions, configuration, and other tasks.

The navigation tree in the left frame displays folders to open windows for configuring, provisioning, and monitoring the system.

The work space in the right pane displays information windows about a folder or component you select in the tree view. For example, when you click on the System Configuration folder, a window is displayed showing configuration properties for the system.

Introduction 24

http://java.sun.com

http://java.sun.com

2Initial Configuration

When you log in to the X-Series Platform for the first time, the system runs the Interview program. This program prompts you to configure various parameters. The Interview does not run on subsequent startups, unless you reset the system to factory defaults.

By default, DHCP and SSH are enabled for quick access and initial setup.

This chapter contains the following topics:

Configuration Overview on page 25

Using the Interview Process on page 28

Changing Default Usernames and Passwords on page 30

Changing Default SSH Keys on page 31

Configuration OverviewThe basic steps to configure the X-Series Platform are as follows:

1. Gather information about your network and security applications.

2. Use the Interview program to configure the basic parameters, such as:

CPM management IP Address and subnet mask

Activate the Web Server to provide access to Greenlight Element Manager and EMS

Create and configure user accounts

Create and configure Access Control Lists (ACLs)

Internal IP network address and subnet mask

NOTE: The internal network IP address must be unique to your network.

CPM default gateway and host name

Time server IP address

System Identifier, which is a unique value from 1 to 255 given to each X-Series Platform

Operating mode

3. Create Virtual Application Processors (VAPs).

4. Define circuits, which are interfaces to the VAP.

5. Define the physical interfaces and map them to the circuits.

6. Define the IP and non-IP flow rules.

The following configuration steps can be performed in any order:

Load and configure the various applications.

Configure facility alarms.

Configure the routing features, such as IP static routes and their cost, and the optional routing protocols (OSPF, OSPFv3, RIP, RIPNG, BGP, and PIM-SM).

Configure High Availability. This can include single or multi-box redundancy, CPM redundancy, and redundant interfaces.


Before You BeginBefore beginning the software installation or configuration of your X-Series Platform, gather the following:

Network topology information

Application information

Network TopologyGather or create the following network topology information:

1. Create a diagram containing as much information as possible about your network topology.

2. List all known IP, VLAN, and media information.

Table 3. Network Topology Information

3. Specify the VAP group default gateway route for all XOS VAP groups.

Table 4. VAP Group Default Gateways

NPMInterface Port Number

IP Address Network Mask Interface Type VLAN Information

Circuit Name

VAP Group Name Default Gateway

Initial Configuration 26

4. Specify all static routes to be configured on the X-Series Platform.

Table 5. Static Route Specifications

5. Specify information about other devices that are directly connected to your X-Series Platform. Add them to your diagram and supply their IP and/or VLAN information if applicable.

______________________________________________________________________________________

______________________________________________________________________________________

______________________________________________________________________________________

______________________________________________________________________________________

______________________________________________________________________________________

______________________________________________________________________________________

6. Specify all information about remote access to your Security Service Switch. For example:

Hostname/IP Address

Username

Password

Application InformationGather the following application specific information:

1. Specify the platform you are using, for example:

Windows

Linux

Solaris

IPSO

HP-UX

AIX

Destination Network Next Hop IP Address


2. Specify the intended VAP group names used with the applications.

Table 6. Applications and Their VAP Group Names

3. Specify all keys for the various applications.

Using the Interview ProcessWhen you start the X-Series Platform for the first time, log in as an administrator (Login ID and Password are both admin). The system automatically runs the Interview program. This program prompts you to configure a number of parameters. The Interview program does not run on subsequent startups, unless you reset the system by erasing the database and rebooting the system. Follow the Interview program to configure the basic parameters, including:

CPM management IP Address

Subnet mask

System Identifier

Interfaces

Internal IP network address and subnet mask

CPM default gateway and host name

Time server IP address

Application VAP Group Name

Application Key


These features can also be configured using the Command Line Interface (CLI) or the graphical Element Management System (EMS) utility. Each feature is described in detail in the appropriate chapter of this configuration guide. Refer to specific chapters as necessary.

Some reasons you would exit the Interview program:

To configure VAP groups and circuits.

To import an existing configuration file using FTP.

If you choose to exit the Interview program, you can refer to the Configuration Overview on page 25 as a guideline to complete the system configuration.

Working with Configuration FilesOn a running system, there are two copies of the X-Series Platform configuration file:

Startup Configuration — The system startup configuration file is applied anytime the system boots or reboots.

Running Configuration — This is the active (running) configuration. This is a copy of the base Startup Configuration with additional configuration changes made since the last system boot or save of the running configuration to the startup configuration.

IMPORTANT: When you save configuration changes, the changes are saved to the Running Configuration. The changes are not applied to the Startup Configuration. When the system reboots, the changes are not seen unless you copy the Running Configuration to the Startup Configuration prior to reboot.

You can save X-Series Platform configuration files locally to the CPM disk as text files consisting of CLI commands. You can copy the Startup or Running configuration to a named configuration file for later analysis and comparison. However, direct booting from a named configuration file is not allowed. Configuration files include sections for each subsystem (system, module, interface, mappings, and services).

The Running Configuration Validation feature allows you to instantly review system configuration settings.

You can perform the following tasks with configuration files:

Display the running and startup configuration files as follows:

show running-config show startup-config

NOTE: If you are using EMS, use the System Config menu to display the running or startup configuration file.

Display the running configuration validation

Copy the running configuration to the startup configuration or to a file

Copy the startup configuration to a named file

Validating the Running ConfigurationTo run the validation using the CLI, enter the following command:

CBS# validate-configuration

In EMS, use the Running Configuration Validation window to instantly review system configuration settings. For example, messages are supplied if no logical interface is configured for a given physical interface. To run the validation using EMS, go to the System Config menu and select Show Running Config Validation. The Show Running Configuration Validation window is displayed.


Saving the Running Configuration The running configuration can be saved to either the startup configuration or a designated file. If you are using EMS, you can copy the running configuration to the startup configuration from the System Config menu. If using the CLI, use the following command:

copy running-config startup-config

You can save the startup or running configuration to a designated file for later analysis and comparison. Direct booting from the designated file is not allowed. To save the running configuration to a designated file from EMS, use the System Config menu. At the CLI, use one of the following commands:

copy running-config <path-file> [-flat] [echo-password] [-sort] [include-default] copy startup-config <path-file> [-flat] [echo-password] [-sort] [include-default]

Where <path-file> is the desired path and file name for the copied running configuration, -flat copies CLI commands with complete context, echo-password includes the encrypted passwords, -sort sorts VAP groups by vap-group-name in the vap-group section and sorts circuits by circuit-name in the circuit section, and include-default includes parameters that still have their default values.

For example:

copy running-config backup-file

Changing Default Usernames and PasswordsThe X-Series Platform has the following default usernames and passwords. For security, you should change them.

To access the Command Line Interface (CLI), the username is admin, and the password is admin. From the admin account, use the following command to change the password. You are prompted for a new password.

CBS# configure password

NOTE: The admin user can be deleted using the configure no username admin command. Before deleting the admin user, make sure that another user name with the same privileges exists. If the admin user is deleted, this command appears in the running configuration file after an XOS upgrade to prevent the admin user from being recreated by default.

To access the X-Series Platform from the EMS web server, the username is admin and the password is admin. Use the configure password command to change the password.

The GRUB password is MD5-encrypted. The password to access GRUB is x40rocks. Perform the following to change the password:

a. Log into the system as root which will place you at the Linux prompt.

b. Run /sbin/grub-md5-crypt to create the MD5-crypted password, for example:

[root@xxxxx root]# /sbin/grub-md5-crypt Password: $1$LiTbt0$30FLNL0TWxFWWyBBhoYw6.

c. Edit the /boot/grub/grub.conf file and replace the old MD5-crypted password with the new MD5-crypted password string.

d. Save the /boot/grub/grub.conf file.

NOTE: When upgrading GRUB, the password is preserved.


Changing Default SSH KeysThe X-Series Platform provides default SSH keys. These should be changed as follows:

1. Log into the system as root.

2. Generate a new RSA1 key:

/usr/bin/ssh-keygen -q -t rsa1 -f /etc/ssh/ssh_host_key -C '' -N ''

The files /etc/ssh/ssh_host_key and /etc/ssh/ssh_host_key.pub are generated. These are the default filenames and location for SSH. You may be prompted to overwrite the existing files.

3. Generate a new RSA key:

/usr/bin/ssh-keygen -q -t rsa -f /etc/ssh/ssh_host_rsa_key -C '' -N ''

The files /etc/ssh/ssh_host_rsa_key and /etc/ssh/ssh_host_rsa_key.pub are generated. These are the default filenames and location for SSH. You may be prompted to overwrite the existing files.

4. Generate a new DSA key:

/usr/bin/ssh-keygen -q -t dsa -f /etc/ssh/ssh_host_dsa_key -C '' -N ''

The files /etc/ssh/ssh_host_dsa_key and /etc/ssh/ssh_host_dsa_key.pub are generated. These are the default filenames and location for SSH. You may be prompted to overwrite the existing files.



3Configuring System Settings

The main topics in this chapter include:

Configuring Basic System Settings on page 33

Configure Facility Alarms on page 36

Configuring and Displaying the System Operational Mode on page 37

Configuring an NTP Server on page 37

Configuring a DNS Server on page 38

Configuring Basic System SettingsWhen you first booted your X-Series Platform, you completed a configuration interview. This section describes how to change those settings using the Command Line Interface (CLI). Specifically, how to:

Change and display device and domain name

Change the network and subnet mask addresses

Change system information

Change time zones

Change external access ability

NOTE: To access the device settings using EMS, expand Configuration in the Navigation Tree and open Wizard. In the Configuration Wizard window, click Open next to Device Configuration. The following sections describe how to access the device settings using the CLI.

Defining the Host NameThe host name is the unique name given to the X-Series Platform during the initial system interview. For systems with two CPMs, each CPM (CP1 and CP2) has a host name. To define the host name, use the following command:

configure hostname <name> [cp1|cp2]

The name is an alphanumeric string, which may include characters such as dashes and symbols. Specify CP1 or CP2 to define a hostname for that CPM.

To display the current hostname, use the following command:

show hostname

The following is an example of the show hostname command:

CBS# show hostname CP1 Hostname: myhost1 CP2 Hostname: myhost2


Defining the IP Domain NameThe IP Domain Name is the name given to the X-Series Platform. The default domain name is crossbeam. Use the no parameter to restore this default setting.

NOTE: To make a domain name searchable by Domain Name Servers (DNS), use configure dns search-name.

To define the IP Domain Name, use the following command:

configure ip domainname <name>

The name is an alphanumeric string, which may include characters, such as dashes, dots, and symbols. The following is an example of the hostname command:

CBS# configure ip domainname mycompany.com

To display the current domain name, use the following command:

show ip domainname

Defining the Control Network IP Address and Subnet MaskThe default value for the XOS system-internal-network is 1.1.0.0/16. IANA has recently (January 2010) allocated 1.0.0.0/8 to APNIC for use on the Internet.

IMPORTANT: Be sure to reset the internal network value to an appropriate unused network, a private address range, or unallocated address blocks defined by IANA. This configuration change requires a chassis reload.

XOS restricts IP routes whose destination network is the same or a more specific network match with the system-internal-network IP address. Traffic that matches this criteria will not pass through the X-Series Platform.

To define the internal control network and subnet mask used by the X-Series Platform to communicate between devices, use the following command:

CBS# configure system-internal-network {<ip-addr> <netmask>|<ip-addr/netmask>}

Where:

The following is an example of this command:

CBS# configure system-internal-network 2.2.0.0 255.255.0.0

NOTE: The configuration of system-internal-network is restricted from Class D or E internal networks. An internal network configured as Class D or E will cause boot problems with the modules in the X-Series Platform. If the configured network is not unique in the system, an error message will display.

Displaying the Current Control Network and Subnet MaskTo display the current control network and subnet mask, use the following command:

show system-internal-network

<ip-addr> IP address for internal control network.<netmask> Network mask for this internal control IP address. The last two bytes are ignored.

Default is 255.255.0.0.<ip-addr/netmask> IP address and network mask in CIDR format, for example, 2.2.0.0/16.

Configuring System Settings 34

This command displays both configured and operational system-internal-network (IP and mask).

Configured System Internal Network : 2.2.0.0Configured System Internal Netmask : 255.255.0.0Operational System Internal Network : 2.2.0.0Operational System Internal Netmask : 255.255.0.0(1 row)

Defining the System IdentifierTo define a system identifier used when connecting multiple X-Series Platforms, use the following command:

configure system-identifier <system-id>

Where system-id must be unique per X-Series Platform. Valid range is 1 to 255.

NOTE: If changes are made to the system identifier and/or internal network address, the X-Series Platform must be rebooted.

Defining the System Time ZoneTo change the time zone specified during the configuration interview, use the following command:

configure timezone <name>

Where name is the case-sensitive time zone. If you hit enter without typing a timezone, a list of available timezones that you can choose from is displayed. This is an actual time zone, as specified in the configuration. The following is an example of the time zone configuration command. This command requires a reboot for the change to implemented.

CBS# configure timezone America/New_York

To display the current time zone setting, use the following command:

CBS# show timezoneTime Zone America/New_York

Setting the System CalendarThe system calendar runs continuously, even if the system is powered off or rebooted. To set the system calendar, use the following command:

calendar <hh:mm:ss> <date> <month> <year>

Where:

The following example sets the system calendar to 1:32 p.m. on December 23, 2010:

calendar 13:32:00 23 dec 2010

NOTE: This command will not run when the system is configured with an NTP server.

Enabling the Greenlight Element ManagerTo enable the Greenlight Element Manager (GEM), use the following command:

configure web-server

<hh:mm:ss> Current time in hours (military format), minutes, and seconds.<day> Current day of the month (1 to 31).<month> Current month, in three letter format.<year> Current year (4 digits - no abbreviation)


To access the Greenlight Element Manager, use a secure connection (https://) to enter the system IP address or name in a Web browser.


Enable Access to EMSUse the configure web-server command to enable access to the legacy EMS application. Use the show web-server command to display whether the Web server is enabled or disabled.

To access EMS, use a secure connection (https://) to enter the system IP address or name in a Web browser followed by /ems.

https://<ip_address>/ems

Configure Facility AlarmsFacility alarms monitor processor usage on each of the modules, and are divided into 3 categories: Minor, Major, and Critical. Each category has an upper and lower threshold, with an upper threshold default value. All threshold values can be set by the user. The following table lists the user-configurable facility alarms.

Table 7. Facility Alarms:

There are additional facility alarms that are not user configurable, but provide output in the message logs. For information about these alarms and how to view the complete list, refer to the show alarms command in the XOS Command Reference Guide.

Access Facility Alarm Settings in the CLITo access the facility alarms threshold settings using the CLI, use the configure facility-alarm command as described in the XOS Command Reference Guide. To display the current threshold alarm settings, use the show facility-alarm command.

Parameter Description and Units

Upper/Lower Threshold Default Values

Minor Major Critical

cpu CPU cycles, percent of capacity 80% / 0% 90% / 0% 99% / 0%

cpu-core CPU core usage, percent of capacity

80% / 0% 90% / 0% 99% / 0%

disk-usage-root /root partition disk usage, percent of capacity

70% / 0% 80% / 0% 97% / 0%

disk-usage-boot /boot partition disk usage, percent of capacity

70% / 0% 80% / 0% 97% / 0%

disk-usage-mgmt /mgmt partition disk usage, percentage of capacity

70% / 0% 80% / 0% 97% / 0%

disk-usage-cbconfig /cbconfig partition disk usage, percent of capacity

70% / 0% 80% / 0% 97% / 0%

disk-usage-tftpboot /tftboot partition disk usage, percent of capacity

70% / 0% 80% / 0% 97% / 0%

disk-usage-var /var partition disk usage, percent of capacity

70% / 0% 80% / 0% 97% / 0%

free-memory APM free memory page count multiplier

N/A / 4 N/A / 2 N/A


Configuring and Displaying the System Operational ModeTo configure the X-Series Platform operational mode, use the following command:

configure operating-mode {single-np|dual-np|quad-np} {series-6}

Where:

To display the current operational mode, use the following command:

CBS# show operating-modeChassis Type : X80Configured Operating Mode : dual-npOperational Mode : dual-npConfigured NPM Mode : series-6Operational NPM Mode : series-6

Configuring an NTP ServerThe Network Time Protocol (NTP) is used to synchronize the time of a computer client or server to another server or reference time source, such as a radio or satellite receiver or modem. It provides accuracies typically within a millisecond on LANs and up to a few tens of milliseconds on WANs relative to Coordinated Universal Time (UTC) through a Global Positioning Service (GPS) receiver, for example. Typical NTP configurations utilize multiple redundant servers and diverse network paths in order to achieve high accuracy and reliability. Some configurations include cryptographic authentication to prevent accidental or malicious protocol attacks and some provide automatic server discovery using IP multicast.

To access the NTP server settings using EMS, open Configuration > System Configuration > NTP Server in the Navigation Tree. The following sections describe how to access the NTP server settings using the CLI.

To configure an NTP server address on the X-Series Platform, use the following command:

configure ntp server <server-address>

Where server-address is the IP address of the NTP server. The following is an example of this command:

CBS# configure ntp server 172.16.31.101

To display the current NTP server configured on the X-Series Platform, use the following command:

show ntp-server

single-np Single NPM mode system. Available on the X20, X30, X45, and X60 platforms.dual-np Dual NPM mode system. This is the default value for the X80-S platform, and

optional on the X45 and X60.quad-np Quad NPM mode system. This is an optional mode for the X80-S platform.


Configuring a DNS ServerThe Domain Name System (DNS) is a global network of servers that translate host names like www.mycompany.com into numerical IP (Internet Protocol) addresses, such as 24.62.13.19.

The DNS server commands add servers in the order they are entered. The first DNS server added will be the first “nameserver” entry in /etc/resolv.conf, the next will be the second listed, and so on. These are equivalent to Windows’ “Preferred DNS Server” and “Alternate DNS Server” respectively. The “search-name” parameters correspond to UNIX “search” directive or Windows’ “appended suffixes” options.

Multiple DNS entries for each can exist, and the entries can be different for CPM and VAP groups.

To access the DNS server settings using EMS, open Configuration > System Configuration > DNS Server in the Navigation Tree. The following describes how to access the DNS server settings using the CLI.

To configure a DNS server address on the X-Series Platform, use the following command:

configure dns server <server-address>

Where server-address is the IP address of the DNS server. For example:

CBS# configure dns server 172.16.31.101

To configure a DNS server address for a VAP group, use the following command:

configure dns server <server-address> vap-group <vap-group-name>

Where server-address is the IP address of the DNS server, and vap-group-name is the name of the VAP group associated with the server address. The following is an example of this command:

CBS# configure dns server 192.168.30.81 vap-group fw

To configure a DNS server search name for a VAP group, use the following command:

configure dns search-name <name> vap-group <vap-group-name>

Where name is the search-name for the specified VAP group, and vap-group-name is the name of the VAP group associated with the server address. The following is an example of this command:

CBS# configure dns search-name mycompany.com vap-group fw

To display the current DNS server configured on the X-Series Platform, use the following command:

show dns-server

DNS Server Address VAP Group 192.168.30.81 192.168.30.81 fw

To display the a DNS server search-name assigned to a VAP group, use the following command:

show dns-search-name

DNS Search Name VAP Group mycompany.com


4Configuring Administration Settings

This chapter contains the following major administration setup topics:

Managing User Accounts on page 39

Changing the CPM Root Password and Expiration Interval on page 41

Changing the Web Server SSL Certificate on page 43

Enabling SSH Access to VAPs on page 44

Managing User AccountsEach system user is defined by the user’s name, password, and access level. Collectively, these properties define each user’s profile. Each user account has both CLI and web access. However, the access is configured differently.

If you are a system administrator, you can:

Create or modify a user profile.

Delete a user profile.

The following sections describe how to perform these tasks using the CLI. To manage user accounts using the Element Management System (EMS), select Administration from the menu bar in the main window then select User Admin. From the Administration menu, you can also change your password, view system users, and view users logged in from the web.

Creating a User AccountTo create or modify a user account using the CLI, complete the following:

1. Create a user name, using the following command:

CBS# configure username <name>

Where name is a case sensitive alphanumeric string.

2. If you are creating a new user, you will be asked to enter a password. The password is a case sensitive alphanumeric string.

3. Retype the password to confirm it.

4. Define the user CLI access level, using the following command:

CBS# configure username <name> privilege <level>

Where the level is the access level assigned to the user. Valid values are from 0 to 15, with 15 being the highest. The default value is 0.


5. Define the user EMS access level, using the following command:

CBS# configure username <name> gui-level <level>

Where level is the access level assigned to the user. The predefined levels restrict the user to view and configure specific windows within EMS in the table below.

Table 8. EMS Access Levels

The administrator level can view and configure all windows. Administration tasks include managing user accounts, upgrading software, and configuring system parameters. Configuration tasks include configuring the system, module, VAP groups, interfaces and protocols. Flow Provisioning tasks include managing rate limiters and IP flow rules.

The following is an example of how to create an account named “rudolph”:

CBS# configure username rudolph privilege 15 gui-level administratorPassword :Retype Password:

To delete a user account from the CLI:

CBS# configure no username <name>

To change a user’s password, use the following command, and enter the new password.

CBS# configure reset-password <username>

NOTE: You can use the enable level command to change your user access level for this session. This command may require a password for higher levels. The no parameter restores the default privilege level.

To create, modify, or delete a user account from EMS, go to the Administration menu and select User Admin. Also use EMS to specify how long web users can remain logged in. If an individual web user does not request a new page within the specified number of minutes, the user session is cancelled and the user must log in again. This setting applies to all users who login to the X-Series Platform via HTTP.

To specify the timeout value for all user web sessions, go to the Administration menu and select Web Session Timeout. The default value is 20 minutes.

Level Perform Configuration

Perform Flow Provisioning

Perform Administration

unauthorized No login No login No login

guest Read only Read only No

network-operator Yes Read only No

service-operator Read only Yes No

administrator Yes Yes Yes

Configuring Administration Settings 40

Displaying User AccountsThe XOS software allows you to view accounts for a single user or all users. To view a single user account, use the following command:

CBS# show username [name]

To view all user accounts, use the following command:

CBS# show usernames

Using EMS, go to the Administration menu and select System Users or Web Users to view users logged in at the CLI or via HTTP, respectively.

Changing the CPM Root Password and Expiration IntervalBy default, the CPM root password expires after 30 days. The password and expiration interval can be changed at the CPM root level. The procedure is different depending on whether the system uses GRUB as the boot loader.

NOTE: The password and its expiration interval are independent of each other. On new installations of XOS V9.x (non-migration from 8.x, a password is required (default is admin).

Changing the Root Expiration IntervalTo modify the number of days before the CPM root password expires, login to the CPM as root and enter the following command followed by the user name, at the root prompt:

[root@xxxxx admin]# chage -M <XX> <username>

Where XX is the number of days before the password expires for the named user. Use 99999 to set the password to never expire.

Changing the Root PasswordTo change the root password, use the following command:

[root@xxxxx admin]# passwd <username> Changing password for user <username>New UNIX password: <password>passwd: all authentication tokens updated successfully[root@xxxxx admin]#

The default username is root, and the default password is admin. For XOS V9.5, a password is required.


Recovering from an Expired CPM Root IntervalTo recover from an expired CPM root interval, perform the following:

1. If the administrator account has not expired, reload the primary CPM as follows. The slot could be 6, 7, 13, or 14.

CBS# reload module <slot>

2. If the administrator account has expired along with the root password, you need to physically pull out and re-seat the primary CPM to reboot the system.

3. Watch carefully for the GRUB menu.

NOTE: To use the backspace function in GRUB, enter CTRL-h.

a. At the GRUB menu, type p on the keyboard then enter the password, x40rocks.

b. At the GRUB prompt, type e to edit.

c. Change the line number by pressing the down arrow until you reach the grub line starting with kernel.

d. Edit the line to add single for single user mode. For example:

grub edit> kernel /vmlinuz-x.x.xx-x ro root=/dev/hda7 console=ttyS0,9600n8 single

e. Type b to boot the CPM in single user mode.

f. At the root prompt, you can use the passwd <password> command to change the password, or refer to Changing Default Usernames and Passwords on page 30 to change the GRUB password.

4. Shutdown and reboot the X-Series Platform, using the following command:

[root@xxxx admin]# shutdown -r now

When the X-Series Platform reboots, use the vap-group-password command if you desire to propagate the new CPM root password to one or more of the VAPs.


Changing the Web Server SSL CertificateSome web browsers, such as Internet Explorer Version 7, give a security certificate warning message when opening an EMS session. This occurs because the certificate for EMS is self-published. The warning message can be ignored.

As an alternative, you can purchase or create a host-specific certificate from a trusted Certification Authority (CA) then install it as described in this section. The process of obtaining a certificate is defined by the third-party CA.

OverviewThe XOS web server resides on Tomcat, which is an open-source implementation of Java Servlet and JavaServer Pages technologies. The web server provides a Secure Socket Layer (SSL) certificate that users must accept to access the XOS web server.

If you choose to purchase a certificate, you can use a public CA such as VeriSign (at www.verisign.com), Thawte (at www.thawte.com), Baltimore (at www.baltimore.com), or various other public CAs. To control the certification process, you can become your own CA, which requires that you install certificate server software and issue your own certificates.

When you submit a request for a server certificate from a CA, you can use the CA’s web site to generate part of the certificate, such as the identification information, the public key, and the private key. You then submit the identification information and the public key to the CA for signing. If you generate the certificate request through a web browser, you submit a .PEM or .DER file that contains the encoded PKCS10 information, which is your identification information and the public key.

Once you generate the certificate request, submit the request to the CA for signing. When the certificate returns, you can download it out of the directory the CA specified and copy it into a text editor. The signed certificate contains your identification information, your public key, and the CA’s signature. This part of the certificate is still in .PEM or .DER format, but it is now encoded X.509 information. If you generated your certificate request in a browser, you can export a PKCS12 file from your browser, which includes your private key and your certificate. You export the file as a password protected PKCS12 file with a .p12 file extension (or a .pfx file extension if you export it using Internet Explorer). You can then import this file into the CA server.

The procedure for getting your certificate signed depends on the CA you use and whether your organization generates and signs its own certificates. If you chose to use a public CA, go to the CA’s web site to get a signed certificate. You can submit the certificate that you generated to your CA over the Internet. They will then contact you with information on how to get your signed certificate.

For instructions on generating a new certificate and key, refer to the following web site:

http://jakarta.apache.org/tomcat/tomcat-3.2-doc/tomcat-ssl-howto.html#s6

Installing the CertificateThe SSL certificate parameters are stored in the /usr/os/web/conf/server.xml file. For example:

<Connector className="org.apache.tomcat.service.PoolTcpConnector"><Parameter name="handler" value="org.apache.tomcat.service.http.HttpConnectionHandler"/><Parameter name="port" value="8443"/><Parameter name="socketFactory" value="org.apache.tomcat.net.SSLSocketFactory"/><Parameter name="keystore" value="/var/tomcat/conf/keystore" /><Parameter name="keypass" value="changeit"/><Parameter name="clientAuth" value="true"/></Connector>


http://jakarta.apache.org/tomcat/tomcat-3.2-doc/tomcat-ssl-howto.html#s6

In this example, the keystore is file /var/tomcat/conf/keystore. The keystore password is changeit.

Once you have the certificate and key, copy the new keystore file to a safe location on the X-Series Platform then modify server.xml file to point to the new keystore file. The following example shows the default location on the X-Series Platform:

<Parameter name="keystore" value="/usr/os/web/conf/keystore" />

If you used a password other than changeit, edit this line in server.xml:

<Parameter name="keypass" value="changeit"/>

Enabling SSH Access to VAPsTo allow individual users to directly access specific VAP members, enable SSH access by executing the following steps. Before starting, ensure that the increment-per-vap parameter has been configured on the management interface for the VAP group so that each VAP member has a unique IP address.

Execute all steps on each member of the VAP group.

1. Create user accounts on each APM.

test_1 (group1):~# useradd <username> test_1 (group1):~# passwd <username> Changing password for user <username>.New UNIX password:passwd: all authentication tokens updated successfully.test_1 (group1): ~#

2. Start SSHd on each member of the VAP by executing service sshd start. For example:

test_1 (group1): service sshd startStarting sshd: [ OK ]

3. Configure SSHd to start after a reboot.

[root@REG80B2 root]# chkconfig --add sshd

4. Configure SSHd to create the host.allow file.

[root@REG80B2 root]# chkconfig sshd on

5. Modify the /etc/hosts.allow file to permit either all hosts or specified hosts access to the SSH daemon. For example, to allow any host connection, specify sshd: ALL or to allow a specific host connection, specify sshd: <host IP address>.

Disable SSH Access

To disable access to the VAP members, use the following procedure.

1. Modify the /etc/hosts.allow file and remove the host from the SSH daemon access list. For example, to disable all hosts from connecting, specify sshd: NONE. To block a specific host connection, remove the host IP address: sshd: <host IP address>.

2. Unconfigure SSHd from starting at boot.

[root@REG80B2 root]# chkconfig --del sshd

3. Stop the service on each VAP.

test_1 (group1): service sshd stopStopping sshd: [ OK ]

4. Delete the user account on the APM.

test_1 (group1):~# userdel <username>


5Configuring VAP Groups

A Virtual Application Processor (VAP) is the application operating environment running on an APM. A VAP consists of the OS, the system software, and an application installed and running on the APM. VAP groups consist of one or more APMs running the same application. XOS provides the ability to perform load balancing and high availability within a VAP group.

An X-Series Platform supports a maximum of 10 VAP groups.

This chapter contains the following main topics:

Overview of VAP Groups in XOS on page 45

Overview of IPv6 Traffic on page 51

Configuring a VAP Group on page 52

VAP Group Example on page 55

Virtual Environment VAP Groups on page 58

Displaying VAP Group Parameters on page 59

Displaying VAP Group Mapping on page 60

Overview of VAP Groups in XOSA Virtual Application Processor (VAP) is an application operating environment that is run on an APM. A VAP consists of the operating system, the system software, and an application.

A VAP group is a collection of VAPs configured with identical policies running the same application, grouped together to provide redundancy and increased throughput. The VAP group modules act in concert with one another as a virtual processing engine. The NPM load balances packet flows across the group based on a feedback loop from the APMs and CPMs.

The assignment of VAPs to physical APMs is controlled using several parameters. The number of VAPs in the group is set using the vap-count parameter. Additional capacity is achieved by adding APMs and increasing the VAP count.

The CPM tracks which VAPs are currently loaded, compared to the VAP count. The operating system and file structure for each VAP is stored on the CPM. To run an application, the APM loads the appropriate image to its memory. This allows for faster reload and reboot when preemption is enabled. The CPM also works in conjunction with the VAPs to provide load balancing information to the NPMs.

The NPM assigns traffic flows to VAPs. Once a flow is assigned, the flow is entered in an Active Flow Table. This table is synchronized between the NPMs and includes information such as source and destination address, time to live, and VAP assignment.

The load and preemption priorities of a VAP group are set to 0 by default. Load priority defines the order in which VAPs are loaded onto APMs. Preemption priority defines which applications can be re-purposed and their APMs reconfigured with a higher priority application. Changing this value must be done carefully because a VAP group with a higher priority value seeks the resources of the other VAP groups if the required resources are not available in its group. For example, a firewall VAP group configured for three VAPs preempts a lower priority VAP group if its resources drop below three VAPs.


Number of VAPs in a GroupThe vap-count parameter determines how many VAPs to create, while the max-load-count parameter determines how many VAPs to load. This makes it possible to create more VAPs than initially required, thus keeping a prepared VAP in reserve. If you set the vap-count to 5 but max-load-count to 4, then an extra VAP will be available and can be brought online with a single command.

NOTE: Some applications require that the vap-count and max-load-count be the same.

The following are guidelines for the vap-count and max-load-count:

1. The vap-count cannot be zero (minimum is 1).

2. The max-load-count cannot be greater than the VAP Count.

When decrementing the vap-count the max-load-count must be adjusted accordingly downward.

Master VAPWhen members of a VAP group load onto APMs, the first VAP to become available is designated the Master VAP. The main function of the Master VAP is to handle routing protocols. For example, the OSPF daemon only runs on the Master VAP to prevent multiple routing tables should the daemon run on all VAPs.

The Master VAP is also used for various protocol responses, such as ARP and Spanning Tree. For example, all VAPs respond to an ARP request, but only the Master VAP’s response is sent out by the X-Series Platform.

Should the Master VAP fail, another VAP in the group is elected Master.

Master VAP ReselectionWhen configuring Layer 2 applications with multiple VAP groups, the master-failover-trigger application parameter must be configured. This parameter tells the system to re-elect a new master VAP if the application fails on the current master VAP, thereby preventing Layer 2 looping issues on the network. Use the master-holddown parameter to specify the amount of time that the current master VAP remains down before XOS gracefully elects a new master VAP. Configure these parameters for each VAP group, after configuring the ap-list.

VAPs and SerializationTwo or more VAP groups running different applications can be connected to one another in series. This allows traffic passing through the X-Series Platform to be inspected by both applications, and then passed on to the internal network or the Internet. For general information about serializing applications, see Configuring Secure Flow Processing on page 151. To learn how to configure specific serialized topologies, see the Multi-Application Serialization Configuration Guide included in this release.

VAPs and APMsThe ap-list parameter defines the list of APMs that are allowed to load VAPs in a group. Generally this list will show ap1 through ap10 when all APMs are identical. However, if an APM is different you may wish to change this list.

You cannot mix APM product types within a VAP group. If your system includes four APM-8650s and two APM-8600s, you would create two VAP groups, one group of four 8650s and another group of two 8600s. It is also important that the APMs in a VAP group have identical hardware, such as processors, hard disk drives, and memory. Use the ap-list parameter and show chassis command to display module name and hardware type information needed for creating unmixed VAP groups.

Configuring VAP Groups 46

Standby VAPThe standby VAP is not part of a VAP group. This VAP runs on an APM, checking the hardware integrity. If an APM running a VAP in a group fails, the standby VAP is dynamically allocated to the VAP group. The standby VAP is preempted and rebooted to run the image of the failed VAP.

As an alternative to configuring a standby VAP, preemption priority can be assigned to each VAP group. This allows you to configure an application in a VAP group as a higher or lower priority compared to another application. Should a higher priority VAP group have a failure, it takes a VAP from the lower priority group. This configuration works well when you have a mission critical application and a non-mission critical application running in the same X-Series Platform.

IPv6 TrafficUse the enable-ipv6 command from the vap-group context to enable IPv6 support on the VAP group. A non-ip-flow-rule is automatically configured and activated, passing IPv6 traffic directly to the Master VAP for processing by the application. After IPv6 is enabled on the VAP group, the user configures circuits, interfaces, and static routes for handling IPv6 traffic. For more information about IPv6 traffic, see Overview of IPv6 Traffic on page 51.

Load BalancingWith load balancing, the NPM distributes the traffic across multiple VAPs, maximizing the overall performance of the X-Series Platform. If a VAP fails, the NPM reclassifies the traffic flow and load balances it among the remaining VAPs. In addition, use the load-balance-vap-list parameter to define the APMs that are allowed to receive new flows. To gracefully shut down a VAP, remove its entry from this list. Existing flows will continue to be handled by the VAP, but once these flows cease the VAP will have no traffic.

Jumbo Frame Support and VAP GroupsVAP groups can be enabled to process Ethernet frames greater than the standard maximum size of 1500 bytes. If you enable jumbo frames for a VAP group, you should set the MTU size on each circuit associated with the VAP group. The minimum should be 1500, and the maximum value is 9000 .

Scatter Gather and Fragmented Ethernet Frame SupportUse the scatter-gather parameter to enable or disable fragmented Ethernet frame support at the physical Ethernet driver.

Flow RulesA flow rule determines how a new traffic flow is processed when it arrives on a logical interface. A traffic flow is identified by a set of parameters in the flow rule, which is configured by the user. There are flow rules for IP and non-IP traffic. Refer to Configuring Flow Provisioning on page 95 for information.

RAID Capability and SettingsAPM-8600s and APM-8650s can accommodate dual hard disk drives. With two drives installed you can, optionally, configure the drives for RAID (redundant array of independent disks) operation. Depending on the RAID configuration, the second drive of the pair can be used either to act as a backup, storing identical information, or act as a second available drive, used whenever the other drive of the pair is busy. The first setting increases data security, the second setting produces maximum drive speed/efficiency.

The RAID setting for the VAP group can be modified at any time as needed.


Requirements for RAID FeatureTo configure the drives for RAID operation, your system must meet the following requirements:

Two hard drives are required in an APM-8600, APM-8650, and APM-9600 to use the RAID feature. Use the show module status disk command to check for the presence and size of the disk drives.

Both disks must be of the same type and have identical partition sizes. Use the cat /proc/partitions (from root) command to check them.

RAID SettingsConfigure RAID settings using the raid parameter of the configure vap-group command.

There are three possible RAID settings:

configure vap-group <vapgroupname> no raid

This command turns raid operation off. The drives function independently. This is the default setting.

configure vap-group <vapgroupname> raid 0

This command configures the system to write to whichever drive of the pair is not currently busy. This setting increases efficiency and speed but does not duplicate data. Also, both drives are required in order to access the full content of the stored data.

configure vap-group <vapgroupname> raid 1

This command configures the system to write the same data to both disks, creating a complete backup in case of one drive’s failure.

See RAID-Related Hard Drive Configuration and Repair on page 241 for more information on RAID configuration.

VAP Operating SystemsThe table shows the VAP Operating Systems and the modules on which they are supported.

Table 9. VAP Operating Systems

You can create different VAP groups running different kernels in one X-Series Platform. Refer to the XOS V9.5 Release Notes or specific application installation guides for additional information.

VAP Backup and RestoreThe VAP backup and restore feature allows you to save your VAP group configurations to the CPM. Should you need to restore your VAP group configurations for any reason, this feature provides a safeguard against unrecoverable failure.

Backup of VAP File SystemsYou can create a VAP group archive for each VAP group on the CPM. Individual archives are named for the VAP group and numbered (from 1 to 99). You can save more than one backup file for each VAP group on the CPM, based on the available disk space.

Name X-Series Kernel Supported Module

xslinux_v3 X-Series Linux v3 Default on APM-8600/8650

xslinux_v5 X-Series Linux v5 APM-8600/8650, APM-9600

xslinux_v5_64 X-Series Linux v5, 64-bit APM-8600/8650, APM-9600

xsve X-Series Virtual Environment APM-9600


Restoring VAP File SystemsWhen you restore a VAP group’s file system, you can specify an archive name, or use the most recent backup (default). However, the archive file used must be from an identical VAP group configuration (number of VAPS and hardware) to the current configuration.

Application MonitoringApplication monitoring is enabled by default, and directs XOS to poll application health on each VAP in a group every five seconds to verify that every application is able to process traffic. If the application is not active on a VAP in the group, the system notifies the NPM to stop new flows (load balancing) to that VAP. The NPM performs this process dynamically. When a VAP is booted, the NPM will wait until the application is fully operational before sending traffic to the VAP.

When Application Monitor is disabled, the NPM sends traffic to the VAP immediately upon the APM being Up, regardless of the application state. If the application goes down, the NPM will continue to send traffic as long as the APM remains up, and there is no change in load balancing.

With the master-failover-trigger application command configured on the VAP group, XOS will re-elect a new master VAP any time application monitoring detects an application failure on the master VAP.

NOTE: Application monitoring cannot detect process hangs. If a process is not functioning but the application is running, the XOS health system continues to report that the application is running.

Configure VAP Group for Ethernet Jumbo Frame SupportTo configure a VAP-group to support jumbo Ethernet frames, use the jumbo-frame parameter from the vap-group context. Once this setting is enabled, you will need to reload the VAP to enable the functionality.

CBS(config-vap-grp)# jumbo-frameCBS(config-vap-grp)# exit

Jumbo Ethernet frame support also needs to be configured at the circuit level using the setting of the MTU on each circuit associated with the vap-group. For example, set the MTU for inf1 to a value of 9000 to enable jumbo frame usage on this VND:

circuit inf1 circuit-id 1026device-name inf1vap-group jumbo

ip-forwardingmtu 9000ip 11.0.0.1/8 11.255.255.255

Once you have set the MTU value, you can observe the usage of jumbo frames. Use an RSH session on the VAP and use the ifconfig command from root:

root# ifconfig inf1inf1 Link encap:Ethernet HWaddr 00:03:D2:E0:0D:01 inet addr:11.0.0.1 Bcast:11.255.255.255 Mask:255.0.0.0 UP BROADCAST DEBUG RUNNING MULTICAST MTU:9000 Metric:1 RX packets:3 errors:0 dropped:0 overruns:0 frame:0 TX packets:4 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:180 (180.0 b) TX bytes:192 (192.0 b)root#


Mapping a Management Circuit to a VAP GroupManaging an application installed on an X-Series Platform is often done using an individual connection (a single physical interface) to the VAP group. This connection allows you to update policies and monitor the application on the VAPs. Some applications require that you explicitly define the circuit as a management circuit (refer to specific application install guides for more information). Configuring the management circuit on a VAP group can be done two ways. If the management circuit has already been configured, you can use a single line command to create the connection:

Command:

CBS# configure circuit mgmt vap-group bluefw management-circuit

In this example, mgmt specifies the management circuit applied to the VAP group, and bluefw is the name of the VAP group to which the management circuit is applied.

The second way to map a management circuit to a VAP group is done while configuring the management circuit.

1. Create a management circuit, so that remote application management utilities can interface with the application.

Command:

CBS# configure circuit mgmtCBS(conf-cct)#

2. Assign a device name to the circuit. For clarity, the device name should be the same as, or based on, the circuit name.

Command:

CBS(conf-cct)# device-name mgmtCBS(conf-cct)#

3. Associate the VAP group with a circuit. Some applications (such as IBM ISS) require that you designate this circuit as the management-circuit. Refer to the application documentation for more information.

Command:

CBS(conf-cct)# vap-group bluefwCBS(conf-cct-vapgroup)# management-circuitCBS(conf-cct-vapgroup)#

4. Assign an IP address for VAP group management. Use increment-per-vap to assign a unique IP address per VAP member, allowing individual management connections. When configuring the management IP addresses it is recommended to leave some unused IP addresses so that additional APMs and VAPs can be added as the system grows.

Command:

CBS(conf-cct-vapgroup)# ip 172.16.19.63/24 increment-per-vap 172.16.19.67CBS(conf-cct-vapgroup-ip)# endCBS#


Overview of IPv6 TrafficXOS supports IPv6 addresses in both an IPv6/IPv4 dual-stack, and a pure IPv6 environment. XOS supports the tunneling of IPv6 traffic through an IPv4 network.

In a simple scenario, IPv6 traffic arrives from an external network, passes through an interface on the NPM to a circuit attached to a VAP group with an application installed and running. The application processes the traffic, and passes it out another circuit and interface on the NPM. The traffic is sent to its destination on the client subnet.

Configure VAP groups to accept IPv6 addresses using the enable-ipv6 command. Applications running on an IPv6-enabled VAP group must be able to process IPv6 traffic. After enabling IPv6 on the VAP group, configure IPv6 addresses on the circuits and the interfaces.

When you enable IPv6 on the VAP group, a default non-ip-flow-rule is automatically created as part of the command:

non-ip-flow-rule ipv6_ruleencapsulation ethernet type 34525action pass-to-masteractivate

The default flow rule passes all IPv6 traffic to the master VAP for processing. Refer to Configuring a VAP Group on page 52 for the steps to configure IPv6 on a VAP group.

Load BalancingIPv6 traffic is not load-balanced across all VAPs in the group. However, IPv6 traffic can be load-balanced across multiple cores on a the master VAP, reducing resource utilization and increasing throughput.

Use the multi-proc-processing parameter under the core-assignment context to modify the default IPv6 non-ip flow rule (ipv6_rule). This parameter will configure load balancing across multiple processors and multiple cores on the master VAP.

For core-assignment configuration steps, refer to Configuring a VAP Group on page 52. For a list and explanation of each of the core-assignment parameters, refer to Chapter 6 of the Command Reference Guide.

IPv6 Circuit Considerations and LimitationsConsider the following information when configuring your X-Series Platform for IPv6:

Enable IPv6 on the VAP group before assigning an IPv6 address to the circuit. If IPv6 is not enabled, a warning will be displayed when the IPv6 address is assigned to the circuit.

IPv6 addressing may be used on VLANs and virtual IP addresses.

Increment-per-vap cannot be configured for an IPv6 address.

IPv6 addressing is not supported on the management network.

You must specify an IP address for a circuit as an IPv4 address before assigning an IPv6 address or alias address on that circuit. If you specify the circuit address as IPv6, all alias addresses for the circuit must be IPv6 addresses.

The minimum MTU size for IPv6 enabled circuits is 1280 and the maximum is 9000. An error message appears when enabling IPv6 on a circuit with an MTU of less than 1280, or when setting the MTU to less than 1280 on an IPv6-enabled circuit. The Crossbeam default MTU value is 1500.

IPv4 / IPv6 mixed mode addresses cannot be configured using the CLI. For example, configuring the following address on a circuit will result in an error.

FC00:0:0:0:0:FFFF:204.9.54.119


Supported IPv4 to IPv6 TunnelsThe following transition mechanisms, or tunnels, are provided to allow early adapters of the IPv6 protocol a method to deliver IPv6 traffic over an IPv4 network segment. Support of these tunnels on the X-Series platform uses a standards (RFC) -based approach and is not Crossbeam proprietary. IPv6 transition tunnels should not be confused with VPN, ssl, or other IPSec tunnels.

There are a number of standard protocols that provide transition mechanisms for allowing IPv6 traffic to traverse an IPv4 network segment. XOS in conjunction with RSW 8.0 supports the following four IPv6 to IPv4 tunneling protocols:

6to4

ISATAP

IPv6IP

GRE

The process uses a dual stack router or a firewall on either side of an IPv4 network. These serve as endpoints for the tunnel. IPv6 traffic is encapsulated within IPv4 traffic, and crosses the network as native IPv4 traffic. When the encapsulated packets reach the destination router or firewall, the IPv6 packets are de-encapsulated and routed appropriately.

For additional tunnel information, refer to IPv6 Tunneling on page 217

Configuring a VAP GroupTo create a VAP group using EMS, open Configuration > Modules and VAP Groups > VAP Groups.

To create a VAP group using the CLI use the following steps. Not all the steps are required, but are shown here to illustrate the commands used for configuration. A simple VAP group configuration follows in the next section.

1. Create a VAP group named bluefw using the default xslinux_v3 operating system.

The command line interface is case sensitive. Different capitalizations (“Bluefw” and “blueFW”) are considered to be different names. To avoid confusion, use unique names. Because commands and keywords are lower case, using at least one upper case letter in a name will reduce confusion.

NOTE: The xslinux_v3 is loaded by default. To install an application that requires a different VAP operating system, specify the Linux kernel in the configure command.

Command:

CBS# configure vap-group bluefw Are you sure you want to create a new vap-group with OS version xslinux_v3? <Y or N> [Y]: Y Creating vap-group bluefw. May take several minutes....................%WARNING: X-Series xslinux_v3 is not supported on APM9600 or above

CBS(config-vap-grp)#

2. Define the number of VAPs in this group. Multiple VAPs provide redundancy and additional capacity.

Command:

CBS(config-vap-grp)# vap-count 3Are you sure you want to adjust vap-count to 3? <Y or N> [Y]: YAdjusting vap-count. May take several minutes............................


3. Specify the maximum number of VAP members in the VAP group. This number cannot exceed the VAP count. In order to install some applications, the max-load-count must match the VAP count.

Command:

CBS(config-vap-grp)# max-load-count 3 CBS(config-vap-grp)#


4. Assign APMs to support the VAP group. This command specifies the list of APMs to be loaded. All VAP members should be running on identical APMs. Use show module status from the CLI to verify the configuration of each APM.

Command:

CBS(config-vap-grp)# ap-list ap3 ap4 ap5CBS(config-vap-grp)#

NOTE: Use the show chassis command to determine the module names and types associated with each APM. Use the ap-list parameter to associate like module types (for example, just APM-8650s) into a VAP group of identical module types.

5. Define the VAP group load-balance-vap-list. The load-balance-vap-list is a list of VAP indexes that the NPM uses to load balance new flows. For example, if you have a VAP group with 3 VAPs (bluefw_1, bluefw_2, and bluefw_3), the NPM load-balances over all three VAPs by default. Use the load-balance-vap-list to force the NPM to load balance over specific VAPs as follows.

Command:

CBS(config-vap-grp)# load-balance-vap-list 1 2 3CBS(config-vap-grp)#

The NPM ignores indexes included in the command that do not exist. For example, the NPM will ignore indexes 4 5 6 7 8 9 in the command for a VAP group that only has 3 indexes.

6. Enable IPv6 address support if required. If you do not require IPv6 support, skip to Step 8.

CBS(config-vap-grp)# enable-ipv6

Enabling IPv6 automatically configures the following non-ip-flow-rule:


7. Use the core-assignment parameter to load-balance IPv6 traffic across multiple processors and multiple cores on the master VAP.

CBS(config-vap-grp)# non-ip-flow-rule ipv6_rule core-assignment multi-proc-processing

8. Configure the default flow-rule for the VAP group and return to the VAP group context. There are four steps to configure the load balancing flow rule.

Create the load balancing flow rule for the bluefw VAP group.

Set flow rule action to load-balance traffic to all available VAP members.

Set the activate flag to enable the action.

Return to the main CLI context to prepare for the next step.

CBS(config-vap-grp)# ip-flow-rule bluefw_lb CBS(ip-flow-rule)# action load-balance CBS(ip-flow-rule)# activate CBS(ip-flow-rule)# exit CBS(config-vap-grp)#

9. Define the VAP group load-priority level. This command determines which VAP group is given priority when assigning VAPs to physical APMs. Valid values are 0 to 255, with 0 being the default value. A zero sets the lowest priority level.

Command:

CBS(config-vap-grp)# load-priority 5CBS(config-vap-grp)#


10. Define the VAP group preemption-priority level. The application running on one VAP group is assigned a higher priority than an application on another VAP group (for example, a firewall assigned a higher priority than an IPS). If the higher priority VAP group requires additional APMs to load its VAPs, this value determines from which group APMs can be taken. If the preemption priority for this group is higher than the load priority for a currently running VAP, that VAP is shutdown and an APM is loaded to the higher priority VAP group instead.

Additionally, should a higher priority VAP group have a failure, it takes a VAP from a lower priority group. This configuration works well when you have a mission critical application and a non-mission critical application running in the same X-Series Platform.

Valid values are 0 to 255, with 0 being the default value. A zero sets the lowest priority level.

Command:

CBS(config-vap-grp)# preemption-priority 5CBS(config-vap-grp)#

11. Define the delay period this VAP group waits before load balancing. Assign the <interval> (number of seconds) a VAP group waits before beginning load balancing. Valid values are from 1 to 3600 seconds, with a default value of no delay.

Command:

CBS(config-vap-grp)# delay-flow 10CBS(config-vap-grp)#

12. Enable or disable the Reverse Path (RP) filter. Disabling rp-filter allows a management server external to the network to enter through the load balancing filter and access the correct VAP. This anti-spoofing mechanism is resident in the Linux kernel. Disable the rp-filter using the no command.

Command:

CBS(config-vap-grp)# no rp-filterCBS(config-vap-grp)#

13. Enable or disable logging of martians (packets with a source IP address but no known route). Packets with source addresses not routable by the system are referred to as martians (packets from Mars) on the grounds that they are of no evident “terrestrial” (that is, normal) source. Martian packets can occur from network equipment malfunction, mis-configuration of a host, or simple coexistence of two logical networks on a single physical layer. For example, if the IP networks 192.168.35.0/24 and 10.1.2.0/24 operate on the same Ethernet segment, packets from 10.1.2.5 are martians to system 192.168.35.8, and vice versa.

When enabled, martians are logged to syslog.

Command:

CBS(config-vap-grp)# log-martiansCBS(config-vap-grp)#

14. Configure vg-reset-wait-time. This command sets the wait time (in seconds) before resetting a VAP when no connectivity to the VAP has been detected. In the case of an application that is busy processing and fails to send heartbeats for a period of time, this command will extend the wait time before the VAP resets.

Command:

CBS(config-vap-grp)# vg-reset-wait-time 15CBS(config-vap-grp)#

15. Configure the reload-timeout value. By default, the system waits 300 seconds for a VAP group to reload. If the reload does not occur, the system resets the slot. For some system configurations, you may need to have the system wait a specific number of seconds for the VAP group to reload. For example, a firewall application may only require the system to wait 120 seconds before reloading. Valid values are 60 to 18000 seconds.

Command

CBS(config-vap-grp)# reload-timeout 120CBS(config-vap-grp)#



VAP Group ExampleThe following is an example of a minimal VAP group configuration:

CBS# configure vap-group bluefwCBS(config-vap-grp)# vap-count 2CBS(config-vap-grp)# max-load-count 2CBS(config-vap-grp)# ap-list ap1 ap2 CBS(config-vap-grp)# ip-flow-rule fw_lbCBS(ip-flow-rule)# action load-balanceCBS(ip-flow-rule)# activate

The following is the same VAP group configuration, with IPv6 and multi-core processing enabled:

CBS# configure vap-group bluefw xslinux_v5CBS(config-vap-grp)# vap-count 2CBS(config-vap-grp)# max-load-count 2CBS(config-vap-grp)# ap-list ap1 ap2 CBS(config-vap-grp)# enable-ipv6CBS(enable-ipv6)# exitCBS(config-vap-grp)# non-ip-flow-rule ipv6_rule core-assignment multi-core-processingCBS(non-ip-flow-rule)# exitCBS(config-vap-grp)#

In Figure 3, the X-Series Platform is configured to handle multiple VAPs, each having access to other networks and the Internet through either VAP group.

Figure 3. VAP Group Example

The following is an example of the commands in a configuration based on this figure:

CBS# configure vap-group redfwAre you sure you want to create a new vap-group with OS version xslinux_v3? <Y or N> [Y]: Y Creating vap-group Firewall. May take several minutes....................%WARNING: X-Series xslinux_v3 is not supported on APM9600 or above

CBS(config-vap-grp)# max-load-count 1CBS(config-vap-grp)# load-priority 2CBS(config-vap-grp)# ap-list ap1CBS(config-vap-grp)# ip-flow-rule Firewall_lb CBS(ip-flow-rule)# action load-balance CBS(ip-flow-rule)# activate CBS(ip-flow-rule)# exit CBS(config-vap-grp)# jumbo-frameCBS(config-vap-grp)# exit

��,�,�

)-.)**.$.)/*

$-.-.-.)/*

��))'��)

!��) !�)'!�)

��)$'��$

�"��+

��,

��,

��)*'��* ��,�,�)0.-.-.)/*

11.-.-.)/0

�"��+

��,

��,

!��$ !�$'!�$

��)1'��1

CBS# configure vap-group bluefwAre you sure you want to create a new vap-group with OS version xslinux_v3? <Y or N> [Y]: Y Creating vap-group Firewall. May take several minutes....................%WARNING: X-Series xslinux_v3 is not supported on APM9600 or above

CBS(config-vap-grp)# max-load-count 1CBS(config-vap-grp)# load-priority 5CBS(config-vap-grp)# preemption-priority 2CBS(config-vap-grp)# ap-list ap2CBS(config-vap-grp)# ip-flow-rule Firewall_lb CBS(ip-flow-rule)# action load-balance CBS(ip-flow-rule)# activate CBS(ip-flow-rule)# exit CBS(config-vap-grp)# exit

CBS# configure circuit purpleCBS(conf-cct)# vap-group redfwCBS(conf-cct-vapgroup)# ip 10.133.2.193 255.255.255.0CBS(conf-cct-vapgroup)# endCBS# configure circuit redCBS(conf-cct)# vap-group bluefwCBS(conf-cct-vapgroup)# ip 20.0.0.193 255.0.0.0CBS(conf-cct-vapgroup)# endCBS# configure circuit yellowCBS(conf-cct)# vap-group redfwCBS(conf-cct-vapgroup)# ip 15.0.0.183 255.0.0.0CBS(conf-cct-vapgroup)# endCBS# configure circuit blueCBS(conf-cct)# vap-group bluefwCBS(conf-cct-vapgroup)# ip 44.0.0.195 255.0.0.0

CBS# configure interface gigabitethernet 1/1CBS(conf-intf-gig)# logical log11CBS(intf-gig-logical)# circuit purpleCBS(intf-gig-log-cct)# endCBS# configure interface gigabitethernet 1/2CBS(conf-intf-gig)# logical log12CBS(intf-gig-logical)# circuit redCBS(intf-gig-log-cct)# endCBS# configure interface gigabitethernet 1/3CBS(conf-intf-gig)# logical log13CBS(intf-gig-logical)# circuit yellowCBS(intf-gig-log-cct)# endCBS# configure interface gigabitethernet 1/4CBS(conf-intf-gig)# logical log14CBS(intf-gig-logical)# circuit blueCBS(intf-gig-log-cct)# endCBS#

You can use the show running-config vap-group <name> command to view the VAP group configuration to verify your steps. Similarly, use the show running-config command with circuit <name>, vrrp-failover-group <name>, or group-interface <name> parameters to view specific running configurations for those parameters.


The following is an example of the commands to create an IPv6 configuration based on Figure 3 on page 55:

CBS# configure vap-group redfw xslinux_v5Are you sure you want to create a new vap-group with OS version xslinux_v5? <Y or N> [Y]:CBS(config-vap-grp)# max-load-count 1CBS(config-vap-grp)# load-priority 2CBS(config-vap-grp)# ap-list ap1CBS(config-vap-grp)# enable-ipv6CBS(enable-ipv6)# end

CBS# configure vap-group bluefw xslinux_v5Are you sure you want to create a new vap-group with OS version xslinux_v5? <Y or N> [Y]:CBS(config-vap-grp)# max-load-count 1CBS(config-vap-grp)# load-priority 5CBS(config-vap-grp)# preemption-priority 2CBS(config-vap-grp)# ap-list ap2CBS(config-vap-grp)# enable-ipv6CBS(enable-ipv6)# end

CBS# configure circuit purpleCBS(conf-cct)# vap-group redfwCBS(conf-cct-vapgroup)# ip 2002:C:4:0:0:0:0:B3/64%WARNING: IPv6 primary address was configured. No IPv4 aliases will be allowed for this circuit.CBS(conf-cct-vapgroup)# endCBS# configure circuit redCBS(conf-cct)# vap-group bluefwCBS(conf-cct-vapgroup)# ip 2002:1:F:0:0:0:0:F5/32%WARNING: IPv6 primary address was configured. No IPv4 aliases will be allowed for this circuit.CBS(conf-cct-vapgroup)# endCBS# configure circuit yellowCBS(conf-cct)# vap-group redfwCBS(conf-cct-vapgroup)# ip 2002:C:4:0:0:0:0:B8/64%WARNING: IPv6 primary address was configured. No IPv4 aliases will be allowed for this circuit.CBS(conf-cct-vapgroup)# endCBS# configure circuit blueCBS(conf-cct)# vap-group bluefwCBS(conf-cct-vapgroup)# ip 2002:1:F:0:0:0:0:11/32%WARNING: IPv6 primary address was configured. No IPv4 aliases will be allowed for this circuit.

CBS# configure interface gigabitethernet 1/1CBS(conf-intf-gig)# logical log11CBS(intf-gig-logical)# circuit purpleCBS(intf-gig-log-cct)# endCBS# configure interface gigabitethernet 1/2CBS(conf-intf-gig)# logical log12CBS(intf-gig-logical)# circuit redCBS(intf-gig-log-cct)# endCBS# configure interface gigabitethernet 1/3CBS(conf-intf-gig)# logical log13CBS(intf-gig-logical)# circuit yellowCBS(intf-gig-log-cct)# endCBS# configure interface gigabitethernet 1/4CBS(conf-intf-gig)# logical log14CBS(intf-gig-logical)# circuit blueCBS(intf-gig-log-cct)# endCBS#


Virtual Environment VAP GroupsThe Virtual Environment provides a platform that allows non-linux based applications to run on XOS. When you configure an X-Series Virtual Environment (XSVE) VAP group, XOS creates a VAP group configured to run one or more virtual machines. During the application install, the virtual machine is created on the APM (the host). The application (guest application or guest) is installed and configured in the same manner as other applications on the X-Series platform.

Use the following command to create the virtual environment VAP:

configure vap-group <name> xsve

For example:

CBS# configure vap-group vbluefw xsveAre you sure you want to create a new vap-group with OS version xsve? <Y or N> [Y]: Y Creating vap-group vfwall. May take several minutes....................%WARNING: X-Series xsve is only supported on APM9600 or above


An APM configured as an X-Series VE VAP supports a single virtual machine running a single guest virtual appliance (application). The X-Series virtual environment is only supported on the APM-9600 or above.

Additional CommandsThe following commands are used when configuring virtual environment VAP groups. These commands fall under the configure vap-group context.

For additional information on these and other virtual environment-specific commands, refer to the XOS Command Reference Guide or to the individual application installation and configuration guide.

Flow ProxyIn a virtualized environment, the flow-proxy parameter changes the flow processing algorithm to improve performance.

IMPORTANT: This parameter is disabled by default, but is required for any VAP group that uses the XSVE VAP operating system.

CBS# configure vap-group vbluefwCBS(config-vap-grp)# flow-proxyCBS(config-vap-grp)#

To disable the flow-proxy command after it has been enabled, use the no flow-proxy parameter.

Fail to HostIn a case where the virtual application (guest) fails, fail-to-host reroutes traffic through the host, allowing packets to be forwarded by the VAP. This parameter is disabled by default and should only be enabled when required by an application (guest). Refer to your application information for specific requirements.

CBS# configure vap-group vbluefwCBS(config-vap-grp)# fail-to-hostCBS(config-vap-grp)#

To disable the fail-to-host command after it has been enabled, use the no parameter.


Incoming Circuit GroupsFor configurations using circuits that are part of an incoming circuit group (ICG), some applications require the use of an incoming circuit group name. For more information about configuring a circuit as part of an incoming circuit group refer to Configuring Circuits on page 68, or the XOS Command Reference Guide. Use the following command to configure an incoming circuit group name.

CBS# configure incoming-circuit-group-name <ICG number> <ICG name>

Configure a Virtual Environment VAP The following is an example XSVE VAP configuration. For the VAP group configuration procedure, refer to Configuring a VAP Group on page 52.

vap-group vbluefw xsvevap-count 2max-load-count 2ap-list ap7 ap8load-balance-vap-list 1 2 3 4 5 6 7 8 9 10flow-proxyfail-to hostip-flow-rule loadbalance

action load-balanceincoming-circuit-group anyactivate

#incoming-circuit-group-name 2 internalincoming-circuit-group-name 3 external#circuit red circuit-id 1025

device-name redincoming-circuit-group 2vap-group vbluefw

ip 50.0.0.2/24 50.0.0.255 increment-per-vap 50.0.0.10circuit blue circuit-id 1026

device-name blueincoming-circuit-group 3vap-group vbluefw

ip 51.0.0.2/24 51.0.0.255 increment-per-vap 51.0.0.10

Displaying VAP Group ParametersTo display configuration information about a VAP group, use the following command:

show vap-group <vap-group-name>

Where vap-group-name is the name of the VAP group. The following is an example display.

CBS# show vap-group bluefwVAP Group : bluefwOperating System : xslinux_v3Load Priority : 0Preemption Priority : 0AP List : ap2 ap3 ap4VAP Count : 2Max Load Count : 2Max Reload Count : 3Load Balance VAP List : 1 2 3 4 5 6 7 8 9 10IP Forwarding (true/false) : fDelay Flow (seconds) : 0


Backup Mode : noneReload Timeout (seconds) : 300RP Filter (true/false) : fLog Martians (true/false) : fDHCP Relay Server List :RAID : noneJumbo Frame (true/false) : fScatter Gather (true/false) : fMaster HoldDown Timer (in seconds) : 0Master Failover Trigger : applicationApplication Monitoring (true/false) : tIPv6 Enabled (true/false) : tIPv6 IP Forwarding (true/false) : fFail To Host (true/false) : fFlow Proxy (true/false) : fReset Wait Time (seconds) : 5(1 row)

Displaying VAP Group MappingYou can view the relationship between VAP groups and the chassis slots occupied by the APMs. To do this with EMS, open Configuration > Modules and VAP Groups > VAP Groups.

To use the CLI, enter the following command:

show ap-vap-mapping

The following is a sample display created by this command.

Where:

Module — Number of the APM.

Slot — APM location in the chassis.

Status —Module status: Active, Up, Down, Initializing, Booting, Diagnostics, Maintenance, or Unavailable.

VAP IP Address — IP address assigned to the VAP

VAP group — Name of the VAP group.

Index — Assignment of the VAP within the VAP group.

Master — True indicates that this is the Master VAP in the group; otherwise, false is shown.

Module Slot Status VAP IP Address VAP Group Index Master

AP1 3 Active 30.1.4.101 redfw 1 true

AP2 4 Active 30.1.4.102 bluefw 1 true

AP3 5 Active 30.1.4.103 bluefw 2 false

AP4 6 Active 30.1.4.104 bluefw 3 false

AP5 7 Booting 30.1.4.105 redfw 2 false


6Configuring Interfaces

The physical data interfaces are located on the Network Processor Modules (NPMs). An XOS system can be configured with only one NPM type; however, the NPM type must also be defined in the software using the configure operating-mode command.

This chapter includes the following major topics:

Data Interface Overview on page 61

Configuring Interfaces on the NPM on page 66

Packet Mirroring on the NPM Interfaces on page 73

Configuring Chassis Resource Protection on page 78

Configuring Redundant Interfaces on page 83

Configuring a Multi-Link Group Interface on page 85

Data Interface OverviewThe data interfaces consist of physical interfaces, logical interfaces, circuits, and Virtual Network Devices (VNDs). The physical interface is mapped to a logical interface, which is mapped to a circuit. The circuit is seen as a VND by the VAPs. Virtual Local Area Networks (VLANs) are associated with a logical interface.

Figure 4. Data Interfaces

VAP Group

Logical Circuit

VND1

VND1

VND1

NPM APMAPMAPM


Physical InterfacesThe physical data interfaces on the NPMs are identified by type (gigabitethernet or 10gigabitethernet), chassis slot number of the NPM, and physical port number. For example, the gigabitethernet interface on the fourth port of the NPM in slot 1 would be designated as gigabitethernet 1/4.

Depending on the NPM model, the physical interface can be a copper or fiber interface and support speeds to 10 Gb.

Logical InterfacesPhysical interfaces are mapped to logical interfaces. A physical interface can be mapped to several logical interfaces.

The application has no knowledge of the physical interface being used. When a physical interface fails, the logical interface and any circuits configured to that logical interface can be moved to another physical interface.

Interface InternalSerial connections between VAP groups, and connections between individual VAPs for synchronization are configured using the interface-internal command.

CircuitsThe circuit provides the path packets follow from the physical interfaces on the NPMs to the VAP groups. These paths are dynamically updated each time a VAP boots onto an APM. When a circuit connects a physical interface to a VAP group, the state of the circuit depends on the physical link state. The circuit goes down when the physical interface fails or is unavailable.

Circuits can also provide internal communication channels between VAPs in a VAP group (synchronization traffic), serial connections for packets to travel between different applications on separate VAP groups, and parallel connections to multiple VAP groups sending traffic to more than one application simultaneously. Circuits between VAPs (sync circuits) or between VAP groups (serial or parallel circuits) are configured on an interface-internal.

When Dual-box High Availability is configured, an external interface is needed to provide connectivity to the sync circuit between VAP groups on separate systems. In this case, the sync circuit is configured using the link-state-resistant parameter. This parameter allows the circuit to continue passing internal traffic between the VAP members, even when the communication between systems stops due to a physical interface failure.

Circuits and IP AddressingFor a circuit servicing a Layer 3 VAP group, IP addresses can be associated with that circuit in different ways:

Multiple addresses associated with a circuit on a VAP is often referred to as aliasing or multi-netting. The first IP address configured on a circuit is the primary IP; the remaining addresses are aliases. These IP addresses can either be the same across VAPs, or different. This configuration is identical for all VAPs within a VAP group. It is possible to have all VAPs use the same primary IP address.

All VAPs within the same group can be assigned the same IP address associated with the circuit.

Create a circuit by specifying a circuit name for the circuit. Then map the circuit to a VAP group and assign an IP address. Optionally, you can place the circuit into a VLAN by specifying a default outbound VLAN tag.

Configuring Interfaces 62

Serialization and FailoverBy using VRRP to identify circuits, serialized applications can be configured into failover groups. VRRP allows you to configure Next Hop Health Check to verify the state of the device on the other end of the circuit. For more information refer to the Mutli-Application Serialization Configuration Guide.

Virtual Network Device (VND) Each VAP in the group sees the circuit as a VND, and it appears as a standard Ethernet interface to any processes or applications running on the VAP. The VNDs are identical on each VAP in the VAP group.

IP addresses are assigned to circuits, and thus to the VNDs. Identical IPs on each VAP allow for load balancing across the VAPs. Optionally, unique IP addresses can be assigned to the VND on each VAP. A unique IP address is usually assigned for application management, since management software must be able to address a specific member of a cluster.

The VND appears as a device name in the circuit configuration. XOS assigns a default device name to each VND; VNDxxxx, where xxxx is a unique number. Use the device-name parameter to change the default VND name; often the circuit-name and device-name are configured as the same.

VLANsA logical interface can be configured to pass traffic with a specific VLAN tag, a range of VLAN tags, or all VLAN and non-VLAN traffic.

The ingress-vlan-tag parameter for the configure interface...logical or configure group-interface...logical command associates a VLAN tag or range of tags to circuits.

NOTE: Logical interfaces with overlapping VLAN ID ranges cannot coexist on the same physical interface. Additionally, when using interface redundancy, the backup interface cannot be assigned a VLAN ID that is assigned to the master interface.

The logical-all parameter for the configure interface command configures a logical interface to pass all VLAN and non-VLAN traffic. Each physical interface can have only one logical interface configured with the logical-all parameter.

In a VLAN configuration using an 802.1q trunk, a single physical interface may handle several subnets. The X-Series Platform can be configured to handle a VLAN trunk by assigning many logical interfaces to a single physical interface. Each logical interface is associated with a VLAN tag or range of VLAN tags.

On ingress, a tagged packet is passed only to the logical interface that allows that VLAN tag. The logical interface then passes the packet to its configured circuit, which sends the packet to a VAP’s VND interface. The VND interface strips the VLAN tag and passes the normal IP packet to the VAP’s IP stack.

By using the default-egress-vlan-tag parameter with the configure circuit command, egress packets can be tagged by associating a VLAN tag with the circuit. When a packet egresses through a particular VND, that VND will assign a tag to that packet based on the circuit’s associated VLAN. The tagged packet is passed to the circuit, then to the logical interface which sends the tagged packet out the corresponding physical interface.

Use the hide-vlan-header optional argument under the default-egress-vlan-tag parameter to remove the VLAN tag from the header. This is used with any application where the VLAN tag must be removed for proper operation.

Use the replace-vlan-tag <id> parameter with the configure circuit <name> vap-group command to replace an existing VLAN tag on the circuit traffic with another VLAN tag. This is useful to map traffic at an incoming port with a VLAN ID of “x” to a VLAN ID of “y,” especially if connected to an external switch.

For security, the X-Series Platform drops any incoming packet tagged with a VLAN tag that is not associated with any of the configured logical interfaces.


In the following example, a single physical interface has 2 logical interfaces, and each logical interface has a single circuit. The physical interface is connected to a VLAN trunk that carries VLANs 400 and 500.

interface gigabitethernet 1/9logical vlan400 ingress-vlan-tag 400

circuit vlan_400logical vlan500 ingress-vlan-tag 500

circuit vlan_500

In this example, when a tagged packet comes into the physical interface, logical interfaces are checked for associated VLAN tags. A packet tagged with VLAN 400 is passed to circuit vlan_400. A packet tagged with VLAN 500 is passed to circuit vlan_500.

MAC Address InheritanceA single MAC address (user-defined or system generated) can be assigned to multiple VLANs on a VAP group using the logical-all parameter under the configure interface command. If you do not specify a MAC address in Step 1, a system generated MAC address will be used.

The following steps describe this process.

1. Create a base circuit with a user-defined MAC address.

circuit int100base device-name int100base vap-group fw

mac-addr 44:44:44:44:44:44 ip 100.100.100.100/24 100.100.100.255

2. Create a VLAN circuit.

circuit int100v circuit-id 1106device-name int100vvap-group fw

ip-forwardingdefault-egress-vlan-tag 100ip 101.101.101.101/24 101.101.101.255

vap-group L2promiscuous-mode active

3. Create the interface, and assign a logical-all to the base circuit.

interface gigabitethernet 1/4 int100logical-all int100base

circuit int100baselogical int100v ingress-vlan-tag 100 100

circuit int100v

Use the show circuit command on the VAP group to verify the MAC address assignment.

Once the base VLAN is configured with the logical-all and a user defined MAC address, the additional VLANs inherit the MAC address automatically. If the user changes the MAC address on the base circuit, the VLANs will inherit the new MAC address.

Assigning a logical to the base circuit in Step 3 prevents changes to the base circuit MAC address from being applied to the VLANs. The MAC addresses on the VLANs remain as assigned in the initial configuration.

To display configured VLAN information, use the show vlan command.


DomainsA domain is used to differentiate traffic that uses overlapping IP addresses. For example, gigabitethernet 1/2 and gigabitethernet 1/3 both receive traffic from an IP address of 10.10.101.10; however, the sources are from different LANs. This can occur in an ISP configuration. To differentiate traffic, assign Domain 1 to the gigabitethernet 1/2 circuit and Domain 2 to the gigabitethernet 1/3 circuit.

A domain ID can also be used to differentiate multicast traffic, where one incoming packet goes out on multiple interfaces. For example, the X-Series Platform configured with PIM-SM passes ICMP multicast traffic. This traffic may be sent out on different interfaces for clients that have joined a multicast group. To differentiate the traffic, you assign a different domain ID for each circuit carrying the multicast traffic. In this case, you would also need to create an IP flow rule that specifies the domain ID or range of domains.

Group InterfacesThe X-Series Platform supports grouping multiple physical ports into a multi-link group interface using LACP (Link Aggregation Control Protocol, IEEE 802.3ad). Configure the group interface as you would a physical interface, by assigning the group to logical interfaces and assigning the logical interfaces to circuits. To create a multi-link group, you must first create a non-VLAN circuit to use for that group. The X-Series Platform supports assigning a multi-link group interface to multiple VAP groups, which allows you to tap traffic from the group interface. There is a limit of 8 ports to a group interface.

Circuits used in one group-interface cannot be used in another group-interface.

For scalability, you can add a multi-link group interface to a bridge or transparent interface. This allows greater throughput of traffic. However, the multi-link group cannot have any configured logical interfaces.

Refer to Configuring Bridge Mode on page 115 for more information on bridge and transparent modes.

Redundant InterfacesA physical interface can be configured as a backup for another physical interface in case of failure. By specifying a backup physical interface, logical interfaces assigned to the master physical interface are also backed up. Switching to the backup interface is transparent to the application. Failover occurs very quickly since the physical interface state is monitored by the X-Series Platform controller daemon.

NOTE: The failover succeeds only if the backup interface is in the UP state.

When failover occurs, the backup port always assumes the master port’s MAC address, and sends a gratuitous ARP to update Layer 2 switch tables.

NOTE: A Master interface cannot be part of a mode multi-link group-interface.

The backup interface can be configured as a Standby or Active interface. If configured as active and the master interface fails, the backup interface assumes the failed interface’s traffic in addition to its own. This is known as an active-active configuration. A backup interface in an active-active configuration cannot be configured to pass the same type of traffic as the master interface. This means that both the master and backup interfaces cannot be configured to pass the same VLAN traffic, or pass untagged traffic. An example of a valid active-active configuration would be to have an interface passing VLANs tagged 100 be a backup to an interface passing VLANS tagged 300.

Restrictions for Overlapping VLANs and Redundancy Interfaces1. The Master interface cannot be in standby-only state.

2. Masters sharing the same backup cannot have overlapping VLANS. Overlapping VLANS include:

a. Completely covering the range (100-200 and 50-250).

b. Includes partially overlapping (100-200 and 200-201).

c. Excludes logical-all.


3. Overlapping VLANS are checked for ALL involved logicals.

4. Masters can be part of a group-interface.

5. Validation is done in all paths including configuring redundancy-interfaces, set/unset standby-only, add/remove VLANs.

VND TapA VND tap enables an application, such as IDS, to view traffic on the circuit specified. Configuring a circuit between two VAP groups allows serial communication between them, as described in Configuring Circuits on page 68. To configure a VND tap, add a third VAP group to the circuit and configure the VAP group for promiscuous mode.

When a tap is configured while there are existing traffic flows, the tap will not see the established flows. If it is necessary to view established flows, use the clear flow-active command.

IMPORTANT: Using the clear flow-active command halts all traffic to the system for approximately 60 to 90 seconds. Do not use this command unless necessary.

A flow rule is required to see any traffic on a tap interface. Adding a tap to a logical line requires that the VAP group be added to the parent circuit first. There are four steps to create a flow-rule for the tap VAP group.

Create the default flow rule for the tap VAP group.

Set flow rule action to load-balance tap traffic to all available VAP members.

Set the activate flag to enable the action.

Return to the main CLI context to prepare for the next step.

CBS(config-vap-grp)# ip-flow-rule tap_lb CBS(ip-flow-rule)# action load-balance CBS(ip-flow-rule)# activate CBS(ip-flow-rule)# end CBS#

Configuring Interfaces on the NPMThe following sections provide the procedures to configure traffic flow on the NPM.

Configuring Physical and Logical InterfacesThere are two types of physical interfaces on the NPM: gigabitethernet and 10gigabitethernet. Each physical interface is identified by the NPM slot number and the port on the NPM. Each physical interface can contain one or more logical interfaces. Generally, there is a separate logical interface for each VLAN tag or range of VLAN tags on an Ethernet interface.

To configure a physical interface using EMS, open Configuration > Interfaces > Data Interfaces. Click on an interface row and click the Edit button.

From the Command Line Interface, use the configure interface command. Not all settings are used on the 10gigabitethernet interface, as noted.


Table 10. NPM Physical Interfaces

You can disable and enable any interface with the [no] enable command.

The following example is a simple gigabitethernet interface configuration. The NPM is in slot 1 and the physical interface is port 4. By default, the interface is enabled, auto-negotiation is enabled, duplex mode is full, and the interface is not in standby mode.

CBS# configure interface gigabitethernet 1/4

Use the show command in the config-intf context to display the current configuration. For example:

CBS# configure interface gigabitethernet 1/4CBS(conf-intf-gig)# showInterface : gigabitethernet 1/4Enable (true/false) : tAuto Negotiate Enabled (true/false) : tMedia Speed (Mbits) : autoDuplex Mode : autoPause Frame (true/false) : tStandby Only (true/false) : f

To display a physical interface’s current operational status, use the show interface command.

Interface-InternalAn interface-internal is created to provide internal connectivity between VAPs (synchronization or sync circuits) or between VAP groups (serialization). An interface-internal can be segmented into logical lines in the same way physical interfaces are assigned logical lines, and handles VLAN traffic, non-VLAN traffic, or all traffic.

The following example configures an interface-internal to pass all traffic.

CBS# configure interface-internal serialCBS(conf-intf-internal)# logical-all log_serCBS(conf-intf-internal-log-all)# circuit serializeCBS(conf-intf-int-log-all-cct)# endCBS#

Setting Definition

auto-negotiate Automatically matches best speed for multi-speed devices and uses automatic sensing to support full duplex operation. A fiber gigabitethernet port must have auto negotiation enabled. Only applicable for gigabitethernet.

duplex-mode Configures half or full duplex mode. Only applicable to gigabitethernet ports configured for a copper connection.

logical <name> ingress-vlan-tag <low> [<high>] [circuit <name>]

Defines a logical interface. Enter a unique name to create a logical interface, or specify an existing logical name to change its settings.

The ingress-vlan-tag passes VLAN traffic with a specific VLAN ID, as specified by <low>, or a range of VLAN IDs, as specified by <low> and <high>. Valid values are 0 to 4094.

Optionally, the circuit parameter assigns an existing circuit to the logical interface.

logical-all Configures a logical interface to pass traffic regardless of VLAN tags. A physical interface can have only one logical interface configured with the logical-all.

media-speed Configures media speed; only applicable to gigabitethernet ports configured for a copper connection. This setting overrides the auto-negotiation speed.

pause-frame Configures pause frame, also known as flow control.

standby-only Only use if the interface is a backup in an interface redundancy configuration. The interface will not be used for active traffic unless the master interface fails.


Configuring CircuitsThe circuit provides the path packets must take from the physical interfaces on the NPMs to the VAP groups.

To configure circuits using EMS, open Configuration > Interfaces > Data Interfaces > Circuits. Use the New button to create a circuit or to map the circuit. Use the Help button for details.

To create a circuit using the CLI, use the configure circuit <circuit-name> command, where circuit-name is the unique name given to the circuit. Use additional parameters to further define the circuit. To configure or re-configure an existing circuit, specify the circuit’s name, and use additional parameters to further define the circuit.

General Circuit Configuration ConsiderationsA circuit cannot be added to more than one logical interface.

Enable IPv6 on the VAP group before you assign an IPv6 address to the circuit. If IPv6 is not enabled, a warning will be displayed when the IPv6 address is assigned to the circuit.

IPv6 addressing may be used on VLANs and virtual IP addresses.

Increment-per-vap cannot be configured for an IPv6 address.

IPv6 addressing is not supported on the management network.

If you specify the IP address for a circuit as an IPv4 address, alias addresses for that circuit can be any mixture of IPv4 and IPv6 addresses. If you specify the circuit address as IPv6, all alias addresses for the circuit must be IPv6 addresses.

The minimum MTU size for IPv6 enabled circuits is 1280. An error message appears when enabling IPv6 on a circuit with an MTU of less than 1280, or when setting the MTU to less than 1280 on an IPv6-enabled circuit. The Crossbeam default MTU value is 1500.

IPv4 / IPv6 mixed mode addresses cannot be configured using the CLI. For example, configuring the following address on a circuit will result in an error.

FC00:0:0:0:0:FFFF:204.9.54.119

Circuits and WRP InterfacesXOS exposes WRP circuits to allow you to configure Next Hop Health Checks and virtual system failover. Circuits can have a device-name starting with 'wrp' to identify that they are part of a Check Point VSX configuration. By using VRRP to assign virtual IP addresses to the WRP circuits and interfaces, XOS can monitor these circuits. A device name prefixed with 'wrp' can be used on multiple circuits.

NOTE: Users cannot change this device name once it is established. Instead, you must delete the circuit and create it again. For information about how WRP circuits are used, see Configuring Multi-System High Availability on page 129.

When you install and configure Check Point VSX, the application script produces additional WRP circuit configurations. These additional circuit configurations are not necessary to the Crossbeam configuration, and can make the configuration difficult to read. If you do not use, or need to review the WRP circuits, use the exclude-wrp parameter to hide the WRP circuits from show-running config.

Circuit ParameterTable 11 lists the parameters available for configuring circuits in the NPM Mode.

Table 11. NPM Circuits

Parameter Definition

domain <1-4095> (Optional) User-defined domain (default is 1).

circuit-id <1-4095> (Optional) Circuit ID to be used, advised to use range <1-1024>


Configuring Circuits on a VAP GroupA VAP group is assigned while creating a circuit using the configure circuit <circuit-name> vap-group <group-name> command. The following settings are applicable after assigning a VAP group:

NOTE: MAC Address Restrictions:

The use of multicast mac-addresses is not allowed, unless the system-reserve option is specified. Source MAC addresses with a (binary) 1 in the least significant bit of the first octet are multicast MACs.

The use of all zero’s (0:0:0:0:0:0) or all one’s (ff:ff:ff:ff:ff:ff) as a MAC address is not supported.

Table 12. Circuit VAP Group Settings

device-name Configures device name, also known as interface name. The device-name of a circuit or interface cannot be lo, gre0, or sit0. Device names cannot begin with eth.

incoming-circuit-group <number>

If necessary, specify an Incoming Circuit Group (ICG) identifier for the circuit. The ICG is used by flow rules as an additional method to identify traffic from one or more specific circuits. Valid values are 2 to 255.

proxy-arp When enabled, Proxy ARP allows the circuit to reply to all ARP requests with known IP addresses.

vap-group Specifies an existing VAP group to assign to this circuit.


ip {<ip-addr> <netmask>|<ip-addr/netmask>} [<broadcast-addr>]

Assign a primary address and broadcast address to the circuit. If an IPv6 address is used, verify that enable-ipv6 has been set on the VAP group, and that the address is entered in CIDR format.

NOTE: When configuring circuits, the broadcast address of the circuit or circuit alias cannot be outside of the subnet. XOS will display an error message if this condition is detected.

increment-per-vap <end-ip-addr>

Defines a range of IP addresses so that each VAP in the group has a unique IP address. The end-ip-addr defines the end of the IP address range.

Increment-per-vap is not supported with IPv6.

alias {<ip-addr netmask>|<ip-addr/0-32>} [broadcast-addr] [increment-per-vap <ip-addr>] | [floating]

Assigns an alias IP address as the primary IP address for the VAP group. The VAP group must have a previously defined IP address. For example, this command assigns an IP address to 3 members of a VAP group:

CBS(conf-cct-vapgroup)# ip 20.2.2.102/24 increment-per-vap 20.2.2.104CBS(conf-cct-vapgroup-ip)# alias 20.2.2.101/24

NOTE: If you specify the IP address for a circuit as an IPv4 address, alias addresses for that circuit can be any mixture of IPv4 and IPv6 addresses. If you specify the circuit address as IPv6, all alias addresses for the circuit must be IPv6 addresses

The floating parameter assigns the alias IP address to the master VAP, allowing traffic, cluster management, and synchronization communication to go directly to the master VAP. If a new master VAP is elected, the address “floats” to the new master.

NOTE: Only one floating address can be used on a circuit. This parameter cannot be used with increment-per-vap.



default-egress-vlan-tag <id> [hide-vlan-header]

(Optional) Define the egress VLAN tag, where id is an integer value ranging from 0-4094. By default, packets are sent without VLAN tags.

NOTE: Some vendors do not tag VLAN 1 packets going over a VLAN trunk because they handle all non-tagged packets as VLAN 1. If the X-Series Platform is configured to look only for tagged packets, it will drop these untagged packets. To avoid this situation, configure two circuits on the same subnet, where one looks for VLAN 1 tagged packets and the other looks for non-tagged packets.

[hide-vlan-header] - Optional argument under default-egress-vlan-tag. Removes the VLAN tag from the header. Used with any application where the VLAN tag must be removed for proper operation.

dhcp-relay-server-list <IP_address> <IP_address>

Specify the IP addresses of the DHCP relay servers that the VAP group will use. See Note below.

dhcp-relay Enables the specified circuit to pass DHCP traffic to a relay server. See Note below.

icmp-redirect Enables ICMP Redirect

ip-flow-rule-priority <value> Change the default IP flow rule priority. Valid values are 0 to 31. Default is 21. Use no to restore the default value.

ip-forwarding Enables IP forwarding on the circuit.

mac-addr <addr> [system-reserved]

System generated MAC addresses are selected from the range 00:03:D2:EX:XX:YY to 00:03:D2:FX:XX:YY. The MAC addresses in the system-reserved address pool have sequential values for X:XX, starting at 0:01. The value for YY is the system ID assigned to the X-Series Platform.

Use the system-reserved parameter to specify a user-defined MAC address that is within the range of MAC addresses generated by the system. Using the existing format, change the X:XX values as required. The YY value must match the system ID, and is updated automatically when the system ID is changed.

The MAC address should be changed only if there are external devices that expect a specific MAC address.

management-circuit Maps a management circuit to a VAP group. Use no to delete the mapping.

mtu <size> Specifies the MTU size for the circuit on that VAP group. The same circuit with a different VAP group can have a different MTU. Default is 1500.

IPv6 circuits must have an MTU size equal to or greater than 1280.

promiscuous-mode [active] Circuits configured in promiscuous mode receive copies of all the data packets on the logical interconnect; other circuits only receive packets destined to that VAP.

Circuits with ip-forwarding should use no promiscuous-mode.

Circuits used as members in a bridge should use promiscuous-mode active.

Circuits used in a tap configuration should use promiscuous-mode.

replace-vlan-tag <id> Specifies a VLAN tag that the circuit will use to replace the existing VLAN tag on the circuit traffic. This is useful to map traffic at an incoming port with a VLAN ID of “x” to a VLAN ID of “y,” especially if connected to an external switch. The <id> is a value in the range of 1 to 4094.



NOTE: To enable a VAP group to use DHCP relay servers, first configure circuits with the dhcp-relay parameter, and then map those circuits to the interfaces for the DHCP server and DHCP client. Then use the dhcp-relay-server-list parameter under configure vap-group to configure the VAP group to use one or more specific DHCP servers.

The following is an example of one-to-one mapping. This example ties the first gigabitethernet interface in slot 1 to a circuit called “dmz” with an IP address of 192.168.30.1. This circuit will appear as an interface on each of the VAPs in the VAP group called “firewall.” Running the ifconfig command from the CLI for each VAP in the group would show the same interface with the same IP address on each VAP.

CBS# configure circuit dmzCBS(conf-cct)# device-name dmzCBS(conf-cct)# vap-group firewallCBS(conf-cct-vapgroup)# ip 192.168.30.1/24CBS(conf-cct-vapgroup-ip)# end

CBS# configure interface gigabitethernet 1/1CBS(conf-intf-gig)# logical dmzCBS(intf-gig-logical)# circuit dmzCBS(intf-gig-log-cct)# end

The following example configures the same circuit, but assigns increment-per-vap and an alias.

CBS# configure circuit dmzCBS(conf-cct)# device-name dmzCBS(conf-cct)# vap-group firewallCBS(conf-cct-vapgroup)# ip 101.1.1.2/24 increment-per-vap 101.1.1.4 alias 101.1.1.1/24CBS(conf-cct-vapgroup-ip)# end


The following example configures a circuit between two VAP groups to serialize traffic:

CBS# configure circuit serializeCBS(conf-cct)# device-name serializeCBS(conf-cct)# vap-group issCBS(conf-cct-vapgroup)# promiscuous-mode activeCBS(conf-cct-vapgroup)# exitCBS(conf-cct)# vap-group firewallCBS(conf-cct-vapgroup)# ip 2.2.2.1/24 increment-per-vap 2.2.2.2CBS(conf-cct-vapgroup-ip)# end

CBS# configure interface-internal serialCBS(conf-intf-internal)# logical-all log_serCBS(conf-intf-internal-log-all)# circuit serialize

The following example assigns a circuit to two VAP groups to parallelize traffic. Traffic coming into the circuit goes to both VAP groups simultaneously.

verify-next-hop-ip <ip-addr> Directs the VAP group that you are configuring to verify connectivity to the specified next-hop IP address and to perform a redundant interface failover in the event that a next-hop IP address health check fails. This option is effective only if the circuit is assigned to an interface that has been configured for redundancy.

IPv6 addresses can be used if at least one address (primary or alias) is an IPv6 address.



CBS# configure circuit circuit1CBS(conf-cct)# vap-group firewallCBS(conf-cct-vapgroup)# ip-forwardingCBS(conf-cct-vapgroup)# ip 100.123.10.1/8CBS(conf-cct-vapgroup-ip)# end

CBS# configure circuit circuit1CBS(conf-cct)# vap-group idsCBS(conf-cct-vapgroup)# promiscuous-mode CBS(conf-cct-vapgroup)# end

The following example configures a circuit for synchronization between VAPs in the VAP group, firewall. The device-name parameter changes the Linux interface name from vndxxxx to sync.

CBS# configure circuit fw_syncCBS(conf-cct)# device-name syncCBS(conf-cct)# vap-group firewallCBS(conf-cct-vapgroup)# ip 2.2.2.1/24 increment-per-vap 2.2.2.2CBS(conf-cct-vapgroup-ip)# end

CBS# configure interface-internal sync_circuitCBS(conf-intf-internal)# logical-all sync_circuitCBS(conf-intf-internal-log-all)# circuit fw_sync

Use the show running-config circuit <name> command to display the circuit configuration parameters.

IPv6 ExamplesTo run traffic using IPv6 addressing, the following example assumes that the VAP group has IPv6 enabled (see IPv6 Traffic on page 47). The difference is the IP address specified for the circuit.

CBS# configure circuit dmzCBS(conf-cct)# device-name dmzCBS(conf-cct)# vap-group firewallCBS(conf-cct-vapgroup)# ip fd00:1545:be72:e5af::cf33:54aa/64CBS(conf-cct-vapgroup-ip)# end


The following example configures an IPv4 address on a VAP group that has previously been IPv6 enabled, and assigns an IPv6 alias.

CBS# configure circuit dmzCBS(conf-cct)# device-name dmzCBS(conf-cct)# vap-group firewallCBS(conf-cct-vapgroup)# ip 101.1.1.2/24CBS(conf-cct-vapgroup-ip)# alias fd00:1545:be72:e5af::cf33:33ff/64CBS(conf-cct-vapgroup-ip)# end


Use the show running-config circuit <name> command to display the circuit configuration parameters.


Packet Mirroring on the NPM InterfacesPacket mirroring provides the ability to selectively copy traffic to another interface. Traffic is filtered by defining specific VLANs, MAC addresses, and other criteria. If the destination is an external NPM interface, an IDS or packet sniffer can be attached to perform an analysis of the traffic. Additionally, you can copy all packets to the eth2 interface on the NPM, and use tcpdump to inspect the packets.

The pass-through feature copies filtered traffic directly out another interface, bypassing any applications or functions on the X-Series Platform. This can be invaluable to maintaining traffic flow in the event you need to troubleshoot an application.

Specific traffic can be dropped at the NPM interface. If traffic coming from a VLAN, or originating from a MAC address has been identified as malicious, the NPM can be instructed to drop the traffic.

All of these filters are defined using an NPM Access Control List (ACL), and assigned to an interface.

Access Control Lists (ACL Interface)An access control list (ACL) consists of a filter and an action. The ACL is applied to an interface on the NPM, where it defines how traffic is handled. When you define an ACL for an interface, the NPM inspects all packets arriving on that interface and performs the ACL defined action on packets that match the criteria.

There are two components to an access control list, the filter that defines the traffic, and the action to be taken on the defined traffic.

NOTE: When you define an ACL for a group interface, the NPM applies the ACL to all physical interfaces in the group.

configure acl-interface The configure acl-interface command creates and configures a new ACL filter. This command is also used to modify an existing ACL filter.

Use the show acl-interface command to display the filtering criteria defined for each ACL filter that is currently configured on the X-Series Platform. Use the no parameter to delete the specified ACL filter.

Before deleting an ACL filter, all interface ACL definitions that include that filter must be deleted. Use the show acl-interface-mapping command to display a list of the ACLs defined for each physical interface and group interface configured on the X-Series Platform.

configure [no] acl-interface <ACL_filter_name>

FiltersYou can define the following filters for an ACL:

Direction

ingress-only: NPM applies ACL action only to traffic flowing into the X-Series Platform. This is the default flow direction filtering criteria defined when you create a new ACL filter.

egress-only: NPM applies ACL action only to traffic flowing out of the X-Series Platform.

bidirectional: NPM applies ACL action to all traffic flowing into and out of the X-Series Platform.

VLAN

<VLAN_tag> Defines VLAN tag filtering criteria, using the specified VLAN tag. The NPM applies the ACL action to a packet passing through the interface only if the packet’s VLAN tag matches the specified VLAN tag. The VLAN tag is specified in decimal (0 to 4095) or hexidecimal (0x000-0xFFF) format.


<VLAN_tag> <mask> Defines VLAN tag filtering criteria, using the specified VLAN tag and mask.The NPM applies the ACL action to a packet passing through the interface only if the packet’s VLAN tag matches the specified VLAN tag when the specified mask is applied. The X-Series Platform applies the mask in binary format, where 0’s indicate “wildcard” bits.

A packet’s VLAN tag matches the specified VLAN tag if all their non-wildcard bits match. To apply the ACL action only to packets with the specified VLAN tag, use a mask of 0xFFF. To apply the ACL action without considering a packet’s VLAN tag, use a mask of 0x000.

Ethernet protocol (ether-type)

<Ethernet_type_code> Defines Ethernet type filtering criteria. Specify the Ethernet type code in hexidecimal format. Valid values are from 0x0000 to 0xFFFF. The NPM applies the ACL action to a packet passing through the interface only if the packet’s Ethernet type code matches the specified Ethernet type code.

<Ethernet_type_code> <mask> Defines Ethernet type filtering criteria, using the specified Ethernet type code and the specified mask. specify the Ethernet type code and mask in hexidecimal format. Valid values for <Ethernet_type_code> are from 0x0000 to 0xFFFF, valid values for <mask> are from 0x0000 to 0xFFFF. The X-Series Platform applies the mask in binary format, where 0’s indicate “wildcard” bits.

A packet’s Ethernet type code matches the specified Ethernet type code if all their non-wildcard bits match. To apply the ACL action only to packets with the specified Ethernet type code, use a mask of 0xFFFF. To apply the ACL action without considering a packet’s Ethernet type code, use a mask of 0x0000.

Source-mac

<source_MAC_address> The NPM applies the ACL action to a packet only if its source MAC address matches the specified source MAC address. Specify the source MAC address using the standard hexidecimal address format (aa:bb:cc:dd:ee:ff).

NOTE: Configure an ACL filter with the source-mac command using the direction ingress-only command. A warning will be displayed if the direction is specified as egress-only or bidirectional.

<source_MAC_address> <mask> You must specify the source MAC address and mask using the standard hexidecimal address format (aa:bb:cc:dd:ee:ff).

NOTE: The X-Series Platform applies the mask in binary format, where 0’s indicate “wildcard” bits.

A packet’s source MAC address matches the specified source MAC address if all their non-wildcard bits match. To apply the ACL action only to packets with the specified source MAC address, use a mask of ff:ff:ff:ff:ff:ff. To apply the ACL action without considering a packet’s source MAC address, use a mask of 00:00:00:00:00:00.

Destination-mac

NOTE: Configure an ACL filter with the destination-mac command using the direction ingress-only command. A warning will be displayed if the direction is specified as egress-only or bidirectional.

<destination_MAC_address> The NPM applies the ACL action only to packets whose destination MAC addresses match the specified filtering criteria. You must specify the destination MAC address using the standard hexidecimal address format (aa:bb:cc:dd:ee:ff).

<destination_MAC_address> <mask> The NPM applies the ACL action to a packet only if its destination MAC address matches the specified destination MAC address when the specified mask is applied. You must specify the destination MAC address and mask using the standard hexidecimal address format (aa:bb:cc:dd:ee:ff).

NOTE: The X-Series Platform applies the mask in binary format, where 0’s indicate “wildcard” bits.

A packet’s destination MAC address matches the specified destination MAC address if all their non-wildcard bits match. To apply the ACL action only to packets with the specified destination MAC address, use a mask of ff:ff:ff:ff:ff:ff. To apply the ACL action without considering a packet’s destination MAC address, use a mask of 00:00:00:00:00:00.


ActionsThe actions direct the NPM about how to handle the traffic that meets the ACL filter criteria, and are specified inline while configuring the ACL with the interface. See Packet Mirroring on the NPM Interfaces on page 73 for details about each of the actions and configuring the interfaces.

Configuring Access Control ListsOnce the ACL has been created and the filters defined, use the following command to assign the ACL to an interface and define the action the NPM will take with the traffic that meets the filter criteria.

acl-interface-mapping <ACL_filter_name> {drop | mirror {gigabitethernet | 10gigabitethernet} <slot>/<port> | pass-through {gigabitethernet | 10gigabitethernet} <slot>/<port> | capture}

The following actions can be specified:

Mirror: This action copies packets to one or more interfaces. Often the interface is connected to an IDS or packet sniffer that performs an analysis of the interface traffic. This functionality is identical to SPAN port functionality.

Pass-through: This action will bypass any applications or functions of the X-Series Platform and will pass traffic directly to an outgoing interface or interfaces. One consideration when using pass-through; Pass-through traffic will not be sent to internal interfaces or VAPs. This includes traffic for protocols that are critical to network and interface connectivity, such as LACP and ARP. Do not configure an ACL to filter and pass-through all traffic on an interface or the network may shut down.

Drop: Instructs the NPM to drop traffic that meets the filter criteria. If traffic coming from a VLAN, or originating from a MAC address has been identified as malicious, the ACL can specify that location and instruct the NPM to drop the traffic. As with the pass-through action, use care when specifying the drop action. If necessary network protocol information is dropped from the interface it may cause the network, or section of the network, to shut down.

Capture: Primarily a troubleshooting function, it instructs the NPM to copy all packets that match the criteria defined in the ACL filter to the local eth2 interface on the NPM. You can use tcpdump to inspect packets captured. By default, the eth2 interface is down. You will have to use ifconfig to bring the eth2 interface up and perform the tcpdump. Once the dump is complete, return the eth2 interface to the down state.

ACL Configuration ConsiderationsACLs can be used in the following manner:

Remote mirroring (the mirrored port is on a different NPM than the source port) is not supported.

Remote pass-through (the destination port is on a different NPM than the source port) is not supported.

An interface or multi-link group-interface must be configured before it is used as a target to pass-through or mirror traffic.

Multiple ACL filters can be configured for the same interface. When a packet arrives on an interface, the NPM applies the filters to the packet one at a time, in the order that the filters were configured.

To reconfigure the precedence of acl-interface-mappings, delete the mappings and redefine them in the order you want.

Multiple interfaces can be configured with the same ACL filter.

An ACL filter can be configured to mirror traffic to multiple interfaces on the same NPM.

An interface cannot mirror or pass-through traffic to itself.

An interface can be configured to mirror or pass-through traffic on a port that is part of a multi-link group-interface, so long as both interfaces are on the same NPM.

A multi-link group-interface can mirror traffic to a single interface, if all group member interfaces and the target mirror interface are on the same NPM.


An interface which is part of a multi-link group interface cannot mirror traffic to another interface. Instead, mirror the traffic from the whole group to another interface

Use the no acl-interface command to remove the specified ACL from a physical interface.

IMPORTANT: If multiple ACLs are defined for the physical interface, the no parameter will only delete the most recently defined ACL.

Use the following show commands to view the criteria for the ACL:

show running-config to determine the order in which ACLs are defined for a physical interface.

show acl-interface to display the conditions defined for each ACL that is currently configured on the X-Series Platform.

show acl-interface-mapping to display a list of the ACLs assigned to each physical interface, and display the action configured for each ACL.

Single Outgoing Interface Example The following example configures an ACL interface filter named single_acl to filter only packets on ingress from VLAN 201. The filter is then applied to the interface below, and assigned an action.

CBS# configure acl-interface single_aclCBS(conf-acl-intf)# direction ingress-onlyCBS(conf-acl-intf)# vlan 201

The following command configures gigabitethernet port number 1 on the NPM installed in slot number 1 to pass traffic between the X-Series Platform and an external network. This command also places you in the conf-intf-gig context in which you can configure settings for the gigabitethernet interface, and create and configure logical interfaces for that interface.

CBS# configure acl-interface-mappingCBS(conf-acl-intf-map)# interface gigabitethernet 1/1CBS(conf-acl-map-intf-gig)# acl-interface single_acl mirrorCBS(conf-acl-map-intf-gig-mirror)# gigabitethernet 1/10CBS(conf-acl-map-intf-gig-mirror)#

The show command displays the ACL configuration:

CBS# show acl-interface-mapping Primary (interface/group) : gigabitethernet 1/1ACL Interface : single_aclaction : mirrorDestination interface(s) : gigabitethernet 1/10

The NPM inspects all packets arriving on gigabitethernet interface 1/1, and copies all packets that match the criteria defined in the ACL interface filter called single_acl to gigabitethernet interface 1/10.

Multiple Outgoing Interface ExampleThe following example configures an ACL interface filter named multiple_acl to filter only packets on ingress. The filter is then configured to mirror traffic to multiple ports on the same NPM.

IMPORTANT: Packet mirroring cannot send traffic to remote NPMs. Packets can only be mirrored to ports on the same NPM.

CBS# configure acl-interface multiple_aclCBS(conf-acl-intf)# direction ingress-onlyCBS(conf-acl-intf)# exitCBS# configure acl-interface-mappingCBS(conf-acl-intf-map)# interface gigabitethernet 1/1CBS(conf-acl-map-intf-gig)# acl-interface multiple_acl mirrorCBS(conf-acl-map-intf-gig-mirror)# gigabitethernet 1/10CBS(conf-acl-map-intf-gig-mirror)# 10gigabitethernet 1/11


CBS(conf-acl-map-intf-gig-mirror)# 10gigabitethernet 1/12CBS(conf-acl-map-intf-gig-mirror)# end

The show command displays the ACL configuration:

CBS# show acl-interface-mapping Primary (interface/group) : gigabitethernet 1/1ACL Interface : multiple_aclaction : mirrorDestination interface(s) : gigabitethernet 1/10, 10gigabitethernet 1/11,

10gigabitethernet 1/12(1 row)

Group Interface ExampleThe following example configures an ACL interface filter named vlan70, and then mirrors all vlan 70 traffic from group-interface mlt to port 2/7.

CBS# configure acl-interface vlan70CBS(conf-acl-intf)# direction ingress-onlyCBS(conf-acl-intf)# vlan 70CBS(conf-acl-intf)# end

CBS# configure group-interface mltCBS(conf-group-intf)# interface-type gigabitethernetCBS(conf-grp-intf-gig)# exitCBS(conf-group-intf)# mode multi-link circuit gr10CBS(conf-group-intf)# interface 2/9CBS(conf-group-intf)# interface 2/10CBS(conf-group-intf)# logical vlan70 circuit vlan70CBS(conf-group-intf)# end

CBS# configure acl-interface-mappingCBS(conf-acl-mapping)# group-interface mltCBS(conf-acl-map-group-intf)# acl-interface vlan70 mirrorCBS(conf-acl-map-group-intf-mirror)# gigabitethernet 2/7CBS(conf-acl-map-group-intf-mirror)# end

Overlapping ACL Filters ExampleThe following example illustrates the behavior when different ACL filters are configured on the same port. In this case, traffic coming into port 2/1 on vlan 70 (the acl filter vlan70 explicitly handles vlan 70) is mirrored to ports 2/6 and 2/9. All other traffic coming in to port 2/1 is mirrored to port 2/5. The order in which the filters are mapped to the ports determines the priority in which they are applied to the traffic on the port.

CBS# configure acl-interface vlan70CBS(conf-acl-intf)# direction ingress-onlyCBS(conf-acl-intf)# vlan 70CBS(conf-acl-intf)# exit

CBS# configure acl-interface allCBS(conf-acl-intf)# direction ingress-onlyCBS(conf-acl-intf)# exit

CBS# configure acl-interface-mappingCBS(conf-acl-mapping)# interface gigabitethernet 2/1CBS(conf-acl-map-group-intf)# acl-interface vlan70 mirrorCBS(conf-acl-map-group-intf-mirror)# gigabitethernet 2/6CBS(conf-acl-map-group-intf-mirror)# gigabitethernet 2/9CBS(conf-acl-map-group-intf-mirror)# exitCBS(conf-acl-map-group-intf)# acl-interface all mirrorCBS(conf-acl-map-group-intf-mirror)# gigabitethernet 2/5CBS(conf-acl-map-group-intf-mirror)# end


Overlapping Port ExampleThe following is an example of overlapping port mirroring. In this case, vlan 101 is mirrored to the interfaces gigabitethernet 4/10, and 10gigabitethernet 4/11, 4/12. An additional acl-interface filter is configured to mirror traffic from port 4/1 to port 4/11. Since the first filter configured (vlan101) includes 4/11, traffic will be mirrored to all ports.

CBS# configure acl-interface vlan101CBS(conf-acl-intf)# direction ingress-onlyCBS(conf-acl-intf)# vlan 101CBS(conf-acl-intf)# exitCBS# configure acl-interface vlan102CBS(conf-acl-intf)# direction ingress-onlyCBS(conf-acl-intf)# vlan 102CBS(conf-acl-intf)# exitCBS# configure acl-interface-mappingCBS(conf-acl-intf-map)# interface gigabitethernet 4/1CBS(conf-acl-map-intf-gig)# acl-interface vlan101 mirrorCBS(conf-acl-map-intf-gig-mirror)# gigabitethernet 4/10CBS(conf-acl-map-intf-gig-mirror)# 10gigabitethernet 4/11CBS(conf-acl-map-intf-gig-mirror)# 10gigabitethernet 4/12CBS(conf-acl-map-intf-gig-mirror)# exitCBS(conf-acl-map-intf-gig)# acl-interface vlan102 mirrorCBS(conf-acl-map-intf-gig-mirror)# 10gigabitethernet 4/11CBS(conf-acl-map-intf-gig-mirror)# end

Configuring Chassis Resource ProtectionChassis resource protection provides configuration parameters to prevent malicious traffic from consuming critical NPM resources. User configurable parameters based on TCP flow validation and flow table limits are set to monitor and filter traffic flow. Unvalidated flows can be dropped or aggressively aged-out, preventing the overflow of buffers and flow tables. Additional settings are provided to make effective use of resources for handling fragmented TCP and UDP packets. Basic per packet validation of TCP/IP packets can be configured to identify and potentially drop invalid packets at line rate to protect APM and end-hosts.

These features must be enabled by the user, and are off by default.

TCP Flow ValidationWhen a TCP packet enters the system, XOS creates a new entry in the flow table and sets the status to unvalidated. In the unvalidated state the uni-directional flow has a short idle time-out of 5-60 seconds. When both directions of a TCP connection pass through the chassis, and the three way handshake has completed, both directions of the connection are validated. They then receive a user-specified idle time-out, which may be as much as one hour. By validating TCP flows in this manner, and aggressively aging out the unvalidated flows, the NPM flow table is protected from TCP-based denial of service attacks such as a SYN flood.

Both directions of a TCP connection must flow through the chassis in order to validate the TCP connection. The TCP packet ordering must also be maintained for TCP flow validation on the chassis. For example, in external TAP configurations where the traffic does not flow through the chassis, the NPM may receive re-ordered TCP frames and fail to validate TCP connections.

After a TCP connection has been validated, the TCP FINs and RSTs are validated before being acted upon. This prevents premature and incorrect termination of TCP connections due to spoofed RSTs.

Flow Table LimitsTCP flow validation alleviates denial of service due to TCP based flood attacks. To protect NPM resources from UDP and ICMP based attacks, flow table limits are configured for individual protocols.


UDP and ICMP flows are stateless, and do not have flow setup or termination schemes. New flows are assigned the idle time-out specified in the ip-flow rule. By specifying flow table limits on each flow type, these flows do not consume critical NPM resources, preventing the setup of other flows.

Establishing a Traffic BaselineUse the Greenlight Element Manager to monitor traffic and establish a traffic baseline. This baseline allows you to profile your traffic and identify anomalous traffic patterns, or put measures in place which will prevent attacks such as TCP SYN floods.

1. Open a Web browser and navigate to the Greenlight Element Manager using a secure connection and the IP address or name of your X-Series Platform (https://<IP_address>).

2. Review the Flows View for an overview of flows per protocol.

Figure 5. GEM Flows View

3. Double-click on the pie chart to open the Flow Utilization by Protocol graph. Use this tool to monitor traffic on the X-Series Platform. For additional information on configuring specific utilization graphs, refer to the Greenlight User Assistance.

Figure 6. GEM Flow Utilization by Protocol


Use the following list to understand the type of traffic your network is processing, and the type of traffic that could be a threat.

What is the most common type of traffic: TCP, UDP, ICMP, or Other-IP.

Identify a baseline for normal traffic. Use the Historical data option to look back over time.

Identify the high end of a range. Are there consistent spikes or low spots in the data?

Use the information gathered to configure parameters to monitor, pass, or drop traffic of a specific protocol based on the number of flows passing through the system. For example, if the flow table limit action is set to drop UDP traffic, the following scenario occurs: When the number of new UDP flows reaches a preliminary threshold, new flows are monitored (counted). As the number of new flows increases toward the user-defined limit, the chance of the preemptive dropping of a new flow increases. When the total number of active flows reaches the predefined limit, all new UDP flows will be dropped. Traffic associated with established flows will continue to pass. The system default is to pass all traffic. If the flow table limit is configured for a protocol, and the action is set to pass, then new flows of that protocol will not be dropped even when the protocol flow table limit is passed. Traffic will continue to pass until the flow table resources are consumed.

Fragment HandlingWith IP traffic, packets are mapped to a flow based on protocol, source and destination IP addresses, and source and destination ports. Mapping TCP and UDP packets to flows requires the source and destination port. When TCP and UDP packets are fragmented, the individual fragments must be associated with source and destination ports before mapping the fragments to flows. Some fragmented TCP and UDP traffic is expected and is handled without impact on the system.

A high number of fragmented packets will consume resources and reduce the effective throughput of traffic. Examples of fragmented or nonconforming packets that may be part of an attack are packets with:

Bad or overlapping offsets

Spoofed packet size

High numbers of missing fragments

Duplicate Head of Packet

A high number of small fragments

Fragment handling employs heuristics to process the fragments. This logic makes decisions whether to drop fragments, aggressively age-out fragmented packets, or forward the fragments. When under processing pressure, nonconforming packets are more likely to be dropped.

The fragment handling heuristics are off by default. Crossbeam recommends enabling fragment handling on your X-Series Platform. Once enabled, it can be specifically disabled on a per-logical line basis. For instance, for an IPS application that inspects all traffic, disable fragment handling on the interfaces or logical lines attached to the application.

NOTE: IPv6, ICMP, and Other-IP traffic are not affected by fragment handling settings.

TCP/IP Packet ValidationTCP/IP Packet validation is an inspection process to detect invalid TCP/IP frames. Packet Validation performs the following checks:

TCP/IP header information

TCP/IP checksum

IP option length

Packet size

Flags

A parameter of drop or pass is configured for invalid packets. The default action is to pass all packets. Statistics are maintained for all packet validation checks regardless of the drop or pass parameters.


Resource Protection Workflow One example of a denial of service attack is a SYN flood. Without TCP flow setup validation enabled, this type of attack will fill the NPM flow table with unvalidated traffic, consuming the available resources. With no space in the flow table for new flows, new traffic is dropped. By using a combination of the Greenlight Element Manager to identify a traffic profile, and resource protection parameters to configure flow table settings, you can limit resources devoted to TCP traffic and prevent a denial of service situation resulting from a SYN flood. This workflow provides configuration steps to monitor and preserve NPM resources in the event of flooding-based denial of service attacks.

Configuration steps for packet validation and fragment handling are included as a best practice. These settings will preserve NPM resources against other malicious traffic.

1. Using GEM and the steps in the Traffic Baseline section, establish a baseline for TCP traffic. See Establishing a Traffic Baseline on page 79.

2. From the command line interface, enable resource protection for the chassis.

Command:

CBS# configure chassis-resource-protectionCBS(conf-resource-protection)# enableCBS(conf-resource-protection)#

NOTE: To restore all the resource protection settings to default values, use the no chassis-resource-protection command.

3. TCP flow validation is enabled by default when you enable chassis resource protection.

With TCP flow validation enabled, you have the option of disabling validation on a per-flow-rule basis, using bypass-tcp-flow-setup-validation. Typically, bypass should be set when the topology prevents TCP packets from being recieved in the correct order, for example, when an external tap is configured. When bypassed, the TCP sequence numbers are updated, and FIN and RST sequence numbers are validated before flow removal.

4. Configure the partitions in the NPM flow table using the information from the baseline exercise.

Use the traffic flow baseline for the threshold value. For instance, if your typical traffic pattern uses 60% of your NPM capacity, set that as the flow table threshold. The TCP threshold value reflects how much more of the NPM resources you want to devote to TCP traffic above the 60%.

If your network typically processes UDP traffic, set a limit value here. The longer timeouts assigned to stateless protocols makes them equally susceptible to flood attacks. Assigning a limit guarantees these flows do not consume all the NPM resources and prevent the setup of other flows.

All protocols must be configured, even if the value entered is 0. The flow table partition threshold range is 0-85%. Individual protocol limits are specified as a percentage of the total flow table, and the total for all values must equal 100.

Command:

CBS(conf-resource-protection)# configure flow table partition threshold 60 tcp 30 udp 10 icmp 0 other-ip 0CBS(conf-flow-table-partition)#

5. Configure the action taken when the protocol (TCP in this case) limit is reached. To protect against a SYN flood, set the action to drop.

Command:

CBS(conf-flow-table-partition)# flow-table-profile tcpCBS(conf-rp-table-profile)# table-limit action dropCBS(conf-rp-table-profile)# exitCBS(conf-flow-table-partition)#


6. Configure the action to be taken when UDP partition limit is reached. Set the action to drop.

Command:

CBS(conf-flow-table-partition)# flow-table-profile udpCBS(conf-rp-table-profile)# table-limit action dropCBS(conf-rp-table-profile)# exitCBS(conf-flow-table-partition)# endCBS#

7. Configure TCP/IP packet validation, and set the following actions to drop.

Command:

CBS# configure packet-validationCBS(conf-pkt-validation)# validate-tcp-packet action dropCBS(conf-pkt-validation)# validate-tcp-checksum action dropCBS(conf-pkt-validation)# validate-ip-packet action dropCBS(conf-pkt-validation)# endCBS#

When packet validation is enabled, the individual validation checks are enabled with a default action of pass. These checks verify that the packets are well-formed, and are not corrupted. Setting the action to drop removes invalid packets at the NPM interface and reduces resource consumption on the APMs and the end-hosts. Statistics are maintained for all checks regardless of the configured action.

These checks are applied to all circuits that are connected to external interfaces. Individual circuits can be configured independently of the global settings.

8. Configure fragment handling parameters.

Fragment handling settings are designed to identify common anomalies and, under stressful conditions allow NPM resources to be directed toward “good” traffic. It is recommended to enable fragment handling on the X-Series Platform. In the case of an IPS application that requires visibility into all the traffic, this functionality can be disabled.

Command:

CBS(conf-resource-protection)# fragment-handling-optionsCBS(conf-rp-frag-handling)# exitCBS(conf-resource-protection)# endCBS#

Fragment handling settings are set globally and may be disabled per logical interface.

Resource protection is now configured globally on your X-Series Platform. All traffic will be subject to these parameters.

Disabling Flow Table Limit Action per InterfaceSome interfaces, such as those handling application management traffic, are not typically subject to attack. However, all NPM network interfaces are subject to the global resource protection settings. In the case of a SYN flood on an external interface, if the flow table partitioning action for the protocol is set to drop when the limit is reached, all TCP traffic on all interfaces is dropped. The normal application management traffic will be blocked.

For internal traffic that passes through an interface on the NPM, you can disable the flow table limit action settings on a per-interface basis.

WorkflowTo disable individual flow table limit action settings on an existing, configured interface, use the following steps:

1. From the CLI, enter the configure interface context.

CBS# configure interface gig 1/1


2. Enter the logical and circuit for the interface.

CBS(conf-intf-gig)# logical mgmtCBS(intf-gig-logical)# circuit mgmt

3. Override the flow table limit action for the individual management circuit.

CBS(intf-gig-log-cct)# no flow-table-limit

4. Use end to return to the main CLI context. Repeat the steps for each interface required.

CBS(intf-gig-log-cct)# endCBS#

Additional resource protection settings can be disabled on individual interfaces using the procedure above.

CBS# configure interface gig 1/1CBS(conf-intf-gig)# logical mgmtCBS(intf-gig-logical)# circuit mgmtCBS(intf-gig-log-cct)# ?

[no] flow-table-limit Overrides the global configuration and disables flow table limit per interface

[no] fragment-handling-options Overrides the global configuration and disables fragment handling options per interface

[no] packet-validation Overrides the global configuration and disables packet-validation per interface

Configuring Redundant InterfacesTo create redundant interfaces, specify the name of the master physical interface being backed up, name of the backup physical interface, and failover mode. Both master and backup must be existing configured interfaces. Upon failover, the original MAC address and circuit IP addresses switch over to the backup port. The gratuitous ARP forces Layer 2 devices to relearn the port on which they have learned this MAC address.

IMPORTANT: When configuring interface redundancy, a backup interface in an active-active configuration cannot be configured to pass traffic in the same broadcast domain as the master interface. This means that both the master and backup interfaces cannot be configured to pass the same VLAN traffic, or both pass untagged traffic. In addition, an interface configured with logical-all cannot be used as a redundant interface. An example of a valid active-active configuration would be to have an interface passing VLANs tagged 100 be a backup to an interface passing VLANs tagged 300.

Use the following command to define a master interface and its backup interface, where slot/port is the actual NPM slot and port location:

configure redundancy-interface master {{gigabitethernet | 10gigabitethernet} {slot/port}} backup {{gigabitethernet | 10gigabitethernet} {slot/port}} mac-usage master failovermode <mode>

The failover mode can be set to the parameters described in Table 13.

Table 13. Failover Mode Parameters


no-failover Failover does not occur.

manual-failover Switch to backup interface now.

preemption-on Upon failure of the master physical interface, the backup interface services traffic. The original Master resumes being Master when it becomes available.



The following example configures the interface at port 1 of the NPM in slot 1 to be backed up by port 2 of the same NPM. Should the master interface fail, the backup interface services traffic but the traffic does not switch back to the original master should the backup fail.

CBS# configure redundancy-interface master gigabitethernet 1/1 backup gigabitethernet 1/2 mac-usage masterCBS(conf-intf-redun)# failovermode manual-swback

NOTE: The mac-usage master parameter is required for the redundancy-interface command.

Here is an example of configuring an interface for redundancy:

CBS# configure interface gigabitethernet 4/6CBS(conf-intf-gig)# logical ispxena2CBS(intf-gig-logical)# circuit ispxena2CBS(intf-gig-logical)# endCBS# configure interface gigabitethernet 4/7

CBS# configure redundancy-interface master gigabitethernet 4/6 backup gigabitethernet 4/7 mac-usage master failovermode preemption-off

If you configure a bridge-mode interface for redundancy, the backup interface must be configured as standby-only, as shown in the following example:

CBS# configure interface gigabitethernet 4/6CBS(conf-intf-gig)# logical ispxena2CBS(intf-gig-logical)# circuit ispxena2CBS# configure interface gigabitethernet 4/7CBS(conf-intf-gig)# standby-only

CBS# configure redundancy-interface master gigabitethernet 4/6 backup gigabitethernet 4/7 mac-usage master failovermode preemption-off

To display the currently configured redundant interface, use the following command:

CBS# show redundancy-interfaceMaster Intf Backup Intf Active Intf MacUsage FailOverMode----------- ----------- ----------- -------- ------------gig 1/3 gig 2/3 Master master preemption-ongig 1/7 gig 2/7 Backup master preemption-off

preemption-off Upon failure of the master physical interface, the backup interface services traffic. The original Master does not resume being Master when it becomes available, unless the backup fails.

manual-swback Upon failure of the master physical interface, the backup interface services traffic. However, the traffic does not switch over to the original master even should the backup interface fail.


Configuring a Multi-Link Group InterfaceYou can aggregate physical links into a group interface using the multi-link mode. All interfaces in a group interface must be of the same physical interface type (gigabitethernet or 10gigabitethernet).

The links are aggregated using Link Aggregation Control Protocol (LACP). To create a multi-link group, you must first create a non-VLAN circuit to use for that group.

NOTE: A multi-link group interface implies a promiscuous-mode and logical-all statement. Multi-link tagged bundles should only be sent across the required VLANs to avoid unnecessary processing.

To view and create group interfaces using EMS, open Configuration > Interfaces > Group Data Interfaces then select Group Physical Interface or Group Logical Interface. Use the Help button for details. In the Command Line Interface, use the configure group interface command.

NOTE: If you have a multi-link group connected to a multi-link group on a different X-Series Platform, each system must have a unique system ID, as set with the configure system-identifier <system-id> command.

Not all settings are used on the 10gigabitethernet interface, as noted.

Table 14. Group Interface Settings for the NPM

Setting Definition

mode {multi-link } circuit <name>

multi-link —Aggregates physical ports into one interface using Link Aggregation Control Protocol (LACP). You must first create a non-VLAN circuit to use for that group.

The circuit <name> assigns an existing circuit to the group interface. Once assigned, this circuit cannot be removed unless you delete then recreate the group interface. Be aware that there are restrictions to the number of circuits that can be assigned to a bridge, as described in Circuit Restrictions for Bridges on page 117.

interface-type Specify the physical interface type for the group. Choices are gigabitethernet or 10gigabitethernet. All interfaces in the group must be the same type.

auto-negotiate Configured under the interface-type context; defaults set based on interface type (gigabitethernet or 10gigabitethernet). Matches best speed for multi-speed devices and uses automatic sensing to support full duplex operation. If using fiber gigabitethernet ports, auto negotiation must be enabled.

duplex-mode Configured under the interface-type context; defaults set based on interface type (gigabitethernet or 10gigabitethernet). Configures half or full duplex mode. For gigabitethernet ports, this is only applicable when the interface is configured for a copper connection.

media-speed Configured under the interface-type context; defaults set based on interface type (gigabitethernet or 10gigabitethernet). Configures media speed. For gigabitethernet ports, this is only applicable when the interface is configured for a copper connection. This setting overrides the auto-negotiation speed.

pause-frame Configured under the interface-type context; defaults set based on interface type (gigabitethernet or 10gigabitethernet). Configures pause frame, also known as flow control.

enable (group interface)

Configured under the interface-type context. The group interface can be disabled or enabled. Default is enabled.

interface slot/port Add individual interfaces to the group. Settings applied to the group are automatically written to the individual interfaces.

enable (individual interfaces)

Configured under each interface context. An interface is enabled if both the group interface and individual interface are enabled. Individual physical interfaces may also be disabled / enabled independent of the group interface.


You can disable and enable a group interface with the [no] enable command:

CBS# config group-interface test interface-type gigabitethernet no enable

You can enable or disable individual physical devices using the enable command, once the device has been specified:

CBS# config group-interface test interface 1/1 no enable

NOTE: When configuring physical link parameters such as speed, MAC address, auto-negotiate, duplex mode, and pause frame for the group interface, the parameters are applied to each physical interface in the group.

By default, the interface is enabled, auto-negotiation is enabled, duplex mode is full, and the interface is not in standby mode.

The following example configures circuit1 for VLAN 2, and configures a second non-VLAN circuit (required). The non-VLAN logical line must be created before the VLAN logical line. When specifying the group interface mode, you must specify a circuit. The circuit is automatically configured with a logical interface that passes all VLAN and non-VLAN traffic.

CBS# configure group-interface labCBS(conf-group-intf)# interface-type gigabitethernetCBS(conf-grp-intf-gig)# exitCBS(conf-group-intf)# mode multi-link circuit circuit2CBS(conf-group-intf)# interface 1/1CBS(conf-grp-intf-intf)# exitCBS(conf-group-intf)# interface 1/2CBS(conf-grp-intf-intf)# exitCBS(conf-group-intf)# interface 1/3CBS(conf-grp-intf-intf)# exitCBS(conf-group-intf)# interface 1/4CBS(conf-grp-intf-intf)# exitCBS(conf-group-intf)# logical logical11 ingress-vlan-tag 2CBS(conf-group-intf-logical)# circuit circuit1

To have traffic coming into this group interface directed to two applications, assign Circuit1 to two VAP groups. For example:

CBS# configure circuit circuit1CBS(conf-cct)# vap-group firewallCBS(conf-cct-vapgroup)# default-egress-vlan-tag 2CBS(conf-cct-vapgroup)# ip-forwardingCBS(conf-cct-vapgroup)# ip 100.123.10.1/8CBS(conf-cct-vapgroup-ip)# exit

CBS# configure circuit circuit1CBS(conf-cct)# vap-group idsCBS(conf-cct-vapgroup)# promiscuous-mode

In the example of directing traffic to two or more VAP groups, you must also assign the LACP circuit to all the VAP groups. Continuing the previous example, Circuit2 is the LACP circuit, and you would assign Circuit2 to both VAP groups as follows:

CBS# configure circuit circuit2CBS(conf-cct)# vap-group firewallCBS(conf-cct-vapgroup)# ip-forwardingCBS(conf-cct-vapgroup)# exit

CBS# configure circuit circuit2CBS(conf-cct)# vap-group idsCBS(conf-cct-vapgroup)# promiscuous-mode

To view existing group interfaces from the CLI, use the show group-interface command.


MAC Address InheritanceIf you apply a user-defined MAC address to the mode multi-link circuit, additional circuits (e.g. VLAN circuits) mapped to the same VAP group inherit the mode circuit’s MAC address automatically. If the user changes the MAC address on the mode circuit, the VLAN circuits will inherit the new MAC address.

If you use the logical command to map additional circuits (e.g. VLAN circuits) to a mode multi-link group interface, the VLAN circuit does not inherit the mode circuit’s MAC address. If the user changes the MAC address on the mode circuit, the VLAN circuits will retain their originally configured MAC addresses.

Interface Status GroupThis command ties the status of interfaces or group-interfaces together. If all interfaces and group-interfaces in an interface-status-group are UP, then the state of the interface-status-group is UP. If any interface or group-interface in the interface-status-group is DOWN, then the interface-status-group state is DOWN.

CBS# configure interface-status-group <group_name>CBS(conf-intf-status-group)# gigabitethernet <slot/port> | 10gigabitethernet <slot/port> | group-interface <name>

When interface-status-group command is enabled, all members of the interface status group will stop passing traffic if one or more members fails.

When interface-status-group is disabled, healthy members will continue to pass traffic if one group interface member fails.

Use show interface-status-group to display the interface status groups and the associated interfaces. Status grouping is disabled by default.



7Configuring Management Interfaces

This chapter provides information on configuring interfaces, including access lists. It includes the following major topics:

Physical Management Interfaces on page 89

Configure Management Physical Interfaces on page 90

Configure Access Lists on page 91

Define an Access List to a Management Interface on page 94

Display Management Access Lists on page 94

Physical Management InterfacesThe physical management interfaces are provided by the Control Processor Modules (CPMs). The management interfaces allow you to manage the configured interfaces. The physical interfaces are identified by chassis slot number of the CPM (13 or 14), physical port number (1 or 2) of the CPM, and interface type (gigabitethernet). Table 15 lists the CPM physical interfaces for each platform.

Table 15. Physical Interfaces for X-Series Platforms

NOTE: Inside NAT does not failover from one CPM management interface to the other CPM. The inside NAT should be explicitly configured identically on both CPM management interfaces.

IMPORTANT: If you configure two management interfaces on the same slot, they cannot be on the same subnet.

X80 / X80-S Chassis Slot #

X60 Chassis Slot #

X20/X30 Chassis Slot

Port # Interface Type Description

13 6 1 gigabitethernet Gigabit Ethernet management port

13 6 2 gigabitethernet Gigabit Ethernet logging port on CP1

14 7 7 1 gigabitethernet Gigabit Ethernet management port

14 7 7 2 gigabitethernet Gigabit Ethernet logging port on CP2


Configure Management Physical InterfacesThe following is an example of displaying the management physical interface configuration:

CBS(conf-mgmt-gig)# showManagement Interface : gigabitethernet 14/1Enabled (true/false) : tIP Address : 172.16.28.42/24Broadcast Address : 172.16.28.255Auto Negotiate Enabled (true/false) : tInput Access List : 1001Output Access List : 1002(1 row)

To configure a management physical interface:

1. Configure the interface type and CPM slot location and port.

Command:

CBS# configure management gigabitethernet 13/2 CBS(conf-mgmt-gig)#

2. Define the MAC address for the management interface using the MAC address of the physical port. If you do not supply the MAC address, the system will either use the NIC MAC, or will automatically generate the MAC address.

Command:

CBS(conf-mgmt-gig)# mac-addr 40:03:d4:55:20:23CBS(conf-mgmt-gig)#

3. Define the Maximum Transfer Unit (MTU) size for the management interface. Specify the size in bytes; the range is 68 to 1536, or use the system default of 1500.

Command:

CBS(conf-mgmt-gig)# mtu 1500CBS(conf-mgmt-gig)#

4. Define the IP address and network mask for the management physical port.

NOTE: You may provide the IP address and the Network Mask in either CIDR format (172.16.10.10/24), or in dotted-decimal format followed by space on the same line (172.16.10.10 255.255.255.0)

Command:

CBS(conf-mgmt-gig)# ip-addr 172.16.10.10 255.255.255.0CBS(conf-mgmt-gig)#

5. Enable the interface as follows:

CBS(conf-mgmt-gig)# enable

6. Define the access list. The access list defines which subnets or specific IP addresses are allowed to connect to and manage the CPM management interface. To create an access list see Configure Access Lists on page 91.

Command:

CBS(conf-mgmt-gig)# access-list 1001 permit ip source-ip 0.0.0.0 255.255.255.255 destination-ip 0.0.0.0 255.255.255.255CBS(conf-mgmt-gig)# access-list 1002 permit ip source-ip 0.0.0.0 255.255.255.255 destination-ip 0.0.0.0 255.255.255.255

Configuring Management Interfaces 90

Configure Access ListsThe access list allows connectivity to the CPM interface from specified networks and hosts, and allows you to specify IP addresses that have access to the CPM management interfaces. By default, all access is denied. In order to manage the X-Series Platform, create an access rule allowing you to perform management functions. An example of a simple access rule would be one that allows:

Access to the system (ssh, telnet, https)

The ability to transfer files or policies (scp/ftp)

ICMP to ping the system when troubleshooting

A list of the management source addresses with access to the system

Permissions for those ip addresses

During the XOS Install Interview, you can choose to allow all traffic, which will automatically create an allow/all rule. This may be safest in a non-production environment, where external access to the system is limited. The following is an example of an allow/all access list.

configure access-list 1001 permit ip source-ip 0.0.0.0 255.255.255.255destination-ip 0.0.0.0 255.255.255.255configure access-list 1002 permit ip source-ip 0.0.0.0 255.255.255.255destination-ip 0.0.0.0 255.255.255.255

TCP/UDP

configure access-list <number> {deny|permit} {tcp|udp} {{source-ip <src-ip-address> <source-wildcard>}|source-any} {source-port-any|source-port-name <name>|source-port <0-65535> [<0-65535>]} {{destination-ip <des-ip-address> <destination-wildcard>}|destination-any} {destination-port-any|destination-port-name <name>| destination-port <0-65535> [<0-65535>]} [log]

ICMP

configure access-list <number> {deny|permit} icmp {{source-ip <src-ip-address> <source-wildcard>}|source-any} {{destination-ip <des-ip-address> <destination-wildcard>}|destination-any} {icmp-message <message>|icmp-type <0-255>} [log]

IP/Protocol Number

configure access-list <number> {deny|permit} {ip|protocol-number <0-255>} {{source-ip <src-ip-address> <source-wildcard>}|source-any} {{destination-ip <des-ip-address> <destination-wildcard>}|destination-any} [log]

NOTE: A wildcard setting of 255.255.255.255 permits any address regardless of the IP address configured. To configure the access list to permit a specific host address, the wildcard used should be 0.0.0.0.

Where:

<number> Number of an access list. This is a decimal number from 0 through 65535.deny Denies access if the conditions are matched.permit Permits access if the conditions are matched.ip|tcp|udp|icmp Protocol name or number.<src-ip-address> IP address of the network or host from which the packet is being sent. Use a 32-bit

quantity in four-part dotted-decimal format.<source-wildcard> Wildcard bits applied to the source. Use a 32-bit quantity in four-part

dotted-decimal format. Places ones in the bit positions you want to ignore.


source-any Abbreviation for a source and source-wildcard of 0.0.0.0 255.255.255.255.source-port-any Configures the source port to be any.source-port <number> Source low/high port-number value. Valid numbers are from 0 - 65535source-port-name <name>

Specifies the source low/high port-name; names are as follows:

ftp-data —File Transfer Protocol Data (20)

ftp — File Transfer Protocol (21)

ssh — Secure SHell (22)

telnet — Telecommunications Network Protocol (23)

smtp — Simple Mail Transfer Protocol (25)

time — Time (37)

domain — Domain Name Server (53)

bootps — Bootstrap Protocol Server Protocol (67)

bootpc — Bootstrap Protocol Client Protocol (68)

tftp — Trivial File Transfer Protocol (69)

http — Hyper Text Transfer Protocol or www or www-http (80)

rtelnet — Remote Telecommunications Network Protocol (107)

pop3 — Post Office Protocol Version 3 (110)

nntp — Network News Transfer Protocol (119)

ntp — Network Time Protocol (123)

imap — Internet Message Access Protocol (143)

snmp — Simple Network Management Protocol (161)

ldap — Lightweight Directory Access Protocol (389)

https — Secure Hyper Text Transfer Protocol (443)

isakmp — Internet Security Association and Key Management Protocol (500)<des-ip-address> IP address of the network or host receiving the packet being sent. There are two

alternative ways to specify the destination:

Use a 32-bit quantity in four-part dotted-decimal format

Use any as an abbreviation for a source and source-wildcard of 0.0.0.0 255.255.255.255

<destination-wildcard> Wildcard bits applied to the destination. Use a 32-bit quantity in four-part dotted-decimal format. Places ones in the bit positions you want to ignore.

destination-any Abbreviation for a destination and destination-wildcard of 0.0.0.0 255.255.255.255.destination-port-any (TCP/UDP Only) Configures the destination port to be any.destination-port-name <name>

(TCP/UDP Only) Specifies the destination low/high port-name. Names are the same as the source-port names.

destination-port <number>

(TCP/UDP Only) Specifies the destination low/high port number. Values are 0 to 65535.


icmp-message <message-name>

Valid message names as listed below:

NOTE: Enter the text string, for example, network-redirect. As an alternative, see the next row in this table for information on how to enter the icmp-type followed by the type number. A list of names, types, and codes is located here: http://www.iana.org/assignments/icmp-parameters

echo-reply —echo-reply (type 0)

destination-unreachable — unreachable (type 3)

network-unreachable — net-unreachable (type 3, code 0)

host-unreachable — host-unreachable (type 3, code 1)

protocol-unreachable — protocol-unreachable (type 3, code 2)

port-unreachable — port-unreachable (type 3, code 3)

fragmentation-needed — packet-too-big (type 3, code 4)

source-route-failed — source-route-failed (type 3, code 5)

network-unknown — network-unknown (type 3, code 6)

host-unknown — host-unknown (type 3, code 7)

network-prohibited — dod-net-prohibited (type 3, code 9)

host-prohibited — dod-host-prohibited (type 3, code 10)

TOS-network-unreachable — net-tos-unreachable (type 3, code 11)

TOS-host-unreachable — host-tos-unreachable (type 3, code 12)

communication-prohibited — administratively-prohibited (type 3, code 13)

host-precedence-violation — host-precedence-unreachable (type 3, code 14)

precedence-cutoff — precedence-unreachable (type 3, code 15)

source-quench — source-quench (type 4)

redirect — redirect (type 5)

network-redirect — net-redirect (type 5, code 0)

host-redirect — host-redirect (type 5, code 1)

TOS-network-redirect — net-tos-redirect (type 5, code 2)

TOS-host-redirect — host-tos-redirect (type 5, code 3)

echo-request — echo (type 8)

router-advertisement — router-advertisement (type 9)

router-solicitation — router-solicitation (type 10)

time-exceeded — time-exceeded (type 11)

ttl-zero-during-transit — ttl-exceeded (type 11, code 0)

ttl-zero-during-reassembly — reassembly-timeout (type 11, code 1)

parameter-problem — parameter-problem (type 12)

ip-header-bad — general-parameter-problem (type 12, code 0)

required-option-missing — option-missing (type 12, code 1)

timestamp-request — timestamp-request (type 13)

timestamp-reply — timestamp-replace (type 14)

address-mask-request — mask-request (type 17)

address-mask-reply — mask-reply (type 18)


http://www.iana.org/assignments/icmp-parameters

Define an Access List to a Management InterfaceTo apply an access list to a management interface, use the following command:

configure management <interface> <slot/port> access-list <list-number> [input|output]

Where:

Display Management Access ListsTo display the current configured access lists, use the following command:

CBS# show access-listAccess List : 1002Access : permitProtocol : ipSource IP and Wildcard : 0.0.0.0 255.255.255.255Destination IP and Wildcard : 172.16.0.0 0.0.255.255Log Enable (true/false) : f

Final StepsIf you are going to be managing the X-Series Platform from an external subnet, you must configure a management port default gateway. While this is generally understood for external management, it is included in the Install Interview questions, and worth noting here.

Command:

CBS# configure management default-gateway 172.16.19.63CBS(conf-mgmt)# endCBS#

icmp-type <type-number>

Configures the ICMP message matching criteria for the ACL to include only packets with the specified message type.

The X-Series Platform applies the ACL’s action (deny or permit) to a packet only if its message type number matches the specified message type number.

Valid message type numbers are from 0 to 255.log Enables logging for this access list.

interface Type of interface, gigabitethernet or high-availabilityslot/port Actual CPM slot and port location of the interfacelist-number Access list to use on this interfaceinput/output Specifies that the access list affects incoming or outgoing traffic on this interface



8Configuring Flow Provisioning

This chapter describes using and creating IP and non-IP flow rules and QoS rate-limiter rules. It contains the following major topics:

Flow Rule Overview on page 95

Configuring IP Flow Rules on page 101

Configuring System IP Flow Rules on page 103

Configuring System Non-IP Flow Rules on page 106

Flow Rule OverviewA flow rule determines how a new traffic flow is processed when it arrives on a logical interface. A traffic flow is identified by a set of parameters in the flow rule, configured by the user. There are four types of flow rules:

IP Flow Rule - processes IP traffic at the VAP level.

Non-IP Flow Rule - processes non-IP traffic at the VAP level.

System IP Flow Rule - affects all IP traffic at the system level, which affects all VAPs.

System Non-IP Flow Rule - affects all non-IP traffic at the system level.

All IP flow rules are part of the IP flow classifier system module. The IP flow classifier is designed to be an IP delivery, policy enforcement, and provisioning module. Its default behavior is to direct IP traffic destined to the X-Series Platform to the correct slot (or VAP) for all the configured IP addresses.

VAP IP Flow RulesThe NPM uses the IP flow rules configured for a VAP group to determine how to process IP traffic destined for the members of that VAP group. Each flow rule is associated with a specific VAP group, and multiple flow rules may be created for each VAP group. These flow rules can identify traffic based on the five fields in the IP packet:

IP source address

IP destination address

Protocol, which indicates the destination upper-layer protocol layer for this datagram. For example, ICMP is 1, IGMP is 2, TCP is 6, and UDP is 17. The default is all protocols (1-255).

Source port

Destination port

In addition, traffic can be identified by the Domain ID and Incoming Circuit Group (ICG) values that define the physical location of the flow in the X-Series Platform.

Each of these values can be a single value or a range of values. For example, a flow rule can be applied to a range of IP source addresses such as 192.100.100.000 to 192.100.100.255. This would be noted in CIDR format as 192.100.100.000/24.

The Incoming Circuit Group (ICG) is used by the NPM as an additional method to classify traffic for one or more circuits. When an ICG number is assigned or changed in the circuit configuration, it will impact flow classification on the NPM. If you change an ICG circuit assignment, verify that the appropriate flow rules have been configured or updated.

The flow of traffic through a VAP or VAP group can be configured using the following actions and other criteria:

load-balance

drop

allow

pass-to-master

pass-to-vap

broadcast

direction

skip-port-protocol

generate-reversed-flow

source-addr

destination-addr

source-port

destination-port

core-assignment

Refer to Chapter 6 of the XOS Command Reference Guide for detailed information configuring flow rules.

Each flow rule is assigned a priority, as described in Priorities for IP Flow Rules on page 100. A flow can match different flow rules. However, only the flow rule with the highest priority is used. It is possible to have many rules at the same priority as long as they do not conflict. Flow rules are considered to be in conflict if they have the same priority, are assigned to the same VAP group, and a packet could match both flow rules. For example, these two flow rules are in conflict if they are applied to the same VAP group:

Rule 1: Dest-IP 20.0.0.1, Priority 10 (default)

Rule 2: Dest-Port 80, Priority 10

Since it is possible to receive a packet destined to 20.0.0.1 on port 80, the rules are in conflict. Simply changing the priority of either rule removes the conflict.

To check for flow rule conflicts, enable the check-flow-rule command. When enabled, each time a flow rule is activated, XOS checks for policy conflicts between that flow rule and the flow rules that are currently activated on the X-Series Platform. If XOS detects a policy conflict, the activate operation fails and the CLI issues an error message.

NOTE: If flow rule checking has been disabled and is then enabled, all the activated flow rules are checked for conflicts.

The following parameters are secondary actions that can be configured for flow rules:

The skip-port-protocol parameter is often used for load balancing purposes. When enabled (default), the NPM ignores the destination port, source port, and protocol in the IP packet when deciding how to load balance a new flow. Therefore, any flows with the same source and destination IP addresses will be assigned to the same VAP.

The timeout for a VAP-level IP flow rule determines the idle timer for a flow. If traffic for any flow is idle for this period of time, the NPM deletes that flow from the Active Flow Table. If another packet belonging to that flow is sent after this deletion, the NPM considers this an unknown flow and processes the packet to determine which flow rules apply and where to send the flow. This could result in moving the flow to another VAP, which could be a problem when using a stateful firewall.

Configuring Flow Provisioning 96

Timeout values can be set from 30 seconds to 1 hour, or auto (default). The auto timeout value is set by the system. By default, the system defines a timeout value of 10 minutes for all TCP packets, 1 minute for UDP packets, and 30 seconds for all other protocols.

VAP IP flow rules are configured with the configure vap-group <group-name> ip-flow-rule <name> command. After creating a flow rule, you must use the activate command to enable it; otherwise the flow rule does not take affect. When you activate a flow rule, it is not applied to existing flows, it only affects new flows.

VAP Non-IP Flow RulesNon-IP flow rules are configured at the VAP group level. The non-IP flow rules are configured with the configure vap-group <group-name> non-ip-flow-rule command and apply to all non-IP traffic coming into the VAP group.

These flow rules identify non-IP traffic by encapsulation type. The encapsulation can be Ethernet, LSAP, or SNAP, along with a protocol number. The Ethernet and SNAP protocols can be 1519 to 65535, and LSAP with DSAP or SSAP can be 0-255. The any keyword can be used with each protocol to allow any protocol number (wildcard). For LSAP, the any keyword denotes all DSAP and SSAP combinations. The non-IP flow rules can be configured using the following criteria:

action drop

action broadcast

action pass-to-master

encapsulation ethernet

encapsulation lsap

encapsulation snap

activate

core-assignment

The VAP non-IP flow rule settings can be seen with the show non-ip-flow command. Refer to Chapter 6 of the XOS Command Reference Guide for detailed information configuring flow rules.

IPv6 Non-IP Flow Rule

When you enable IPv6 on the VAP group, a non-ip flow rule is automatically created. The non-ip flow rule ipv6_rule is configured as follows:


IPv6 traffic is classified as non-ip traffic and passed to the master VAP. Because this traffic is passed to the master VAP, traffic is not load balancing across multiple VAPs. If you anticipate a high volume of IPv6 traffic, use the core-assignment parameters under the VAP group context to load-balance IPv6 traffic across multiple cores on the master VAP.

For VAP group and core-assignment configuration steps, refer to Configuring a VAP Group on page 52.

Default VAP Flow RulesThe IP flow classifier automatically generates an IP flow rule when you assign an IP address to a VAP group, using the configure circuit command. Depending on the IP address, one of the following rules is created:


The IP Flow Rule action will be set to broadcast when the IP address subnet is set to a broadcast address. Secondary actions are set to default and the priority is 21.

For other IP addresses, the action is to load balance, using the IP address as the destination address. Secondary actions are set to default and the priority is 21.

When using the IP address with increment-per-vap, the original address is sent to VAP index 1. The next IP address in sequential order is sent to VAP index 2, and so on. The flow is not load balanced. Secondary actions are set to default, timeout is set to auto, and the priority is 21.

When using the IP address with increment-per-vap, the IP flow rule action is set to broadcast for packets with a broadcast address. Otherwise, the default flow-rule is used. Priority is 21.

These rules cannot be modified directly; however, you can modify the default priority using the ip-flow-rule-priority parameter when assigning a circuit to a VAP group. For example, you can change the default priority of 21 using the following command:

configure circuit <name> vap-group <name> ip-flow-rule-priority <value>

Use the show default-ip-flow-rule command to view the default IP flow rule settings.

System IP Flow RulesSystem-wide IP flow rules are configured with the configure system-ip-flow-rule command and apply to all traffic coming into the system. The system IP flow rules can be configured using the following actions and other criteria:

drop

allow

pass-to-masters

broadcast

direction

skip-port-protocol

generate-reversed-flow

source-addr

destination-addr

source-port

destination-port

Refer to Chapter 6 of the XOS Command Reference Guide for detailed information configuring flow rules.

System IP flow rules have the same parameters as the VAP IP flow rules, which are discussed in the previous section. When a flow matches a system flow rule and a VAP-level flow rule, the flow rule with the higher priority takes precedence. If the priority is the same, the system IP flow rule applies. The exceptions, such as the timeout value, are explained in Interaction Between System and VAP IP Flow Rules on page 99.

System IP flow rules must be configured when local IP host addresses (or pools of addresses) exist on the X-Series Platform by virtue of a third party proxy or VPN application that are not otherwise visible to the X-Series Platform configuration.


The default system IP flow rule settings can be seen with the show default-ip-flow-rule command. You can also change the default timeout values for specific IP addresses, destination ports, and so on. Refer to the XOS Command Reference Guide for details on XOS command usage, syntax, options, and output.

CBS# show system-ip-flow-ruleSystem IP Flow Rule : myruleDestination Address : 0.0.0.0Destination Address High : 255.255.255.255Destination Port : 0Destination Port High : 65535Source Address : 0.0.0.0Source Address High : 255.255.255.255Source Port : 0Source Port High : 65535Incoming Circuit Group : 1Protocol : 1Protocol High : 255Domain : 1Domain High : 4095Action : dropActivate (true/false) : fPriority : 10Skip Protocol (true/false) : tSkip Port (true/false) : tSkip Port Protocol (true/false) : tTimeout : auto

If the default timeout value for a specific rule is changed, you must verify that the timeout is set properly for all return flows. In the following example, the system default has been changed to 1 hour for all TCP packets. Because HTTP (TCP port 80) typically has short timeouts, create a higher priority rule with a destination port of 80 for the TCP packets, and assign a timeout value of 3 minutes. You must also create a flow rule using source port 80 with a timeout of 3 minutes. Without this flow rule, the return flow, which is not bound for destination port 80, resets the timeout of the active flow entry from 3 minutes to 1 hour. This occurs because the original flow and the return flow share the same active flow entry even though they use two different rules.

Interaction Between System and VAP IP Flow RulesIP flow rules are matched only if all of the corresponding IP fields and incoming circuits are within the range of the IP flow rule. If more than one IP flow rule is matched, the flow rule with the highest priority rule (with primary action other than “none”) is selected and only independent secondary actions matching the flow rule are incorporated or merged into this rule.

The timeout value is normally set by the system IP flow rule. However, if a flow matches both a VAP-level flow rule and system IP flow rule that ONLY defines the timeout value, the timeout value of the VAP-level flow rule applies regardless of priority. For example, you create and activate a VAP group flow rule that has a timeout interval of 30 seconds for TCP packets and a priority of 15. You also have a system IP flow rule with a timeout of 20 minutes for TCP packets and a priority of 25. The system IP flow rule does not define any parameter other than the timeout value. The result is that all TCP packets for that flow timeout in 30 seconds.

If a flow only matches a VAP-level IP flow rule and system flow rule that both have a timeout value of auto, the IP flow classifier decides the best timeout from a scaling point of view, currently set at 1 minute.

Multi-VAP Group IP Flow Rule MatchAn IP flow rule is assigned to a specific VAP. If an IP flow rule is not specifically assigned, it is a system rule. In multi-VAP group environments, the flow rule classifier builds basic flow rule matches per VAP group, then merges these matches from each VAP group and system VAP group.


Merging matched IP flow rules from different VAP groups results in different actions, selected based on the circuit of the classified packet. Classification always occurs on a particular interconnect of the X-Series Platform and is a function of the VAP group reachability from this interconnect.

Table 16 summarizes the merger/match process.

Table 16. Summary of Merger/Match Process

When matches from multiple groups are merged, secondary actions are merged based on the secondary action merge flow rule. Each primary action from selected matches is considered in the context of the corresponding VAP group.

System Non-IP Flow RulesThe system non-IP flow rules are configured with the configure system-non-ip-flow-rule command and apply to all non-IP traffic coming into the system.

These flow rules identify non-IP traffic by encapsulation type. The encapsulation can be Ethernet, LSAP, or SNAP, along with a protocol number. The Ethernet and SNAP protocols can be 1519 to 65535, and LSAP with DSAP or SSAP can be 0-255. The any keyword can be used with each protocol to allow any protocol number (wildcard). For LSAP, the any keyword denotes all DSAP and SSAP combinations. The system non-IP flow rules can be configured to take the following actions:

The system non-IP flow rule settings can be seen with the show system-non-ip-flow-rule command.

Priorities for IP Flow RulesEach IP flow rule has an associated priority level. This priority is used to differentiate rules with over-lapping criteria. Valid priority level values are from 0 (lowest priority) to 31 (highest priority) with some being allocated to users and others to the IP flow classifier. Only priorities in the range of 10 through 20 and 25 through 30 are configurable. The rest are for IP flow classifier use only.

VAP Group Directly Reachable through Ingress Circuit

VAP Group NOT Directly Reachable

System Rules

Circuit of the VAP group is in promiscuous mode.

Result of the basic match from this VAP group is considered.

Only the primary action from this match is considered but not applied on the current circuit.

Result of the basic match is considered.Circuit of the VAP group is in

non-promiscuous mode; destination MAC-address is local or broadcast.

Result of the basic match from this VAP group is considered only if the primary action is of the highest priority in comparison with other matches; otherwise, the result is the same as for a NOT directly reachable VAP group.

Circuit of the VAP group is in non-promiscuous mode; destination mac-address is external.

None of the rules are considered.

broadcast Send the traffic flow to all VAPs. Should be used when non-IP traffic is meant to terminate on the XOS system.

drop Drop packets. This is the default action.

pass-to-masters Pass the traffic to the master VAPs of all VAP groups. Should be used when non-IP traffic is meant to pass through the XOS system.


NOTE: The ip-flow-rule-priority parameter is used when assigning a circuit to a VAP group. This is the only time you can configure a priority for a flow rule. The priority ranges are 10-20 and 25-30.

The lower range of priority (10 through 20) is designed for all normal operations of the X-Series Platform. You can define flow rules to load-balance traffic across particular VAPs of the VAP group or select VAPs based on other criteria, like load. The higher range (25 to 30) is designed to over-write the system default configuration; to redefine normal IP forwarding rules; or to restrict a particular type of the traffic. Flow rules with priorities in this range are considered critical rules.

NOTE: If there is an overlap of a VAP group level flow rule and a system non-IP flow rule where both handle protocol or encapsulated traffic, the VAP group flow rule always has the higher priority. In addition, a flow rule that contains a specific protocol or encapsulation type has a higher priority than a flow rule that uses the any parameter to specify the protocol or encapsulation type.

Configuring IP Flow RulesTo configure VAP group-specific IP flow rules using the EMS, open Configuration > Flow Provisioning > IP Flow Rules. The following example configures IP flow rules using the CLI. In this example, there are three flow rules, all assigned to VAP group firewall.

Flow rule fw_lb load balances all IP traffic. It will match any protocol, and source and destination addresses and ports. It is configured for a 10 minute timeout interval, where if traffic for this flow is idle for 10 minutes then the NPM will delete that flow from the Active Flow Table.

Flow rule bad_host drops all traffic from source address 10.1.2.3, and flow rule no_icmp drops all traffic destined for ICMP (protocol 1).

CBS# configure vap-group firewall ip-flow-rule fw_lbCBS(ip-flow-rule)# action load-balanceCBS(ip-flow-rule)# timeout 10-minutesCBS(ip-flow-rule)# activateCBS(ip-flow-rule)# end

CBS# configure vap-group firewall ip-flow-rule bad_hostCBS(ip-flow-rule)# action dropCBS(ip-flow-rule)# priority 29CBS(ip-flow-rule)# source-addr 10.1.2.3CBS(ip-flow-rule)# activateCBS(ip-flow-rule)# end

CBS# configure vap-group firewall ip-flow-rule no_icmpCBS(ip-flow-rule)# action dropCBS(ip-flow-rule)# priority 30CBS(ip-flow-rule)# protocol 1CBS(ip-flow-rule)# activate

All of these rules can match ICMP; the first 2 rules default to protocols 1-255 while the last rule matches protocol 1 explicitly. The rules “fw_lb” and “bad_host” both match the source address of 10.1.2.3. If these rules had the same priority, they would conflict. Therefore, the priority of the “fw_lb” rule is at the default of 10, “bad_host” is at 29, and “no_icmp” is at 30. Refer to the XOS Command Reference Guide for the full syntax of the ip-flow-rule command.

The following IP flow rule example shows how to ensure that all FTP traffic (which uses ports 20 and 21) is sent to VAP #1 in the group. Since a packet can possibly have both a source AND destination of port 20, the rules must be set to different priorities. The same is true for port 21. However, since a packet cannot have a destination port of both 20 and 21, these rules can be the same priority.


vap-group vap1 xslinux_v3 max-load-count 1 ap-list ap3 ap4 ap5 ap6 ap7 ap8 ap9 ap10 load-balance-vap-list 1 2 3 4 5 6 7 8 9 10 ip-forwarding ip-flow-rule vap1_lb action load-balance activate ip-flow-rule ftp_data_src action pass-to-vap 1 priority 11 source-port 20 20 activate ip-flow-rule ftp_data_dst action pass-to-vap 1 priority 12 destination-port 20 20 activate ip-flow-rule ftp_ctrl_src action pass-to-vap 1 priority 11 source-port 21 21 activate ip-flow-rule ftp_ctrl_dst action pass-to-vap 1 priority 12 destination-port 21 21 activate

Use the show ip-flow-rule <flow-rule-name> command to display an IP flow rule and its parameters. If no flow rule is specified, all system IP flow rules are displayed.

CBS# show ip-flow-ruleIP Flow Rule : flow1VAP Group : fwDestination Address : 0.0.0.0Destination Address High : 255.255.255.255Destination Port : 0Destination Port High : 65535Source Address : 0.0.0.0Source Address High : 255.255.255.255Source Port : 0Source Port High : 65535Incoming Circuit Group : 1Protocol : 1Protocol High : 255Domain : 1Domain High : 4095Action : load-balanceActivate (true/false) : tPriority : 10Skip Protocol (true/false) : tSkip Port (true/false) : tSkip Port Protocol (true/false) : tTimeout : autoTrace (true/false) : fGenerate Reversed Flow (true/false) : fDirection : bothBypass-tcp-flow-setup-validation (true/false) : fCore Assignment : random-single-core


Configuring Non-IP Flow RulesTo configure VAP group-specific non-IP flow rules using the EMS, open Configuration > Flow Provisioning >Non IP Flow Rules. The following example configures a non-IP flow rule using the CLI. In this example, the drop_traffic flow rule drops snap traffic.

CBS# configure vap-group red non-ip-flow-rule drop_trafficCBS(ip-flow-rule)# encapsulation ethernet type 2000CBS(ip-flow-rule)# action dropCBS(ip-flow-rule)# activateCBS(ip-flow-rule)# end

Configuring System IP Flow RulesTo configure system level IP flow rules using the EMS, open Configuration > Flow Provisioning > System IP Flow Rules. The following example configures IP flow rules using the CLI. In this example, the tcp_traffic flow rule sets the timeout period to 20 minutes for all TCP (protocol 6) traffic.

CBS# configure system-ip-flow-rule tcp_trafficCBS(conf-system-ip-flow)# action dropCBS(conf-system-ip-flow)# timeout 20-minutesCBS(conf-system-ip-flow)# protocol 6CBS(conf-system-ip-flow)# activate

Use show system-ip-flow-rule <flow-rule-name> to display a system IP flow rule and its parameters. If no flow rule is specified, all system IP flow rules are displayed.

CBS# show system-ip-flow-ruleSystem IP Flow Rule : testruleDestination Address : 0.0.0.0Destination Address High : 255.255.255.255Destination Port : 0Destination Port High : 65535Source Address : 0.0.0.0Source Address High : 255.255.255.255Source Port : 0Source Port High : 65535Incoming Circuit Group : 1Protocol : 1Protocol High : 255Domain : 1Domain High : 4095Action : dropActivate (true/false) : fPriority : 10Skip Protocol (true/false) : tSkip Port (true/false) : tSkip Port Protocol (true/false) : tTimeout : autoTrace (true/false) : fGenerate Reversed Flow (true/false) : fDirection : bothCore Assignment : random-single-core(1 row)


Troubleshooting IP FlowsYou can trace VAP group and system IP flow rules using the trace option. When the trace option is enabled on an ip-flow-rule (configure vap-group <group-name> ip-flow-rule <name> -t), the system marks all new flows established using the specified flow rule with a “trace” flag. The “trace” flag is cleared only when the flow is cleared from the flow table by either timing out or by issuing the clear flow-active command.

Flow Processing Log Information

The flow processing daemon logs debug messages related to flow setup, timeout, and re-routing activity on flows marked with the “trace” flag.

IMPORTANT: When troubleshooting flows, it is important that you create specific IP flow rules with the “trace” option enabled for running in a production environment. Avoid enabling the trace option on a flow unnecessarily. The logging activity will cause a severe reduction in performance.

The flow processing daemon on the NPM logs trace messages to the syslog using daemon.notice facility/severity level. You can control the logging level per NPM using the following command (issued from the NPM):

/ # cbs_query_flowd log fp notice [on|off]

The default setting for notice log level under fp (flow processing) is on. Setting it is not persistent (does not survive an NPM reboot). Changing the NPM logging level does not change the trace functionality, but filters any outgoing log messages. System performance will not suffer as significantly with notice set to off while you are tracing flows.

On the CPM, these messages go to /var/log/messages by default but this can be controlled by editing the /etc/syslog.conf file and restarting the syslog daemon. The APM also logs messages during the setup and re-routing of flows that are marked with the “trace” flag. The APM will log a message every time a flow (re)setup is needed for a packet marked with the trace flag. The trace logs are stored in /var/log/messages as kernel-level entries from a specific VAP.

If a flow is originated from an APM, the flow initially cannot be traced. However, if the flow experiences a policy lookup on the NPM, the NPM may identify that the flow should be traced. If so, the APM will store this fact, and will then log a message every time a flow (re)setup occurs for the APM originated flow.

Clear-Flow Active Command

The clear-flow active command deletes all active flow connection and load-balancing information from all Network Processor Modules (NPMs) installed in the X-Series Platform, or from a specified NPM. This command stops all traffic, and in most cases, causes a complete service interruption that may last for several minutes. Consider these risks carefully before entering this command. Crossbeam Systems recommends that you use this command only after consulting with a Crossbeam Systems Customer Support or Professional Services representative.

Handling Flows When an Application is Down

When the Application Monitor identifies an application as down on an APM, it will stop flows to that VAP. Flows are automatically reassigned to active applications, and new flows are balanced across the active VAPs. When the application returns to an Active state, it is eligible to receive new flows distributed by the NPM.


Dropping Flows

Flows can be dropped for several reasons, as listed below. Use the show flow active command to display information currently stored in the Active Flow Table (AFT) on the Network Processor Modules (NPMs) running on the X-Series Platform. The verbose parameter display additional information, including Drop reasons about all active flows. For a list of filter parameters to use with the show flow active command, see Chapter 6 of the XOS Command Reference Guide.

Possible values for <drop_reason_ID> are:

No L2 policy match — The destination MAC in the packet did not match the MAC address of any VND on the circuit where the packet entered the system.

NOTE: Circuit information is not displayed for flows with this drop reason ID.

No L3 policy match — There are no IP flow rules that apply to this layer 3 flow.

NOTE: Circuit information is not displayed for flows with this drop reason ID.

L3 drop policy — This layer 3 flow matches the conditions defined in the access control list (ACL) for an IP flow rule configured with the action, drop.

PS2Master failed — A VAP group IP flow rule configured with the action, pass-to-master, or a system-level IP flow rule with the action, pass-to-masters, applies to this flow. The NPM attempted to send this flow to one or more master VAPs, but the operation failed because none of the master VAPs were in the Active state.

PS2IDX failed — A VAP group IP flow rule configured with the action, pass-to-vap, applies to this flow. The NPM attempted to send this flow to the appropriate VAP, but the operation failed because the VAP was not in the Active state.

Load-balance failed — A VAP group IP flow rule configured with the action, load-balance, applies to this flow. The NPM attempted to load balance this flow across the VAPs in the appropriate VAP group, but the operation failed because there were no active VAPs in the group or because there were no VAPs in the VAP group’s load-balance VAP list.

Broadcast failed — A flow rule configured with the action, broadcast, applies to this flow. The NPM attempted to broadcast this flow to all VAPs in one VAP group or to all VAPs in all VAP groups, but the operation failed because none of the VAPs to which the NPM sent the flow were in the Active state.

No Reason — One or more flow rules were successfully applied to this flow, and none of those IP flow rules are configured with the action, drop.


Configuring System Non-IP Flow RulesTo configure system non-IP flow rules using the EMS, open Configuration > Flow Provisioning > System Non-IP Flow Rules. Configuring system non-IP flow rules is similar to configuring system IP flow rules with the exception of the packet encapsulation used and the lack of IP addresses.

The following example configures system non-IP flow rules using the CLI. In this example, the snap_traffic flow rule drops all SNAP packets with a protocol number of 2000 and an Organization Unique Identifier (OUI) of 10.

CBS# configure system-non-ip-flow-rule snap_trafficCBS(conf-system-non-ip-flow)# encapsulation snap type 2000 oui 10CBS(conf-system-non-ip-flow)# action dropCBS(conf-system-non-ip-flow)# activate

Use the show command to view the system non-IP flow rule snap_traffic:

CBS(conf-system-non-ip-flow)# showSystem Non IP Flow Rule : snap_trafficEncapsulation : snapType : 2000OUI : 10Action : dropActivate (true/false) : t(1 row)


9Configuring Protocols

This chapter contains the following major topics:

Configuring Static IP Routes on page 107

Configuring ARP Entries on page 109

Displaying ARP Entries on page 109

Configuring Neighbor Discovery on page 109

Displaying Neighbor Discovery on page 110

Configuring a RADIUS Server Host on page 110

Configuring an LDAP Server on page 111

Defining the LDAP Version Number on page 111

Configuring RSA SecurID to Authenticate Administrative Users on page 112

Configuring RSA SecurID to Authenticate Remote Users on page 113

Configuring Static IP RoutesStatic IP routes are user-defined routes that cause packets moving between a source and a destination to take a specific path. Static IP Routes are defined per routing domain and apply to all VAP groups which have circuits in this domain, unless the VAP group is explicitly specified. Static routes are automatically load-balanced using an Equal-Cost Multi-Path (ECMP) algorithm.

NOTE: Static IP routes are configured in the same manner for IPv4 and IPv6 protocols.

To configure static IP routes, you need to specify the routes Destination IP Network Address and Network Mask. The Destination Network IP Address is the IP address of the network or host to which the packet is being sent. You also need to specify the Next Hop IP address. This is the IP address of the next hop that can be used to reach that network.

When specifying an IP route you can also specify to verify the health of the next hop. When specified, probes (ARP requests) are sent to the next hop IP address. If the probes fail, route cost is increased so that an alternative route with a lower cost may be chosen.

Use the following command to configure static IP routes:

configure ip route {<ip-addr> <netmask>|<ip-addr/netmask>} <next-hop-ip-addr> [domain <domain-id>] [circuit <circuit-name>] [vap-group <groupname>] [metric <metric-value>] [verify-next-hop]

Where:

<ip-addr> IP address of the route.

<netmask> Network Mask associated with this IP address.

<ip-addr/netmask> Destination IP address and network mask. Can be either CIDR or decimal format. IPv6 addresses use the CIDR format: 2001:BA95:AC10:0:CF6A:D:2145:3713/64


The following example configures a static route, its netmask, and the address of the next hop:

CBS(config)# ipCBS(config-ip)# route 190.1.1.0 255.255.255.0 192.213.212.111

The following example configures the same static route, but also verifies the health of the next hop:

CBS(config)# ipCBS(config-ip)# route 190.1.1.0 255.255.255.0 192.213.212.111 vap-group firewall verify-next-hop

The following example configures an IPv6 static route and verifies the health of the next hop:

CBS(config)# ip CBS(config-ip)# route 2002:BA95:AC10:0:CF6A:D:2145:3000/64 FC00:C959:CC10:0:CF6A:D:2145:0700 vap-group firewall circuit thru1 verify-next-hop

To display configured static routes, use the following command:

show ip route [<destination-ip> | <first-destination-ip> <last-destination-ip>] [sort-by-destination-address]

The following is an example display of the show-ip-route command.

If you have IPv6 configured, show ip route will display the IPv6 routing table.

NOTE: XOS will restrict IP routes whose destination network is the same or a more specific network match with the system internal network IP address. For example, the route a.b.c.0/24 is a more specific match than route a.b.0.0/16. XOS displays the following error when a conflict is found:

CBS# config management ip-route 1.1.1.234 255.255.255.255 192.168.32.2 %CONF-ERR: Invalid value Detail: Conflict found with internal network 1.1.0.0 255.255.0.0 CBS#

<next-hop-ip-addr> IP address of the next hop that is used to reach that network.

<domain-id> User defined domain, ranging from 1 to 4095, with a default value of 1.

<circuit-name> Name of the circuit.

<groupname> VAP group for this address. Default value is all VAP groups.

<metric-value> Metric value that you want to assign to the static IP route that you are configuring.

Valid values are:

IPv4: 1 to 255, inclusive, with a default of 0 (zero)

IPv6: 257 to 511, inclusive with a default value of 256

verify-next-hop Verifies a next hop route for a default network before using it.

Module Destination Gateway Metric Device

group1_1 192.168.20.30/32 30.1.0.20 0 eth0

group1_1 30.1.0.0/16 0.0.0.0 0 eth0

group1_1 20.0.0.0/8 0.0.0.0 0 vnd1

group1_1 21.0.0.0/8 0.0.0.0 0 vnd2

group1_1 22.0.0.0/8 0.0.0.0 0 vnd3

group1_1 23.0.0.0/8 0.0.0.0 0 vnd4

Configuring Protocols 108

Configuring ARP EntriesDefine static Address Resolution Protocol (ARP) entries by relating an IP address to a MAC address. To define ARP entries, use the following command:

configure arp <ipaddr> <mac-addr> [domain <domain-id>] [vap-group <groupname>] [circuit <circuit-name>]

Where:

The following is an example of this command:

CBS(config)# arp 192.178.25.10 00:05:c2:13:44:71

Displaying ARP EntriesTo display ARP entries, use the following command:

CBS# show arpModule Address Hardware Addr Type Interfaceprimarycpm 1.1.11.20 00:05:c2:00:0b:05 dynamic eth015Net_1 1.1.11.101 00:05:c2:00:0b:04 dynamic eth0npm1 1.1.11.1 00:05:c2:00:0b:11 dynamic eth0npm2 1.1.11.2 00:05:c2:00:0b:12 dynamic eth01.1.11.35 1.1.11.35 00:05:c2:00:0b:15 dynamic eth0192.178.25.10 192.178.25.10 00:05:c2:13:44:71 static firewall192.178.25.15 192.178.25.15 00:05:c2:15:0c:a8 dynamic eth2192.178.25.47 192.178.25.47 00:d1:c2:b5:71:e5 dynamic eth2

Configuring Neighbor DiscoveryNeighbor discovery specifies a static association between an IPv6 address and an ethernet layer MAC address to ensure proper forwarding of frames.

By default, an NPM can apply a neighbor-discovery static entry to any packet passing between the members of a VAP group and a physical interface on an NPM.

Use the following command to configure neighbor discovery:

configure neighbor-discovery <IP_address> <MAC_address> [domain <domain_ID>] [vap-group <VAP_group_name>] [circuit <circuit_name>]

configure no neighbor-discovery <IP_address> <MAC_address>

For additional details on how to configure the parameters of this command, please refer to Chapter 7 of the Command Reference Guide.

The following command configures a new static entry that maps the IP address fd00::330:f3cb:fb31:56ab to the MAC address 00:03:d2:00:02:0d. This command also configures the new entry to apply only to packets destined for VNDs that XOS creates for the circuit called testcct on the VAP group called testvapgroup:

<ipaddr> IP address of the ARP entry.

<mac-addr> MAC address of the ARP entry.

<domain-id> Domain for this host. The range is 1to 4095, with a default of 1.

<groupname> VAP group to which this IP address is assigned. Default is all VAP groups. The CLI displays the existing VAP group names.

<circuit_name> Specifies a circuit. The CLI displays existing circuit names when the user types ?


CBS# configure neighbor-discovery fd00::330:f3cb:fb31:56ab 00:03:d2:00:02:0d vap-group testvapgroup circuit testcctCBS#

NOTE: You cannot specify a MAC address that contains only 0’s or only f’s.

Displaying Neighbor Discovery This command displays the entries in the neighbor discovery table, along with status information for each discovered node.

Use the following command to display the neighbor discovery table:

show neighbor-discovery

For additional details about the output of this command, please refer to Chapter 12 of the Command Reference Guide.

The following is a sample output of this command.

CBS# show neighbor-discovery

Neighbor Entry Module one_2 Domain 0 IP address 2002:1::3 State REACHABLE Type UNICAST Flags - HW Address 00:03:d2:20:0c:69 Device enter

Neighbor Entry Module one_2 Domain 0 IP address 2002:2::1 State REACHABLE Type UNICAST Flags - HW Address 00:03:d2:1f:fb:b9 Device thru

Neighbor Entry Module one_2 Domain 0 IP address fe80::203:d2ff:fe1f:fbb9 State STALE Type UNICAST Flags - HW Address 00:03:d2:1f:fb:b9 Device thru

Configuring a RADIUS Server HostThe X-Series Platform supports RADIUS Authentication by selecting a host to use as the RADIUS server and the desired authentication port. Define the RADIUS authentication host and its attributes as follows:

configure radius-server host {<hostname>|<ip-address>} [auth-port <auth-port-value>] [timeout <timeout-value>] [key <key-word>]


Where:

Configuring an LDAP ServerThe X-Series Platform supports the Lightweight Directory Access Protocol (LDAP), which is an Internet protocol that email programs use to look up contact information from a server.

LDAP servers index all the data in their entries, and filters may be used to select just the person or group you want, and return just the information you want.

For example, search for all people located in Chicago whose name contains “Fred” that have an email address and return their full name, email, title, and description.

To configure an LDAP Server, use the following command:

configure ldap-server {<hostname>|<ip-address>} [auth-port <auth-port-value>]

Where:

NOTE: Configuration for the LDAP Server are written to the /etc/ldap.conf.This file will be overwritten when the CPM is rebooted. To preserve these files, execute the CLI command copy running-config.

Defining the LDAP Version NumberTo define an LDAP Host Server's Version number, use the following command:

configure ldap-parameter version {2|3} [distinguished-name <name>]

Where:

<hostname> DNS name of the RADIUS server host.

<ip-address> IP address of the RADIUS server host. The IP address of the RADIUS server host can be on the same network as a management interface or circuit. However, the IP address:

Cannot be on the XOS system's internal network, as defined by the configure system-internal-network command

Cannot be an address that is already defined in the XOS configuration, such as a circuit’s address or a management address.

<auth-port-value> UDP destination port for authentication requests. Valid values are 0 to 65535. Default is 1812.

<timeout-value> Number of seconds the X-Series Platform waits for a RADIUS server host to reply before timing out. Valid values are 0 to 3600 seconds. Default is 3 seconds.

<key-word> Authentication and encryption key used for all RADIUS communications between the host and the RADIUS daemon. This key must match the encryption used on the RADIUS daemon.

<hostname>| <ip-address>

Enter the LDAP server DNS name or IP address.

<auth-port-value> Specifies the UDP destination port for authentication requests. Valid values are 0 to 65535, with a default value of 389.

version {2|3} The LDAP server’s version number, 2 or 3. Default is 3.

<name> List of distinguished names to search.


Configuring RSA SecurID to Authenticate Administrative UsersRSA SecurID is an authentication solution built on a method called “two-factor authentication” using the RSA ACE/Server authentication engine. The normal password authentication methods provides a low level of authenticity. The SecurID solution adds a second level of authentication in addition to a predefined PIN. A user is required to provide an additional passcode which is dynamically generated during the authentication process making it hard to break into a SecurID system.

The SecurID system issues individually registered devices to authorized employees, that generate single-use passcodes (tokens), which change based on a timecode algorithm. A new tokencode is generated every 60 seconds and the RSA ACE/Server authentication engine validates these changing tokencodes. A user is then required to provide this dynamic tokencode in combination with a predefined PIN assigned to them during the authentication process.

The RSA ACE/Server is used to issue authenticators (token generator devices) to individuals, setup security policies and create logs of user accesses to the network resources managed by the system. In addition, RSA Security provides device specific software agents called “RSA Ace/Agents,” to enable strong authentication in network resources and web servers.

The RSA Security system also provides the RSA ACE/Server Application Programming Interface that allows developers to integrate RSA SecurID Authentication into your applications.

Prerequisites: Before starting this procedure, obtain the ACEAgent_50_PAM.tar from the SecurID Website. This is not included as part of the XOS distribution.

IMPORTANT: After enabling the SecurID pam authentication, the only method available to recover from a loss of communication between the Ace server and CPM is by logging in as root via the console.

To configure RSA SecurID, complete the following:

1. On the CPM, create the following directory:

mkdir /var/ace

2. Transfer the file sdconf.rec from the RSA server to the /var/ace directory.

3. Open the sdconf.rec file and edit the following line:

edit /etc/profile

Add the line:

VAR_ACE=/var/ace/sdconf.rec

4. On the CPM, install the ACEAgent_50_PAM.tar, using the following command:

install_pam.sh

5. Open the /etc/sshd/sshd_config file and complete the following:

a. Set the PAMAuthenticationViaKbdInt parameter to Yes.

b. Set the Restart sshd: line to Restart sshd: "service sshd restart”

6. Open the /etc/pam.d/sshd file.

a. Comment out the following:

auth required /lib/security/pam_stack.so service=system-auth

b. Add the following:

auth required /lib/security/pam_securid.so reserve

7. Execute the CLI command copy running-config to preserve these files. Otherwise, the files in the /etc/pam.d/ directory will be overwritten when the CPM is rebooted.


8. In some cases, the agent libraries (client side) use the wrong interface IP address in the decryption and authentication fails. To overcome this issue, place a new text file “sdopts.rec” next to “sdconf.rec” with the following line included:

“CLIENT_IP=x.x.x.x”

Where x.x.x.x is the agent’s primary IP address, as defined on the server (the IP of the interface to which the server is routed).

Configuring RSA SecurID to Authenticate Remote Users

Integrating an ACE/Server with VPN-1/FireWall-1To integrate an ACE/Server with the Check Point VPN-1/FireWall-1:

1. Copy the sdconf.rec file from the ACE/Server system to /var/ace on each VAP within the FireWall-1 VAP group.

2. On each VAP in the FW-1 VAP group, edit the /etc/services file, adding the lines:

securid 5500/udp securidprop 5510/tcp

3. On the ACE/Server, define the VPN-1/Firewall-1 system using the Add Client option. If you did not define Sites (optional), leave this field blank. If you want to authenticate all users on the ACE Server, in the Client Type select Communication Server and Open to All Locally Known Users.

4. Make sure you allow these services in the VPN-1/Firewall-1 policy and that the clocks are synchronized on both systems (ACE/Server and VPN-1/Firewall-1).

Configuring ACE/Server in a VPN-1/Firewall-1 Clustered Environment1. Define each cluster member separately on the ACE server with its unique IP address.

2. To prevent the cluster member from hiding (NATing) its unique IP address, add the following line to the "$FWDIR/lib/table.def" file on the mgmt and install policy:

"no_hide_services_ports = { .., <5500, 17> };". (Note: this line is already a part of the table.def file as of NG AI R55 based SmartCenter).

3. If the agent libraries (client side) use the wrong interface IP address in the decryption and authentication fails, place a new text file, sdopts.rec, next to sdconf.rec with the following line added:

"CLIENT_IP=x.x.x.x" where "x.x.x.x" is the agent's primary IP as defined on the server (the IP of the interface that the server is routed to).

NOTE: After a SmartCenter upgrade, any changes to the *.def files are lost.


Configuring an ACE Server to Work with a High Availability Cluster When the ACE Server first communicates with Check Point VPN-1/Firewall-1, it sends a Node Secret. The Node Secret is sent to the published Name and IP address of each VAP in the FW-1 VAP group. To define the X-Series Platform name and IP address:

1. Determine each VAP’s name and IP address, using the ping command as follows:

On Windows: ping <the host name of the machine> On Solaris: ping -s <the host name of the machine>

2. Add each VAP name and IP address to the hosts file on the VPN-1/Firewall-1 ACE server. The X-Series Platform name will be the first name and all other interfaces follow. You also have to rename it; for example, if the computer name is "Bob," assign the name to the published IP (the IP shown while pinging the system). Then name the rest of the interfaces “bob-1”, “bob-2” and so on.

3. On the ACE Server, add a client with the name of the system on which VPN-1/Firewall -1 is installed.

4. Under the Secondary Node, add the rest of the interfaces. Each time a different interface.

5. Complete the previous step for all VAPs in the FW-1 VAP group.


10Configuring Bridge Mode

You can configure the X-Series Platform for Layer 2 bridging. This chapter describes the components used to configure bridges in XOS.

Configuring Bridge-Mode Bridges on page 115

Additional Configurations on page 117

Configuring Bridge-Mode BridgesThe X-Series Platform supports bridging multiple circuits on different segments of the same LAN as part of a bridge-mode device. To create a bridge-mode bridge, three steps need to be completed.

1. Create required circuits: A bridge circuit and at least two interface circuits in promiscuous-mode active.

2. Assign the interface circuits to physical ports.

3. Group the circuits using the bridge-mode command.

NOTE: There are restrictions to the number of circuits that can be assigned to a bridge. Refer to Circuit Restrictions for Bridges on page 117.

If you are using EMS, open Configuration > Interfaces > Data Interfaces > Bridge Mode.

To create a bridge-mode bridge using the CLI, complete the following:

1. Configure a circuit to be used as the bridge-mode bridge.

CBS# configure circuit <bridge_circuit_name>CBS(conf-cct)#

2. Configure a device name for the bridge.

CBS(conf-cct)# device-name <device_name>CBS(conf-cct)#

3. Assign a VAP group to the circuit:

CBS(conf-cct)# vap-group <vapgroupname>CBS(conf-cct-vapgroup)#

NOTE: For applications requiring an IP address on the bridge, assign the IP address to the template circuit before creating the bridge. If the application does not specifically require an IP address on the bridge, do not assign a primary IP address to the template circuit. Refer to your application documentation for IP address requirements.

4. Configure the IP address if required.

CBS(conf-cct-vapgroup)# ip 10.70.10.1/24 10.70.10.255CBS(conf-cct-vapgroup-ip)# endCBS#

5. Configure the first of two member circuits.

CBS# configure circuit <name1>CBS(conf-cct)#


6. Configure device name to the circuit.

CBS(conf-cct)# device-name <device_name>CBS(conf-cct)#

7. Assign the bridged VAP group to the circuit.

CBS(conf-cct)# vap-group <vapgroupname>CBS(conf-cct-vapgroup)#

8. Set the circuit on the VAP group to promiscuous mode active. Do not configure IP information, as these circuits will be grouped into the above circuit by using the bridge-mode feature.

CBS(conf-cct-vapgroup)# promiscuous-mode activeCBS(conf-cct-vapgroup)#

9. Repeat steps 4 through 7 for the second member circuit. Proceed with Step 9 after the second circuit has been configured.

10. Assign these circuits to the physical interfaces, using the following command:

CBS# configure interface gigabitethernet 1/1 logical-all <logical name> circuit <name1>CBS(intf-gig-log-cct)# endCBS# configure interface gigabitethernet 1/2 logical-all <logical name> circuit <name2>CBS(intf-gig-log-cct)# endCBS#

11. Group the circuits into bridge-mode, using the following command:

CBS# configure bridge-mode <bridge_circuit_name>CBS(conf-bridge-mode)# circuit <name1>CBS(conf-bridge-mode)# circuit <name2>CBS(conf-bridge-mode)# endCBS#

The following is an example of a bridge-mode configuration:

CBS# configure circuit bridgecctCBS(conf-cct)# device-name bridgecctCBS(conf-cct)# vap-group idsCBS(conf-cct-vapgroup)# ip 10.70.10.1/24 10.70.10.255CBS(conf-cct-vapgroup-ip)# end

CBS# configure circuit circuit1 vap-group ids promiscuous-mode activeCBS# configure circuit circuit2 vap-group ids promiscuous-mode activeCBS# configure interface gigabitethernet 1/1 logical logical_1 circuit circuit1CBS(intf-gig-log-cct)# endCBS# configure interface gigabitethernet 1/2 logical logical_2 circuit circuit2CBS(intf-gig-log-cct)# end

CBS# configure bridge-mode bridgecctCBS(conf-bridge-mode)# circuit circuit1CBS(conf-bridge-mode)# circuit circuit2


Additional ConfigurationsThe following section provided examples of additional bridge-mode bridge configurations. These include a basic bridge configuration, a VLAN configuration, and a bridge configuration using multi-link group interfaces.

A bridge can be connected to another application in a different VAP group. To do this, configure an interface-internal to connect the bridge to the application’s VAP group. Also, you can direct traffic to the application’s VAP group circuits based on VLAN tags using a combination of ingress-vlan-tag in the logical line and default-egress-vlan-tag in the circuit configuration.

Circuit Restrictions for BridgesA circuit should not be assigned to more than two bridges. More specifically, a circuit and VAP group pair with promiscuous-mode active should not be included in more than two bridging configurations. (This applies to bridge-mode transparent, as well as bridge-mode bridge.)

In a VAP group with multiple VAPs, traffic on promiscuous circuits cannot be sent from one VAP to another VAP. This is true even when the IP address on the bridge-mode circuit is configured as increment-per-vap.

Furthermore, a circuit is restricted to one bridge in the following configurations:

A circuit should not be assigned to more than one bridge if it also mapped to a physical interface.

A circuit should not be assigned to more than one bridge if the circuit and VAP group pair is configured for no promiscuous-mode.

NOTE: If you disable a bridge that has attached multi-link group-interfaces, the attached interfaces are not disabled automatically.

NOTE: If you enable an MLT group-interface that is attached to a bridge that is disabled, the bridge will remain disabled.

Before creating a bridge configuration, you need to determine:

Whether the application requires bridge or transparent mode for the bridge-mode configuration

Whether VLAN support is required

Basic ConfigurationThe following configuration example provides a basic bridge configuration using the transparent mode.

vap-group is1 xslinux_v3 vap-count 2 max-load-count 2 ap-list ap4 ap5 load-balance-vap-list 1 2 3 4 5 6 7 8 9 10 11 12 ip-flow-rule def_rule_is1 action load-balance priority 15 activate

circuit bridge10 device-name bridge10vap-group is1

circuit vpt1_1 device name vpt1_1vap-group is1

promiscuous-mode activecircuit vpt1_2

device-name vpt1_2


vap-group is1promiscuous-mode active

interface gigabitethernet 1/1logical-all vpt1_1

circuit vpt1_1 interface gigabitethernet 1/2

logical-all vpt1_2circuit vpt1_2

bridge-mode bridge10 transparentcircuit vpt1_1circuit vpt1_2

VLAN ConfigurationThe following configuration example expands the basic configuration so that traffic is serialized from an IPS application to a firewall. VLANs are also supported.

vap-group is2 xslinux_v3 vap-count 5 max-load-count 2 ap-list ap7 ap8 ap9 ap10 load-balance-vap-list 1 2 3 4 5 6 7 8 9 10 delay-flow 120 reload-timeout 1200 ip-flow-rule ips_def_rule_is2 action load-balance priority 15 activatevap-group fw xslinux_v3 vap-count 3 max-load-count 2 ap-list ap2 ap3 ap4 load-balance-vap-list 1 2 3 4 5 6 7 8 9 10 delay-flow 120 reload-timeout 1200 ip-flow-rule fw_def_rule action load-balance priority 15 activate

system-non-ip-flow-rule IS_stp encapsulation lsap dsap 66 ssap 66 action broadcast activate

circuit fwext device-name fwext vap-group fw ip-forwarding ip 172.16.11.10/24 172.16.11.255circuit vlanbridge1 device-name vlanbridge1 vap-group is2circuit intvlancirc1

device-name intvlancirc1 vap-group fw


ip-forwarding vap-group is2 promiscuous-mode activecircuit vlan20 device-name vlan20

vap-group fwdefault-egress-vlan-tag 20ip-forwardingip 172.16.20.10/24 172.16.20.255

circuit vpt1_5device-name vpt1_5vap-group is2

promiscuous-mode active

interface gigabitethernet 1/2 logical FWEXT circuit fwextinterface gigabitethernet 1/5

logical vpt1_5circuit vpt1_5

interface-internal GRBR5logical-all GRBR5

circuit intvlancirc1logical vlan20 ingress-vlan-tag 20 20

circuit vlan20

bridge-mode vlanbridge1 transparentcircuit intvlancirc1circuit vpt1_5

Configuration Using Multi-Link GroupsThe following configuration creates two multi-link groups, MLT1 and MLT2, then adds these groups instead of physical interfaces to the bridge group. This allows greater throughput of traffic. However, the multi-link group cannot have any configured logical interfaces.

vap-group is1 xslinux_v3vap-count 2max-load-count 2ap-list ap4 ap5load-balance-vap-list 1 2 3 4 5 6 7 8 9 10 11 12ip-flow-rule ips_def_rule_is1

action load-balancepriority 15activate

circuit MLTa device-name MLTa vap-group is1


circuit MLTbdevice-name MLTbvap-group is1



circuit MLTBRIDGE1device-name MLTBRIDGE1 vap-group is1

group-interface MLT1interface-type gigabitethernetmode multi-link circuit MLTa

gigabitethernet 1/1 gigabitethernet 1/2gigabitethernet 1/3

group-interface MLT2interface-type gigabitethernetmode multi-link circuit MLTb

gigabitethernet 2/1gigabitethernet 2/2gigabitethernet 2/3

bridge-mode MLTBRIDGE1 transparentcircuit MLTacircuit MLTb


11Configuring CPM Redundancy

This chapter explains how to configure redundant CPMs. It includes the following major topics:

Overview on page 121

Installing CPMs and Configuring Redundancy in a New System on page 124

Installing a Second CPM into an Existing System on page 125

Managing CP Redundancy on page 125

Troubleshooting CP Redundancy on page 126

Using the CBS PSI Tool on page 127

Configuring a Redundant CPM Management Virtual IP Address on page 128

NOTE: APM redundancy is covered by creating redundant VAPs, as described in Configuring VAP Groups on page 45.

NPM redundancy is covered by creating redundant interfaces, as described in Configuring Interfaces on page 61

Multi-system redundancy allows one X-Series Platform to be redundant for another, which is accomplished using VRRP, as described in Configuring Multi-System High Availability on page 129.

OverviewCPMs provide physical resources, data, and services to the X-Series Platform, as well as outside management clients. To provide continuous operation of the system in the event of a CPM failure, CP redundancy should be configured between CPMs.

The X-Series Platform architecture supports a Primary and a Secondary CPM. Whenever two CPMs are present in the X-Series Platform, the CPMs (and other modules in the system) learn about each other’s states through periodic heartbeat messages. Either CPM can act as the Primary/Secondary (Active/Standby) CPM from either slot. However, at any given time, the system can have only one Primary CPM.

If the Primary CPM detects a catastrophic module failure and crashes; or if the Secondary CPM detects that the Primary CPM is no longer operational, the following occurs:

Secondary CPM forces a hardware reset of the Primary CPM slot.

Secondary CPM declares itself as the new Primary CPM.

Primary CPM IP address is installed on the control bus IP interface of the Secondary CPM.

New Primary CPM sends out a gratuitous ARP message with its own MAC address.

New Primary CPM starts all daemons that run on the Primary CPM.

To make the failover virtually transparent to the other modules in the system, the Primary and Secondary CPMs are assigned unique IP addresses on the control network. These addresses are role based, not slot based. This makes the Primary CPM control network IP address the same, independent of the slot where the CPM resides. (This is different than the management interface’s IP address, which is slot-based.) To the APMs and NPMs, the CPM failover looks like a quick CPM reboot.


NOTE: When using a Primary/Secondary in the admin state for two CPMs and the Primary CPM is rebooted, the Secondary CPM attempts to become the Primary. During this process, if the Primary finishes rebooting before the Secondary becomes the Primary, the Secondary is put into the single user mode, making it inaccessible and displaying the following message:

An error occurred during the file system check. Dropping you to a shell. Use the fsck command ...

Use caution when using the Primary/Secondary admin state, and if this scenario occurs, use the commands displayed in the error message to place the Secondary CPM back online.

Configure the IP NAT inside and outside settings on each management interface. When configuring for redundancy, make sure that the IP NAT settings are the same for the interfaces on both CPMs.

CP Redundancy Software ComponentsThe XOS CP redundancy software uses the following components:

Heartbeat Driver - Maintains the state of all the modules in the system and implements the Primary CPM election protocol.

Two-Port Multiplexor Driver (TPM) - Creates an abstraction of one reliable control network out of two control Ethernets.

CPRM (Control Processor Redundancy Management) Daemon (cbscprmd) - Executes the CP redundancy state machine.

Disk Mirroring Driver (drbd) - Executes disk mirroring.

Shell Scripts - Execute procedures necessary for CP redundancy state transitions.

CP Redundancy Hardware RequirementsTo use CP redundancy you must have the following hardware installed on your X-Series Platform:

Backplane EEPROM should be identical on both CPM slots. To check the backplane EEPROMs, use the following command:

/usr/os/bin/eepromdiag -c backplane -rs

NOTE: Execute this command on both CPMs to ensure that the results are the same.

Valid board EEPROM on both CPMs. To check the board EEPROMs, use the following command:

/usr/os/bin/eepromdiag -c board -rs

Partition size must be identical on both CPMs. Use the following command to perform the check:

cat /proc/partitions

CP Redundancy and RAID LimitationsThis section describes requirements and limitations regarding RAID and CP redundancy configurations.

Both CPMs must have identical RAID hard disk drive (HDD) configurations A mix of non-RAID and RAID configurations is not supported in CP redundancy.

The mirrored partition sizes on each HDD on both CPMs must be identical.

If you add a CPM-8600 to an X-Series Platform with an existing CPM-8600, you must make sure the new CPM-8600 has the same configuration as the original.

To configure CP redundancy with RAID, perform a fresh install on each CPM-8600.

Configuring CPM Redundancy 122

CP Redundancy Runtime StatesThe following are possible runtime states for the Primary and Secondary CPMs:

Unknown — The CPM is in an unknown state and may be in the process of booting. If the offline CPM is in an Unknown state for a long period of time, use the cp-unknown-state command to instruct the primary CPM to ignore the offline CPM.

Initialization — The CPM is in its initial state.

Un-synchronized

Occurs when the CPM crashes during re-synchronization of the mirrored disk partitions. After going into this state, the CPRM daemonizes itself and allows normal Linux initialization to complete.

Occurs during the repair process, if the Primary CPM crashes.

In both cases, the CPRM waits for another CPM to become the Primary CPM. The CPRM learns about this event from the heartbeat driver. The CPRM on the un-synchronized CPM makes sure the control network interface has the same subnet mask as that of the Primary CPM and reconfigures it if it does not. The CPRM then goes into the Repair state and waits for the connection from the Primary CPM.

Repair — The Secondary CPM is being synchronized to the Primary CPM. When the CPRM goes into the Repair state it allows Linux initialization to complete. The CPRM then waits for a TCP connection to be established from the Primary CPM. After TCP connection is established, the CPMs exchange information to access what repairs are needed.

During synchronization, any commands issued against the CPM are blocked to prevent the disruption of the synchronization process. This includes attempting to execute the following commands:

upgrade

reload cpm modules

change cp-redundancy setting, including disabling or setting either CPM offline.

Offline — The X-Series Platform is operating in the stand-alone CPM state. The CPRM goes into the Offline state either during initialization or when the System Administrator requests it through the CLI. When the CPM is in this state, other system processors ignore it as if the module is not plugged into the chassis. The only XOS subsystem that runs on the CPM in this state is the heartbeat driver, as it needs to communicate the Offline state to the other CPM to avoid the other CPM from causing a reboot. The Offline state is useful in preventing hard drive data corruption when a CPM is accidentally plugged into the wrong chassis or when the System Administrator needs to perform diagnostics or maintenance operations. To return the Offline CPM to another state, the module must be rebooted or plugged into the proper chassis.

Primary — The Primary CPM runs all software subsystems and replicates all data on mirrored partitions and configuration data on the CPM system partition. At 10 second intervals, the CPRM updates the Persistent State Information (PSI) with a new timestamp.

The CPRM, on the Primary CPM, also learns through the heartbeat driver the state of the other CPM. States include:

a. CPM is in the Offline state or not plugged in. This module is ignored.

b. CPM is in the Un-synchronized or Election state. If this persists for the configured timeout (the default value is 60 seconds), the other CPM is reset.

c. CPM is in the Primary state. A major error has occurred. When this occurs there is connectivity to other processing elements and many of them agree that the current primary is correct and that the second CPM is reset. If however, the majority of elements point to the second CPM as the primary, the current primary is reset. If no other processing elements are running, the CPM with a smaller slot number resets the other CPM.

d. CPM is in the Repair state. In this case, the CPRM establishes a TCP connection to the other CPM and exchanges information to determine what repairs are needed. The CPRM then starts data replication once it learns from the other CPM that it is ready to do so. It also informs the other CPM when data replication is complete.


e. CPM is in the Secondary state. In this case, the CPRM watches the Secondary CPM to determine if it is healthy. The CPRM resets the Secondary CPM if it is found to be unhealthy or failed.

f. CPM is plugged in but does not come up for more than the configured timeout (Default value is 20 minutes). The CPM is reset.

Secondary — The CPM is a standby, ready for fail-over state. The CPM in this state maintains a replicated copy of data on mirrored partitions and configuration data on the CPM system partition. At 10 second intervals, the CPRM updates the CP_PSI with a new timestamp. The goal of the Secondary CPM is to determine when the Primary CPM fails or becomes unavailable in another manner. When the CPRM on the Secondary CPM detects such a condition, it resets the Primary CPM and sets the Secondary CPM to the Primary state. This change of the system state is propagated through the heartbeat to other processing elements. The CPM components also learn about the state change from the heartbeat driver and take appropriate component specific actions.

In particular, the CPRM instructs the mirroring driver (DRBD) to run in primary mode and mount the corresponding devices. It then assigns a CPM the Primary CPM IP address to a control network interface.

Installing CPMs and Configuring Redundancy in a New SystemPerform the following procedure to install and configure the X-Series Platform for CP Redundancy. Depending on your hardware platform, the slots for the CPMs are 6 and 7, or 13 and 14.

NOTE: When configuring CP Redundancy, dual CPMs will synchronize only if each CPM is running the same XOS Version and Kit number. Previous versions of XOS allowed kit variations, as long as the release version was the same.

1. Insert the CPM that is intended to be the Secondary CPM into the desired slot. If unfamiliar with this process, refer to the hardware installation guide for your hardware platform.

2. Install the XOS software.

3. Log into XOS as root:

CBS# unix su Password: [root@xxxxx admin]#

4. Halt the CPM:

# halt

5. Insert the CPM that is intended to be the Primary CPM into the desired slot.

6. Install the XOS software. Refer to the Install Server User Guide for detailed instructions.

7. When prompted, make sure to enable CP Redundancy on this CPM.

Configure CP redundancy <Y or N>[N]: y

8. Reload the entire X-Series Platform.

CBS# reload allThis command may take a few minutes.

Any unsaved configuration will be lost.Do you want to save it to startup-config? <Y or N>[Y]:

#Start Configuration Validation###End Configuration Validation

Do you still want to save the current configuration? <Y or N>[Y]:Proceed with reload? <Y or N> [Y]:

Make sure that the Primary CPM is up and running before continuing.

9. At the secondary CPM, press the reset pin located at the front to reboot, or remove and re-seat the CPM.


Installing a Second CPM into an Existing SystemTo install a second CPM and configure CP Redundancy on an existing system, complete the following:

1. Prevent the current CPM from monitoring the second CPM for redundancy:

CBS# cp-unknown-state {cp1|cp2} ignore

Enter the Primary CPM in this command. For example, use the following command when the Primary CPM is currently in the first CPM slot and the intention is to install a CPM in the second CPM slot:

CBS# cp-unknown-state cp1 ignore

2. Insert the new CPM into the chassis.

3. Using the Install Server or USB Installer, install the same version of XOS software that is on the primary CPM. Do not reboot the new CPM when the installation process has completed.

4. Configure CP redundancy on the current primary CPM using the following command:

CBS# configure cp-redundancy

5. Reload all modules. Choose yes to save the configuration when prompted.

CBS# reload allThis command may take a few minutes.

Any unsaved configuration will be lost.Do you want to save it to startup-config? <Y or N>[Y]:Y

#Start Configuration Validation###End Configuration Validation

Do you still want to save the current configuration? <Y or N>[Y]:YProceed with reload? <Y or N> [Y]:Y

6. When the primary CPM has been reloaded, reboot the newly installed CPM.

When the new secondary CPM boots up the first time, it will discover the chassis system ID and reboot. After this reboot, it will start the synchronization process.

7. Have the CPMs monitor each other for redundancy:

CBS# cp-unknown-state {cp1|cp2} monitor

Managing CP RedundancyFor redundancy purposes, each CPM can be administratively configured to the following states:

Election —The CPMs determine the correct Primary CPM.

Offline — A CPM is set to offline for offline work.

Setting the CPM state to Offline takes effect immediately.

Displaying the CP Redundancy StateTo determine the current administrative state, use the following command:

CBS# show cp-redundancyAdministrative State:CP1 (this cp) is ELECTION CP2 (other cp) is ELECTION CP Redundancy is ENABLED


Operational State:CP1 (this cp) is PRIMARY CP2 (other cp) is REPAIR CP Redundancy is ENABLEDSynchronization Status:Disk synchronization is 20% completed

NOTE: If the show cp-redundancy command is executed during an In Service Upgrade, the following warning message will be displayed:

WARNING: In-Service Upgrade Is in Progress.

Configuring the CP Redundancy Administrative StateTo change or manage CP redundancy, use the following command:

configure cp-redundancy set {cp1|cp2|this_cp|other_cp} {election|offline}

Where:

To take a CPM out of the offline state, execute the following commands on the Offline CP:

CBS# configure cp-redundancy set this_cp electionCBS# copy running-config startup-config

If there is a Primary CPM in this system, the configuration on the Primary CPM must match the configuration on the other CPM. To ensure this, execute the following command on the Primary CPM:

CBS# configure cp-redundancy set other_cp election

After executing the previous command, reboot the offline CPM. If the configuration of the CPM being rebooted does not match that of the Primary CPM, the CPM being rebooted comes up in an offline state.

Troubleshooting CP RedundancyTo troubleshoot potential CP redundancy, use the following commands and tools:

show cp-redundancy command from the CLI

Persistent State Information tool (PSI)

CPRMD log files located in /usr/os/logs/cbscprmd.log and cbscprmd.log.saved.

To use the PSI tool use the following command:

[root@xxxxx /root]# /usr/os/bin/cbspsitool <options>

The PSI usage options are:

-r, —rbcnt <value>. Sets the reboot count to the designated value.

-x, — xos <ser_num>. Sets the XOS serial number to the designated serial number, which is described in Using the CBS PSI Tool on page 127.

-g, — debug <level>. Executes the program with debug set to the specified level.

cp1 Configures CP1

cp2 Configures CP2

this_cp Configures the CPM from which you are issuing the command

other_cp Configures the other CPM

election Sets specified CPM to election state

offline Sets specified CPM to offline state


-v, — version. Displays the version of this program and exits the PSI tool.

-h, — help. Prints this help message and exits the PSI tool.

The PSI.dat file contains the count of successive unsuccessful reboots. If there are excessive reboots, the CPM automatically goes into the Offline state. If this occurs, use the following command to reset the count to zero:

/usr/os/bin/cbspsitool -r 0

The /usr/os/logs/cbscprmd.log and cbscprmd.log.saved files contain important CPRM log messages for the current and previous versions of the CPRM daemon, respectively.

Using the CBS PSI ToolDuring the installation of a new CPM into an existing chassis, it may be necessary to reset the PSI (Persistent State Information) value on the CPM hard disk. This value is a record of the chassis backplane serial number which is learned by, and stored on, the CPM. At boot time, the CPM compares this value to the current backplane serial number. If the numbers match, the CPM boots completely. This means that a primary CPM boots, starts daemons, manages the other modules in the chassis; and a secondary CPM boots and starts synchronizing with the primary CPM.

If the numbers do not match, the CPM boots in the standby mode only. This means that the CPM boots but does not synchronize, nor attempt to manage the other modules in the chassis. This process prevents a loss of configuration. However, if the PSI value is zero, the CPM learns the current backplane serial number and boots completely.

To run the CBS PSI tool, complete the following:

1. Exit to the UNIX prompt, using the following command:

CBS# unix su password: UNIX_password [root@xxxx admin]#

2. Run the CBS PSI tool to reset the box_ser_num value to zero as follows. This causes the CPM to relearn the current backplane serial number upon the next boot.

[root@xxxx admin]# /usr/os/bin/cbspsitool -x 0

A display similar to the following should be displayed on the console. The backplane serial number is identified by the box_ser_num.

CCpNvPsi class information psi_fname: /usr/os/.PSI.dat------ Persistent State Information ------------ version: 1 box_ser_num: 0 state: CP_PSI_PRIMARY admin_state: CP_ADMIN_ELECTION timestamp: 1035216100 - Mon Oct 21 12:01:40 2002 update_seqn: 27306 reboot_cnt: 0 xsum: 0x00000bfa---- mirror id ---- ocp_ser_num: A2330003 timestamp: 1034973673 random: 1804289383

This command can be used before you unplug the CPM or after you plug the CPM into a different X-Series Platform. In the first case, you should stop the CPRM daemon before resetting the serial number. In the latter case, reboot the CPM after resetting the serial number. When CPRM finds the X-Series Platform serial number in the file is zero, it replaces it with correct serial number that it reads from the EEPROM.



Configuring a Redundant CPM Management Virtual IP AddressA single virtual IP address (VIP) can be configured to persist from CPM to CPM during failover. This virtual IP address can be used for any communication to the device (SSH, EMS, CLI, etc.) as well as a single-source virtual IP address for SNMP traps, irrespective of which CPM is the active Primary.

NOTE: Configuration of a management VIP address is allowed either in the Primary or offline state. If configured in an offline state, the management VIP address will not become active until the CPM becomes Primary.

The virtual management IP address utilizes the Active CPM interface MAC address.

LimitationsThe following lists the management VIP address restrictions:

The underlying management interface must already be defined and assigned a static IP address.

If an active / standby CPM is present, both the primary and the secondary static management IP address must be on the same network as each other.

The management VIP address must have the same network as the underlying static management IP address. Although the management VIP address inherits its subnet mask from the static management IP address, the management VIP address itself must be different from the static management IP address.

The specified management VIP address must not already be in use:

On another management interface

As a hop in the management routing table

In the management host table

As a management ip-alias

The management VIP address is only used for the management interface, it is not used for any of the other interfaces on the CPM.

In a dual-system configuration, the management VIP address cannot be the same as any CPM IP address used in the configure remote-box command.

Redundant CPM Management Virtual IP Address ConfigurationPerform the following:

1. Execute the following command:

CBS# configure management vip-addr <ip address>

Where <ip address> is the desired virtual IP address.

2. Validate the configuration of the virtual IP address by executing the following command:

CBS# show management-vip

3. To modify the virtual IP address that you configured, delete it using the following command then configure the new virtual IP address:

CBS# configure management no vip-addr <ip address>

Where <ip address> is the IP address to be removed.

Virtual IP Address Interaction during In-Service Upgrade (ISU)When performing an ISU with a virtual management IP address configured, the virtual IP address will remain associated with the CPM running the original version of the XOS software. The virtual IP address will only transition to the CPM running the new version of the XOS software when ISU completes and the old CPM is upgraded to the new XOS software. This transition occurs when the user selects Option 3 during the final question of the Interactive In-Service upgrade. Refer to the In-Service Upgrade (ISU) Overview on page 233.

12Configuring Multi-System High

Availability

This chapter provides information about configuring two or more X-Series platforms in a redundant manner using VRRP and Failover Groups to achieve a Multi-System High Availability configuration. The following topics are covered in this chapter:

VRRP Components on page 129

VRRP Configurations on page 133

Example Dual Box High Availability Configuration on page 143

Example VRRP Configurations on page 144

Configuring VRRP and Automatic ARP for NAT on page 148

You can configure multiple X-Series Platforms for High Availability (HA) using the Virtual Router Redundancy Protocol (VRRP). However, you do not need to configure one chassis as master and another as standby. Instead, create a failover group containing one or more VAP groups and circuits, and create a similar failover group with the same ID on one or more separate chassis. Should one group fail, the counterpart group on another chassis becomes master.

For detailed steps to configure an Active-Standby or an Active-Active Dual-box, high availability configuration, refer to the Multi-System High Availability Configuration Guide provided with your XOS documentation.

VRRP ComponentsThe following sections describe the various components used when configuring VRRP.

Failover GroupFailover groups and Virtual Routers (VRs) are used only in High Availability configurations. A failover group is a grouping of one or more VRs. A VR identifies the circuits and associated VAP groups for high availability. Only a failover group, not the entire system or an individual VAP group, can fail over to a backujp failover group on another system. Failover groups operate in pairs, one on each chassis, and are usually assigned different VRRP priorities. The failover group with the higher priority is the master.

Each failover group needs a counterpart failover group on another chassis participating in VRRP multi-system redundancy. For example, a failover group on an X-Series Platform may contain the VAP groups and circuits used for a firewall application. The same firewall application and configuration would be on a second X-Series Platform, also contained in a failover group. Both failover groups must have the same group identifier; however, they must have different priority values. The failover group with the higher value assumes Master status.

You cannot use two failover groups on the same chassis to back up one another.


Virtual Router (VR)A virtual router can be attached to a single circuit only, and can include only one VAP group attached to that circuit. In addition, the VR can assign individual IP addresses to the circuit and the VAP group interface. For circuits already configured with an IP address, the VR can also assign a virtual IP address. This virtual IP address allows you to configure failover groups using the same virtual IP address on more than one chassis.

VRRP PriorityEach failover group is assigned a VRRP priority. The failover group with the higher priority is designated as the Master. All failover groups with the same ID must be configured with different VRRP priorities.

A failure within a failover group does not necessarily cause a failover to another chassis. Instead, the VRRP priority is reduced by a pre-configured value, called a priority-delta. This minimizes or eliminates the problem of failing over to a chassis that has an even more diminished capacity. After the failure has been rectified, the VRRP priority returns to its configured value. Each of the following events can be assigned a priority-delta:

Failure of an interface that is associated with a monitored VR circuit

Examples include a redundant interface, an LACP interface, or a bridge interface.

If the chassis has been configured with redundant interfaces, as described in Redundant Interfaces on page 65, an interface failure that causes a failover to a redundant interface in an UP state does not cause the VRRP priority to be decremented.

Unreachable next hop IP address

Failure to reach the next-hop IP address is based on the ability of the address to answer ARPs or, for IPv6, neighbor solicitations. The next-hop IP to be verified is configured within the VRRP/virtual-router/VAP group.

If you configure a failover group with a next hop health check and you also configure a monitored circuit with an associated interface, an unreachable next hop caused by a failure of the interface does not cause two decrements of the VRRP priority delta (one for the interface failure and one for the next hop becoming unreachable). The VRRP priority delta is decremented once for the interface failure and XOS prevents two decrements of the VRRP priority for a single cause.

NOTE: If you configure redundant interfaces and associate them with a monitored interface, and both the master and backup interfaces fail, the situation is the same as that described in the previous paragraph.

If the redundant interface passes a subsequent next hop health check, the VRRP priority is restored to its previous value.

Less than the desired number of active VAPs per VAP group

You can configure the minimum number of active VAPs that you require in the VAP group. If the VAP group is configured as part of the failover group, then when the number of VAPs falls below the limit, the VRRP priority of the failover group is decremented.

Interface failure in a monitored circuit.

Circuits that connect physical interfaces to VAP groups can be monitored using VRRP. If a failure on the monitored circuit is detected, the priority delta (value) decreases the configured priority (value) of the failover group. If the priority delta drops below the priority of the failover group on the other chassis, the failover group loses its master status, and fails over to the backup. With preemption enabled, after the monitored circuit recovers, the failover group resumes its role as master.

NOTE: By default, preemption is OFF to prevent the second failover (with its associated delay) that would restore the original Master Failover Group as Master.

When an event causes a failover, the Interface Redundancy Manager (IRM) daemon deactivates IP addresses in the Master failover group then activates those IP addresses in the backup failover group.

Configuring Multi-System High Availability 130

OSPF Failover in Conjunction with VRRPCrossbeam uses VRRP to provide failover between devices that do not participate in a routing protocol exchange and do not notice link or node-level failures. However, the VRRP status is not propagated to dynamic routing protocol entities such as OSPF and this may result in broken or asymmetric IP forwarding.

To solve this problem, Crossbeam enables the VAP to communicate per-circuit VRRP status to the associated OSPF daemon. This is done through the ospf-cost-increment parameter in the vrrp failover-group CLI context.

When the VRRP status of an interface is backup, XOS increases the cost of the corresponding OSPF interface. When the VRRP status of the interface changes to master, XOS removes the incremented cost from the OSPF configuration.

For more information about OSPF configuration, refer to the RSW Installation Guide.

High Availability (HA) Port and Management InterfacesThe X-Series Platform requires a communication link between all the X-Series Platforms in a High Availability (HA) configuration. Crossbeam recommends using the CPM High Availability port and both Management ports on the primary CPMs and both Management ports on the secondary CPMs when connecting between chassis. In a dual-system configuration, each pair of ports (HA, Management 1, and Management 2) are connected to separate network broadcast domains. Crossbeam recommends that customers do not connect the High Availability ports or Management ports directly to each other. Connecting the ports directly can lead to scenarios that do not provide full redundancy.

NOTE: Typically, you configure High Availability and Management ports for auto-negotiation. If you connect any of these ports to a switch and auto-negotiation does not work, use the configure management high-availability or configure management gigabitethernet command to manually set up the communication parameters.

VRRP ExampleSystems do not have to be configured with identical failover groups. For example, as shown in Figure 7 System A has these failover groups:

Failover Group 1 contains the VAP group and circuits used for a firewall and has VRRP priority 150.

Failover Group 2 contains the VAP group and circuits used for an intrusion prevention system application and has VRRP priority 150.

Failover Group 3 contains the VAP group and circuits used for a URL filtering application and has VRRP priority 100.

Failover Group 4 contains the VAP group and circuits used for a web security application and has VRRP priority 100.

System B has:

Failover Group 1 contains the VAP group and circuits used for a firewall and has VRRP priority 100.

Failover Group 2 contains the VAP group and circuits used for an intrusion prevention system application and has VRRP priority 100.

Failover Group 5 contains the VAP group and circuits used for an e-mail security application and has VRRP priority 150.

Failover Group 6 contains the VAP group and circuits used for an intrusion detection system application and has VRRP priority 150.


System C has:

Failover Group 5 contains the VAP group and circuits used for an e-mail security application and has VRRP priority 100.

Failover Group 6 contains the VAP group and circuits used for an intrusion detection system application and has VRRP priority 100.

Failover Group 3 contains the VAP group and circuits used for a URL filtering application and has VRRP priority 150.

Failover Group 4 contains the VAP group and circuits used for a web security application and has VRRP priority 150.

Figure 7. VRRP Example

In this example, each of the systems hosts two Master failover groups and two backup failover groups. Under normal operating conditions, System A runs the Firewal and IPS applications, System B runs the E-mail Security and IDS applicaitons, and System C runs the URL Filter and Web Security applications.. With all systems running properly, a catastrophic problem with System A causes Failover Groups 1 and 2 to fail over to System B. Similarly, with all systems running properly, a catastrophic failure on System B causes Failover Groups 5 and 6 to fail over to System C.

If System A suffers an interface failure within Failover Group 1, reducing its VRRP priority to 125, no failover would occur. If additional failures on System A reduce the VRRP priority of Failover Group 1 below 100, a failover to System B would occur.

NOTE: This example shows independent applications, where none of the configurations serialize traffic from one application to another. When traffic is serialized between two applications, such as Firewall and IPS, both applications are placed in the same failover group.

System B System CSystem A

Layer 2 Switch

Failover Group 1: FirewallFailover Group 2: IPSFailover Group 3: URL Filter

Failover Group 1: FirewallFailover Group 2: IPSFailover Group 5: E-mail Security

Failover Group 5: E-mail Security Failover Group 6: IDSFailover Group 3: URL FilterFailover Group 4: Web SecurityFailover Group 6: IDSFailover Group 4: Web Security

Priority 150 Priority 150 Priority 100 Priority 100




VRRP ConfigurationsThere are two basic configurations: Active-Standby and Active-Active. In both configurations, one failover group is Master and the counterpart failover group is in backup mode. With Active-Standby, the failover group on one chassis passes traffic while the backup group on the other chassis does not. With Active-Active, the Master failover group on each chassis passes traffic. The corresponding failover groups on each chassis are in backup mode.

To achieve an Active-Active configuration, attach one circuit to two VRs, where each VR is in a different failover group. The basic configuration, as shown in Figure 8, includes:

Failover Group 1:

On System A, this failover group has a VR attached to a circuit connected to a VAP group. Since the circuit does not have an IP address assigned, the VR assigns a primary IP address.

On System B, the failover group has a VR with the same configuration; however, the VR assigns a virtual IP address, which is the same as the primary IP address on System A’s VR.

The failover group on System A is given a higher VRRP priority than System B’s failover group. Thus System A’s failover group is initially the Master.

Failover Group 2:

On System B, this failover group has a VR attached to a circuit connected to a VAP group. Since the circuit does not have an IP address assigned, the VR assigns a primary IP address.

On System A, the failover group has a VR with the same configuration; however, the VR assigns a virtual IP address, which is the same as the primary IP address on System B’s VR.

The failover group on System B is given a higher VRRP priority than System A’s failover group. Thus System B’s failover group is Master.

In this configuration, each Master group has a unique IP address; thus, traffic can flow to both addresses. Should one group fail, the failover group on the other chassis automatically handles all traffic going to both addresses.

Should the circuit be configured with an IP address in this configuration, both VRs would assign virtual addresses instead of one VR assigning a primary address.

Figure 8. VRRP Active-Active Configuration


Before creating a VRRP configuration, determine:

Which X-Series Platforms are to be involved

What applications or configurations require redundancy

Definition of each failover group

Which X-Series Platform will host the master failover group, and which will host the bacup failover group

Next, perform the following steps, described in the subsequent sections:

1. Physically cable and configure the High Availability ports.

2. Create failover groups.

3. Configure VRRP on the VAP groups.

IMPORTANT: Each X-Series Platform must have a unique identifier to be in a VRRP configuration, which is set with the configure system-identifier <system-id> command.

NOTE: A VRRP Virtual Router (VR) configuration can have only one VAP group assigned. A VAP group is assigned to a VR using the vrrp virtual-router vrrp-id <id> vap-group <name> command. During the XOS upgrade you will be prompted to change your configuration. If you do not make this change, the system keeps only the first vap-group and issues a warning message.

Configuring High Availability PortsThere are a few options when cabling the High Availability ports:

In a dual-system configuration, the High Availability ports can be connected directly to each other or can be connected using a switch.

For 3 or more systems, the High Availability ports are connected to a switch as shown in Figure 7 on page 132.

If you have CP redundancy, you can connect all four High Availability ports (2 chassis, with primary and secondary CPMs on each chassis) into a single switch. This allows physical connectivity between the two primary CPMs, one in each chassis. Should a primary CPM fail in one chassis, the backup CPM will still be able to communicate with the remote chassis. To prevent packet loops, the secondary CPM automatically drops all packets coming into its High Availability port.

Figure 9. Multi-System VRRP Hardware Configuration


Redundant Interfaces for the High Availability PortsFor additional High Availability port redundancy, you can configure the two CPM management ports as redundant interfaces for the High Availability port. However, each management interface must be reachable via IP routing from the corresponding remote chassis.

You can use the remote-box command to specify up to five IP addresses that could be used to connect to each remote chassis, each IP address corresponding to a port on one CPM on the remote chassis. The first address is used to establish the initial UDP session to the remote box. By specifying more than one IP address, you provide redundancy in the event the UDP session over the first IP address fails.

NOTE: The remote-box command requires that you have interconnected CPMs on the two chassis. The IP address that you specify depends on how you have interconnected the CPMs. If you use the High Availability port, specify the internal IP address ( for example, 1.1.45.20) associated with the remote Primary CPM (obtained by running show internal-ip on the remote chassis). If you use either or both of the management ports, specify the external IP address(es) associated with the port(s). Crossbeam recommends that you connect the CPMs using separate network broadcast domains for each pair of ports (the HA ports, the Management 1 ports, and the Management 2 ports). Connecting the ports directly can lead to scenarios that do not provide full redundancy.

With this method, in the absence of any remote chassis configuration, VRRP exchanges between the IRM daemons still occur.

Show Remote BoxUse the show-remote-box command to view the path and path status for each of the interconnected ports on the two systems.

CBS# show remote-box 22Local System ID: 85

Remote System ID: 22Remote IP Local Intf Local IP Status Time In State Link Qual 192.168.211.89 14/1 192.168.211.85 Active 0 days, 00:00 0 10.10.22.20 HA port 10.10.85.20 Standby 0 days, 00:00 0 (2 rows)


Creating an Active-Standby ConfigurationThis section describes one way to use VRRP to build an Active-Standby failover configuration. The steps assume an existing single VAP configuration and configure virtual routers, virtual IP addresses, and assign failover priority values to the circuits and interfaces.

NOTE: IPv6 addressing can be used for circuits and interfaces attached to a VAP group where IPv6 is enabled, including virtual IP addresses. However, management networks are configured with IPv4.

Configuring the Remote System IDA remote system is identified by its system ID. Use the configure remote-box command to configure the remote system ID and IP address.

configure remote-box <system-id> <1st-ip-address> [<2nd-ip-address>] [<3rd-ip-address>] [<4th-ip-address>] [<5th-ip-address>]

If you connect Chassis 1 to Chassis 2 using only the High Availability (HA) port, you do not need to configure the remote system identifier. The two chassis use the HA port to exchange system-id information. However, if you do this, the HA port connection represents a single point of failure. Crossbeam recommends that you connect the HA ports and both management interfaces on both the primary and secondary CPMs in both chassis. Then use the remote-box command on both chassis to configure all five ports on the remote chassis.

Configure the remote system ID and IP address using the configure remote-box command.

NOTE: The remote-box command requires that you have interconnected CPMs on the two chassis. Crossbeam recommends that you specify the following IP addresses:

For the High Availability port, specify the internal ip address (1.1.46.20) associated with the remote Primary CPM (obtained by running show-internal-ip on the remote chassis).

For the management ports, specify the external IP addresses associated with the ports.

Ideally, specify all three addresses associated with the remote primary CPM and the two management addresses associated with the remote secondary CPM.

Crossbeam recommends that you connect the CPMs using separate network broadcast domains for each type of port (the HA ports, the Management 1 ports, and the Management 2 ports) on both the Primary CPMs and the Secondary CPMs. Connecting the ports directly can lead to scenarios that do not provide full redundancy.

CBS# configure remote-box 46 10.10.46.20 192.168.50.46 192.168.51.56 CBS(conf-remote-box)# end

NOTE: The example configure remote-box command specifies only the IP addresses of the management 1 and 2 interfaces for the primary CPM on chassis 2. Crossbeam recommends that you connect the management interfaces of the other CPM on Chassis 2 and that you add the IP addresses for those interfaces to the configure remote-box command.

Configuring VRRP Systems Linked Through a Switch

NOTE: When two CPMs on two different X-Series Platforms are linked through a switch in a multi-box redundancy configuration, both X-Series Platforms could become VRRP master if the switch is not configured for multicast.

Routers are responsible for forwarding multicast traffic to other subnets in the network but use multicast routing protocols other than IGMP. To guarantee that multicast forwarding requirements are met for the other subnets, multicast traffic must always be forwarded to the router ports. The port on the switch that has a router connected to it can be configured so that it receives all multicast traffic streams.

Normally, switches discover the router ports automatically. However, there may be circumstances where you must manually add router ports. The following example shows how to manually define router ports for a 3COM 4300 switch by using the addPort command on the routerPort menu:


1. At the Top-level menu, enter:

bridge multicastFilter routerPort addPortSelect router unit (1):

2. Enter the unit number (1). The following prompt is displayed:

Select router port (1-48):

3. Enter the desired router port number.

Create Failover GroupsBefore creating a failover group, you need to decide which configuration elements to include, specifically the circuits and VAP groups. The following procedure describes how to create a failover group.

NOTE: When configuring VRRP, the broadcast IP address for the virtual router must be on the same subnet as the virtual router. XOS displays an error message if this condition is not met.

To create a failover group using EMS, open Configuration > Redundancy > VRRP > VRRP Failover Group. Use the Help button for details.

1. Create the failover group by assigning it a name and ID. The ID must be unique on this chassis, and must be the same on its counterpart failover group on the remote chassis (Chassis 2).

Syntax:

configure vrrp failover-group <name> failover-group-id <1-255>

Command:

CBS# configure vrrp failover-group vrrp_FW failover-group-id 200CBS(conf-vrrp-group)#

2. Set the VRRP priority. VRRP priority should be different for each chassis; the failover group with the higher priority is the master. Certain events such as an interface failure or a change in VAP group member count can be configured to decrement the VRRP priority. A chassis failover occurs when the VRRP priority value of one failover group drops below the priority of the failover group on the other chassis. VRRP priority values are 1 to 255, and the default is 100.

Syntax:

priority <1-255>

NOTE: In this sample configuration, set the VRRP priority to 150 on Chassis A, and 100 on Chassis B. Chassis A is the master.

Command:

CBS(conf-vrrp-group)# priority 150CBS(conf-vrrp-group)#

3. If you are using OSPF with RSW, assign an OSPF cost increment to the circuits configured in the VRRP failover group. This will force OSPF to fail over to the backup chassis at the same time that a decremented priority value causes VRRP to fail over.

Syntax:

CBS(conf-vrrp-group)# ospf-cost-increment circuit <circuit_name> <increment-cost>

Command:

CBS(conf-vrrp-group)# ospf-cost-increment circuit lan02 10CBS(conf-vrrp-group)# ospf-cost-increment circuit lan04 10CBS(conf-vrrp-group)#

4. Specify the interval time (in seconds) between VRRP advertisements to other systems. Only the Master failover group sends advertisements. The number of seconds can be 1 to 255. The default is 2 seconds. Since a backup failover group can become Master, it is recommended that you use the same setting on all chassis.


Syntax:

advertise-interval <seconds>

Command:

CBS(conf-vrrp-group)# advertise-interval 30CBS(conf-vrrp-group)#

5. Configure preemption to specify how you want VRRP to function on failure. When preemtion is enabled, failure of the master failover group causes traffic to be re-routed to the backup failover group (which then becomes Master). The original Master resumes Master status when it becomes available. When disabled (using no), upon failure of the master failover group, the backup failover group services traffic. The original Master does not resume being Master when it becomes available, unless the backup fails.

NOTE: By default, preemption is OFF to prevent the second failover (with its associated delay) that would restore the original Master Failover Group as Master. The examples in this chapter do not enable preemption.

6. Determine which circuits to monitor and set a priority-delta for each circuit. A monitored circuit can affect the failover group VRRP priority; however, monitored circuits (usually a physical interface) do not fail over when VRRP switches to a backup failover group. The priority-delta decrements the failover group VRRP priority whenever an interface on the monitored circuit fails. In this example, physical interfaces are given a priority-delta of 30. When an interface failure occurs, the priority value (150) is decremented by the priority-delta (30) to 120. Since this is not less than 100 (priority on Chassis B), failover will not occur until more than one circuit has failed. Only then does the failover group on Chassis B becomes the master. The priority-delta is added back to the priority when the interface returns to the Up state.The priority-delta can be 0 to 255. Default is 1.

Syntax:

monitor-circuit <circuit_name> priority-delta <0-255>

Command:

CBS(conf-vrrp-group)# monitor-circuit lan02CBS(conf-vrrp-failover-cct)# priority-delta 30CBS(conf-vrrp-failover-cct)# exitCBS(conf-vrrp-group)# monitor-circuit lan03CBS(conf-vrrp-failover-cct)# priority-delta 30CBS(conf-vrrp-failover-cct)# exitCBS(conf-vrrp-group)# monitor-circuit lan04CBS(conf-vrrp-failover-cct)# priority-delta 30CBS(conf-vrrp-failover-cct)# exitCBS(conf-vrrp-group)#

NOTE: If your system incorporates group interfaces, use the monitor-group-interfaces parameter in this context to set a priority delta. Additionally, you can set a minimum number of ports (in the group interface) using the dist-port-threshold parameter to monitor. If the number of ports in that state falls below this threshold, the priority delta value is subtracted from failover group priority. See the XOS Command Reference Guide for specific command information.

7. Create a Virtual Router, assign an ID, and attach it to an existing circuit. If more than one VR is attached to the same circuit, each VR must have a unique ID.

Syntax:

virtual-router vrrp-id <1-4096> circuit <circuit_name>

Command:

CBS(conf-vrrp-group)# virtual-router vrrp-id 101 circuit v101CBS(conf-vrrp-failover-vr)#

a. If you require the VR circuit interface to remain in an UP state when the failover group is in backup mode, use the backup-stay-up command. The circuit does not send or receive traffic while in the backup state.

CBS(conf-vrrp-failover-vr)# backup-stay-up


NOTE: In a VSX configuration, VSX automatically enables backup-stay-up for all VLAN interfaces that are part of a VRRP group.

b. Assign a MAC address to the VRRP circuit. Use the vrrp-mac feature to automatically generate a unique vrrp-mac. In the event of a failover, the MAC address moves with the virtual IP address to the new chassis. By keeping the MAC address consistent, it reduces the time it takes for upstream routers to converge routing tables.

Alternatively, use interface (default) to use the MAC address configured on the physical interface.

Syntax:

mac-usage {vrrp-mac|interface}

Command:

CBS(conf-vrrp-failover-vr)# mac-usage vrrp-mac

c. Assign a priority-delta to the VR. In this serialized topology, each virtual router is given a priority-delta of 55. When a VR fails, the failover group’s VRRP priority of 150 is decremented by the priority-delta value of 55 to a new priority value of 95. Since this is less than 100 (the priority on Chassis B), the failover group on Chassis B becomes the master. The priority-delta is added back to the priority when the VR recovers.

Syntax:priority-delta <1-255>

Command:

CBS(conf-vrrp-failover-vr)# priority-delta 55CBS(conf-vrrp-failover-vr)#

d. Assign a VAP group to the VR. Specifying the VAP group of the VR allows you to assign a virtual IP address to the VAP group. The circuit must already be mapped to the VAP group with the configure circuit <name> vap-group <name> command.

Syntax:

vap-group <vap-group-name>

Command:

CBS(conf-vrrp-failover-vr)# vap-group fwCBS(conf-vrrp-vr-vapgroup)# exitCBS(conf-vrrp-failover-vr)#

e. Assign a unique IP address to the virtual router only when its circuit is NOT configured with an IP address. This address is considered to be the primary address. The no parameter deletes the IP address.

Syntax:

ip {<ip-addr> <netmask>|<ip-addr/netmask>} [<broadcast-ip-addr>] [increment-per-vap <ip-addr>]

This command assigns the fw VAP group to the virtual router and an IP address of 172.16.20.103 with a mask of 24 in CIDR format.

Command:

CBS(conf-vrrp-failover-vr)# vap-group fwCBS(conf-vrrp-vr-vapgroup)# ip 172.16.20.103/24CBS(conf-vrrp-vr-vapgroup)#

f. If the circuit already has an assigned IP address, assign a virtual IP address as follows. Note that the number of virtual IP addresses assigned to an IP address is limited to 99.

NOTE: Use the same format to assign virtual IPv6 addresses.

Syntax:

virtual-ip {<ip-addr> <netmask>|<ip-addr/netmask>} [<broadcast-ip-addr>] [increment-per-vap <ip-addr>]}


Command:

CBS(conf-vrrp-vr-vapgroup)# virtual-ip 10.0.101.1/24CBS(conf-vrrp-vr-vapgroup)# exit

The optional floating parameter assigns the virtual IP address to the master VAP, allowing traffic, cluster management, and synchronization communication to go directly to the master VAP. If a new master VAP is elected, the address “floats” (is assigned) to the new master.

In a VRRP configuration, the floating parameter assigns the virtual IP address to the master VAP on the new master chassis in the event of a failover.

g. Configure a next hop health check and a priority-delta. The next hop health check is an optional setting to verify IP connectivity to the next hop gateway. In this example, the gateway is given a priority-delta of 55. If the next hop health check fails, the failover group VRRP priority of 150 is decremented by the priority-delta value of 55 to a new priority value of 95. Since this is less than 100 (the priority on Chassis B), the failover group on Chassis B becomes the master. The priority-delta is added back to the priority when the interface recovers.

Specify the IP address of an external host (external to the X-Series Platform). The IP address 10.10.101.20 is used here as an example. Using exit twice returns you to the correct CLI context to repeat the VR configuration process.

NOTE: An IP route whose next-hop destination network is exactly the same (same network and netmask) as the circuit's or virtual circuit's network is considered to be invalid. If you try and set such a next hop destination, an error message is displayed.

Command:

CBS(conf-vrrp-vr-vapgroup)# verify-next-hop-ip 10.10.101.20CBS(conf-vrrp-vr-verify-next-hop)# priority delta 55CBS(conf-vrrp-vr-verify-next-hop)# exitCBS(conf-vrrp-vr-vapgroup)# exitCBS(conf-vrrp-group)#

h. Repeat Step 6, a through g, to add additional VRs to the failover group. Note that each VR can be assigned to only one VAP group.

8. Create this failover group on one or more additional X-Series Platforms.

Configure the FW VAP group for Failover

VRRP monitors the fw VAP group for failure of individual VAPs. By setting the active-vap-threshold and the priority-delta, individual VAP failures can decrement VRRP priority and cause a comparison with the VRRP priority on the remote chassis. Failover occurs when the priority-delta value decrements the priority value of the master failover group below the priority value of the associated failover group on the remote chassis. In our configuration, the priority value for the master failover group is 150 and the priority value for the backup group is 100. A priority-delta of 60 is used for each of the VAP groups.

To configure the fw VAP group for failover:

1. Enable VRRP on the fw VAP group.

CBS# configure vrrp vap-group fwCBS(conf-vrrp-vap-group)#

2. Assign the VRRP enabled fw VAP group to a failover group list (required).

CBS(conf-vrrp-vap-group)# failover-group-list failover1CBS(conf-vrrp-vap-group)#

3. Optionally, set the hold down timer.

Configure the hold-down-timer to 120, which forces the VAP group to wait for two minutes while the application fully boots. This wait prevents the failover group from becoming VRRP master before the application is fully active.

CBS(conf-vrrp-vap-group)# hold-down-timer 120CBS(conf-vrrp-vap-group)#


4. Optionally, set the active VAP threshold and return to the main CLI context.

The active-vap-threshold monitors the number of active VAPs in the VAP group. If the number of active VAPs drops below the threshold, the failover group’s priority value is decremented by the priority-delta (defined in Step 5) and a comparison is done between the priorities of the failover groups on the two chassis. Failover occurs when the priority-delta decrements the priority value of the master failover group below the priority value of the backup group.

CBS(conf-vrrp vap-group)# active-vap-threshold 3CBS(conf-vrrp vap-group)# endCBS#

5. Set the priority-delta value for the VAP group (optional).

Assign a priority-delta to the VAP group. VRRP decrements the priority of the failover group whenever the number of active VAPs falls below the active-vap-threshold.

The priority-delta value can be any number between 1 and 255 and the default value is 1. When the VAP returns to the Active state, the priority-delta value is added back to the priority value.

CBS(conf-vrrp-vap-group)# priority-delta 60CBS(conf-vrrp-vap-group)#

Manually Force a VRRP FailoverIt may be useful to force a VRRP failover from the Master failover group to the backup (for example, if you want to update the policies on the VAP group in the primary failover group). To force a failover, use the vrrp-relinquish-master <VRRP_failover_group> command.

Command:

CBS# vrrp-relinquish-master vrrp_fw_primaryDo you really want to relinquish VRRP Master?[N]

You must enter Y for the command to execute.

To manually return the primary failover group to master status, issue the vrrp-relinquish-master command on the currently active master:

CBS# vrrp-relinquish-master vrrp_fw_backupExecuting this command will force a VRRP failover. Are you sure? [N]


View VRRP StatusVRRP (DBHA) component status can easily be reviewed using the DBHA View in the Greenlight Element Manager (GEM). The DBHA View displays failover group status and remote box connections, including alarm conditions. Any alarm conditons are linked to details in the GEM Alarms Panel, simplifying identification and classification of the alarms. Refer to the GEM user assistance for additional information about the view.

Figure 10. DBHA View in GEM

You can also view the status of VRRP components and connections using the following XOS CLI commands:

show vrrp status [<failover_group_id>]

show vrrp detail-status [<failover_group_name>]

show remote-box

See View VRRP Status on page 178 for details on using these commands. Also, see the XOS Command Reference Guide for all information regarding usage, options, and syntax for these commands.


Example Dual Box High Availability ConfigurationThe following is an example of configuring two chassis in a Dual Box High Availability system, where the two management ports are configured as redundant interfaces to the High Availability ports.

NOTE: IPv6 addressing can be used for circuits and interfaces attached to a VAP group where IPv6 is enabled, including virtual IP addresses. However, management and control networks are configured with IPv4.

System 1system-identifier 1remote-box 2 1.1.2.20 192.168.30.2 192.168.100.2 192.168.30.22 192.168.100.22management gigabitethernet 13/1

ip-addr 192.168.30.1/24 192.168.30.255 enable access-list 1001 input access-list 1002 output

management gigabitethernet 13/2ip-addr 192.168.100.1/24 192.168.100.255 enable access-list 1001 input access-list 1002 output


management gigabitethernet 14/2ip-addr 192.168.100.11/24 192.168.100.255 enable access-list 1001 inputaccess-list 1002 output

System 2system-identifier 2remote-box 1 1.1.1.20 192.168.30.1 192.168.100.1 192.168.30.11 192.168.100.11management gigabitethernet 13/1

ip-addr 192.168.30.2/24 192.168.30.255 enable access-list 1001 input access-list 1002 output


management gigabitethernet 14/1 ip-addr 192.168.30.22/24 192.168.30.255 enable access-list 1001 input access-list 1002 output

management gigabitethernet 14/2ip-addr 192.168.100.22/24 192.168.100.255 enable access-list 1001 inputaccess-list 1002 output


Example VRRP ConfigurationsThis section presents two example VRRP configurations - a simple Active-Active example, and a complex Active-Standby example.

NOTE: IPv6 addressing can be used for circuits and interfaces attached to a VAP group where IPv6 is enabled, including virtual IP addresses. However, management and control networks are configured with IPv4.

Simple ConfigurationThe following is a simple example of two chassis in an active-active configuration:

System Avrrp failover-group firewall failover-group-id 1 priority 101 monitor-circuit firewall virtual-router vrrp-id 100 circuit gig21 vap-group fw virtual-ip 192.250.10.100/24 192.250.10.255 virtual-router vrrp-id 101 circuit gig22 vap-group fw virtual-ip 192.251.10.100/24 192.251.10.255

vrrp failover-group ips failover-group-id 2 priority 201 virtual-router vrrp-id 200 circuit gig31 vap-group ips virtual-ip 192.150.10.100/24 192.150.10.255 virtual-router vrrp-id 201 circuit gig32 vap-group ips virtual-ip 192.151.10.100/24 192.151.10.255#vrrp vap-group fw priority-delta 2

active-vap-threshold 3#vrrp vap-group ips

priority-delta 2active-vap-threshold 3

System Bvrrp failover-group firewall failover-group-id 1

priority 201virtual-router vrrp-id 100 circuit gig11

vap-group fw virtual-ip 192.250.10.100/24 192.250.10.255

virtual-router vrrp-id 101 circuit gig12vap-group fw virtual-ip 192.251.10.100/24 192.251.10.255

vrrp failover-group ips failover-group-id 2priority 101virtual-router vrrp-id 200 circuit gig31

vap-group ips virtual-ip 192.150.10.100/24 192.150.10.255

virtual-router vrrp-id 201 circuit gig32vap-group ips virtual-ip 192.151.10.100/24 192.151.10.255


#vrrp vap-group fw


#vrrp vap-group ips


Complex ConfigurationThe following is a example of two chassis in an Active-Standby configuration with VLANs.

Figure 11. Two System Active-Standby Configuration

System Avap-group fw xslinux_v3#circuit inside3 circuit-id 1025

device-name inside3vap-group fw

default-egress-vlan-tag 3 hide-vlan-headerip 192.168.1.10/24 192.168.1.255

circuit inside4 circuit-id 1026device-name inside4vap-group fw

default-egress-vlan-tag 4 hide-vlan-header


ip 4.4.4.10/24 4.4.4.255circuit outside5 circuit-id 1027

device-name outside5vap-group fw


circuit outside6 circuit-id 1028device-name outside6vap-group fw


interface gigabitethernet 1/1logical inside3 ingress-vlan-tag 3 3

circuit inside3logical inside4 ingress-vlan-tag 4 4

circuit inside4interface gigabitethernet 1/2

logical outside5 ingress-vlan-tag 5 5circuit outside5


vrrp failover-group left failover-group-id 1priority 200monitor-circuit inside3

priority-delta 10monitor-circuit inside4

priority-delta 10monitor-circuit outside5


priority-delta 10virtual-router vrrp-id 103 circuit inside3

priority-delta 20mac-usage vrrp-macvap-group fw

virtual-ip 192.168.1.50/24 192.168.1.255virtual-router vrrp-id 104 circuit inside4


virtual-ip 4.4.4.50/24 4.4.4.255virtual-router vrrp-id 105 circuit outside5




virtual-ip 10.10.1.50/24 10.10.1.255#vrrp vap-group fw


System Bvap-group fw xslinux_v3


#circuit inside3 circuit-id 1025

device-name inside3vap-group fw


circuit inside4 circuit-id 1026device-name inside4vap-group fw






interface gigabitethernet 1/1logical inside3 ingress-vlan-tag 3 3

circuit inside3logical inside4 ingress-vlan-tag 4 4

circuit inside4interface gigabitethernet 1/2



vrrp failover-group right failover-group-id 1priority 190monitor-circuit inside3

priority-delta 10monitor-circuit inside4



priority-delta 10virtual-router vrrp-id 103 circuit inside3


virtual-ip 192.168.1.50/24 192.168.1.255virtual-router vrrp-id 104 circuit inside4





priority-delta 20


mac-usage vrrp-macvap-group fw

virtual-ip 10.10.1.50/24 10.10.1.255#vrrp vap-group fw


Configuring VRRP and Automatic ARP for NATIn some specific VRRP configurations, if a firewall application is configured with automatic ARP for NAT both chassis may reply to the NAT ARP requests. This section describes configuration options for DBHA using VRRP and dynamic routing so that both chassis do not reply to the NAT ARP requests.

NAT MAC UsageThe Checkpoint VPN-1 Power NGX application uses the interface MAC as the virtual MAC when creating NAT entries. On the X-Series Platform, the interface is a circuit or VND (virtual network device). This VND MAC is either a circuit defined MAC, or an interface level MAC.

Beginning with XOS 8.0, the MAC address manual definition is at the circuit VAP group (mac-addr - Sets MAC address) level:

From the CBS CLI

CBS# show circuit insideCircuit Name : insideCircuit-Id : 1Device Name : inside...MAC Address : 00:03:d2:f5:48:02 (system-reserved)...

Ifconfig from the VAP

inside Link encap:Ethernet HWaddr 00:03:d2:f5:48:02

To prevent an incorrect NAT associated MAC configuration between the two chassis, the same VRRP MAC is defined at circuit level.

With the VND level MAC address set the same as the VRRP IP address, VRRP gratuitous ARP generation upon failover will update the adjacent switch CAM table. This will ensure NAT automatic ARP failover is also transparent and performed efficiently.

Interface MAC LimitationsWhen configuring the X-Series Platform, you can associate multiple VNDs with a single interface differentiated by VLAN tags. These VNDs may belong to the same or different VAP groups. When using NAT with automatic ARP within VPN-1, the VND MAC address can be configured with the same address as the virtual VRRP MAC.

When configuring Interface level MAC or circuit level MAC options with the same MAC address as the VRRP virtual MAC address, all of the associated VRRP failover group’s priorities have to be the same, to provide consistent master or backup status.

The MAC replication configuration mentioned above, in addition to having no support for a VND level IP address, prevents the user from switching VRRP master/backup and backup/master statuses between chassis and VAP groups.


VRRP ConfigurationTo prevent the VRRP backup circuit from responding to automatic ARP requests, legacy style VRRP should be used.

Legacy VRRP uses the IP address at the circuit level. There is no concept of virtual and interface level IP addresses; only the virtual IP (VIP) address. When the circuit is in backup, the VND remains down, not populating the routing table or responding to ARP/IP requests.

The user also has the option of configuring both a circuit level IP address and a VIP.

However, in this scenario the circuit level ip address should not be defined, nor should the VRRP backup-stay-up command be used. When configuring legacy VRRP the IP is no longer defined at the circuit level, but at the VRRP VAP group level as an IP address (not a virtual-ip.).

virtual-router vrrp-id 2 circuit outsidevap-group fw

ip 192.168.102.1/24 192.168.102.255

After the legacy VRRP option has been defined, allowing the addition of more IP addresses, you can create more VRRP failover groups and define additional virtual-ip addresses. Both the legacy and virtual (sub interface) circuits will remain down when VRRP is in backup.

From the CLI:

vrrp failover-group fw failover-group-id 1virtual-router vrrp-id 2 circuit outside

vap-group fw ip 192.168.102.1/24 192.168.102.255

vrrp failover-group fw2 failover-group-id 2virtual-router vrrp-id 3 circuit outside

vap-group fw virtual-ip 192.168.102.2/24 192.168.102.255

One drawback to this type of configuration is not being able to use the VRRP next hop health check. VRRP next hop health check uses the circuit level IP address to check the availability of a pre-configured gateway. If the IP address is not reachable, a priority-delta is deducted.


Dynamic Routing, VRRP, and NAT CircuitsThere are several ways to configure dynamic routing on the X-Series Platform: one example is to configure OSPF on the same IP stack as VRRP, with OSPF configured at the circuit level, and VRRP using a sub circuit or virtual router. This does not work with NAT and Automatic ARP.

The most effective method is to keep VRRP and dynamic routing separate. When designing a solution with OSPF, VRRP, and automatic ARP, VRRP circuits should be defined as OSPF passive. Or they should be configured as an AS external and different circuits used for OSPF. When VRRP migrates the OSPF process will send an update LSA to its adjacent devices advertising the stub networks availability.

With OSPF running on different circuits, the update by the VRRP master and backup chassis would be immediate. The worst circumstance is the VRRP master chassis dies so hello packets are no longer received from the original master, and the VRRP backup chassis switches VRRP from backup to master. In this case the OSPF transitive paths have already been calculated because these circuits had previously formed adjacency, now the second chassis is the only one advertising the VRRP circuit network.

To prevent the earlier automatic ARP issue, the circuit level IP address cannot be active, this prevents the user from configuring OSPF, VRRP and enabling automatic ARP for NAT on the same circuit(s).

As described above, you can define OSPF on an alternative transitive pair of circuits and VRRP in legacy mode on another circuit, enabling the user to have all three working together on the same firewall.


13Configuring Secure Flow Processing

You can configure the X-Series system to run multiple applications in series and in parallel. This chapter includes the following major topics:

What Is Secure Flow Processing on page 151

Serial Traffic and Circuits on page 151

Managing Serialized VAPs on page 153

Secure Flow Processing on Layer 3 on page 154

Parallel Traffic Flow on page 156

What Is Secure Flow ProcessingOne aspect of Secure Flow Processing is the concept of configuring VAP groups in series and parallel to more efficiently pass traffic. Serializing data traffic through multiple applications allows you to inspect incoming traffic at multiple levels. There are two significant advantages to serializing applications on the Crossbeam X-Series Platform:

The ability to move traffic / data between VAP groups within the X-Series Platform. There is no requirement of a physical interface to pass traffic between the APMs.

Once IP traffic has been classified by the NPM, serialized traffic uses the speed of the data plane to move between APMs; it is not limited to the speed of the physical interface.

Serialization provides the ability to configure and link multiple VAP groups running different applications. These specific applications are linked in series, allowing traffic to be inspected by each application. Multiple VAP groups are created (one per application) with circuits configured between the VAP groups. Traffic is passed from one VAP to the next, allowing for multi-layered, in-depth inspection by best-of-breed security applications within a single device.

Previous versions of XOS that supported serialization were restricted to certain applications working on a Layer 3 routing level. XOS provides numerous enhancements to allow secure flow processing by bridging Layer 2 applications linked to Layer 3 applications. For detailed serialization configuration information, refer to the Multi-Application Serialization Configuration Guide.

Serial Traffic and CircuitsTo create a virtual connection between two or more VAPs, a circuit is configured on two VAP groups, and assigned to an interface-internal. In a serial configuration, traffic is directed through one application, across the circuit, and then through a second application.

XOS exposes these internal interfaces and virtual circuits to the CLI, allowing users to to configure serial connections from Layer 2 to Layer 3, and across Layer 3 applications. Additionally, user access to the internal interfaces and virtual circuits allows the serialized configuration to be included in a VRRP / Dual Box High Availability configuration, and in the case of module or interface failures, failed over to a back up system.

For information about configuring VRRP and failover groups, refer to the Multi-System High Availability Configuration Guide.


IPv6 addressing is supported on serialized configurations, with an IPv6-enabled VAP group. This configuration can include VLANs and virtual IP addresses. IPv6 is not supported on the management network.

Configuring a Serial Circuit Serial circuits between VAP groups are created in the same manner as regular circuits. The circuit is attached to two VAP groups, and later assigned to an interface-internal. The following configuration example shows the circuit between connecting two Layer Three VAP groups, and then defined as part of an interface-internal.

circuit between device-name between vap-group fw ip 9.0.0.1/24 vap-group ips ip 9.0.0.2/24

interface-internal betweenlogical-all between

circuit between

The following is an example of IPv6 addressing:

circuit between device-name between vap-group fw ip 2002:1:F:0:0:0:0:F5/32 vap-group ips ip 2002:1:F:0:0:0:0:E9/32

interface-internal betweenlogical-all between

circuit between

To configure serialization between a Layer 2 and a Layer 3 application, the serial circuit is configured as part of the Layer 2 bridge. The configuration is slightly different from the Layer 3 configuration, since the Layer 2 VAP group does not have an IP address, and must be in promiscuous mode. The following example assumes that the two VAP groups (IPS and FW) have already been configured, and shows only the high level steps for configuring serialization. For detailed procedures, refer to the Multi-Application Serialization Configuration Guide.

1. Configure the circuits.

LAN (on the client subnet side)

Br1 (the layer two bridge)

Ser1 (the serial circuit between the VAP groups)

WAN (connected to an external network)

2. Configure Bridge Mode

Br1 (bridge-mode transparent)

3. Configure Interfaces

LAN (physical interface)

WAN (physical interface)

Interface-Internal (the interface that connects the circuit internally to the VAP groups and the bridge)

Configuring Secure Flow Processing 152

A sample Layer 2 to Layer 3 serialization configuration:

circuit LAN device-name LANvap-group IPS

promiscuous-mode active circuit Br1

device-name Br1vap-group IPS

circuit Ser1device-name Ser1vap-group IPS

promiscuous-mode activevap-group FW

circuit WANdevice-name WANvap-group FW

ip 172.16.20.7/24#bridge-mode Br1 transparent

circuit LANcircuit Ser1

#interface gigabitethernet 1/1

logical LANcircuit LAN

interface gigabitethernet 2/1logical WAN

circuit WANinterface-internal Br1

logical-all Ser1circuit Ser1

Managing Serialized VAPsManaging of multiple VAPs is often done using individual physical connections to the modules. With serialized applications it is more efficient to manage both VAP Groups using a single physical interface. To accomplish this, you only need to assign both VAP Groups to the same circuit.

circuit mgmt device-name mgmt vap-group fw ip 192.168.0.1/24 increment-per-vap 192.168.0.3 vap-group ips ip 192.168.0.6/24 increment-per-vap 192.168.0.8

When configuring the management IP addresses it is recommended to leave some unused IP addresses to allow for future growth. In the example above, the addresses 192.168.0.4 and 192.168.0.5 are left unassigned, thus allowing for additional VAPs to be added to the “fw” VAP group.


Secure Flow Processing on Layer 3The following example explains a simple serial traffic flow, and provides basic circuit and interface configuration steps. For detailed steps to configure a basic serialized topology, please refer to the Multi-Application Serialization Configuration Guide.

Figure 12. Example Layer 3 Serialization Topology

1. Create individual VAP groups.

vap-group L3_VAP_1vap-count 2max-load-count 2ap-list ap1 ap2ip-flow-rule L3_default

action load-balanceactivate

#

vap-group L3_VAP_2vap-count 1max-load-count 1ap-list ap3 ip-flow-rule L3_default_2


#

2. Create a circuit for incoming traffic.

circuit incoming device-name incomingvap-group L3_VAP_1

ip 10.0.0.1/24


3. Create a circuit between VAP groups, and associate the circuit with both VAP groups.

circuit l2l3 device-name l2l3vap-group L3_VAP_1

ip 5.5.5.1/24 vap-group L3_VAP_2

ip 5.5.5.2/24

4. Create a circuit for outgoing traffic, and associate it with the VAP group it will be leaving through.

circuit outside device-name outsidevap-group L3_VAP_2

ip 20.0.0.1/24

5. Define an interface-internal and assign the serial circuit l2l3.

interface-internal l2l3logical-all l2l3

circuit l2l3

6. Configure the route between VAPS

config ip route 20.0.0.1/24 ip 5.5.5.2 vap-group L3_VAP_1config ip route 10.0.0.1/24 ip 5.5.5.1 vap-group L3_VAP_2

7. Create and configure management circuit.

circuit mgmtdevice-name mgmtvap-group L3_VAP_1

ip 192.168.0.1/24 increment-per-vap 192.168.0.3vap-group L3_VAP_2

ip 192.168.0.6/24 increment-per-vap 192.168.0.8

8. Create the interfaces for each circuit.

interface gigabitethernet 1/1logical incoming

circuit incominginterface gigabitethernet 2/1

logical outsidecircuit outside

interface gigabitethernet 1/8 logical management

circuit mgmt

This will create a unique IP address on each VAP in the group, which is required for a management circuit. The circuit must have the increment-per-vap parameter, even if the VAP group contains only one VAP.

Once the VAP groups, circuits, management circuits, and all interfaces are configured, you can install the applications on each VAP. Refer to your application documentation for installation information.


Parallel Traffic FlowParallel traffic flow copies traffic flow to one or more passive applications, simultaneously. This flow is achieved by mapping a circuit from the physical interface to multiple VAP groups.

IP flow rules at the VAP level can be used to move traffic simultaneously through multiple security engines, such as a Layer 3 firewall and IDS.

IPv6 addressing is supported on parallel configurations, with an IPv6-enabled VAP group. This configuration can include VLANs and virtual IP addresses. IPv6 is not supported on the management network.

Figure 13. Example Parallelization Topology

1. First, create individual VAP groups.

vap-group L3_FWvap-count 2max-load-count 2ap-list ap1 ap2load-balance-vap-list 4 5 6 7 8 9 10 1 2 3ip-flow-rule L3_default


#vap-group IDS_VAP

vap-count 1max-load-count 1ap-list ap3 load-balance-vap-list 4 5 6 7 8 9 10 1 2 3ip-flow-rule L3_default_2


#


2. Next create a circuit for incoming traffic.

circuit incoming device-name incoming

3. The following example assigns a circuit to two VAP groups to parallelize traffic. Traffic coming into the circuit goes to both VAP groups simultaneously.

configure circuit incomingvap-group L3_FW

ip 10.10.10.20/24vap-group IDS_VAP

promiscuous-mode

4. Create a circuit for outgoing traffic, and associate it with the VAP group it will be leaving through.

circuit outgoing device-name outgoingvap-group L3_FW

ip 5.5.5.15/24

5. Create and configure management circuit.

circuit mgmt device-name mgmtvap-group L3_FW ip 192.168.0.1/24 increment-per-vap 192.168.0.3 vap-group IDS_VAP ip 192.168.0.6/24 increment-per-vap 192.168.0.8

6. Associate virtual circuits with physical interfaces.

Interface gigabit 1/1logical incoming

circuit incoming

interface gigabit 1/2 logical outgoing

circuit outgoing

interface gigabit 1/3 logical mgmt

circuit mgmt

7. Once the VAP groups, circuits, and management circuits are configured, you can install the applications on each VAP. Refer to your application documentation for installation information.



14Configuring RMON and SNMP Traps

The X-Series Platform supports Remote Monitoring (RMON) Event and Alarm groups. You can configure Remote Monitoring (RMON) and simple network management protocol (SNMP) to monitor your X-Series Platform. Use RMON for industry-standard management information base (MIB) entries and use SNMP traps to implement Crossbeam MIB entries for system monitoring.

The following topics are covered in this chapter:

System Owned RMON Alarms and Events on page 159

Configuring RMON Events on page 160

Configuring RMON Alarms on page 160

Displaying RMON Alarms, Events, and Logs on page 161

Displaying Disk Monitoring Event and Alarm Entries on page 162

Configuring Simple Network Management Protocol (SNMP) on page 163

Supported SNMP Traps on page 164

SNMP Traps and Informs on page 164

Configuring Trap Destinations on page 165

Displaying the SNMP Trap Log on page 165

SNMP MIB Dependencies on page 166

Allowing SNMPv3 User Access on page 167

Refer to Crossbeam SNMP MIB Reference on page 267 for a comprehensive guide to Crossbeam MIB entries and their corresponding object identifiers (OIDs).

Configuring RMONNOTE: RMON configuration is not supported on an offline CPM.

System Owned RMON Alarms and EventsSome RMON alarm and event entries are system owned and can not be modified or deleted by the user. The “owner” string for these entries reads “system” and their alarm index range is 65000 and above.

A user defined alarm entry should use indices lower than 65000. If a “system” owned alarm entry’s type indicates “trap” then a trap is sent out to all the configured trap destinations, irrespective of the community string used in that system owned alarm entry.

Look for the industry standard RMON MIBs in the /usr/share/snmp/mibs directory.


Configuring RMON EventsEach RMON event entry includes one of the following action types:

Log and Trap

Log only

Trap only

Use the event action types to configure the RMON Alarm entries, and dictate the action taken by the RMON agent when an alarm entry crosses its configured threshold.

After you configure RMON, save the configuration settings with copy running-config start-config so that the configuration persists across system restarts.

To configure the RMON event action type, use the following commands:

configure rmon event 1 trap <community-string> description “trap only”configure rmon event 2 log description “log only” owner <name>configure rmon event 3 log trap <community-string> description “log and trap”

For example:

CBS# configure rmon event 1 trap public description "trap only" CBS# configure rmon event 2 log description "log only" owner user1 CBS# configure rmon event 3 log trap public description "log and trap"

NOTE: The community string cannot contain whitespace characters (space or tab).

Configuring RMON AlarmsRMON alarms provide for periodical statistical sampling of variables, and comparing them with previously configured thresholds. If the monitored variables cross the specified thresholds, RMON generates an event. For each alarm entry, you must specify the following:

SNMP variable

Sampling interval

Delta vs. absolute value

Rising and falling threshold values

Rising and falling event entries

Optionally the owner for the entry

To configure RMON alarms, use the following command:

configure rmon alarm <number> <variable> <interval> {delta|absolute} rising-threshold <value> [<event-number>] falling-threshold <value> [<event-number>] [owner <owner>]

For example:

CBS# configure rmon alarm 1 ifEntry.ifInOctets.1 10 absolute rising-threshold 80 3 falling-threshold 60 3 owner user1

CBS# configure rmon alarm 2 ifOutUcastPkts.1 10 delta rising-threshold 80 3 falling-threshold 60 3 owner user1

Configuring RMON and SNMP Traps 160

Displaying RMON Alarms, Events, and LogsTo display an RMON trap or log, use the following commands:

show rmon log show rmon alarms show rmon events

For example:

CBS# show rmon log

CBS# show rmon alarmsAlarm No. : 65000Interval (secs) : 60Variable : .1.3.6.1.4.1.2021.9.1.9.1Sample Type : AbsoluteValue : 28Startup : risingRising Threshold : 70Falling Threshold : 60Rising Event : 65000Falling Event : 65001Owner : systemEntry Status : valid

Alarm No. : 65001Interval (secs) : 60Variable : .1.3.6.1.4.1.2021.9.1.9.2Sample Type : AbsoluteValue : 26Startup : risingRising Threshold : 70Falling Threshold : 60Rising Event : 65000Falling Event : 65001Owner : systemEntry Status : valid

Alarm No. : 65002Interval (secs) : 60Variable : .1.3.6.1.4.1.2021.9.1.9.3Sample Type : AbsoluteValue : 29Startup : risingRising Threshold : 70Falling Threshold : 60Rising Event : 65000Falling Event : 65001Owner : systemEntry Status : valid

Event Index Log Index Log Time Log Description

3 1 00:00:01 Alarm [index 1] crossed FALLING threshold. alarmValue=0, threshold=60

3 2 00:00:11 Alarm [index 2] crossed FALLING threshold. alarmValue=0, threshold=60


Alarm No. : 65003Interval (secs) : 60Variable : .1.3.6.1.4.1.2021.9.1.9.4Sample Type : AbsoluteValue : 17Startup : risingRising Threshold : 70Falling Threshold : 60Rising Event : 65000Falling Event : 65001Owner : systemEntry Status : valid(4 rows)CBS# show rmon eventsEvent Number : 65000Event Type : Log_n_trapCommunity :Last Time Sent : 00:00Owner : systemDescription : Disk Usage Crossed Upper Threshold

Event Number : 65001Event Type : Log_n_trapCommunity :Last Time Sent : 00:00Owner : systemDescription : Disk Usage Crossed Lower Threshold(2 rows)

Displaying Disk Monitoring Event and Alarm EntriesThe X-Series Platform tracks two system owned events for disk usage crossing the upper and lower thresholds values and four alarms (for four partitions) for monitoring disk usage on CPM. Additional information about the partitions whose disk-usage crosses these thresholds can be obtained through SNMP by reading the diskTable MIB or by using the CLI/GUI commands/views on disk usage/history.

The four partitions are (using the snmpwalk dump on diskTable):

dskTable.dskEntry.dskPath.1 = / dskTable.dskEntry.dskPath.2 = /boot dskTable.dskEntry.dskPath.3 = /cbconfig dskTable.dskEntry.dskPath.4 = /tftpboot

dskTable.dskEntry.dskDevice.1 = /dev/hda7 dskTable.dskEntry.dskDevice.2 = /dev/hda1 dskTable.dskEntry.dskDevice.3 = /dev/hda6 dskTable.dskEntry.dskDevice.4 = /dev/hda8

The X-Series Platform also monitors the dskPercent for these four partitions using system owned RMON alarm entries.

dskTable.dskEntry.dskPercent.1 dskTable.dskEntry.dskPercent.2 dskTable.dskEntry.dskPercent.3 dskTable.dskEntry.dskPercent.4

You can display RMON alarms and events, as described in Displaying RMON Alarms, Events, and Logs on page 161.


Configuring Simple Network Management Protocol (SNMP)In order for the X-Series Platform to communicate with a SNMP-based network management system on your network, you must configure the SNMP settings appropriately.

Access the SNMP configuration parameter in the EMS GUI from navigating to Configuration > Protocols > SNMP. The basic command in the XOS CLI

Configuring an SNMP ServerUse the configure snmp-server community command to define an SNMP community on the server consisting of this X-Series Platform (the SNMP server) and one or more SNMP management stations (the SNMP clients). The SNMP management station(s) included in the community use the specified community string as a password to read from the SNMP agent on the X-Series Platform.

configure snmp-server community <community_string> {<IP_address> | {<IP_address> <subnet_mask> | <IP_address>/<0-32>}}

See the XOS Command Reference Guide for detailed syntax, usage, and parameter explanations of this command.

Table 17 lists the parameters used with this command.

Table 17. configure snmp-server parameters

Parameter Description

<community_string> Community string, expressed as a text string, assigned to the SNMP community that you are defining.

The SNMP management stations use the specified community string as a password to read the SNMP agent on the X-Series Platform.

NOTE: The community string cannot contain whitespace characters.

<IP_address> Configures the SNMP community to include the SNMP agent on the X-Series Platform (the SNMP server) and the single SNMP management station (SNMP client) with the specified IP address.

{<IP_address> <subnet_mask> | <IP_address>/<0-32>}

Configures the SNMP community to include the SNMP agent on the X-Series Platform (the SNMP server), and the SNMP management station within the specified IP address range.

You can specify the IP network and subnet mask address in CIDR format (for example, 10.15.0.0/16), or non-CIDR format (for example, 10.16.0.0 255.255.0.0).


Configuring SNMP TrapsThe X-Series Platform supports SNMP Versions 1, 2c, and 3 while supporting the notification types of either Trap or Inform (Version 2c only). This section describes how to configure the X-Series Platform to capture and view traps generated by the X-Series Platform.

The main topics in this section are:

SNMP Traps and Informs on page 164

Supported SNMP Traps on page 164

Configuring Trap Destinations on page 165

Displaying the SNMP Trap Log on page 165

SNMP MIB Dependencies on page 166

Allowing SNMPv3 User Access on page 167

Refer to Crossbeam SNMP MIB Reference on page 267 for a list of Crossbeam-specific MIB entries and their corresponding object identifiers (OIDs).

Refer to SNMP MIB Dependencies on page 166 for a list of the SNMP MIB dependencies and compile orders required by some SNMP applications. Refer to the SNMP application documentation for specific requirements.

SNMP Traps and Informs

There are two types of SNMP notification messages: SNMP traps and SNMP informs. You can configure the SNMP notification receiver host to accept one of these two types of messages.

The host confirms receipt of SNMP informs, but does not confirm receipt of SNMP traps. Therefore, if you want to send critical notifications to the host, you should configure the host to receive SNMP informs.

NOTE: SNMP informs are only available for use with devices that support SNMP version 2c.

Supported SNMP Traps

The XOS software supports the following SNMP traps. For a complete list of all SNMP traps, refer to the Crossbeam SNMP MIB Reference on page 267:

Cold and warm starts.

Chassis temperature exceeded/normal - indicates the chassis temperature status, as measured at the upper fan tray. This is indicated as normal or exceeded.

System alarm state - indicates the highest current alarm state of the X-Series Platform.

Module temperature exceeded/normal - indicates the module temperature status as normal or exceeded.

Module insert/remove - indicates that a module has been removed or inserted in the XOS backplane.

Module state changes - indicates the various states of each module in the X-Series Platform. Valid states include: unavailable, down, initializing, up, standby, and bootwait.

CPU load exceeded/normal - indicates the CPU load of the processor modules and indicates whether the CPU is seeing normal or excessive loads. This is the current value of the one minute load on the CPU module that is experiencing normal or excessive loads.

Link State - indicates the operational state (up or down) of the control link.

Power supply failed/recovered - indicates that a redundant power supply has either failed or been replaced.

Power supply feed failed/recovered - indicates that a redundant power supply feed has either failed or become operational.

Fan tray insert/remove - indicates that a fan tray has been removed or inserted in the X-Series Platform.


Fan failed and recovered - indicates the various states of each fan in the fan trays.

Voltage - Indicates whether CPU and Module voltages are within or exceeding normal ranges.

Module Memory Usage - indicates memory usage per module.

CPU Utilization - indicates usage of all cores on the specified module.

CPU Core Utilization - indicates percentage of total use of CPU core, per core, on a specified module. CPU core utilization can be tracked for APMs and CPMs.

NPM Flow Table Usage - indicates total flow table usage.

NPM Flow Table Protocol - indicates the flow table limit status (exceeded / met / normal) of flows by protocol. Protocols include TCP, UDP, ICMP, and Other.

Module SDRAM check

Configuring Trap Destinations

Use the following to configure a trap destination:

The SNMP Host IP Address

Specify the SNMP Version (V1 or V2c)

Notification Type (traps or informs)

Community String to identify the trap.

To configure a trap destination, use the following command:

configure snmp-server host <host_ip_addr> [traps|informs] [version 1|2c] <community-string> [udp-port <port-number>]

NOTE: The community string cannot contain whitespace characters (space or tab).

For example:

configure snmp-server host 10.1.1.29 traps version 1 private configure snmp-server host 10.1.1.29 informs version 2c public

To delete a host, use:

configure no snmp-server host <host_ip_addr> <community-string>

NOTE: If the host that you wish to delete currently receives informs, you must specify the informs parameter with this command.

Displaying the SNMP Trap Log

XOS maintains a rotating log of the last 100 SNMP traps issued by the X-Series Platform. A trap is counted only once even though it may have been sent to several destinations. Associated trap variables, for example, sysUpTime, as well as Date and Time are also recorded in the log.

The following example shows a partial display of an SNMP trap log:

CBS# show traplogTrap Description : cbsHwModuleStatusChangedTrap OID : .1.3.6.1.4.1.6848.4.1.14sysUpTime : 23:12:10Time & Date : 2006-11-02 23:00:44.90Num of variables : 1Variable 1 : cbsHwModuleStatus.1 = up(4)Variable 2 :Other Variables :


Trap Description : cbsHwModuleStatusChangedTrap OID : .1.3.6.1.4.1.6848.4.1.14sysUpTime : 23:10:50Time & Date : 2006-11-02 22:59:25.09Num of variables : 1Variable 1 : cbsHwModuleStatus.1 = initializing(3)Variable 2 :Other Variables :

NOTE: The sysUpTime value is the time accumulated since the SNMP agent was configured.

SNMP MIB DependenciesCertain SNMP applications require that SNMP MIB dependencies and compile orders be defined. The following lists detail these dependencies for Crossbeam-specific MIBs (CBS-*).

Base SNMP MIBs required for CBS MIBs, with dependencies:

SNMPv2-SMI: root

SNMPv2-TC: SNMPv2-SMI

SNMPv2-CONF: SNMPv2-SMI

SNMPv2-MIB: SNMPv2-SMI, SNMPv2-TC, SNMPv2-CONF

IANAifType-MIB: SNMPv2-SMI, SNMPv2-TC

IF-MIB: SNMPv2-SMI, SNMPv2-TC, SNMPv2-CONF, SNMPv2-MIB, IANAifType-MIB

INET-ADDRESS-MIB: SNMPv2-SMI, SNMPv2-TC

HOST-RESOURCES-MIB: SNMPv2-SMI, SNMPv2-TC, SNMPv2-CONF, IF-MIB

CBS MIBs with dependencies:

CROSSBEAM-SYSTEMS-MIB: SNMPv2-SMI

CBS-HARDWARE-MIB: SNMPv2-SMI, SNMPv2-TC, HOST-RESOURCES-MIB, CROSSBEAM-SYSTEMS-MIB

CBS-MODULE-RESOURCES-MIB: SNMPv2-SMI, SNMPv2-TC, HOST-RESOURCES-MIB, CROSSBEAM-SYSTEMS-MIB, CBS-HARDWARE-MIB

CBS-VAPGROUP-MIB: SNMPv2-SMI, SNMPv2-TC, HOST-RESOURCES-MIB, CROSSBEAM-SYSTEMS-MIB, CBS-HARDWARE-MIB

CBS-VRRP-MIB: SNMPv2-SMI, SNMPv2-TC, HOST-RESOURCES-MIB, CROSSBEAM-SYSTEMS-MIB, CBS-HARDWARE-MIB, INET-ADDRESS-MIB, CBS-VAPGROUP-MIB

Compile order based upon above dependencies (items on same line may be compiled in any order):

1. SNMPv2-SMI

2. SNMPv2-TC, SNMPv2-CONF, CROSSBEAM-SYSTEMS-MIB

3. SNMPv2-MIB, IANAifType-MIB, INET-ADDRESS-MIB

4. IF-MIB

5. HOST-RESOURCES-MIB

6. CBS-HARDWARE-MIB

7. CBS-MODULE-RESOURCES-MIB, CBS-VAPGROUP-MIB

8. CBS-VRRP-MIB


Allowing SNMPv3 User AccessYou can allow SNMPv3 users to have read-only access to the X-Series Platform. To configure access:

configure snmp-user <username> [no-passwords] [auth-type [md5|sha|none]] [priv-type [des|none]] [oid <access>]

Where:

<username> Must be a unique name.

no-passwords Do not prompt the user for a password. By default, the user must enter the passwords configured with auth-type and priv-type.

auth-type [md5|sha|none]

Specifies the authentication method for the user. Choices include no authentication (none), MD5 checksum, and SHA authentication. The default is none.

priv-type [des|none] Use des to encrypt data or none (default) to not encrypt data. If using des, auth-type must be md5 or sha.

oid <access> Specifies the MIB subtree that the user can access. For example, specify “iso” for the whole tree, or “mib-2” to limit access to just the MIB objects that are part of the mib-2 tree. Smaller portions of the MIB can be selected, such as entering “interfaces” to restrict access to just the interface table. The default is .iso.

The following OID formats are allowed:

numeric oids, such as .1.3.6.1

fully qualified oid names, such as .iso.org.dod

names directly under mib-2, such as “system”, “interfaces”, “at”, and “ip”



15Using UNIX Commands on VAP Groups

and Upgrading Applications

This chapter describes the UNIX commands used on all VAPs within a VAP group. It contains the following major topics:

Executing UNIX Commands on a Designated VAP on page 169

Executing UNIX Commands on All VAPs on page 170

The /usr/os/bin/cbs_rsh tool can be used to execute any interactive UNIX command on all available VAPs in a VAP group or on a particular VAP within a VAP group.

Executing UNIX Commands on a Designated VAPTo execute UNIX commands on a designated VAP in a VAP group, use the following command:

/usr/os/bin/cbs_rsh <VapGroupName> <VapIndex>

This command takes you to an rsh session on the specified VAP. Once executed, you can execute any UNIX command on that VAP.

The following is an example for a VAP with an index of 2 in a VAP group named vgSync. In this example, the command would be entered as follows:

[root@xxxxx]# /usr/os/bin/cbs_rsh vgSync 2 Last login: Mon Mar 08 17:33:09 from primarycpmvgSync_2(xxxxx):root$vgSync_2(xxxxx):root$ ps

vgSync_2(xxxxx):root$vgSync_2(xxxxx):root$ exitlogout

rlogin: connection closed.[root@xxxxx]#

PID TTY TIME CMD

1651 pts/0 00:00:00 login

1652 pts/0 00:00:00 bash

1672 pts/0 00:00:00 ps


Executing UNIX Commands on All VAPsTo execute UNIX commands on a all VAPs within a VAP group, use the following command:

/usr/os/bin/cbs_rsh <vapGroupName>

This command causes the following prompt to display:

vapGroupName>>

Commands entered at this prompt are either specific to cbs_rsh, or executed directly on each VAP within the VAP group. Once executed, the command results are displayed. When the execution of all commands has completed you are returned to the VAP Group Name prompt.

Commands specific to the cbs_rsh are as follows:

Table 18. Commands Specific to cbs_rsh

Copying a File from the CPM to all the VAPs in a VAP GroupTo copy a file from the CPM to all the VAPs within a VAP group, use the cbs_cp command at the VAP group prompt. The following example copies the file SomeFile to the VAPs in the FW VAP group.

1. Issue the cbs_rsh command with the desired VAP group (FW in our example) to access the VAP group prompt. Help for the cbs_rsh commands is displayed along with the VAP group prompt:

[root@xxxxx admin]# /usr/os/bin/cbs_rsh FW !p --- Execute previous UNIX command quit/q --- Exit this program cbs_cp --- Copy a file on CPM to all vaps within a vap-group cbs_start_s --- Start saving commands for VAPs currently not present cbs_stop_s --- Stop saving commands for VAPs currently not present FW>>

2. Enter the cbs_cp command at the VAP group prompt; the system prompts for the source of the file to be copied. Enter the source with its full path as shown in the example below.

FW>>cbs_cp

Source File on CPM (with full path) :/tftpboot/.private/home/admin/SomeFile

3. When prompted, enter the full destination path.

Destination File on VAP (with full path) :/root/SomeFile

Command Description

quit /q Exits the cbs_rsh session.

? /help Displays help menu.

cbs_start_s Starts saving commands for those VAPs that are down. When these VAPs come back up, all the save commands are executed on that VAP.

cbs_stop_s Stops saving commands for those VAPs that are down.

cbs_cp Copies a file on the CPM to all VAPs within the VAP group.

!p Execute previous UNIX command.

Using UNIX Commands 170

4. The system then issues a copy command for each VAP in the VAP group by appending VAP group index number to the VAP group name (in this example: FW_1, FW_2, etc).

The full example is shown below:

[root@xxxxx bin]# /usr/os/bin/cbs_rsh FW !p Execute previous UNIX commandquit/q Exit this program cbs_cp Copy a file on CPM to all vaps within a vap-group cbs_start_s Start saving commands for VAPs currently not present cbs_stop_s Stop saving commands for VAPs currently not presentFW>>cbs_cp Source File on CPM (with full path) :/tftpboot/.private/home/admin/SomeFile Destination File on VAP (with full path) :/root/SomeFile Copying :/tftpboot/.private/home/admin/SomeFile to /tftboot/FW_1//root/SomeFile Copying :/tftpboot/.private/home/admin/SomeFile to /tftboot/FW_2//root/SomeFile FW>> quit [root@xxxxx admin]#


Using UNIX Commands 172

16Viewing Performance, Statistics, and

StatusThis chapter describes how to view the XOS performance, statistics, and status. It contains the following major topics:

Using GEM to View Module Information and Status on page 173

Viewing Module Information and Status on page 174

Viewing Disk Usage on page 175

Viewing Heartbeat Status on page 175

Viewing Module Utilization and Load Averages on page 176

Viewing Interface Statistics on page 177

Viewing Switch Data-Path (SDP) Statistics on page 178

View VRRP Status on page 178

Viewing Flow Distribution on page 180

Using GEM to View Module Information and StatusThe Greenlight Element Manager (GEM) provides a view into the components and health of your X-Series Platform. GEM is a Web application that displays system and component operational information in an easy to reference format. GEM provides system monitoring capabilites only, and does not allow you to configure your system.

Individual views provide module status, connection rates, load averages, flow rates, VAP group information, DBHA and Failover Group status, application data, and more in a single glance. The tabbed interface can be customized to show specific module information or system historical data.

User assistance provides additional details about the views and steps to add or customize the display for your specific needs.

To access the Greenlight Element Manager, configure the Web server during the XOS software installation, or from the command line after install. Use a secure connection (https://) to enter the system IP address or name in a Web browser.



Viewing Module Information and StatusTo view the status of all modules in the chassis, use the following command:

CBS# show module status

The following is a partial display. This display continues for each module in the chassis.

NA = Not Available, DP = Data Plane, CP = Control Plane cp = Control Processor, ap = Application Processor, np = Network ProcessorSlot 3: Board Type AP8650 Board Part Number 004910 Board Serial Number L829H004 Board Revision 8 Control FPGA Revision 0x600 Focus FPGA Revision 0x600 Board OCODE A000 Dual-CPU Capacity Available CPU Presence Present CPU2 Presence Not Present CPU Voltage 1.14(V) 3.3v Supply 3.31(V) 1.8v Supply 1.82(V) 1.5v Supply 1.52(V) CPU Temperature 31(C) Intake Air Temperature 28(C) FPGA Temperature 38(C) Exhaust Air Temperature 33(C) SDRAM 1 Size 1048576(KB) SDRAM 2 Size 1048576(KB) SDRAM 3 Size 0(KB) SDRAM 4 Size 0(KB) Total Memory 2073904(KB) Used Memory 430116(KB) Free Memory 1643788(KB) Shared Memory 0(KB) Buffers Memory 0(KB) Cached Memory 111604(KB) Memory Utilization 15.36% Total High Memory 262144(KB) Free High Memory 15860(KB) Total Low Memory 1811760(KB) Free Low Memory 1627928(KB) Active LED On Standby LED Off Failure LED Off Hard Disk N/A Second Hard Disk N/A Flash N/A Hard Drive Error None Second Hard Drive Error None RAID Status Not Active, RAID none Line 3/4 Up Line 3/5 Up Control Bus A Up Control Bus B Up ap2a Link Down ap2b Link Up ap2c Link Down

Viewing Performance, Statistics, and Status 174

ap2d Link Down CPU Speed 2327 MHz CPU Up Time Threshold 96179 secs Acceleration Card 1 Not Present Acceleration Card 2 Not Present Ethernet Daughter Card Not Present

EMS provides a Module Memory Usage graph under Status > System Status in the navigation tree.

Viewing Disk UsageThe X-Series Platform tracks two system events for disk usage crossing the upper and lower thresholds values and four alarms (for four partitions) for monitoring disk usage on the CPM. It also lists the 5 largest files on each VAP. This information is collected once a day.

To display the disk usage history using EMS, open Status > System Status > Disk Usage History Stats in the navigation tree.

To display the current disk usage using the CLI:

CBS# show disk-usage

To display a history of disk usage using the CLI:

CBS# show disk-usage history======================================================================Top Disk Users Report for Thu Mar 18 04:02:03 EDT 2010======================================================================

Filesystem 1K-blocks Used Available Use% Mounted on/dev/sda12 11812024 1902472 9309528 17% //dev/sda9 102740 7772 89752 8% /boot/dev/mapper/d2vg0-lv0 1971472 127200 1744128 7% /cbconfig/dev/mapper/d2vg1-lv0 37353008 7801048 27654488 23% /tftpboot/dev/mapper/d2vg1-lv1 2011792 83136 1826464 5% /mgmt

In case of critical disk errors, you can configure the CPM to automatically go offline as follows:

CBS# configure cp-action {cp1|cp2} disk-error offline

Viewing Heartbeat StatusThe Heartbeat is a mechanism to determine the state of the system connections (on the control and data planes) between modules, which verifies whether these modules can communicate with one another. The Heartbeat Status reports the condition of these connections. The system processor modules (CPM, APM and NPM) use Heartbeat Status to determine the state of their connections to other processor elements in the system.

The Heartbeat Status shows whether each processing element is exchanging heartbeats with another processor. The CPMs run heartbeats with all other processors over the control plane. The NPMs run heartbeats with APMs over the control plane, as well as over the data plane. APMs run heartbeats among themselves over the data plane. Eight missed heartbeats in a row causes a connection to be declared unusable.


To view the Heartbeat Status using EMS, open Status > System Status > Heartbeat Status in the navigation tree. For the CLI, use the following command:

CBS# show heartbeatLink Quality TO: 1FROM 1 2 3 4 5 6 7 8 9 10 11 12 13 14ON portsCB A: NA 100% 100% 100% 100% NA 100% 100% 100% 100% 100% NA 100% NACB B: NA NA NA NA NA NA NA NA NA NA NA NA NA NADP A: 100% 100% 100% 100% NA NA NA 100% NA 100% 100% NA 100% NADP B: NA NA NA NA NA NA NA NA NA NA NA NA NA NADP C: NA NA NA NA NA NA NA NA NA NA NA NA NA NADP D: NA NA NA NA NA NA NA NA NA NA NA NA NA NA

This display continues for each module in the chassis.

Viewing Module Utilization and Load AveragesModule Load Averages show the CPU % load average for all APMs and CPMs in the system during the previous 1, 5, and 15 minute intervals, indicating the number of computable processes. EMS provides a Module Utilization Averages graph under Status > System Status in the navigation tree.

To view the Module Load Averages from the CLI, use the following command.

CBS# show cpuCPU utilization average for np1: for last 1 minute: 1.00 for last 5 minutes: 1.00 for last 15 minutes: 1.00

CPU utilization average for np2: for last 1 minute: 1.00 for last 5 minutes: 1.00 for last 15 minutes: 1.00

CPU utilization average for ap2: for last 1 minute: 0.00 for last 5 minutes: 0.00 for last 15 minutes: 0.11

CPU utilization average for ap4: for last 1 minute: 0.00 for last 5 minutes: 0.00 for last 15 minutes: 0.00

CPU utilization average for cp2: for last 1 minute: 0.98 for last 5 minutes: 1.64 for last 15 minutes: 1.22

CPU load average for np1: for last 1 minute: 1.00 for last 5 minutes: 1.00 for last 15 minutes: 1.00

CPU load average for np2: for last 1 minute: 1.00 for last 5 minutes: 1.00 for last 15 minutes: 1.00


CPU load average for ap2: for last 1 minute: 0.00 for last 5 minutes: 0.00 for last 15 minutes: 0.00

CPU load average for ap4: for last 1 minute: 0.00 for last 5 minutes: 0.00 for last 15 minutes: 0.00

CPU load average for cp2: for last 1 minute: 0.00 for last 5 minutes: 0.00 for last 15 minutes: 0.00Slot Module CPU User Nice Syst Idle Irq SfIrq Iowt---- ------ ----- ----- ----- ----- ----- ----- ----- ----- 3 ap1 CPU 0.0 0.0 0.0 100.0 0.0 0.0 0.0 3 ap1 CPU0 0.0 0.0 0.0 100.0 0.1 0.1 0.0 3 ap1 CPU1 0.0 0.0 0.0 100.0 0.0 0.0 0.0

Slot Module CPU User Nice Syst Idle Irq SfIrq Iowt---- ------ ----- ----- ----- ----- ----- ----- ----- ----- 4 ap2 CPU 0.0 0.0 0.0 100.0 0.0 0.0 0.0 4 ap2 CPU0 0.0 0.0 0.0 100.0 0.0 0.0 0.0 4 ap2 CPU1 0.0 0.0 0.0 100.0 0.0 0.0 0.0

Slot Module CPU User Nice Syst Idle Irq SfIrq Iowt---- ------ ----- ----- ----- ----- ----- ----- ----- ----- 13 cp1 CPU 0.1 0.0 0.0 0.0 0.0 0.1 98.4

Viewing Interface StatisticsView the statistics for the interfaces on the CPM or NPM as follows:

CBS# show interface detailGigabitethernet 6/1 is upInterface is in use Hardware address is 00:03:d2:2f:52:7a MTU 1500 bytes, BW 100 Mbits, full-duplex, auto-negotiation is enabled Last clearing of "show interface" counters never PHY stats: Statistics on physical line Received: Total frames 53610 (bytes 42983430) Broadcast frames 0 Undersized frames 0 Oversized frames 0 Throttles 0 Total errors 0 Frame check sequence (FCS) errors 0 Frame errors 0 Overrun errors 0 Ignored errors 0 Transmitted: Total frames 17662 (bytes 1594894) Underrun errors 0 Total errors 0 Collisions 0


Viewing Switch Data-Path (SDP) StatisticsSDP Statistics are the statistics for each APM and CPM. View SDP statistics as follows:

CBS# show switch-data-path In In In Out Out OutSlot Mod SDPx Packets Errors Drops Packets Errors Drops10 VAP 0 268708390 0 0 307895769 0 010 VAP 1 251429612 0 0 293391354 0 010 VAP 2 891383778 266 266 5438364 0 010 VAP 3 209923713 0 0 6220223 0 010 VAP 4 0 0 0 0 0 010 VAP 5 0 0 0 0 0 010 VAP 6 0 0 0 0 0 010 VAP 7 0 0 0 0 0 010 VAP total 1621445493 266 266 612945710 0 0

To clear these counters, use the following command:

clear switch-data-path {all|module <low> [<high>]}

Where:

View VRRP StatusIf you have two or more X-Series Platforms in a VRRP configuration, you can determine the state of VRRP on an individual system with the following commands:

show vrrp status [<failover-group-id>]

show vrrp detail-status [<failover-group-id>]

show-remote-box

See Configuring Multi-System High Availability on page 129 for details on configuring VRRP.

show vrrp statusThe following is an example of this command.

Slot Physical slot number

mod Module type

SDPx Switch data path number

In Packets Number of packets received

In Errors Number of packets received with errors

In Drops Number of packets dropped while receiving packets

Out Packets Number of packets sent

Out Errors Number of errors occurring during transmission

Out Drops Number of drops occurring during transmission

all Clears the counters from all modules.

<low> A specific module. Valid values are 1 to 14. If high is not specified, this is the only module.

<high> The low parameter specifies the first module (lowest number) and high specifies the last module (highest number). All modules in the range are included. Valid values are 1 to 14.



CBS# show vrrp statusPriority is Actual/ConfiguredFG-ID Priority Status Preempt Master Sys ID Master Priority2 200/200 Master on 222 2005 0/100 Down off 1 (2 rows)

The Status column displays the state of the failover group, which can be:

backup —Failover group is in backup mode.

down — Failover group is not functioning.

init — Failover group is initializing.

master — Failover group is in master mode.

unknown — The state of the failover group cannot be determined.

The Priority column displays the failover group’s current priority / configured VRRP priority, in that order. If everything is running normally, these numbers are the same. Otherwise, one or more events caused the priority to be decremented by pre-configured priority-deltas, as described in VRRP Priority on page 130.

If there is a problem, use the ID in the Sys ID column and enter the show vrrp virtual-router <id> or show vrrp circuit-ip vr-id <id> command. These displays will provide additional information to help you determine which event caused the priority to be decremented. If necessary, use the show vrrp vap-group <vap-group-name>, show vrrp verify-next-hop, and show vrrp monitor-circuit commands to find the problem.

If using EMS, you can access the VRRP status by opening Status > Network Status > VRRP Stats in the navigation tree.

show vrrp detail-statusThe following is an example of this command.

CBS# show vrrp detail-status Execute “show vrrp detail-status-help” to see help for this command.

FG_ID Status Priority Delta Type Component 200 Master 198/198 1 vr ymlt/20 200 Master 198/198 1 vr ser/10 200 Master 198/198 1 mi gigabitethernet 4/4 200 Master 198/198 1 mi gigabitethernet 2/4 200 Master 198/198 50 vg vsx(5 rows)CBS#

Show Remote BoxUse the show-remote-box command to view the path and path status for each of the interconnected ports on the two systems.

CBS# show remote-boxSystem ID : 45First IP Address : 1.1.45.20Second IP Address : 192.168.50.45Third IP Address : 192.168.51.55Fourth IP Address : 0.0.0.0Fifth IP Address : 0.0.0.0Active IP Address : 1.1.45.20(1 row)

Viewing Flow DistributionUse the following commands to monitor traffic flows from the NPMs to VAPs on the APMs:

show flow active

show flow distribution

show flow-path active

show flow activeThe show flow active command displays information currently stored in the Active Flow Table (AFT) on the NPMs running on the X-Series Platform.

Use this command to determine whether flows are arriving on the NPMs, and if they are being dropped or sent to the appropriate VAP groups.

Use the verbose parameter to change the format that the CLI displays the output of the command, and to display additional information about each active flow.

The show flow active command is a snapshot of the current state of the active flows when the command is issued. Information is not updated when the state of an existing flow changes or when a new flow arrives on the X-Series Platform.

Use the poll parameter to continuously poll the NPMs and display updated information at regular intervals. Use this parameter to continuously monitor traffic over time, observing changes in the states of existing flows and obtaining information about new flows that arrive on the X-Series Platform.

NOTE: Press Ctrl-y to stop updating the command output and return to the CLI prompt.

By default, this command displays information about all active flows. Use one or more of the following parameters to filter the command output and display specific information:

By default, this command lists the active flows in the order in which they appear in the AFT. Use the sort parameter to sort the list of flows.

source-address

destination-address

source-port

destination-port

protocol

domain

circuit-id

module

master-npm

fast-path-only

verbose

poll

sort

validated

validation-pending

no-validation


Syntaxshow flow active [source-address {<IP_address> | <lowest_IP_address> <highest_IP_address>} ] [destination-address {<IP_address> | <lowest_IP_address> <highest_IP_address>}] [source-port {<port_number> | <lowest_port_number> <highest_port_number>}] [destination-port {<port_number> | <lowest_port_number> <highest_port_number>}] [protocol {<protocol_number> | <lowest_protocol_number> <highest_protocol_number>}] [domain {<domain_ID_number> | <lowest_domain_ID_number> <highest_domain_ID_number>}] [circuit-id {<circuit_ID_number> | <lowest_circuit_ID_number> <highest_circuit_ID_number>}] [module {<npm_slot_number> | <lowest_npm_slot_number> <highest_npm_slot_number>}] [master-npm {<npm_slot_number> | <lowest_npm_slot_number> <highest_npm_slot_number>}] [fast-path-only] [verbose] [poll <polling_interval>] [sort] [validated] [validation-pending] [no-validation]

Default Output

By default, the show flow active command displays information in a table, using the following format: .

Modules <VAP_name1>, <VAP_name2> ...rx circuit <ID#> Master <NPM_name> Fast Path {Y|N} rx packets <#>

{Re-routing | Drop(<drop_reason_ID>)}rx circuit <ID#> Master <NPM_name> Fast Path {Y|N} rx packets <#>

The format shown in the entry for <NPM_or_VAP_name1> is used if the flow is received by an NPM or VAP and then transferred to a VAP for processing.

The format shown in the entry for <NPM_or_VAP_name2> is used if the flow is received by an NPM or VAP and then rerouted to an external system or dropped.

For detailed information about this command and parameters, refer to the XOS Command Reference Guide.

show flow distributionThe show flow distribution command displays the number of flows that each Network Processor Module (NPM) has assigned to each virtual application processor (VAP) in each VAP group, and displays the rates at which each NPM assigns new and existing flows to each VAP.

The following table describes the information provided in each column/row.

Module Source Destination Prot Dom TTI/MAX<NPM_or_VAP_name1> <IP>:<port> <IP>:<port> <#> <ID#> <mm:ss>/<mm:ss>

<NPM_or_VAP_name2> <IP>:<port> <IP>:<port> <#> <ID#> <mm:ss>/<mm:ss>

Column/Row Heading Information Provided

NP Name of the NPM.

Use the show chassis command to display the NPM names assigned to the NPMs installed in your X-Series Platform.


There are two parameters for sorting the show flow distribution output:

sort vap-group sorts flows by vap-group name, then index-in-group, then apm slot

sort apm-slot sorts flows by apm-slot, then vap-group name, then index-in-group

The following is an example:

CBS# show flow distribution New Flows Aged Flows Flows Rate RateNP Uptime Slot VAP ========== ========== =========np1 0 days, 00:00 7 testvapgroup_1 0 0 0np2 3 days, 19:17 7 testvapgroup_1 7344 5133 72043 np3 0 days, 00:00 7 testvapgroup_1 0 0 0np4 3 days, 19:17 7 testvapgroup_1 6340 5538 79002np1 0 days, 00:00 8 testvapgroup_2 0 0 0np2 3 days, 19:17 8 testvapgroup_2 5733 5670 69182np3 0 days, 00:00 8 testvapgroup_2 0 0 0np4 3 days, 19:17 8 testvapgroup_2 5223 5041 68423

show flow-path activeThe show flow-path active command displays the flow paths for the active flows that the X-Series Platform is currently processing.

A flow path is the path that a flow takes when it goes through the X-Series Platform. A flow path has the following basic elements:

Flow classification information — source and destination IP addresses, source and destination port numbers, protocol number, and domain ID number

Network Processor Module (NPM) on which the active flow enters the X-Series Platform

Circuit(s) on which the active flow enters an active virtual application processor (VAP) group(s)

Active VAP(s) that processes the flow

NPM interface on which the active flow leaves the X-Series Platform

Uptime Amount of time that the NPM has been in the UP state, in days, hours, and minutes.

Hours and minutes are expressed in the format: mm:ss.

For example, 9 hours and 7 minutes is 09:07.

Slot Slot number for the APM assigned to the VAP to which the NPM assigns flows.

VAP Name of the VAP to which the NPM assigns flows.

A VAP name has the format:

<VAP_group_name>_<VAP_Index_Number>

Use the show ap-vap-mapping command to display the index numbers assigned to the VAPs in each VAP group configured on the X-Series Platform.

New Flows Rate The change in the number of new flows assigned to the VAP, since the last second.

Aged Flows Rate The change in the number of existing flows assigned to the VAP, since the last second.

Flows Total number of flows currently assigned to the VAP.

Column/Row Heading Information Provided


By default, this command displays information only about the initial entry path and the final egress NPM interface for each flow. That is, the command displays information only about the NPM, circuit, and VAP on which the flow first enters the X-Series Platform and the NPM interface on which the flow exits the X-Series Platform.

Use the show flow-path active command output to determine whether NPMs are dropping flows when they arrive on the X-Series Platform, to make sure traffic is successfully passing through the X-Series Platform, and to determine where the NPM sends flows that it does not drop.

Use the verbose parameter to display information about the full path for each active flow, from the ingress NPM to the egress NPM interface. Use this parameter to monitor flows when:

The flows pass through more than one VAP group configured on the X-Series Platform.

The flows pass through a VAP group that is configured with separate circuits and NPM interfaces for ingress and egress traffic.

The show flow-path active command is a snapshot of the current state of the active flows when the command is issued. Information is not updated when the state of an existing flow changes or when a new flow arrives on the X-Series Platform.

Use the poll parameter to continuously poll the NPMs and display updated information at regular intervals. Use this parameter to continuously monitor traffic over time, observing changes in the states of existing flows and obtaining flow path information for new flows that arrive on the X-Series Platform.

NOTE: Press Ctrl-y to stop updating the command output and return to the CLI prompt.

By default, this command displays flow path information for all active flows. You can use one or more of the following parameters to filter the command output to display flow path information only for active flows that match the criteria that you specify with the parameters:

By default, this command lists the active flows in the order in which they appear in the AFT.

Syntaxshow flow-path active [verbose] [source-address {<IP_address> | <lowest_IP_address> <highest_IP_address>}] [destination-address {<IP_address> | <lowest_IP_address> <highest_IP_address>}] [source-port {<port_number> | <lowest_port_number> <highest_port_number>}] [destination-port {<port_number> | <lowest_port_number> <highest_port_number>}] [protocol {<protocol_number> | <lowest_protocol_number> <highest_protocol_number>}] [domain {<domain_ID_number> | <lowest_domain_ID_number> <highest_domain_ID_number>}] [circuit-id {<circuit_ID_number> | <lowest_circuit_ID_number> <highest_circuit_ID_number>}] [module {<npm_slot_number> | <lowest_npm_slot_number> <highest_npm_slot_number>}] [master-npm {<npm_slot_number> | <lowest_npm_slot_number> <highest_npm_slot_number>}] [fast-path-only]

source-address

destination-address

source-port

destination-port

protocol

domain

circuit-id

module

master-npm

fast-path-only


Default Output

By default, the show flow-path active command displays information in a table, using the following format:

rx circuit <ID#> rx active <VAP_name> [rx passive]tx_port <slot>_<port> master <NPM_name>

rx circuit <ID#> Drop(<drop_reason>)master <NPM_name>

The output has the format shown in the entry for <NPM_name1> if the originating NPM transfers the flow to a VAP for processing.

The output has the format shown in the entry for <NPM_name2> if the originating NPM drops the flow.

The following example displays the initial entry path and the egress NPM interface for every active flow that the X-Series Platform is currently processing. In this example, a VAP group called testvapgroup is currently configured on the X-Series Platform and is running a firewall application.

CBS# show flow-path active

This command may take a few minutes. Do you want to continue? <Y or N> [Y]: Y

rx circuit 1027 rx active testvapgroup_2 rx passivetx_port 4_2 master np2

rx circuit 1028 rx active testvapgroup_1 rx passivetx_port 2_2 master np2

rx circuit 1029 Drop(PS2IDX failed)master np2

CBS#

For additional detailed information about the parameters used with this command, as well as additional examples, refer to the XOS Command Reference Guide.

Module Source:port Destination:port Prot Dom<NPM_name1> <IP>:<port> <IP>:<port> <#> <ID#>

<NPM_name2> <IP>:<port> <IP>:<port> <#> <ID#>

Module Source:port Destination:port Prot Domnp2 172.16.10.100:2009 172.16.20.240:80 6 1

np4 172.16.20.240:80 172.16.10.144:53814 6 1

np2 172.16.10.207:31754 172.16.20.240:80 6 1


17Viewing Alarms, Events, and Logs

This chapter describes how to view the X-Series Platform message and audit logs. It contains the following major topics:

Viewing Alarms on page 185

Configuring a Remote Syslog Server on page 190

Viewing Message Logs on page 190

Viewing Audit Logs on page 192

Viewing AlarmsXOS records alarm data and the Alarms Manager provides the data to GEM, the CLI, SNMP, and system log files for end user consumption. GEM provides extensive alarm information in multiple views in the user interface including the Alarms Panel, the Alarm Details view, and the XOS Alarms Model Information Pages.

The CLI provides several options under the show alarms command, providing the same level of granularity as the GEM Alarm views.

The Message Log maintains a history of software events that have occurred on the XOS console. This historical log can be searched and the information displayed as well.

SNMP traps and MIB tools can be configured to output system alarms into third party tools.

Figure 14. XOS Alarm Notification

XOS provides consistent alarm notifications through a variety of interfaces.


Using GEMIn GEM, the Alarms Panel displays information about each of the alarms logged on the X-Series Platform. The list includes active alarms and optionally, historical alarms. To show historical alarms, check the Include Historical Alarms check box.

The alarm list can be filtered to show all alarms, or only alarms related to Dual-box high availability (DBHA). Click the Reset Filter button to remove the filter from the alarm list.

The Alarms Panel displays the columns listed below by default, and is updated when a new alarm is generated. To sort a column, click on the header.

ID: The number assigned to the alarm by XOS.

Date: Date and Time of alarm.

Severity: Critical, Major, Minor, Info, or Clear (indicates the alarm has been cleared).

Source: The location where the alarm occurred. The location can be a module or an assembly.

Description: A brief description of the alarm.

Colors highlight the alarm severity; red indicates a critical alarm, orange indicates a major alarm, yellow indicates a minor alarm, green indicates a cleared alarm, and gray indicates an informational alarm. To mark one or more alarms as read, right-click the alarm and select Mark as Read. For additional details on an alarm, double-click the alarm to open the Alarm Details view.

Additional columns are available by clicking the down arrow in the a column header and selecting from the secondary list. These columns display information that may be specific to certain types of alarms. For more information, refer to the GEM user assistance.

Alarm DetailsThe Alarm Details window displays additional information to support the data shown in the Alarms panel, including a probable cause and suggested repair actions.

The Probable Cause is a starting point for investigating the alarm, and the information should be used with the Suggested Repair Actions to resolve the issue.

Repair information may include how to clear the alarm, CLI commands to gather more information about the alarm conditions, and suggestions to prevent the alarm from occurring again.

Clearing AlarmsA small set of informational alarms are clearable using the XOS CLI. These are listed in the optional User Clearable column.

Other alarms can only be cleared through user action, such as a configuration change or rebooting a module. For alarm repair actions, refer to the Suggested Repair information in the Alarm Details view. Double clicking any alarm in the Alarms View will display the Alarm Details view.

Viewing Alarms, Events, and Logs 186

Reviewing Alarm InformationXOS and GEM provide information about the current and historical alarms on the X-Series Platform. An additional resource for alarm information is the XOS Alarm Model, which is a complete list of all the alarms implemented in XOS. The Alarm Model can also be viewed by using the show alarm model command from the Command Line Interface.

Historical DataDisplaying historical data allows you to look back over a period of time, up to and including the current GEM session. It provides perspective on current conditions, and can help identify trends or events that resulted in an alarm.

Using the CLIUse the CLI to display currently active alarms, or an alarm history that includes both active and past alarms. Use the active parameter to display currently active alarms. Use the history parameter to display all active and past alarms. Use the model parameter to display the XOS alarms model. The alarms model provides detailed information about every alarm that can be raised on an X-Series platform.

CBS# show alarms activeCBS# show alarms historyCBS# show alarms model

When you use the show alarms active or show alarms history command, you can use a number of additional parameters to filter the output by severity, source, id, and date/time. You can add the verbose parameter to view additional detail, including suggested repair actions. No additional parameters are available for show alarms model.

Show Alarms ActiveThe active parameter displays an active alarm status summary. Specify one or more of the minor, major, or critical filters to display the active alarm summary, followed by a list of the conditions that triggered each severity of active alarm.

Use the following CLI command to view active alarms:

CBS# show alarms activeActive Alarms Summary:

Source Critical Major Minor ------ -------- ----- ----- cp2 1 3 0 np1 0 0 2 uprfan 0 1 0 Total 1 4 2

XOS displays a summary of all alarm levels, showing the module and the number of alarms for each alarm severity. View specific alarm information by including the alarm category in the show alarm command, as follows:

CBS# show alarms active majorActive Alarms Summary:



* indicates an alarm that can be cleared with the 'clear alarms' CLI command

Major:

ID Date Source Description -- ---- ------ ----------- 96 Nov 11 16:02:43 cp2 Failover group xyz priority 49*95 Nov 11 15:34:16 cp2 Failover group xyz status master 94 Nov 11 15:34:10 cp2 No remote box configured 69 Nov 10 09:18:23 uprfan Fan tray mismatch

XOS displays the alarm status summary for each module. The alarm status summary is followed by information for each active alarm in the category specified.

Use the verbose parameter to display complete alarm information, including suggested repair actions:

CBS# show alarms active major verboseActive Alarms Summary:


Major:================================================================================ Alarm Id : 94 Brief Description : No remote box configured Date : Thu Nov 11 15:34:10 2010 Severity : Major Alarm Name : noRemoteBoxConfigured Alarm Source : cp2 Slot : slot 7 Module : cp2 Information : Probable Cause : Configuration Or Customization Error Event Type : other User Clearable : false Extended Description : VRRP is configured but no remote box is configured, : which could indicate an incomplete configuration. Repair Action : If you have VRRP configured, you must configure at : least one remote box. Either add a remote box or : delete the VRRP configuration.--------------------------------------------------------------------------------

You can further filter the data by source, ID, or date and time by using additional parameters. See the XOS Command Reference Guide for detailed syntax, usage, and parameter explanations of this command.

Show Alarms HistoryThe history parameter displays a list of the last 1000 alarms on the system, active and historical. The most recent alarms appear first.

Use the following command to view all alarms, active and historical:

CBS# show alarms history


ID Date Severity Source Description -- ---- -------- ------ ----------- 425 Oct 7 17:20:10 Minor bay1 Power supply missing 424 Oct 7 17:01:23 Clear pwrA Power supply failure 423 Oct 7 11:01:20 Minor cp1 Firmware mismatch slot: 1,5,13 422 Oct 7 10:44:59 Info system New system alarm level (major) 421 Oct 7 10:43:54 Major lwrfan Fan tray mismatch 420 Oct 7 10:35:59 Minor np1 Flow table median threshold(tcp)

XOS displays a summary of all alarms, starting with the most recent. Use additional parameters to filter the output of the show alarms history command by ID, severity, source, and date. Use the verbose parameter to display complete alarm information, including suggested repair actions.


Show Alarms ModelThe Alarms Model provides detailed information about all alarms that can be generated on the XOS Platform, including an extended description and suggested repair actions. This information is not related to the currently active or historical alarms on your X-Series Platform. It is a list of all the alarms that can be generated by the X-Series Platform, and is provided for informational purposes.

Use the following command to view the Alarms Model:

CBS# show alarms model

XOS displays the alarms model in the following format:

CBS# show alarms model Alarm Name : applicationDown Managed Objects : Slot : Module Parameters : APP CPM Host Name : APP CPM IP Address : APP Name : APP New State : APP Old State : APP Release : APP VAP Group Name : APP VAP Index : APP Version Information : Default Severity : minor Probable Cause : Out Of Service Event Type : Processing Error Alarm User Clearable : false Brief Format : Application down Extended Description : The application running on the specified APM is : down. Repair Action : Verify the state of the application on the APM (use : GEM System view). Restart the application if : necessary. Targets : GEM : LOG : SNMP


User Clearable AlarmsA small number of alarms can be cleared from the CLI by using the clear alarms command.

Active alarms that can be cleared by using the clear alarms command are indicated by an asterisk in the ID column of the output of the show alarms active command when used with one or more parameters.

NOTE: You must use at least one parameter (e.g. critical, major, or minor) in the show alarms active command to see this output. Otherwise the show alarms active command displays only the summary table.

In this example, the alarm with ID 95 is user-clearable.

ID Date Source Description -- ---- ------ ----------- 96 Nov 11 16:02:43 cp2 Failover group xyz priority 49*95 Nov 11 15:34:16 cp2 Failover group xyz status master 94 Nov 11 15:34:10 cp2 No remote box configured 69 Nov 10 09:18:23 uprfan Fan tray mismatch

Cleared alarms remain in the alarms history, and can be viewed by executing the show alarms history CLI command. Use the clear-alarms command to clear these alarms. You can clear all alarms by using the all parameter, or you can specify an alarm or group of alarms by using the id parameter.

The following command clears the user-clearable alarm with ID 95.

CBS# clear alarms id 95CBS#

When you clear an alarm, a new alarm appears in the alarms history with a severity of Clear. The cleared alarm (ID 95) remains in the alarm history.


NOTE: The system also clears alarms automatically, either because the condition that generated the alarm no longer exists, or because the alarm has been superseded by a later alarm.

Configuring a Remote Syslog ServerThe X-Series Platform supports a remote logging server. To configure the remote server, enter the following:

CBS# configure logging server {<hostname>|<ipaddress>}

To view any configured remote logging servers:

show logging server

Viewing Message LogsThe Message Log maintains a history of software events that have occurred on the XOS console. Use it to store a record of system events or to diagnose operational problems. Log files are stored in the /var/log directory of the CPM system. All APMs and NPMs are configured to log their events to this location.

Log file rotation can occur on a periodic basis or when the log file reaches a specified size. Logs are rotated by saving the current log file to an archive copy. The current log or any of the archive copies can be retrieved by a Network Management Server for long-term storage.

You can also create several log files containing messages from different subsystems, or different severity levels to be directed to different files. To support proactive notification of error condition log messages, a generic method is provided to send log messages as traps. To display the Message Log:


CBS# show logging consoleJul 7 20:33:33 crossbeam cbssysctrld[1777]: [W] int CHmSession::read () Received msg from hm with opcode=5 0 entriesJul 7 20:33:33 crossbeam cbssysctrld[1777]: [W] HM Reallocating sbd 512->1024Jul 7 20:33:33 crossbeam cbssysctrld[1777]: [W] pollHiChange Poll rx'd from unknown slot 3Jul 7 20:33:33 crossbeam cbssysctrld[1777]: [W] int CHmSession::read () Received msg from hm with opcode=5 0 entriesJul 7 20:33:33 crossbeam cbssysctrld[1777]: [W] int CHmSession::read () Received msg from hm with opcode=5 0 entriesJul 7 20:33:35 crossbeam cbssysctrld[1777]: [W] pollHiChange Poll rx'd from unknown slot 7Jul 7 20:33:35 crossbeam cbssysctrld[1777]: [W] pollHiChange Poll rx'd from unknown slot 3Jul 7 20:33:35 crossbeam cbssysctrld[1777]: [W] pollHiChange Poll rx'd from unknown slot 7

Console debug messages for the APMs and NPMs are disabled by default. To enable console messaging from the APM enter the Linux command:

echo "6 4 1 7" > /proc/sys/kernel/printk

NOTE: To make this change permanent following a reboot, edit the APM /etc/rc.local file by commenting out the line echo "1 2 1 1" > /proc/sys/kernel/printk.

Reporting messages to the console may cause performance degradation. Excessive verbose messaging may cause the module to reset.

Kernel messages can always be viewed in dmesg on the APM and the system log on the CPM.

Viewing Alarm Details from the Message LogTo view alarm details in the message log, you can grep the log. For example, start by grepping the log by AlarmID, then to see more detail about major alarms, grep the term "| major |".

To see all of the alarms in the messages file, grep the output for “AlarmID”.

CBS# unix su Password: [root@TechPubs-X45 admin]# [root@TechPubs-X45 admin]# grep AlarmID /var/log/messages

Nov 20 15:08:31 TechPubs-X45 cbsalarmlogrd: AlarmID 82 | Sat Nov 20 16:08:31 2010 | major | cp2 | vrrpFailGroupStatusChange | Failover group vrrp_fw status downNov 20 15:08:31 TechPubs-X45 cbsalarmlogrd: AlarmID 83 | Sat Nov 20 16:08:31 2010 | critical | cp2 | vrrpFailGroupPriorityChange | Failover group vrrp_fw priority 0Nov 20 15:09:23 TechPubs-X45 cbsalarmlogrd: AlarmID 84 | Sat Nov 20 16:09:23 2010 | major | cp2 | vrrpFailGroupPriorityChange | Failover group vrrp_fw priority 199Nov 20 15:09:23 TechPubs-X45 cbsalarmlogrd: AlarmID 85 | Sat Nov 20 16:09:23 2010 | major | cp2 | noRemoteBoxConfigured | No remote box configuredNov 20 15:09:29 TechPubs-X45 cbsalarmlogrd: AlarmID 86 | Sat Nov 20 16:09:29 2010 | major | cp2 | vrrpFailGroupStatusChange | Failover group vrrp_fw status masterNov 20 15:09:39 TechPubs-X45 cbsalarmlogrd: AlarmID 87 | Sat Nov 20 16:09:39 2010 | major | cp2 | vrrpFailGroupPriorityChange | Failover group vrrp_fw priority 100


Nov 20 17:37:34 TechPubs-X45 cbsalarmlogrd: AlarmID 88 | Sat Nov 20 18:37:34 2010 | clear | uprfan | fanTrayIncompatible | Fan tray mismatch | CorrelationID 53Nov 20 17:37:37 TechPubs-X45 cbsalarmlogrd: AlarmID 89 | Sat Nov 20 18:37:37 2010 | major | uprfan | fanTrayIncompatible | Fan tray mismatch

To see the CLI command show alarms in the messges file, grep the output for "| major |"

[root@TechPubs-X45 admin]# grep AlarmID /var/log/messages | grep majorsystemAlarm | New system alarm level (major)Nov 20 15:08:31 TechPubs-X45 cbsalarmlogrd: AlarmID 82 | Sat Nov 20 16:08:31 2010 | major | cp2 | vrrpFailGroupStatusChange | Failover group vrrp_fw status downNov 20 15:09:23 TechPubs-X45 cbsalarmlogrd: AlarmID 84 | Sat Nov 20 16:09:23 2010 | major | cp2 | vrrpFailGroupPriorityChange | Failover group vrrp_fw priority 199Nov 20 15:09:23 TechPubs-X45 cbsalarmlogrd: AlarmID 85 | Sat Nov 20 16:09:23 2010 | major | cp2 | noRemoteBoxConfigured | No remote box configuredNov 20 15:09:29 TechPubs-X45 cbsalarmlogrd: AlarmID 86 | Sat Nov 20 16:09:29 2010 | major | cp2 | vrrpFailGroupStatusChange | Failover group vrrp_fw status masterNov 20 15:09:39 TechPubs-X45 cbsalarmlogrd: AlarmID 87 | Sat Nov 20 16:09:39 2010 | major | cp2 | vrrpFailGroupPriorityChange | Failover group vrrp_fw priority 100Nov 20 17:37:37 TechPubs-X45 cbsalarmlogrd: AlarmID 89 | Sat Nov 20 18:37:37 2010 | major | uprfan | fanTrayIncompatible | Fan tray mismatchNov 20 20:37:39 TechPubs-X45 cbsalarmlogrd: AlarmID 91 | Sat Nov 20 21:37:39 2010 | major | uprfan | fanTrayIncompatible | Fan tray mismatchNov 22 14:06:37 TechPubs-X45 cbsalarmlogrd: AlarmID 92 | Mon Nov 22 15:06:37 2010 | major | cp2 | vrrpFailGroupStatusChange | Failover group xyz status down

Viewing Audit LogsThe Audit Log allows you to track user-configuration events showing which users are making what changes to the system. To display the Audit Log, use the following command:

show audit-trail

The following is an example of this command.

Aug 23 14:04:59 crossbeam cli: USER: admin, COMMAND: CBS# configure timeout > configure no timeout

Aug 23 14:05:08 crossbeam cli: USER: admin, COMMAND: CBS# configure timeout > configure no timeout

Aug 23 14:05:10 REG80A1 cli: USER: admin, COMMAND: CBS# configure hostname > configure hostname REG80A1 cp1

Aug 23 14:05:11 REG80A1 cli: USER: admin, COMMAND: CBS# configure hostname > configure hostname REG80A2 cp2

Aug 23 14:05:11 REG80A1 cli: USER: admin, COMMAND: CBS# configure ip telnet > configure ip telnet


Aug 23 14:05:12 REG80A1 cli: USER: admin, COMMAND: CBS# configure ip ftp > configure ip ftp

Aug 23 14:05:13 REG80A1 cli: USER: admin, COMMAND: CBS# configure system-identifier > configure system-identifier 101 #Warning: WARNING: Command takes effect next system reload

Aug 23 14:05:14 REG80A1 cli: USER: admin, COMMAND: CBS# configure operating-mode > configure operating-mode quad-np series-6 #Warning: WARNING: Command takes effect next system reload

Aug 23 14:05:15 REG80A1 cli: USER: admin, COMMAND: CBS# configure access-list ip source-ip destination-ip > configure access-list 1001 permit ip source-ip 0.0.0.0 255.255.255.255 destination-ip 0.0.0.0 255.255.255.255

Aug 23 14:05:16 REG80A1 cli: USER: admin, COMMAND: CBS# configure access-list ip source-ip destination-ip > configure access-list 1002 permit ip source-ip 0.0.0.0 255.255.255.255 destination-ip 0.0.0.0 255.255.255.255



18Upgrading XOS Software and Firmware

This chapter provides migration and upgrade information. It includes the following major topics:

XOS Automated Workflow System on page 195

XOS shar Upgrade on page 196

Upgrading the XOS Firmware on page 200

XOS Migration on page 204

If you are upgrading to XOS 9.5.x on an X-Series Platform currently running XOS 9.0 or above, use the Automated Workflow System to perform an upgrade.

If you are upgrading to XOS 9.5.x on an X-Series Platform currently running XOS 8.x, use the Migration Process to upgrade to XOS 9.5.x.

XOS Automated Workflow SystemThe Automated Workflow System (AWS) combines multiple manual steps into simple menu selections to automate the upgrade of XOS software and firmware. Crossbeam recommeds using the Automated Workflow System to perform upgrades. This automated process performs system checks and upgrades automatically, saving significant time over the manual upgrade process. AWS allows you to upgrade XOS software versions beginning with XOS V9.0.

To view the currently installed XOS version prior to upgrading, use the show release command from the upgrade context.

CBS# upgrade show releaseCrossbeam: 9.0.0-131 (current)CBS#

To verify system compatibility for an upgrade, perform the following system check. After copying the .shar file onto your system, run the verify-system command from the upgrade context.

CBS# upgrade verify-system 9.x.x

If incompatibilities are detected, error messages will be displayed. Perform the necessary changes (if any) and start the Automated Workflow System to perform the upgrade.

The Automated Workflow System will not upgrade XOS V8.x to XOS V9.0; use the Migration Process. The Automated Workflow System provides a method to upgrade software versions after V9.0, and can be used to upgrade firmware on modules in the X-Series Platform after XOS 9.0 has been installed.

To access the Automated Workflow Menu, enter the following command from the XOS CLI:


Welcome to the X-Series Platform Automated Workflow System!Version: 1.1.0-x

1. Configure XOS... 2. Upgrade XOS software and firmware... 3. View system configuration and status... 4. Applications... 5. Custom...



Please Enter Selection:

Enter your selection to begin the automated process. Help is available at each level by using the “?”.

AWS supports five automated workflows:

Software upgrade: This feature is available to upgrade from XOS V9.0 to a future release. You cannot upgrade to XOS V9.0 using AWS.

Firmware compatibility check: The firmware compatibility check compares the current versions of installed firmware against the recommended version, and displays the result (Up to date, or Update required).

Firmware update: This workflow will perform the firmware update automatically.

Prepare for Rollback: This workflow saves the currently installed XOS version to a separate distribution (distribution two) on the disk. Distribution one is prepared for the installation of a new version of XOS.

Rollback: This workflow will return the X-Series Platform to the previous version of XOS should an upgrade fail.

Information generated during an automated workflow is saved to the AWS log file, available at /var/log/aws.log. You can also watch the progress during a workflow using the show automated-workflow-progress command.

During the upgrade process, AWS will lock the configuration. If a user attempts to access the configuration file to make changes during the upgrade, an error message stating the configuration is “locked by the Automated Workflow System” will be displayed.

XOS shar UpgradeWith XOS V9.x installed, you can use AWS to upgrade your XOS versions. However, there may be situations where you choose to perform a shar upgrade of the software or firmware. Those procedures are included in the next sections.

IMPORTANT: The shar upgrade procedure will not work to upgrade from pre-V9.0 to XOS V9.x. To upgrade to XOS V9.x, you must use the XOS Migration on page 204.

To view the currently installed XOS version prior to upgrading, use the show release command from the upgrade context.

CBS# upgrade show releaseCrossbeam: 9.0.1-xxx (current)Crossbeam: 9.5.x-xxxCBS#

Copying and Extracting the shar FileTo FTP and extract the XOS software, complete the following:



2. Change to the rpm directory, using the following command:

# cd /usr/os/rpm

Upgrading XOS Software and Firmware 196

3. FTP or copy the xos-upgradepack-A000-X.Y.Z-W.shar.gz file from your XOS CDROM or software package to /usr/os/rpm. Note that A000-X.Y.Z-W represents Ocode-Major.Minor.Maintenance-Kit.shar

4. Enter the following command at the root prompt while in /usr/os/rpm:

# gzip –d xos-upgradepack-A000-X.Y.Z-W.shar.gz

5. Change the extracted file to an executable as follows:

# chmod 777 xos-upgradepack-A000-X.Y.Z-W.shar

6. Once the above command completes, enter the following command:

# ./xos-upgradepack-A000-X.Y.Z-W.shar

Upgrading with the Extracted XOS SoftwareTo avoid upgrade problems during the procedure, do not execute any CLI commands that are not within the following upgrade procedure.

During the upgrade, the system displays informational messages regarding verifications and checks. As these verifications and checks are being performed, you may temporarily be unable to issue any CLI commands.

Single CPM Configurations1. Start a CLI session.

2. If using CP redundancy, skip to Dual CPM Configurations below.

3. From within the CLI, enter the Upgrade context using the following command:

CBS# upgrade

4. Display the available software versions as follows:

CBS(upgrade)# show new-release

5. Enter the desired software version. For example:

CBS(upgrade)# install <xos_version>

Answer the questions as prompted. If in the Offline state, saving the configuration is optional.

6. After the successful upgrade message is displayed, reload the system as follows:

CBS# reload all

Dual CPM Configurations—Upgrading XOS

NOTE: Changing the CPM Redundancy configuration may require a reboot of the CPMs before the configuration changes take effect.

You cannot have two CPMs synchronized in the X-Series Platform while running the XOS upgrade. Installations to CPMs must be performed one at a time. The recommended process for upgrading an XOS system with Control Processor redundancy is to:

1. Take the secondary CPM offline.

2. While the primary CPM is still running, upgrade the secondary CPM.

3. Configure the platform so that, upon reload, the current secondary CPM boots as the primary and the original primary is taken offline.

4. Now upgrade the offline CPM, which had been the original master, and re-enable election/sync.

5. Reboot the offline CPM. The two CPMs synchronize and CPM redundancy is restored.

To upgrade a Dual CPM configuration:

1. Start a CLI session on the primary CPM.


2. If you are using CP redundancy, bring the secondary CPM offline.

CBS# configure cp-redundancy set other_cp offline

3. Start the CLI on the secondary CPM. Note that it may take a moment or two for the secondary CPM to come offline.

4. Perform the Copying and Extracting the shar File on page 196 to copy the file to the offline CPM.

5. From within the CLI on the offline CPM, enter the Upgrade context using the following command:

CBS# upgrade

6. From the secondary CPM, display the available software versions as follows:


7. From the secondary CPM, enter the desired software version. For example:


Where xos_version is the specific version of the software that you wish to load.

Answer the questions as prompted.

8. Configure the secondary CPM for election.

CBS# configure cp-redundancy set this_cp election

9. Save the configuration on the secondary CPM.

10. Start the CLI on the primary CPM then configure the secondary CPM for election from the primary CPM.


11. Reload the secondary CPM:

CBS# reload offline-cp

12. Reload all APM and NPM modules. For the X80 system, reload modules 1 to 12 as follows; otherwise, reload modules 1 to 5. This command takes all modules offline; no traffic is processed during this time.

CBS# reload modules 1 12

13. Take the Primary CPM offline.

CBS# configure cp-redundancy set this_cp offline

14. When prompted, log back in to the CPM that was the original Primary CPM.

15. Save your configuration.

CBS# copy running-config startup-config

NOTE: The secondary CPM reloads with the new version of the XOS software and becomes the Primary CPM.

16. Connect to CLI on the current Primary CPM and verify that the system is operating correctly before upgrading the other CPM.

17. Repeat the procedure in Copying and Extracting the shar File on page 196 on the offline CPM.

18. From within the CLI on the offline CPM, enter the Upgrade context using the following command:

CBS# upgrade

19. From the offline CPM, display the available software versions as follows:


20. From the offline CPM, enter the desired software version. For example:


Where xos_version is the specific version of the software that you wish to load.

Answer the questions as prompted.

21. Set the offline CPM to Election mode.





23. Connect to the current Primary CPM and reload the offline CPM. The CPM becomes the secondary CPM.


After the reload is complete CPM redundancy is restored. The CPM roles are reversed from when you started this procedure.

Dual CPM Configurations—Installing ApplicationsFor dual CPM configurations, to install an application, do the following:

For RPM:

1. Execute the rpm -i command for the application on both CPMs.

2. Execute the application install command from the CLI only on the primary CPM.

For CBI:

1. Copy the .cbi bundle to /usr/os/apps/archive/ on the primary CPM only.

2. Install the application from the CLI as normal.

Files Generated During the Software Upgrade ProcedureThe following configuration files are created during the upgrade process.

Table 19. Files Created During an Upgrade

Directory File Name Description

/tmp upgrade-saved-cfg.<release>.orig Original configuration before upgrade.

upgrade-saved-cfg.<release>.flat Original configuration before upgrade in flat format.

upgrade-saved-cfg Configuration after upgrade in flat format.

upgrade-cfg-error(s) All CLI commands plus errors.

/usr/os/logs cli_conversion_log CLI Upgrade log file.

SystemSoftwareReleaseHistory History log file.

/var/log cli-upgrade-errors CLI errors only.


Upgrading the XOS FirmwareCrossbeam recommeds using the Automated Workflow System to perform firmware upgrades. This automated process performs system checks and upgrades automatically, saving significant time over the manual upgrade process. Refer to the XOS V9.5.x Release Notes for the updated list of the minimum firmware version requirements.

Caution: Manual Firmware and FPGA upgrades must be executed at the same time before reloading. Attempting to reboot the system after a firmware upgrade without the corresponding FPGA upgrade will fail.

Use /crossbeam/bin/revs_check to display the current status of the firmware on your system. Information is provided for each installed module (NPM, CPM, APM). Additionally, you can review the Firmware Status view in the Greenlight Element Manager for the same information. There are two types of firmware to upgrade: FPGA and FLASH.

To see the current firmware status, the minimum revision necessary, and the recommended version from the command line, execute the revs_check command from the /crossbeam/bin directory with the following parameters:

-c: Display the current firmware revision.

-m: Display modules which do not meet minimum requirement.

-u: Display modules whose firmware revisions are not up-to-date.

-h: Display latest version of the files.

CBS# unix su[root@xxxx admin]# cd /crossbeam/bin[root@xxxx admin]# ./revs_check -c -m -u

The flashit tool needed to perform an upgrade is located within the /crossbeam/bin directory. If there are multiple flash or FPGA files, the file ending in the highest number is the latest. If you are unsure of the type of firmware, enter -h to determine the usage.

Manually Upgrading Firmware on an NPM Caution: Manual Firmware and FPGA upgrades must be executed at the same time before reloading. Attempting to reboot the system after a firmware upgrade without the corresponding FPGA upgrade will fail.

Upgrade the firmware on an NPM-86xx using a CPM as follows:

1. Set the NPM to maintenance state:

CBS# configure module <slot-number> maintenance

NOTE: The module must be in maintenance mode; otherwise, the procedure may leave the module in an unstable state.

2. Go to the unix prompt.

CBS# unix su

3. Change directory to /usr/os/etc/Firmware/NP.

[root@xxxx admin]# cd /usr/os/etc/Firmware/NP

4. Locate then copy the Flash and FPGA files to the /tftpboot/npm6_shared directory. For example:

[root@xxxx NP]# cp npm6xbprc_R0010_130.dat /tftpboot/npm6_shared/[root@xxxx NP]# cp npm6FlashImage0014.dat /tftpboot/npm6_shared/[root@xxxx NP]# cp npm6sysctrl_R04.dat /tftpboot/npm6_shared/


5. Telnet to the NPM.

[root@xxxx admin]# telnet npm<slot>

The NPM can be in slots 1-4. For example, enter npm1 to connect to the NPM in slot 1.

6. Run the flashit -w <filename> command for each file. The following filenames are examples only:

/# flashit -w /cbs/shared/npm6xbprc_R0010_130.dat/# flashit -w /cbs/shared/npm6FlashImage0014.dat/# flashit -w /cbs/shared/npm6sysctrl_R04.dat

7. When the upgrade is completed, return to the CPM prompt:

/# exit

8. Verify that the upgrade completed successfully.

a. Make sure that there were no NPM error messages during the flashit command.

b. Check the syslog for the message, npm<slot> flashit: program terminates normally.

9. Exit the unix prompt and return to the CLI.

10. Enable the module.

CBS# configure module <slot-number> enable

11. Use the show version detail command to verify that the firmware is upgraded. For example:

CBS# show version detailCopyright (c) 2000-2011 by Crossbeam Systems, Inc. All rights reserved.Version: XOS 9.5.x [Mar 1 2011 13:56:26] (bldmgr)gcc: gcc version 2.96 20000731 (Linux 7.3 2.96-112)CVS_Label: XOS-9_5_x_x-20101205_2Kit_Number: 88

CPU at 1995 Mhz processor with 3592316K bytes of memory3566660K bytes of memory in useUptime is 3 day(s) 21 hour(s) 40 min(s) 36 sec(s)Hard disk is 120(GB)Second Hard disk is not presentFlash is not present

Details per slot:

Revision for slot 1

Boot Strap version : 2.0.0.10Bootloader version : 2.0.0.8Diagnostics version : 2.1.0.2SysCtl FPGA version : 0x4Focus FPGA version : 0xaCPLD version : 0x5Board version : FBoard serial number : G801B022Board type : NP8600Board part number : 003927


Manually Upgrading Firmware on an APMCaution: Manual Firmware and FPGA upgrades must be executed at the same time before reloading. Attempting to reboot the system after a firmware upgrade without the corresponding FPGA upgrade will fail.

To upgrade APM firmware, the APM must be part of a VAP group. Upgrade the APM firmware, as follows:

1. Determine the APM VAP mapping structure:

CBS# show ap-vap-mapping

2. Set the APM to maintenance state:

CBS# configure module <slot-number> maintenance

NOTE: The module must be in maintenance mode; otherwise, the procedure may leave the module in an unstable state.

3. Wait momentarily for the APM to reboot. Verify that the APM is fully booted in Maintenance mode:

CBS# show ap-vap-mapping

4. Log into the VAP mapped to the APM.

[root@xxxx admin]# rsh <vap_index>

5. The firmware image files are located under the /usr/os/etc/Firmware/AP directory. The flash image file has Flash in the filename. The following filenames are examples only:

apcp6FlashImage0010.dat (flash image file for an APM-8600) apcp6_fpga_60.dat (FPGA image file for an APM-8600)

For 86xx modules, from the VAP UNIX prompt, execute the following commands:

# /usr/os/bin/flashit -w <flash image file> # /usr/os/bin/flashit -w <image file>

6. After the success messages display, exit out of the VAP rsh.

7. After the reload completes, enable the APM:

CBS# configure module <slot-number> enable

Manually Upgrading Firmware On a CPMCaution: Manual Firmware and FPGA upgrades must be executed at the same time before reloading. Attempting to reboot the system after a firmware upgrade without the corresponding FPGA upgrade will fail.

The firmware upgrade procedure for a CPM depends on the whether a single or dual CPMs are present.

Upgrading Firmware for a Single CPMThe firmware image files are located in the /usr/os/etc/Firmware/CP directory. The flashit image file has Flash in the filename. The following filenames are examples only:

apcp6FlashImage0010.dat (flash image file for a CPM-8600) apcp6_fpga_60.dat (FPGA image file for a CPM-8600)

To upgrade the firmware on a single CPM, complete the following steps:

For CPM-86xx modules, from the UNIX prompt, execute the following commands:



1. After the success messages display, halt the CPM as follows:

# halt

2. Press the reset pin located at the front of the CPM to reboot it, or remove and re-seat the CPM.

3. After the CPM reboots successfully, verify that the firmware is upgraded:

CBS# show module status revision {cp1|cp2}

Upgrading Firmware for Dual CPMsThe following upgrade procedure is valid for systems running XOS V8.0 or later. For earlier versions, refer to the XOS Configuration Guide for that release. Also, the dual CPMs must be in a Primary / Secondary state before the upgrade starts.

The firmware image files are located in the /usr/os/etc/Firmware/CP directory. The flashit image file has Flash in the filename. The following filenames are examples only:

apcp6FlashImage0010.dat (flash image file for a CPM-8600) apcp6_fpga_60.dat (FPGA image file for a CPM-8600)

To upgrade the FPGA firmware for a dual CPM system, complete the following:

1. Make sure that CP redundancy is enabled and that both CPMs are set to the Election state:

CBS# show cp-redundancy

Take note which CPM is Primary and which is Secondary. If the Administrative state of the CPMs is not set to Election, perform the following at the Primary CPM:

CBS# config cp-redundancy set this_cp electionCBS# configure cp-redundancy set other_cp electionCBS# wr

NOTE: Although a message warns that the system must be reloaded, you do not need to reload the modules.

2. Upgrade the Secondary CPM firmware. For CPM-86xx modules, from the UNIX prompt, execute the following commands:


The firmware image files are located under the /usr/os/etc/Firmware/CP directory.

3. After the success messages display, halt the CPM:

# halt

4. Reboot the CPM by pressing the reset pin located in the front of the CPM. If necessary, you can also remove and re-seat the CPM to reboot it.

5. After the CPM reboots successfully, verify that the firmware is upgraded. From the primary CPM use the following CLI command:


6. Reboot the Primary CPM so the upgraded CPM becomes Primary.

7. Upgrade the non-upgraded CPM firmware. For CPM-86xx modules, from the UNIX prompt, execute the following commands:


The firmware image files are located under the /usr/os/etc/Firmware/CP directory.

8. After the success messages display, halt the CPM as follows:

# halt

9. Reboot the CPM by pressing the reset pin located in the front of the CPM. If necessary, you can also remove and re-seat the CPM to reboot it.


10. After the CPM reboots successfully, verify that the firmware is upgraded. From the primary CPM use the following CLI command:


NOTE: At the completion of this procedure, the CPMs have swapped their Primary and Secondary states. To restore the Primary state to the original CPM, wait for CPM synchronization, and then reboot the current Primary CPM.

XOS MigrationStarting with XOS V9.0, Crossbeam implemented the RHEL5 64-bit kernel and compatible userspace packages. These improvements provide updated support and additional security provisions. Versions of XOS prior to V9.0 use a 32-bit operating system. Upgrading from XOS V8.x or earlier to V9.5.x requires running the XOS migration feature. The migration process combines elements of a fresh installation with the upgrade process to automatically migrate your applications and system configuration to the latest XOS software.

The following migration scenarios are supported:

XOS V8.5 to XOS V9.x

XOS V8.x with Series-6 (86xx and higher) hardware to XOS V9.x

XOS V7.3 on a CPM-86xx with no configuration to XOS V9.x

X-Series Platforms with older modules (pre-86xx) require upgrades prior to migration.

Migration is an automated process that takes place in three stages.

1. Stage One saves the existing configurations and necessary files, and prepares the new distribution.

2. Stage Two copies the required files to the new distribution, and converts files to the RHEL5 kernel.

3. Stage Three converts the configuration file, upgrades all VAPs, saves the configuration, and reboots the system.

Because the migration process is an automated task, user input must be provided for the following items before the migration begins.

A system reboot is required at the end of the first stage of the process. Choose whether to automatically reboot the system, or to halt the system and perform the reboot manually.

If an error occurs during migration, choose whether to halt the migration and remain incomplete with the new version while repairing the error, or revert to the previous version.

Throughout the migration process, the current state of the CPM is saved in a system file. The CPM invokes different scripts during the process, using responses to the above questions when required.

XOS V9.x Migration Minimum Requirements:Hardware modules (APM, CPM, and NPM) of 8600 or above

CPM-8600 or later with 4GB RAM (four 1-Gigabit DIMMs)

If your X-Series Platform is not running 86xx-series modules, you must upgrade the modules before migrating to V9.x. Additionally, the CPM-86xx must have four 1-Gigabit DIMMs installed to meet the memory requirements. Refer to the hardware upgrade instructions and RAM upgrade instructions provided with the modules for upgrade procedures.

Once your system meets these minimum requirements, follow the appropriate procedure in the Workflows section to perform the migration.

The migration process performs a number of system checks, and provides information when user action is required. If the application versions currently installed on your X-Series Platform are not supported on 9.x, you will be asked to exit the migration process and upgrade the applications. See Applications Supported for Migration for application information.


If your current configuration requires changes, you will be asked to exit the migration process and update the configuration. For information on configuration requirements for migration, see Upgrade Considerations on page 208.

Applications Supported for MigrationThe following applications are supported for migration to XOS 9.0. For information regarding application migration during an upgratde to XOS V9.X. and higher, please refer to the XOS Release Notes.

Check Point SecurityGateway R70/R71

Check Point VPN-1 Power NGX R65

Check Point VPN-1 Power VSX NGX R65/R67

Imperva SecureSphere 7.0

IBM Proventia N-IPS 2.0

Trend IWSS 3.1

Trend IMSS 7.0

Websense 6.3.2

RSW 8.0

WorkflowsThe following workflows provide information and steps to execute the migration process.

Migration from XOS V8.5.x to V9.xThe migration process performs a number of system checks, and provides information when user action is required. If your system does not meet the minimum requirements, an error message will be displayed, and you will be given options to make the necessary upgrades before the migration.

Single CPM Migration1. Copy the following file to the /crossbeam/rpm directory on your X-Series Platform.

CBS# unix suPassword:[root@xxxxx admin]# mv xos-migratepack-9.5.x-xxx /crossbeam/rpm/

2. Execute the following commands to begin migration:

[root@xxxxx admin]# cd /crossbeam/rpm[root@xxxxx rpm]# chmod 777 xos-migratepack-9.5.x-xxx[root@xxxxx rpm]# ./xos-migratepack-9.5.x-xxx

The script executes. Several system checks are performed, and messages are displayed when the migration requirements are not met. The system provides information in many cases about the recommended course of action. After each requirement is met, restart the migration process.

3. When all checks are passed, the following general messages will appear:

!!!!!!!!!!!!!!!!!! WARNING !!!!!!!!!!!!!!!!!!

Distribution 2 will be overwritten by the migration process.Would you like to proceed? (y/n):y

!!!!!!!!!!!!!!!


!!! WARNING !!! !!!!!!!!!!!!!!!

If you use acl-interface to mirror or pass-through traffic from thesource interface to the target interface, you must ensure that the targetacl-interface is configured. If you fail to configure the targetinterface, the migration will proceed but your acl-interfaceconfiguration will not transfer into XOS V9.5.x.Please see the Release Notes or Configuration Guide for more information.

Would you like to proceed? (y/n):y

Any unsaved configurations will be lost during migration process.Do you want to save the configuration before continuing? (y/n):y[I] Saving configuration.


Saving configuration ... Please be patient....[I] Performing pre-migration checks.[I] Executing pre_upgrade_check.

WARNING !!!

WARNING: XOS 9.5.x only supports the following modules:

* CPM-8600 * APM-8600, APM-8650, APM-9600 * NPM-8600, NPM-8620, NPM-8650, NPM-9600

Only supported modules will boot after upgrading to XOS 9.5.x.

Do you want to reboot the CPM automatically, when the migration processrequires a reboot? (y/n):y

If an error occurs during the migration process, do you want to revert to theexisting XOS release? (y/n):y

The system performs the migration, saves your configuration and reloads all modules at the completion of the process.

Dual CPM MigrationIf you have a redundant dual CPM configuration, use the following procedure to migrate to XOS V9.5.x.

1. Switch to the linux prompt and login to the system as root.

CBS# unix suPassword:

2. Copy the following file to the /crossbeam/rpm directory on both the primary and secondary CPMs on your X-Series Platform.

[root@xxxxx admin]# mv xos-migratepack-9.5.x-xxx /crossbeam/rpm/

3. Start a CLI session on the primary CPM.

4. Verify that both CPMs are fully sync’ed using the show cp-redundancy command.

CBS# show cp-redundancy

5. From the primary CPM, bring the secondary CPM offline.

CBS# configure cp-redundancy set other_cp offline


6. Start the CLI on the offline (secondary) CPM. Note that it may take a moment or two for the secondary CPM to come offline.

7. Execute the following commands to begin migration:

[root@xxxxx admin]# cd /crossbeam/rpm[root@xxxxx rpm]# chmod 777 xos-migratepack-9.5.x-xxx[root@xxxxx rpm]# ./xos-migratepack-9.5.x-xxx

The script executes. Several system checks are performed, and mesages are displayed when the migration requirements are not met. The system provides information in many cases about the recommended course of action. After each requirement is met, restart the migration process.

8. When all checks are passed, the following general messages will appear:

!!!!!!!!!!!!!!!!!! WARNING !!!!!!!!!!!!!!!!!!

Distribution 2 will be overwritten by the migration process.Would you like to proceed? (y/n):y

Any unsaved configurations will be lost during migration process.Do you want to save the configuration before continuing?(y/n):y[I] Saving configuration.


Saving configuration ... Please be patient....[I] Performing pre-migration checks.[I] Executing pre_upgrade_check.

WARNING !!!

WARNING: XOS 9.5.x only supports the following modules:

* CPM-8600, CPM-9600 * APM-8600, APM-8650, APM-9600, APM-2030 * NPM-8600, NPM-8620, NPM-8650, NPM-9600, NPM-20, NPM-30

Only supported modules will boot after upgrading to XOS 9.x.x.

Do you want to reboot the CPM automatically, when the migration processrequires a reboot? (y/n):y

If an error occurs during the migration process, do you want to revert to theexisting XOS release? (y/n):y

The system performs the migration.

9. From the CLI on the offline CPM, configure the offline (secondary) CPM for election.


10. Save the configuration on the offline (secondary) CPM.


11. From the primary CPM, configure the offline (secondary) CPM for election.


12. From the primary CPM, reload the offline (secondary) CPM:


13. From the primary CPM, verify the offline (secondary) CPM is rebooting.


CBS# show chassis

14. From the primary CPM, set the primary CPM to offline. This may take a few moments.

CBS# configure cp-redundancy set this_cp offline

The secondary CPM reloads with the new version of the XOS software and becomes the primary CPM.



16. Connect to the CLI on the new primary CPM and reload the NPMs and APMs to run with the XOS 9.x software.

CBS# reload all

17. Verify that the system is operating correctly before upgrading the offline CPM.

18. Repeat Step 7 and Step 8 on the offline CPM.

19. Enter the CLI on the primary CPM, and set the secondary CPM to election.


20. From the primary CPM, reload the secondary CPM.


Migration from XOS V7.3.x or V8.x with Series- 6 HardwareIf your X-Series Platform has XOS V8.x and Series-6 hardware, the migration procedure is the same as the previous workflow. You are not required to upgrade to 8.5 prior to migration.

If XOS V7.3 is installed, and there are no VAP groups or interfaces configured on the system, and the system identifier is a value between 1-255, it is possible to migrate directly to XOS V9.x using the Workflows on page 205.

Migration with an NPM-82x0 or an APM-8400If your X-Series Platform has a mixture of Series-6 and non-Series-6 hardware, the migration will fail. XOS V9.x requires Series-6 hardware. In most cases, the upgrade process to Series-6 hardware requires application and configuration updates. Before deciding to perform a migration, consider if it is more efficient to migrate your current configuration or to perform a clean installation of XOS V9.x and reconfigure your system.

Perform the following upgrades to bring your system up to the base-level configuration for Migration.

1. Upgrade the modules according to the Series-6 upgrade instructions included with your hardware.

2. Upgrade your configuration. See Upgrade Considerations on page 208 for more information.

3. Upgrade or uninstall/reinstall the applications. See Applications Supported for Migration on page 205.

4. After the upgrades are complete, perform the XOS V9.x Migration procedure.

Upgrade ConsiderationsWhen migrating XOS, a script checks the configuration file for compatibility problems. If any issues are discovered, you have the option of stopping the upgrade and correcting them. You can also perform these checks manually before migrating. The following checks are performed by the migration script:

Interfaces with overlapping ingress VLANs cannot be redundant and must be reconfigured. A backup interface in an active-active configuration cannot be configured to pass the same type of traffic as the master interface. This means that both the master and backup interfaces cannot be configured to pass the same VLAN traffic, or pass untagged traffic. An example of a valid active-active configuration would be to have an interface passing VLANs tagged 100 be a backup to an interface passing VLANS tagged 300. Refer to VLANs on page 63 for more information.


If your configuration uses the acl-interface command to mirror or pass-through traffic from a source interface to a target interface, the following configurations must be updated. If these configurations are not updated, your acl-interface configuration will not transfer into XOS V9.x:

The target acl-interface must be configured.

Remote mirroring and pass-through (when the target port is on a different NPM than the source port) is not supported.

An interface cannot mirror or pass-through traffic to itself.

The MTU setting is no longer applicable to interfaces. Instead, the MTU must be assigned to circuit and VAP group with the circuit <name> vap-group <name> MTU command.

Virtual router configurations can no longer have multiple VAP groups mapped to a single circuit. Make sure that all virtual routers have only one VAP group assigned. A VAP group is assigned to a virtual router using the vrrp virtual-router vrrp-id <id> vap-group <name> command.

The promiscuous-mode active parameter should be used for circuits used as members in a bridge. Previously, the member circuits were set to promiscuous-mode using the circuit <name> vap-group <name> command.

The system ID must be configured if upgrading to the NPM-8600. You can configure the system ID with the configure system-identifier <system-id> command, where valid values are 1 to 255.

CPM versions prior to CPM-8600 are not supported.

Upgrades from a version prior to XOS V8.0 are not supported, with the exception of an unconfigured XOS V7.3.

The series-2 operating-mode is not supported. Please configure the operating-mode to series-6 prior to upgrading.

The xslinux_v1 VAP OS is not supported. Please remove any VAP groups using the xslinux_v1 OS.

The xslinux_v4 VAP OS is not supported. Please remove any VAP groups using the xslinux_v4 OS.

UNI kernels are not supported. Please change these VAP groups to 'kernel-smp auto'

A system-identifier of 0 is not supported. Please configure the system-identifier to <1-255> prior to upgrading.

During an upgrade or migration, if your running configuration includes a flow rule with either a source or destination IP address in the form a.b.c.d/0 where a.b.c.d is not 0.0.0.0, the address will be converted to 0.0.0.0/0.

Only the following modules are supported on XOS V9.5.x:

CPM-8600, CPM-9600

APM-8600, APM-8650, APM-9600, APM-2030

NPM-8600, NPM-8620, NPM-8650, NPM-20, NPM-30

Refer to the XOS Command Reference Guide for a list of all commands that have changed in XOS V9.5.x.

Log FilesMajor milestones on the migration process are logged in the syslog of the running distribution. More detailed information is logged into the following files:

/crossbeam/logs/migration_s1.log – This log file will contain information for stage one of the migration process. It will be created and updated on the old distribution, and then copied to the new distribution at the end of stage one.

/crossbeam/logs/migration_s2.log – This log file will contain information for stages two and three of the migration process. It will be created and updated on the new distribution. The log files include a time and date stamp for each event.



19Configuring Routing Protocols

This chapter describes how to install routing protocols on the X-Series Platform. The routing protocols include:

Routing Information Protocol Version 2 (RIPv2)

Routing Information Protocol Version Next Generation for IPv6 (RIPng)

Open Shortest Path First (OSPF)

Open Shortest Path First for IPv6 (OSPFv3)

Border Gateway Protocol (BGP)

Protocol Independent Multicast-Sparse Mode (PIM-SM)

Crossbeam provides these routing protocols in a Routing Software (RSW) package that is sold and installed separately from XOS. Refer to the RSW Installation Guide for detailed information.


Routing Software Overview on page 211

Installing Routing Protocols on page 213

Installing a Routing Protocol on a VAP Group on page 214

Managing Routing Configurations on page 214

IPv6 Tunneling on page 217

Routing Software OverviewRegardless of the protocol that you install and use on the Crossbeam Platform, RSW installs a Network Services Module (NSM) that coordinates the routing information between the routing daemons and the operating system on the X-Series Platform. NSM shares the routing information between all members in a VAP group.

NSM collects and distributes all routing information from the routing protocols, the kernel, and static routes. In XOS V9.x, the routing software only uses static routes configured in the XOS CLI.

NSM is installed automatically with the routing protocols. Therefore, a routing protocol such as OSPF cannot run independently without an active NSM daemon. If the NSM daemon accidentally stops, all routing protocols on the platform stop.

The routing daemons run on a single, master-VAP in a VAP group so that external routers see the VAP group as a single neighbor. The routing daemons communicate with neighbor devices and pass calculated routing information to the routing information base (RIB). NSM also collects information about directly connected networks and static routes from the XOS operating system.

After NSM on the master VAP accumulates routing information, it writes the routing information to the XOS Forwarding Information Base (FIB), and sends collected routing information to the NSM instances running on all other members in the VAP group. The NSM instances on all other VAPs write the routing information to the respective XOS FIBs. This architecture ensures that every member in a VAP group contains identical routing information.


Understanding Routing Services Failover on the X-Series PlatformRouting services stop when failures occur on the master VAP that cause a routing protocol daemon or NSM to stop. To minimize the impact of the failure, XOS selects another master VAP and starts the routing protocol on the newly elected master. The NSM daemon on the new master VAP begins relearning routing information and distributing it within the VAP group.

During the transition to a new master, all existing routes are stale and unused and the new master VAP must relearn all routes. During the relearning period, the FIB Retain feature in NSM allows the use of stale routes (default of 90 seconds).

See Configuring FIB Retain on page 215 for more information.

Converting Static IP Routes From a Previous RSW VersionIn earlier versions of RSW, static routes were created and managed by NSM. In XOS V9.x, XOS writes static routes into the VAP operating system and NSM uses only XOS CLI-created static routes.

The conversion script in XOS V9.x, nsm_routes_to_cli.sh, converts routes configured in the nsm.conf file on the first VAP in the VAP group into configuration commands for XOS. The conversion script validates each route in nsm.conf for a next hop IP address. The script discards any route that does not have a next hop IP address and records the discarded route in the conversion script output.

Usage: /crossbeam/bin/nsm_routes_to_cli.sh [OPTIONS] Convert NSM IP routes to XOS IP routes. Options: -o <file>: Output file to store result -h : Print out this help

Converting Static IP Routes to XOS CommandsFollow these steps to convert static IP routes configured in nsm.conf to XOS route configuration commands.

NOTE: Make backup copies of the XOS running-config and each NSM and protocol configuration before converting static IP Routes to XOS commands. See Configuring Static IP Routes on page 107 for more information.

1. Log into the CPM: CBS#

2. Change to root: CBS# unix su [root@test-chassis admin]#

3. Change directory to /crossbeam/bin: [root@test-chassis admin]# cd /crossbeam/bin [root@test-chassis bin]#

4. Execute the route conversion command: [root@test-chassis bin]# ./nsm_routes_to_cli.sh -o <file> [root@test-chassis bin]#

The script finds the first VAP and converts the routes in nsm.conf to XOS commands saved in the specified <file>.

The script output indicates any routes that do not have a next hop IP address specified.

Executing Converted Static IP Routes to Configure XOS1. Log into the CPM:

CBS#

2. Execute the commands in the file with converted routes: CBS# exec /<path>/<file> CBS#


NOTE: If any routes are invalid, the XOS CLI returns an error message.

3. Copy the running-config to startup-config: CBS# copy running-config startup-config CBS#

Verifying Static IP RoutesVerify the static routes before and after the conversion and configuration into XOS. Do this by comparing the static routes defined in the XOS running-config and the NSM configuration file.

Installing Routing ProtocolsTo install the routing protocols onto the XOS CPM:

1. Log into the X-Series Platform as root, using the following commands:

CBS# unix suPassword:[root@xxxxx admin]#

For the xslinux_v3 VAP operating system, copy the following RPMs to the /usr/os/rsw/rpm directory:

zebos-common-x.x.x-x.x.x.x.EL.i686.rpmzebos-bgp-x.x.x-x.x.x.x.EL.i686.rpmzebos-ospf-x.x.x-x.x.x.x.EL.i686.rpmzebos-ospf6-x.x.x-x.x.x.x.EL.i686.rpmzebos-rip-x.x.x-x.x.x.x.EL.i686.rpmzebos-ripng-x.x.x-x.x.x.x.EL.i686.rpmzebos-pim-x.x.x-x.x.x.x.EL.i686.rpm

For the xslinux_v5 or xslinux_v5_64 VAP operating system (designated by an EL5 in the file name), copy the following RPMs to the /usr/os/rsw/rpm directory:

zebos-common-x.x.x-x.x.x.x.EL5.i686.rpmzebos-bgp-x.x.x-x.x.x.x.EL5.i686.rpmzebos-ospf-x.x.x-x.x.x.x.EL5.i686.rpmzebos-ospf6-x.x.x-x.x.x.x.EL5.i686.rpmzebos-rip-x.x.x-x.x.x.x.EL5.i686.rpmzebos-ripng-x.x.x-x.x.x.x.EL5.i686.rpmzebos-pim-x.x.x-x.x.x.x.EL5.i686.rpm

2. Exit from the root:

[root@xxxxx rpm]# exit


Installing a Routing Protocol on a VAP GroupPerform the following to install and configure a routing protocol:

1. Create a VAP group for the routing protocol (refer to Configuring a VAP Group on page 52). If a VAP group was already created, you need to know which kernel is installed as displayed by the show vap-group command.

2. Install a protocol on a VAP group:

CBS# routing-protocol <protocol> vap-group <vap-group-name> install

Where <protocol> is rip, ripng, ospf, ospf6, pim, or bgp.

3. After the protocol has been installed, enable and start the protocol:

CBS# configure routing-protocol <protocol> vap-group <vap-group-name> start

4. Configure the protocol:

CBS# routing-protocol <protocol> vap-group <vap-group-name> configure

The last command launches the ZebOS CLI. You are prompted for a password. Refer to the RSW Installation Guide for the default password, information on how to configure the routing protocols, and to change the default password. When you have finished using the ZebOS CLI, type exit to return to the XOS CLI.

Managing Routing ConfigurationsThe following commands allow you to manage the routing protocol configurations:

routing-protocol <protocol> vap-group <vap-group-name> installrouting-protocol <protocol> vap-group <vap-group-name> configurerouting-protocol <protocol> vap-group <vap-group-name> uninstallrouting-protocol <protocol> vap-group <vap-group-name> updaterouting-protocol <protocol> vap-group <vap-group-name> save <path_and_file>routing-protocol <protocol> vap-group <vap-group-name> restore <path_and_file>routing-protocol <protocol> vap-group <vap-group-name> status

Where:

The following commands write the protocol status to the XOS running-config and take the related action on the protocol daemon.

configure routing-protocol <protocol> vap-group <vap-group-name> {start|restart|stop}

<protocol> Can be rip, ripng, ospf, ospf6 pim, or bgp.

install Installs a protocol on the VAP group.

configure Launches the routing protocol prompt. Refer to the RSW Installation Guide to configure the protocol.

uninstall Removes the protocol from the VAP group.

update Updates routing information for the vap-group. Use this command after members have been added to the VAP group or after the VAP group has been deleted and recreated.

save <path_and_file> Saves an existing routing configuration on a VAP group to a file.

restore <path_and_file> Restores a saved routing configuration from a file to a VAP group.

Caution: Restoring an RSW configuration file from a lower or higher RSW version may cause unpredictable results.

status Displays the status of a routing protocol on a VAP group.



Where:

The following command enables access to the routing protocol NSM daemon on the master VAP from the CLI.

routing-protocol-services vap-group <VAP_group_name> configure

Any changes made to NSM are written to all active VAP group members. In NSM you can modify ACLs, configure FIB Retain, and troubleshoot general routing issues.

Accessing NSMUse the following steps to access the CLI for NSM.

1. From the XOS CLI command prompt, enter: CBS# routing-protocol-services vap-group <VAP_group_name> configure Password:

2. Enter the password (default is admin): Password: ***** Router>

3. Enter privileged mode: Router>enable Password:

4. Enter the privileged mode password (default is admin): Password: ***** Router#

Configuring FIB RetainCrossbeam recommends configuring FIB Retain on the VAP group to allow for graceful failover if the NSM daemon restarts or if the master VAP in a VAP group goes down. The default FIB Retain time is 90 seconds.

Because the master VAP builds the RIB used in all VAP members, any interruption to the operation of the master VAP can result in stale routes in the FIBs on slave VAP members.

When XOS elects a new master VAP member, the routing protocols must start, form adjacencies, and exchange routing information. The new master VAP learns routes for the entire VAP group. Slave VAP members must wait for RIB updates from the new master VAP. The resulting delay can cause traffic to be dropped or misrouted.

FIB Retain minimizes the impact of this delay, by allowing you to specify a period of time during which existing routes are preserved while the routing protocol on the newly elected master refreshes routes in the FIB.

If your system uses a single VAP, FIB Retain can temporarily preserve existing routes for IPv4 and IPv6 traffic in the event NSM restarts.

NOTE: Only IPv4 routing information is synchronized across VAPs in the VAP group. IPv6 traffic is processed only on the master VAP. IPv6 routes learned by the master VAP are not shared with other members of the VAP group, but must be relearned by the new master VAP in the event of a failure.

Follow these steps to configure FIB Retain if the default time of 90 seconds is not suitable for your configuration.

<protocol> Can be rip, ripng, ospf, ospf6, pim, or bgp.

start Starts the protocol.

stop Stops the protocol.

restart Restarts the protocol. This forces a running protocol, such as OSPF, to rebuild its routing table.

1. From the XOS CLI command prompt, enter: CBS# routing-protocol-services vap-group <VAP_group_name> configure

2. Enter the password (default is admin): Password: *****

3. Enter privileged mode: Router>enable Password:

4. Enter the privileged mode password (default is admin): Password: ***** Router#

5. Enter the configure terminal context to configure the FIB Retain value: Router#configure terminal Router(config)#

6. Enter the FIB Retain configuration context: Router(config)#fib retain {time <number_of_seconds> | forever} Router(config)# Enter either the number of seconds or forever to retain the FIB values.

7. Exit the terminal configuration context: Router(config)#end Router#

8. Write the configuration to nsm.conf: Router#write Configuration saved to /usr/os/etc/zebos/nsm.conf Router#

9. Exit NSM to synchronize the changes to all VAP members: Router#exit CBS#

10. CBS#

Redistributing RoutesProtocols in the X-Series Platforms can be configured to redistribute routes to other protocols. Refer to the RSW Installation Guide for information on how to access the protocol and turn on route redistribution.



IPv6 TunnelingIPv6 tunneling provides a method to transmit and receive IPv6 traffic over an IPv4 network. Typically, this kind of tunnel is used to connect to an IPv6-enabled destination (host or network).

In the Crossbeam implementation of IPv6 tunneling, one end of the tunnel is a Crossbeam X-Series chassis and the other end is a dual stack router. IPv6 traffic is encapsulated in IPv4 packets and passed over the IPv4 network. When an IPv4 packet reaches the destination, it is unencapsulated and the IPv6 packet is routed appropriately. On the X-Series chassis, the traffic-processing VAP group must be configured to handle IPv6 traffic.

6to4 TunnelOne IPv6 tunneling protocol is 6to4 tunneling. Using 6to4, you do not have to specify the destination address of the tunnel. You configure the local (source) end of tunnel with an IPv4 address and an IPv6 address. The IPv6 address contains an encoding of the IPv4 address.

The major steps to configure the local/source end of the tunnel are:

Create a VAP group and enable it to handle IPv6 traffic.

Create a circuit with an IPv6 address and associate it with the VAP group.


Create the tunnel, specifying 6to4 as the tunnel type, associating the tunnel with the VAP group and the IPv6 circuit, and specifying the address of the IPv4 circuit as the source-address.

Figure 15. 6to4 Tunnel

ExampleTo configure a 6to4 tunnel using the parameters in Figure 15, perform these steps:

1. On the X-Series chassis configure a VAP group, and enable IPv6 traffic processing.

CBS# configure vap-group vg1 enable-ipv6

NOTE: A non-ip flow rule is automatically configured with these parameters:

non-ip-flow-rule ipv6_rule encapsulation ethernet type 34525 action pass-to-master activate

2. Identify the IPv4 address that you want the VAP group to use to send traffic over the IPv4 network. In this example, the local IPv4 address is 172.16.20.30.

NOTE: This address will be the source address that you specify when configuring the 6to4 tunnel.

3. Configure the first of two circuits with these characteristics:

a. Choose a name for the circuit (in this example, tunnel_a).

CBS# configure circuit tunnel_a

b. Associate the circuit with the VAP group (in this example, vg1)

CBS(conf-cct)# vap-group vg1

c. Assign an IPv6 address to the circuit. The address must be constructed as follows:

The first 16-bit field must be 2002 to identify the IPv6 address as a 6to4 tunnel address.

The next two fields must contain a decimal-to-hexadecimal conversion of the IPv4 address that you identified in Step 2. In this example, converting 172.16.20.30 to two hexadecimal numbers results in AC10:141E (172 converts to AC, 16 converts to 10, and so on).

In the last 16-bit field, specify the host address that you want to associate with this circuit (for example, 3).

CBS(conf-cct-vapgroup)# ip 2002:AC10:141E:0:0:0:0:3/48%WARNING: IPv6 primary address was configured. No IPv4 aliases will be allowed for this circuitCBS(conf-cct-vapgroup-ip)# end

4. Configure a second circuit with these characteristics:

Choose a name for the circuit (in this example, tunnel_b).

CBS# configure circuit tunnel_b

Associate the circuit with the VAP group (in this example, vg1)


Assign the local IPv4 address that is being used by the VAP group to send traffic to the external IPv6 network.

CBS(conf-cct-vapgroup)# ip 172.16.20.30/24CBS(conf-cct-vapgroup-ip)# end

5. Configure the 6to4 tunnel.

CBS# configure ipv6-tunnel 6to4 circuit tunnel_a vap-group vg1CBS(conf-tunnel-6to4)# source-address 172.16.20.30CBS(conf-tunnel-6to4)#

6. Optionally, configure a time-to-live value for packets sent through the tunnel. Values range from 1 to 256 seconds, and the default is 64.

CBS(conf-tunnel-6to4)# time-to-live 120

7. Optionally, enable Maximum Transmission Unit (MTU) discovery for the path that the tunnel uses. Discovery of the MTU helps to avoid fragmentation.

CBS(conf-tunnel-6to4)# path-mtu-discoveryCBS(conf-tunnel-6to4)# endCBS#


ISATAP TunnelISATAP (Intra-Site Automatic Tunnel Address Protocol) is used for a site that has not fully implemented an IPv6 network. An ISATAP host talks to an ISATAP router through the ISATAP automatic tunnel over the existing IPv4 network. Typically, this kind of tunnel enables IPv6-enabled hosts to communicate over a local IPv4 network.

The ISATAP tunnel interface treats underlying IPv4 as an NBMA (Non Broadcast Multi Access) network. Each ISATAP interface is assigned the same IPv6 prefix. The Router Advertisements (RA) are used to automatically assign prefixes.

The major steps to configure the local/source end of the tunnel are:




Create the tunnel, specifying isatap as the tunnel type, associating the tunnel with the VAP group and the IPv6 circuit, and specifying the address of the IPv4 circuit as the source-address.

Figure 16. ISATAP Tunnel

ExampleTo configure an ISATAP tunnel using the parameters in Figure 16, perform these steps:



NOTE: A non-IPv4 flow rule is automatically configured with these parameters:



2. Identify the IPv4 address that the you want the VAP group to use to send traffic over the IPv4 network. In this example, the local IPv4 address is 172.32.30.40.

NOTE: This address will be the source address that you specify when configuring the ISATAP tunnel.


a. Choose a name for the circuit (in this example, tunnel_1).

CBS# configure circuit tunnel_1



c. Assign an IPv6 address to the circuit. The address must be constructed as follows:

The prefix fields can be any valid IPv6 prefix, but must include the reserved identifier for ISATAP, which is 0:5efe. This example uses FC00:0:0:0:0:5EFE: as the prefix.

The last four fields must contain a decimal-to-hexadecimal conversion of the IPv4 address that you identified in Step 2. In this example, converting 172.32.30.40 to two hexadecimal numbers results in AC20:1E28: (172 converts to AC, 32 converts to 20, and so on).

CBS(conf-cct-vapgroup)# ip FC00:0:0:0:0:5EFE:AC20:1E28/16%WARNING: IPv6 primary address was configured. No IPv4 aliases will be allowed for this circuitCBS(conf-cct-vapgroup-ip)# end


Choose a name for the circuit (in this example, tunnel_2).

CBS# configure circuit tunnel_2





5. Configure the isatap tunnel.

CBS# configure ipv6-tunnel isatap circuit tunnel_1 vap-group vg2CBS(conf-tunnel-isatap)# source-address 172.32.30.40CBS(conf-tunnel-isatap)#


CBS(conf-tunnel-isatap)# time-to-live 120


CBS(conf-tunnel-isatap)# path-mtu-discoveryCBS(conf-tunnel-isatap)# endCBS#

NOTE: For ISATAP hosts to obtain automatic IPv6 addresses, you must turn on Router Advertisements in the RSW software on the X-Series chassis. See the RSW documentation for details.



IPv6IP TunnelAn IPv6IP tunnel requires that you configure an IPv4 address for both ends of the tunnel (on the X-Series chassis and on the destination router). Typically, this kind of tunnel is used as a static connection between two IPv6-enabled networks that are separated by an Ipv4 network.

The major steps to create the local/source end of the tunnel are:




Create the tunnel, specifying ipv6ip as the tunnel type, associating the tunnel with the VAP group and the IPv6 circuit, specifying the address of the IPv4 circuit as the source-address, and specifying the IPv4 address of the destination router as the destination-address.

Figure 17. IPv6IP Tunnel

ExampleTo configure an IPv6IP tunnel using the parameters in Figure 17, perform these steps:

1. On the X-Series platform configure a VAP group, and enable IPv6 traffic processing.





NOTE: This address will be the source address that you specify when configuring the IPv6IP tunnel.


a. Choose a name for the circuit (in this example, tunnel_x).

CBS# configure circuit tunnel_x



c. Assign an IPv6 address to the circuit (in this example, FD00:BC00:FFFF:2:0:0:0:2/48).

NOTE: Unlike the IPv6 addresses that are associated with 6to4 and ISATAP tunnels, this address has no pre-defined prefix and does not contain any encoding of the IPv4 source-address.

CBS(conf-cct-vapgroup)# ip FD00:BC00:FFFF:2:0:0:0:2/48%WARNING: IPv6 primary address was configured. No IPv4 aliases will be allowed for this circuitCBS(conf-cct-vapgroup-ip)# end


Choose a name for the circuit (in this example, tunnel_y).

CBS# configure circuit tunnel_y





5. Configure the IPv6IP tunnel.

CBS# configure ipv6-tunnel ipv6ip circuit tunnel_x vap-group vg3 source-address 172.48.40.50 destination-address 172.48.40.51CBS#


CBS# configure ipv6-tunnel ipv6ip circuit tunnel_x vap-group vg3 time-to-live 120CBS#


CBS# configure ipv6-tunnel ipv6ip circuit tunnel_x vap-group vg3 path-mtu-discoveryCBS#

GRE TunnelA GRE tunnel requires that you configure an IPv4 address for both ends of the tunnel (on the X-Series chassis and on the destination router). Typically, this kind of tunnel is used as a static connection between two IPv6-enabled networks that are separated by an Ipv4 network.

The major steps to create the local/source end of the tunnel are:




Create the tunnel, specifying gre as the tunnel type, associating the tunnel with the VAP group and the IPv6 circuit, specifying the address of the IPv4 circuit as the source-address and the IPv4 address of the destination router as the destination-address.


Figure 18. GRE Tunnel

ExampleTo configure a GRE tunnel using the parameters in Figure 18, perform these steps:






NOTE: This address will be the source address that you specify when configuring the IPv6IP tunnel.


a. Choose a name for the circuit (in this example, tunnel_m).

CBS# configure circuit tunnel_m



c. Assign an IPv6 address to the circuit (in this example, FD00:DE00:AAAA:5:0:0:0:5/48).

NOTE: Unlike the IPv6 addresses that are associated with 6to4 and ISATAP tunnels, this address has no pre-defined prefix and does not contain any encoding of the IPv4 source-address.

CBS(conf-cct-vapgroup)# ip FD00:DE00:AAAA:5:0:0:0:5/48%WARNING: IPv6 primary address was configured. No IPv4 aliases will be allowed for this circuitCBS(conf-cct-vapgroup)# end


Choose a name for the circuit (in this example, tunnel_n).

CBS# configure circuit tunnel_n






5. Configure the GRE tunnel.

CBS# configure ipv6-tunnel gre circuit tunnel_m vap-group vg4 source-address 172.64.50.60 destination-address 172.64.50.61


CBS# configure ipv6-tunnel gre circuit tunnel_m vap-group vg4 time-to-live 120CBS#


CBS# configure ipv6-tunnel gre circuit tunnel_m vap-group vg4 path-mtu-discoveryCBS#


20Backups, Upgrades, and Reinstallation

This chapter describes how to backup the configuration, upgrade XOS hardware, and reinstall XOS software. This information is provided to assist with upgrading to 86xx modules so that migration to XOS V9.x can take place. You cannot use the upgrade or reinstallation procedures to upgrade to XOS V9.x.

The major topics in this chapter include:

VAP Backup on page 225

Preparing for Safe Upgrade on page 226

Upgrading to CPM-8600 or APM-8600 Modules on page 229

Creating a New Configuration Database on page 230

Reinstallation on page 230

VAP BackupThe Backup tool can backup everything under the specified directory tree for a CPM or all members of a VAP group. This information is tarred and zipped, with the output being placed in /root. To use the Backup tool, complete the following:

1. Log in as root.


2. Enter the following command at the root prompt.

/usr/os/bin/archive_dir -v <vap-group-name> -d <directory-name>

Command line options:

If you do not provide any options at the command line, the Backup tool runs in interactive mode and prompts for the VAP group name, directory to be backed up, and archive name. If you leave VAP group name blank, the directory from CPM is backed up. In backing up a VAP group, enter the path as it would show while logged in to a VAP.

To restore a file:

1. Unzip the file

gzip –d <targz_file>

2. Move to:

/tftpboot/[<vap-group-name>_<index>]

-v <vap-group-name>

If the -v option is used, you must specify the <vap-group name>. If the -v option is not used, then the CPM will be archived.

-d <directory-name> Default = /usr/os (the directory may show as /crossbeam; it is the same as /usr/os)

-h Help

-V version


3. Untar the file:

tar xvf <tar_file>

If restoring a VAP group directory, repeat this process for all the VAP group indexes, including the <vap-group-name>_common directory.

NOTE: If you backed up a <vap-group-name> / without specifying another directory, you may see a Cannot mkdir: No such file or directory message. This can be ignored.

Preparing for Safe UpgradeThe Automated Workflow System can be used to upgrade from XOS V9.0 to XOS V9.x. See XOS Automated Workflow System on page 195 for more information.

You cannot perform an upgrade to XOS V9.0. Upgrading to XOS V9.0 from pre-V9.0 can only be done using the Migration process. See XOS Migration on page 204 for more information.

The Safe Upgrade feature partitions the disks on the CPMs into two distributions so that two different XOS configurations can be installed at the same time, and the CPM can boot in either distribution. This allows you to install and use a later XOS release while keeping the existing release. It also allows two different configurations using the same XOS release. Dividing the CPM hard disk into 2 distributions does not degrade system performance.

The Automated Worflow System (AWS) provides a simple procedure for preparing for an upgrade, and upgrading your XOS version. In the case where an upgrade fails, having a snapshot of your system also allows for an easy rollback using AWS.

AWS automatically performs prerequisite checks and notifies you of any incompatibilities. All backup procedures are then performed automatically.

To perform the safe upgrade process prior to an XOS upgrade, use the following procedure.

1. Open the Automated Workflow System.

Command:


Welcome to the X-Series Platform Automated Workflow System!Version: 1.1.0-11

1. Configure XOS... 2. Upgrade XOS software and firmware... 3. View system configuration and status... 4. Applications... 5. Custom...


2. Select 2. Upgrade XOS software and firmware...

Please Enter Selection: 2

Automated Workflow Menu: Upgrade XOS software and firmware

1. Upgrade XOS software and install recommended firmware 2. Prepare system for a possible rollback 3. Install XOS firmware only 4. Roll back XOS software and firmware

Backups, Upgrades, and Reinstallation 226

5. Verify XOS software and firmware compatibility

Select a task to execute an automated workflow.Enter b to go back, x to exit or ? for help.

3. Choose 2. Prepare system for a possible rollback

Please Enter Selection: 2

Automated Workflow: Prepare system for a possible rollback

Press ? for assistance on questions

Analyzing system configuration...

VRRP State : non-masterCPM Redundancy Status : singleCurrent XOS Software Release : 9.5.x-67

All external management interfaces will be shut off.Please use the console interface to monitor the system.

WARNING: Rollback preparation will automatically reload the chassis!

Begin rollback preparation now (y/n)? [n]: y

Once the rollback (safe upgrade) procedure completes, return to the main AWS menu and perform the XOS upgrade.

Manual Safe Upgrade ProcedureIn a single CPM system, take the system offline. In a dual CPM system, the system can remain active. To partition the CPM disk drive for the safe upgrade feature, you need to run in single user mode which requires a console connection to CPM. If you have a Dual Box High Availability configuration, repeat this procedure on each system.

PrerequisitesA minimum 80GB hard drive on the CPM.

Perform the following steps:

1. Reboot the CPM.

2. Watch carefully for the GRUB menu. At the GRUB menu, type p on the keyboard then enter the password, x40rocks.

3. At the GRUB prompt, type e to edit.

4. Change the line number by pressing the down arrow until you reach the grub line starting with kernel. Add single to the end of this line, and press Enter to save the edit.

5. Type b to boot the CPM in single user mode.

6. Run the following command to copy from distribution 1 (d 1) to distribution 2 (d 2):

/usr/os/bin/xos-copy-dist -d 2

Everything on the running partition is copied to the other partition. This could take up to two hours.

7. Reboot the CPM. The GRUB boot loader will show the option to boot from the second distribution.


8. In dual CPM system, repeat this procedure to partition the disk in the second CPM. However, the CPM being partitioned must not be monitored by the other CPM for redundancy.

To prevent monitoring, enter:

CBS# cp-unknown-state {cp1|cp2} ignore

For example, the following command forces CP2 to take no action while CP1 is being partitioned:


You would repeat this step when the other CPM is being partitioned, for example:


After the partitioning, you need to have the CPMs monitor each other for redundancy:

CBS# cp-unknown-state {cp1|cp2} monitor

To change or list the kernel from either partition (d1 or d2) to be used on next boot:

/usr/os/bin/xos-toggle-boot {d1|d2}

When using ./xos-toggle-boot from the /crossbeam/bin directory to switch distributions and boot into another version of XOS, use ./xos-toggle-boot -l to see all the kernel versions available. When you are in XOS V8.x, V9.x will not be specified in the list. The kernel will be available, but XOS V8.x will not recognize it as V9.x. Always choose the first kernel listed without an XOS version number.

Under XOS V8.x:

[root@xxxxx bin]# ./xos-toggle-boot -l

System is running on Distribution 1(Release 8.5.0)

Default kernel for next boot is Distribution 11: Linux (2.4.21-47.ELuni) (Distribution 1 - 8.5.0)

Available kernels:0: Linux (2.4.21-47.EL) (Distribution 1 - 8.5.0)1: Linux (2.4.21-47.ELuni) (Distribution 1 - 8.5.0)2: Linux (2.4.21-47.ELuni) (Distribution 1 - 8.5.0)3: Linux (2.6.18-53.el5_64smp) (Distribution 2) <-- Choose this kernel4: Linux (2.6.18-53.el5_64smp) (Distribution 2)[root@TechPubs-X45 bin]#

Under XOS V9.x:

[root@xxxxx bin]# ./xos-toggle-boot -l

System is running on Distribution 2 (Release 9.x.x)

Default kernel for next boot is Distribution 23: Linux (2.6.18-53.el5_64smp) (Distribution 2 - 9.x.x)

Available kernels:0: Linux (2.4.21-47.EL) (Distribution 1 - 8.5.0)1: Linux (2.4.21-47.ELuni) (Distribution 1 - 8.5.0)2: Linux (2.4.21-47.ELuni) (Distribution 1 - 8.5.0)3: Linux (2.6.18-53.el5_64smp) (Distribution 2 - 9.x.x)4: Linux (2.6.18-53.el5_64smp) (Distribution 2 - 9.x.x)[root@xxxxx bin]#

To change the default kernel from the CLI, use the cp-next-boot {cp1|cp2} distribution x command, where x is 1 or 2, then reboot. The show cp-next-boot CLI command displays which partition will be used to boot the CPM the next time the CPM boots.



NOTE: If you are using xos-copy-dist to copy from D2 to D1, the default boot disk will be D2 at the completion of the command. This is not the case if you are copying from D1 to D2.

Partitioning the CPM Hard Disk

You can divide the CPM hard disk into two partitions and use the xos-copy-dist command. Creating a second partition provides the following benefits:

Enable the Safe Upgrade feature

This feature enables you to install a future upgrade on one partition of the CPM hard disk, while your existing XOS version and configuration remain on the other partition. You can then choose to boot the CPM in either partition. This enables you to try a new version while having the option to easily return to your existing version in case of issues.

Run the same XOS version on both partitions and use the partitions to try different configurations.

Create a backup of the partition.

NOTE: Dividing the CPM hard disk into 2 partitions does not degrade system performance.

For a detailed partitioning procedure, refer to Manual Safe Upgrade Procedure on page 227.

NOTE: If you are using xos-copy-dist to copy from D2 to D1, the default boot disk will be D2 at the completion of the command. This is not the case if you are copying from D1 to D2.

Upgrading to CPM-8600 or APM-8600 ModulesThe following instructions outline the steps needed to upgrade to CPM-8600 or APM-8600 hardware.

NOTE: If installing CPM-8600s to replace earlier CPMs and you want to enable the RAID feature on the new CPMs (assuming the boards have dual disk drives installed), or to reconfigure the RAID setting on previously installed CPM-8600s, you must perform a reinstallation. Refer to Reinstallation on page 230.

Upgrade a CPM-8400 to a CPM-8600NOTE: XOS V9.x requires that a CPM-8600 have 4 GB of memory

To upgrade your CPM-8400 to CPM-8600, complete the following steps:

1. If you currently have CPM redundancy configured, skip this step. If you have a single CPM, configure CPM redundancy to facilitate the upgrade process. Refer to Configuring CPM Redundancy on page 121 for more information. This process takes about eighty (80) minutes.

NOTE: You must reload the chassis when changing from non-CPM redundancy to CPM redundancy.

2. Upgrade all existing CPM modules to XOS Version 8.1 or later.

3. Insert the new CPM-8600 module into the secondary CPM slot. Refer to the X80 Platform Hardware Installation Guide for more information about inserting modules.

4. After the synchronization is complete, halt the old CPM module and remove it from the chassis. This causes CPM failover to occur. The secondary CPM (the 8600 module) becomes the primary CPM.

5. Insert a new CPM-8600 module.

6. Configure this CPM-8600 module as the backup.

Upgrade an APM-8400 to an APM-8600NOTE: A single VAP group cannot contain APM-8400 modules and AMP-8600 modules, except during the

upgrade process described below.

The following procedure assumes there are two empty slots. To upgrade APM-8400 to APM-8600, complete the following steps:

1. Upgrade all existing APM modules to XOS Version 8.1 or later.

2. Install the first new APM-8600 into an empty APM slot. Refer to the X80 Platform Hardware Installation Guide for more information about installing modules.

3. Configure the APM-8600 into the VAP group.

4. Bring down an APM-8400 module using the following command:

configure module <slot-number> maintenance

This command automatically initializes an APM-8600 into the VAP group.

NOTE: For more information about the use of maintenance mode, see Maintenance Mode on page 297.

5. Remove the APM-8400 that is in maintenance mode and replace it with another APM-8600.

6. Repeat steps 3 through 5 for each APM-8400 until all new members in the VAP group are APM-8600 modules.

Creating a New Configuration Database

To create a new configuration database and delete the old which includes deleting all VAPs, use the following command:

CBS# reset-configuration

ReinstallationReinstallation is accomplished through the use of either the Install Server, or the USB Installer (USBI).For complete information on using the Install Server, refer to the Install Server User Guide. For complete information on using the USB Installer, refer to the USB Installer User Guide.

If you have previously configured and run the Install Server, you may use the following procedure to re-install the XOS software. If you have not created your Install Server, refer to the Install Server User Guide.

1. From the system console on the CPM, reboot the CPM and issue an <Esc> <Ctrl-r> command during the reboot as follows:

a. To reboot the CPM, issue the command:

reload module <slot # x>

Where x the slot number of your master CPM.

b. At the start of the reboot process and during the reboot you will need to respond to prompts including:

Do you want to save (your configuration) to startup-config? ... Proceed with reload?

c. During the reboot process, there is a momentary pause with a message similar to the following:

Crossbeam System, Inc. Bootstrap Complete Rev x.x.x.x

This is described as the spinning “|” prompt.

d. At the spinning “|” prompt, press <Esc> <Ctrl-r>.

Caution: The following command erases all configuration data, followed by a system reboot going through the first-time boot sequence.


e. From the system console, use the following command to boot the XOS diskless CPM:

ROM> boot net

The boot net command sends out a broadcast message to locate the Install Server. The X-Series Platform is forced to boot from the Install Server, using the diskless partition with the matching MAC address.

2. From the system console, login as root. The default password is admin.

3. Install the XOS system software, as follows, where <system dirname> is the directory name containing the system software release to be installed.

# /usr/os/bin/install-xos /common/<system dirname> [-r1]

NOTE: The -r1 is a RAID option. Prerequisites and RAID details are provided in RAID-Related Hard Drive Configuration and Repair on page 241.

RAID options can only be enabled on a CPM during an install or reinstallation, as follows:

default, no switch included, the system writes only to the primary drive.

-r1, dual disk drives in a CPM-8600, the system writes identical data to both drives, creating a complete backup in case of one drive’s failure. You can use the RAID-1 option (-r1) in combination with the CPM SYNC feature.

4. When the install-xos script completes, enter reboot at the prompt. The CPM then reboots from its own disk and prompts you for a login name and password.

Changing the Disk Partitioning SchemeBegininning with XOS V9.5.0, you can change the disk partitoning scheme on the CPM using the cp-disk-scheme command. The CPM supports partitions of 80, 120, 250, and 500 Gigabytes.

cp-disk-scheme 500

The partitioning scheme that you select must be equal to or smaller than your CPM disk size and must be larger than the current partitioning scheme.

If you want to configure cp-redundancy between two CPMs, both must have the same disk partitioning scheme.

After you configure a new partitioning scheme, you must reboot the CPM.

Use the show cp-disk-scheme command to see the current and the configured disk partitioning schemes.

CBS# show cp-disk-schemeCurrent scheme: 250GBConfigured scheme: 500GB



AIn-Service Upgrade

This appendix describes In-Service Upgrades (ISU), an alternate method of upgrading XOS software while minimizing downtime. It includes the following major topics:

In-Service Upgrade (ISU) Overview on page 233

In-Service Upgrade Process Overview on page 234

Modes of In-Service Upgrade on page 234

Understanding Optimal Batch Configuration on page 235

Executing a Sample In-Service Upgrade on page 236

Executing a Non-Interactive In-Service Upgrade on page 240

In-Service Upgrade (ISU) OverviewYou cannot perform an In-Service Upgrade to bring your X-Series Platform up to XOS V9.x. You must use the Migration process. See XOS Migration on page 204 for more information.

The process of an ISU virtually splits the chassis in half. One half of the chassis is responsible for forwarding traffic, while the other half is upgraded. The first step in the process is upgrading the secondary CPM to the new version of XOS software and rebooting into primary mode. During this time, there will be no interruption in traffic. Once the upgraded CPM has initialized as a primary CPM, APMs and NPMs will be moved — in batches — to the half of the chassis controlled by the upgraded CPM. The batches of APMs and NPMs will be grouped to minimize downtime and impact to traffic flows. The final stage of ISU includes upgrading the old primary CPM and reloading it into secondary mode.

NOTE: During the upgrade, the NPMs and APMs are rebooted by the system.

PrerequisitesThe ISU feature has several requirements for successful completion, including redundant CPMs, APMs, and NPMs. The following are the minimum prerequisites to perform an In-Service Upgrade.

XOS V8.0 or later.

Two CPMs.

Two (or more) APMs per VAP group are recommended.

Circuits configured with redundancy across two (or more) NPMs are recommended.

If needed for applications, configure Application Monitor.

Resolve any issues listed below before starting In-Service Upgrade.

Verify that CPM redundancy is enabled and that CPMs are synchronized. Note that CPM synchronization may take between 45 and 90 minutes depending on your system’s configuration.

Verify that both CPMs are in election mode.

Verify that the current boot partition is the same as cp-next-boot setting.

Verify that adequate disk space (greater than 2GB) is available.


In-Service Upgrade 234

LimitationsThe following are limitations of In-Service Upgrade:

Applications are not upgraded during the process — only the XOS operating system.

Dynamic routing tables are not preserved during upgrade.

The XOS chassis does not operate at full capacity during the upgrade.

No CPM redundancy exists during the upgrade.

There is no state sharing between the upgraded and un-upgraded CPMs within a chassis.

The configuration database is locked during the In-Service Upgrade.

Interfaces without redundancy are not supported.

In-Service Upgrade Process OverviewThe basic steps for In-Service Upgrade are:

Prepare for the upgrade — Copy, unzip, set permissions, and unpack the shar file

Enter In-Service Upgrade

Configure batches

Issue the install command through CLI

Save system configuration

Save the batch configuration

Transition the secondary CPM into offline mode and upgrade

Reload the offline CPM into primary state

Move batches

Upgrade the old CPM

Modes of In-Service UpgradeIn-Service Upgrade has two modes; Interactive and Non-Interactive. Interactive mode allows the user to control the timing of each step during upgrade. Interactive mode is the default mode. Non-Interactive mode executes the upgrade with no user input.

The Interactive In-Service Upgrade can be aborted at any time by entering Ctrl-Y or answering “N” at any prompt in the upgrade interview process. Once the In-Service Upgrade is aborted, all APMs and NPMs are transitioned back to the original state.

Non-Interactive In-Service Upgrades can be executed from either the CLI or from the XOS Graphical User Interface. Non-Interactive In-Service Upgrades begun using the XOS Graphical User Interface are executed automatically using the default settings. Those Non-Interactive In-Service Upgrades begun through the CLI can use either default or custom settings.

The Non-Interactive In-Service Upgrade can be aborted at any time. To end a Non-Interactive In-Service upgrade, switch to an Interactive In-Service Upgrade by executing the CLI command upgrade in-service install <XOS version> and then entering Ctrl-Y.

NOTE: If there is a possibility that the CPM may need to be rolled-back to the previous version, Safe Upgrade is the recommended procedure. For more information on Preparing for Safe Upgrade on page 226.

NOTE: If the show cp-redundancy command is executed during an In Service Upgrade, the following warning message will be displayed: WARNING: In-Service Upgrade Is in Progress.

Understanding Optimal Batch ConfigurationIn-Service Upgrade (ISU) uses the concept of batches to collect APMs and NPMs into groups. A batch consists of one or more APMs or NPMs selected to be taken out of active service and upgraded to the new XOS software. These batches are automatically created through the Configuration Analyzer. The results of the Configuration Analyzer should be reviewed in the context of your current system topology.

As APMs and NPMs are upgraded and transitioned to the new CPM, there is a potential for interruption in traffic. By grouping specific APMs and NPMs into specific batches, the disruption can be minimized. The default batch configuration can be modified as required to a maximum of ten custom batches.

Sample Configuration Analyzer OutputThis section describes the typical Configuration Analyzer output.

Batch1 will consist of one half of the APMs associated with each VAP group, excluding the master VAP. These APMs will be transferred to control of the new CPM and will be upgraded to the new XOS software in the process. Any flows currently processed by the APMs in this group will fail-over to the remaining APMs for that VAP group during the transition as defined by the flow-rules for that group. By design, all down and standby VAPs are moved as part of Batch1.

Batch2 will consist of one half of the NPMs associated with redundant circuits. These NPMs will be transferred to control of the new CPM and will be upgraded to the new XOS software in the process. Any flows processed by the NPMs in this group will fail-over to the remaining NPMs during the transition. Optimally, NPMs controlling LACP circuits, bridges and active redundancy-interfaces will be processed during Batch3 to minimize traffic interruption.

Batch3 will consist of the remaining NPMs. These NPMs will be transferred to control of the new CPM and will be upgraded to the new XOS software in the process. At the start of this batch, all active flows will be transitioned to the upgraded half of the chassis. There will be an interruption in traffic due to this transition.

Batch4 will consist of the remaining APMs in the chassis. Theses APMs will be transferred to control of the new CPM and will be upgraded to the new XOS software in the process. There is no interruption in traffic during this transition. Once all APMs have initialized on the upgraded half of the chassis, the system will be operating at full capacity on the upgraded software.

Creating In-Service Upgrade BatchesThere are two methods for batch creation — Configuration Analyzer and custom batch lists.

Configuration Analyzer

The Configuration Analyzer creates a default batch list using a predefined logic. Review the Understanding Optimal Batch Configuration on page 235 before proceeding.

The default batch list can be viewed by executing the following commands:

CBS# upgrade in-serviceCBS(in-service-upgrade)# show default-batches

To use the default batch list, execute the following command to set the batch list:

CBS# upgrade in-serviceCBS(in-service-upgrade)# batch-default


Standby and “down” modules are the first to be moved since these modules are not passing traffic. These modules are placed in the first batch, which cannot be modified. To view the modules in this batch, enter the following commands:

CBS# upgrade in-serviceCBS(in-service-upgrade)# show standby-modules

Custom Batch List

A custom batch list can be created if required. It is important to understand the impact specific APMs and NPMs have on traffic flows before assigning a slot to a batch. Review Understanding Optimal Batch Configuration on page 235 before proceeding.

Before defining custom batches, clear the current batch list using the following commands:

CBS# upgrade in-serviceCBS(in-service-upgrade)# clear-batches

Once the batches have been cleared, you can define the custom batch configuration. Any NPMs and APMs that are not assigned to a specific batch will be placed in a default batch and moved after the last defined batch has been moved.

The following example assigns np2 (slot 2) to Batch2:

CBS(in-service-upgrade)# batch-2 np2Batch2: np2(2)

The following example assigns multiple slots to a batch:

CBS(in-service-upgrade)# batch-1 fw_1 fw_2 Batch1: fw_1 fw_2Batch2: np2(2)

NOTE: A slot can only be assigned to one batch. If moving a slot that is already defined in another batch, remove the slot from the previous batch before assigning it to the new batch.

Executing a Sample In-Service UpgradeThis section describes a sample In-Service Upgrade that uses a four-batch scenario. The following illustration represents the state of the sample XOS chassis before the upgrade begins.

Figure 19. Sample Chassis Before In-Service Upgrade


The example configuration contains a single VAP group consisting of two members and redundant circuits split across NPM1 and NPM2.

1. The shar file contains the files needed to execute the upgrade. These files must be copied to the correct location (/usr/os/rpm), unpacked, and have the correct permissions applied (chmod 777). This step must be performed on the primary CPM only. For more information, see Backups, Upgrades, and Reinstallation on page 225.

2. At the unix prompt on one of the CPMs, execute the following commands:

[root@xxx admin]# echo isu_child_tmo=10800 >> /usr/os/etc/cbscprmd.cf[root@xxx admin]# killall -HUP cbscprmd

3. Repeat the previous step for the other CPM.

4. Begin an Interactive In-Service Upgrade as shown below. The command batch-default clears any custom batches, and uses only system generated batches.

CBS# upgrade in-serviceCBS(in-service upgrade)# batch-default Batch1: FW_1(3)Batch2: np2(2)Batch3: np1(1)Batch4: FW_2(4)CBS(in-service upgrade)# install <XOS version>

The system displays informational messages regarding verifications and checks. Correct any issues before proceeding with the In-Service Upgrade.

5. Save the configuration to ensure that the upgraded half of the chassis is synchronized with the running configuration.

Any unsaved configuration will be lost.Do you want to save it to startup-config? <Y or N>[Y]:

Enter Y to save the configuration.

6. The current batch configuration is displayed as shown in the following example:

Do you want to save the current batch configuration? <Y or N>[Y]:Batch1: fw_1(3)Batch2: np2(2)Batch3: np1(1)Batch4: fw_2(4)

Done with batches? <Y or N> [Y]:

Enter Y to indicate that you are done with batches or enter N to reconfigure the batches.

NOTE: Please review the batch order, it cannot be changed during the In-Service Upgrade following completion of this step.

7. To continue the In-Service Upgrade, the secondary CPM will be reloaded into offline mode and upgraded, enter Y at the system prompt to continue.

Proceed with install? <Y or N>[Y]:

NOTE: The system automatically copies the shar file to the new CPM, the new CPM goes offline and performs the upgrade. This step takes about 30 minutes to complete. During this time, a series of informational messages are displayed to indicate progress.

8. You are prompted to bring the other CPM to Primary. Enter Y to continue.

Continue with in-service upgrade to bring other CP to Primary? <Y or N>[Y]:

NOTE: The new CPM will be rebooted into Primary CPM mode running the upgraded XOS software.

9. You are prompted to execute the next batch. This is the first batch to be moved that consists of the resources configured in Batch1.

Continue with in-service upgrade to execute next batch? <Y or N>[Y]:


The module in Batch1 is moved from the old CPM to the new CPM as shown in the following illustration.

Figure 20. Batch1 Operation

NOTE: When all resources in the batch have been successfully moved to the upgraded half of the chassis, a successful completion message is displayed. The resources are now running the upgraded software, but are not processing traffic.

10. You are prompted to continue with the next batch. This is the second batch consisting of the resources configured in Batch2. The following illustration shows Batch2 moved to the new CPM.


NOTE: During this batch, the first NPMs are moved to the upgraded half of the chassis. After this transition, the interfaces are active, however, traffic is not processed by the new half of the chassis. Restart applications that do not tolerate an interface state transition after completion of this batch.

11. You are prompted to continue with the next batch. This is the third batch consisting of the resources configured in Batch3. The following illustration shows Batch3 moved to the new CPM.



NOTE: During this step, the second half of the NPMs are transitioned to the upgraded half of the chassis. This transition triggers the traffic flows to move from the old half to the new half of the chassis. All traffic is then processed by the upgraded half of the chassis and full interface redundancy is restored.

12. You are prompted to continue with the next batch. This is the fourth batch consisting of the resources configured in Batch4. The following diagram shows Batch4 moved to the new CPM.


NOTE: During this step, the remaining APMs transition to the new half of the chassis. This restores full APM capacity to the chassis.

13. Choose an option to complete or abort the In-Service Upgrade:

Option 1 aborts the In-Service Upgrade transitioning all NPMs and APMs to the old CPM and transitioning the new CPM offline. An interruption in service occurs during this transition.

Option 2 transitions the old CPM into offline state, however the In-Service Upgrade remains in progress.

Option 3 results in the old CPM upgrading to the new software and reloading into CPM secondary mode. This restores full redundancy to the chassis running the new software.


Executing a Non-Interactive In-Service UpgradeA Non-Interactive In-Service Upgrade performs the same steps as an Interactive In-Service Upgrade. After you have confirmed the batch configuration and confirmed that you want to continue with the upgrade, there is no further interaction required. The Non-Interactive In-Service Upgrade continues automatically from that point.

To execute a Non-Interactive In-Service Upgrade:

1. The shar file contains the files needed to execute the upgrade. These files must be copied to the correct location (/usr/os/rpm), unpacked, and have the correct permissions applied (chmod 777). For more information, see Backups, Upgrades, and Reinstallation on page 225.

2. At the admin prompt on one of the CPMs, execute the following commands:

[root@xxx admin]# echo isu_child_tmo=10800 >> /usr/os/etc/cbscprmd.cf[root@xxx admin]# killall -HUP cbscprmd

3. Repeat the previous step for the other CPM.

4. Begin an Interactive In-Service Upgrade as shown below:

CBS # upgrade in-serviceCBS(in-service upgrade)# install <XOS version> non-interactive

The system displays informational messages regarding verifications and checks. Correct any issues before proceeding with the In-Service Upgrade.

5. When the dependency checks are complete, you are prompted to save the system configuration. Save the configuration to ensure that the upgraded half of the chassis is synchronized with the current configuration.

Do you want to save it to startup-config? <Y or N>[Y]:

Enter Y to save the configuration.

6. The current batch configuration is displayed as shown in the following example:

Batch1: FW_1(3)Batch2: np2(2)Batch3: np1(1)Batch4: FW_2(4)

Review the batch order. Once the batch order is agreed to, it cannot be modified during the rest of the In-Service Upgrade.

Enter Y to indicate that you are done with batches or enter N to reconfigure the batches. For more information, see Custom Batch List on page 236.

7. You are then prompted to continue with the upgrade, CLI sessions will be terminated, and there will be a loss of CPM redundancy if you continue at this point. Enter Y to continue.

The Non-Interactive In-Service Upgrade continues automatically.

Aborting a Non-Interactive In-Service UpgradeTo abort a Non-Interactive In-Service Upgrade, close all open CLI sessions. Then perform the following:

1. Start a new CLI session.

2. Execute the following command:

CBS # upgrade in-service install <XOS version>

3. Press CTL-Y, once in the interactive session to execute the In-Service Upgrade abort.


BRAID-Related Hard Drive Configuration and

Repair

With the dual drive and RAID capabilities in the APM-8600, APM-8650, APM-9600, CPM-8600, and CPM-9600, there exists a number of possible scenarios for upgrading, repairing, replacing, swapping, and otherwise changing drive configurations in these modules. This appendix provides information and procedures that deal with the following scenarios:

Going from a non-RAID configuration to a RAID configuration

Going from one RAID configuration to a different RAID configuration

Going from a RAID configuration to a non-RAID configuration

Correcting RAID drive failures

NOTE: Configuring RAID 1 with only a single drive installed is not supported.


RAID Feature Definitions on page 242

Additional Documentation on page 242

Prerequisites for RAID on page 242

Important RAID Information on page 243

Obtain Hard Drive Information on page 244

Error Situations and Error Messages on page 247

Configure RAID on an Existing Non-RAID System on page 248

Change the Existing RAID Setting on a CPM-8600 on page 250

Change the Existing RAID Setting on the APM-8600s, APM-8650s, or APM-9600s in a VAP Group on page 251

Configuring Non-RAID on a CPM-8600 with Existing RAID 1 on page 251

Detect and Replace Failed Drives on an Existing RAID System on page 253

Identifying Drives in an APM-8600, APM-8650, or APM-9600 with Non-RAID Configured on page 255


RAID Feature DefinitionsThere are three possible RAID settings:

Non-RAID — The system writes only to the primary disk (on CPM-8600s), or to the disks independently (on APM-8600s, APM-8650s, and APM-9600s).

RAID 0 — The APMs write to both drives in a process called striping. This arrangement produces maximum speed/efficiency of operation, but the data is not duplicated and both disks are required to access the full content of the data.

RAID 1 — The CPMs or APMs write the same data to each disk at the same time, providing a complete, ongoing backup in case of one drive’s failure.

You can use the RAID 1 option in combination with the CPM SYNC feature.

Additional DocumentationThis appendix only covers RAID configuration. For related information, refer to the following information:

For XOS and module upgrades, refer to the upgrade sections of this configuration guide.

For information about removing and reinserting CPM and APM modules and physically inserting hard drives, refer to the APM-9600, APM-8650, APM-8600, CPM-9600, and CPM-8600 Hard Disk Replacement and Upgrade Guide.

For Install Server information, refer to the Install Server User Guide.

For information on using the USB Installer, refer to the USB Installer User Guide.

Prerequisites for RAIDThis section provides hardware, software and other requirements to use RAID on an XOS system.

CPM-8600s, CPM-9600s, APM-8600s, APM-8650s, and/or APM-9600s (with various RAID and non-RAID standard configurations).

RAID 0 or RAID 1 requires two hard drives of the same capacity.

Console connection to issue CLI and UNIX commands to modules.

One of these methods of installing XOS software

Install Server Software with diskless partition that allows the CPM-8600 to boot from the server

The latest XOS USB Installer, plugged into the CPM USB port.

RAID-Related Hard Drive Configuration 242

Important RAID InformationThe following sections contain important general and specific RAID information and general information about RAID issues and errors. For more information about error situations and error messages, refer to Error Situations and Error Messages on page 247.

Disk Drive and RAID Information for CPM-8600s, APM-8600s, and APM-8650sThe following information will help you understand the various RAID scenarios explained in this document.

On CPM-8600s, APM-8600s, and APM-8650s, the two drive slots are labeled SATA1 (the inside slot) and SATA2 (the outside slot). Refer to the hard drive diagram in Identifying Drives in an APM-8600, APM-8650, or APM-9600 with Non-RAID Configured on page 255.

If you are installing only a single drive into a CPM-8600 or APM-8600, install the drive into the SATA1 position (inside slot).

Disk Drive and RAID Information for CPM-9600s and APM-9600sThe CPM-9600 comes from the factory with two disk drives configured for RAID 1. Only RAID 1 is supported on the CPM-9600.

The APM-9600 comes from the factory with no disks installed and can operate without a disk, with a single disk, or with two disks configured for either RAID 0 or RAID 1. Choose the configuration based on the requirements of your application.

NOTE: The APM-9600 uses serial SCSI (SAS) disks.

RAID Information for CPM-8600s OnlyThe following information applies to RAID configurations on a CPM-8600:

To install RAID, change the RAID setting to a different RAID type, or uninstall RAID, you must do an XOS reinstallation. The RAID/non-RAID setup is part of the installation process and part of the installation command.

If you are converting to a RAID configuration from non-RAID, you must add a blank drive to the SATA2 drive slot.

If you are converting back to non-RAID from a RAID configuration, Crossbeam recommends that you remove the SATA2 hard drive.

If you do not remove the second drive, and later the non-RAID SATA1 drive fails, you must move the SATA2 bootable drive to the SATA1 position. If you replace the failed SATA1 drive with a blank drive, the CPM-8600 will mistakenly boot from the original SATA2 RAID 1 drive, or display an error message.

Always use a blank drive to repair a failed RAID 1 drive situation. This ensures that the system will boot from the existing, working RAID 1 drive as intended.

RAID Information for CPM-9600s OnlyThe CPM-9600 comes from the factory with two disks installed in a RAID 1 configuration. Only the RAID 1 configuration is supported.


Important RAID Information for APM-8600s, APM-8650s, and APM-9600sThe following information applies to RAID configurations on an APM-8600:

On an APM-8600, APM-8650, or APM-9600, RAID (or non-RAID) is configured for an entire VAP group, not for individual APMs, and is set up using the configure vap-group command, not by XOS reinstallation.

When an installed APM-8600, APM-8650, or APM-9600 becomes an active part of the VAP group, its drives, if present, are automatically set up for the RAID operation established for that VAP group.

NOTE: APM-8600s, APM-8650s, and APM-9600s cannot be configured in the same VAP group. Crossbeam recommends that all APMs in a VAP group have the same configuration.

Obtain Hard Drive InformationUse the following commands and resources to help with RAID configuration and operation.

Table 20. RAID Configuration and Operation Commands

Using mdstat to Determine RAID-Related Drive InformationThe file /proc/mdstat provides status on the drives in a RAID pair, and whether or not they are in sync. The following file was generated by an APM-8600 with two hard drives set up for RAID 1 operation:

Some of the file entries (shown in bold below) indicate:

Type of RAID set up (raid1)

Names, numbers, and positions of the drives as follows:

sda which is SATA1, the inside drive slot sdb which is SATA2, the outside drive slot

Refer to the diagram in Identifying Drives in an APM-8600, APM-8650, or APM-9600 with Non-RAID Configured on page 255.

Total number of drives in the RAID group ( [2/2] )

v3_2 (cpm): root$ cat /proc/mdstatPersonalities : [raid0] [raid1] read_ahead 1024 sectors Event: 3md1 : active raid1 sdb1[1] sda1[0] 93433920 blocks [2/2] [UU]

Command or Tool Used For

show module status Shows whether there is a “Hard disk” and a “Second hard disk” and if there have been errors on either disk.

show module status disk Shows number of drives in each module.

cat /proc/mdstat The /proc/mdstat file provides information on the status of the disk drives and RAID.

Refer to Using mdstat to Determine RAID-Related Drive Information on page 244.


md5 : active raid1 sdb5[1] sda5[0] 2104448 blocks [2/2] [UU]md6 : active raid1 sdb6[1] sda6[0] 2104448 blocks [2/2] [UU]...... unused devices: <none>

NOTE: MD1 is always configured as raid1, regardless of the RAID0 or RAID1 configuration.

An indication that both drives are in sync: [UU] or, as in the following partial example, one drive is degraded [_U] (not in sync or failed):

md1 : active raid1 sdb1[1] sda1[0] 93433920 blocks [2/1] [_U] [===>.................] recovery = 17.8% (16638196/93433920) finish=127.9

NOTE: If a RAID drive has failed, the output of the cat /proc/mdstat command includes (F) next to the failed drive information. For example: Personalities : [raid1]md0 : active raid1 sda1[0] sdb1[1](F) 71681920 blocks [2/1] [U_] In this example, the [U_] indicates that sdb1 is either degraded or has failed. The (F) indicates that it is failed, not degraded.

The following example shows a CPM-8600, previously set up for RAID 1. One drive failed and was replaced (Part 1) but the drives are not synchronized. The xos-raid-add command is issued (Part 2) and the system begins the synchronization process for the two drives. In the example, md1 is 47% synchronized (Part 3).

Part 1: New drive inserted but drives are not synchronized

[root@mysysCPM2 admin]# cat /proc/mdstat Personalities : [raid0] [raid1] read_ahead 1024 sectorsEvent: 5 md1 : active raid1 sda1[0] 104320 blocks [2/1] [U_]md5 : active raid1 sda5[0] 505920 blocks [2/1] [U_]md6 : active raid1 sda6[0] 1003904 blocks [2/1] [U_]md7 : active raid1 sda7[0] 8008256 blocks [2/1] [U_] md8 : active raid1 sda8[0]28001152 blocks [2/1] [U_] unused devices: <none>

Part 2: Second drive is added to the RAID1 pair

[root@mysysCPM2 admin]# /usr/os/bin/xos-raid-add Duplicating firsSCSI device sdb: 195371568 512-byte hdwr sectors (100030 MB) t drive partition Table... SCSI device sdb: 195371568 512-byte hdwr sectors (100030 MB) Hot adding secondary drive... /dev/md1 /dev/md5 /dev/md6 /dev/md7 /dev/md8


Part 3: Check synchronization progress, md1 shows 47% complete

[root@mysysCPM2 admin]# more /proc/mdstat Personalities : [raid0] [raid1] read_ahead 1024 sectors Event: 11 md1 : active raid1 sdb1[2] sda1[0] 104320 blocks [2/1] [U_] [=========>...........] recovery = 47.0% (50096/104320)


Error Situations and Error MessagesIf, when configuring or repairing RAID and non-RAID instances, you do not use blank disk drives as recommended, or fail to follow other recommended procedures, there are a number of error situations that can occur. Refer to the table below for explanations and remedies.

Table 21. RAID Error Messages

WARNING: This is a non-RAID CPM install and a second drive is present. To make the first drive bootable, we must rewrite the second drive's MBR to make it non-bootable. Respond 'y' to continue with the rewrite, respond 'n' to leave the MBR alone (you know it is non-bootable), or 'q' to quit the install and remove the drive. (y|n|q)? [y]

SATA1 SATA2 Error Message Recovery Information

CPM-8600 Drive Mismatch Situations: “Disk config mismatch”

RAID 1 non-RAID SATA1:RAID1 and SATA2:non-RAID If you are configuring RAID, reverting to non-RAID, or changing to a different RAID setting, unless there is a drive you need to remove to preserve, just continue with the configuration process and the system will reconfigure the mismatched drive as needed.

non-RAID RAID 1 SATA1:non-RAID and SATA2:RAID1

CPM-8600 Conversion from RAID to non-RAID without Removing the SATA2 Drive

RAID 1 RAID 1 Warning Message (see text below)a

a. Warning message received when converting from RAID to non-RAID on a CPM-8600 and user fails to remove the SATA2 RAID-configured drive.

If the system proceeds with a normal XOS installation, the first drive will become non-RAID and you will end up with a non-RAID/RAID 1 drive mismatch. So the system stops the reinstallation process with the warning message (detailed below the table).

At this point, if you want to preserve SATA2, respond q to quit the install process, then remove the drive. Otherwise, respond y to continue the install and allow the rewrite. If the second drive was bootable, it will be made non-bootable.

CPM-8600 RAID 1 Failed Drive and Missing Drive Situations

Failed RAID 1 Replacing the drive with a non-blank drive will cause a mismatch error, or the CPM will boot from the wrong drive.

You must replace the failed drive with a blank drive. If you place another bootable drive in the SATA1 position, the system will not boot off your good, SATA2 RAID 1 drive.


Configure RAID on an Existing Non-RAID SystemThis section explains how to change an existing, non-RAID, XOS V8.0 or later system, to a RAID configuration. This section assumes that you have CPM-8600s, CPM-9600s, APM-8600s, APM-9650s, and/or APM-9600s installed.

NOTE: This section does not provide the instructions to physically remove the board, insert the hard drives, and reinsert the board. It also assumes that you understand how to take antistatic precautions. For these instructions, refer to the document that is included with the disk drive(s).

Adding a Second Drive to a CPM-8600 and Enabling RAIDTo install RAID on an existing CPM-8600 you must perform an XOS reinstallation which requires shutting down the CPM-8600. Complete the following steps to remove a CPM-8600, insert its second drive, and perform the reinstall/RAID configuration. (The following procedure assumes you are using the Install Server.)

1. Log on to the console port as admin.

2. If desired, back up the System’s Running Configuration using the CLI:

CBS# copy running-config <file-name>

3. FTP the configuration file to a workstation or server. Transfer the configuration file (stored by default at the following location) to an external storage device.

/tftpboot/.private/home/admin/<file-name>

4. Shut down the system from the console of the CPM-8600 using the CLI command:

CBS# shutdown

5. Remove the module and insert a blank hard drive into the SATA2 position (outside slot).

If you need physical installation instructions, refer to the more complete instructions in the APM-9600, APM-8650, APM-8600, CPM-9600, and CPM-8600 Hard Disk Replacement and Upgrade Guide.

6. When you reinsert the CPM-8600, the CPM-8600 boots as usual and you must stop the boot process so you can issue the install command. In this case, from the console, as soon as the Spinning Pipe prompt displays, type:

<ESC><Ctrl-r>

You will see the ROM> prompt.

NOTE: If the CPM-8600 does not boot as usual, contact Crossbeam Customer Support.

7. At the ROM> prompt, you have two ways to install the XOS software:

Use an Install Server. See Installing XOS Using an Install Server, next.

Use the Crossbeam USB Installer (USBI). See Installing the XOS Software Using the Crossbeam USB Installer (USBI) on page 249.

Installing XOS Using an Install Server1. At the ROM> prompt, type boot net to start the XOS install process.

2. Log in as root (there is no password).

3. Install the XOS system software, as follows, where <system dirname> is the directory name containing the system software release to be installed. Enter the install command with the RAID 1 switch (-r1):

# /usr/os/bin/install-xos /common/<system dirname> -r1

4. When the install-xos script completes, enter reboot at the prompt. The CPM-8600 then reboots from its own disk and prompts you for a login name and password.


Repeat these steps if you are installing RAID on a second CPM-8600.

NOTE: If you have two CPM-8600s, follow the CPM redundancy process as detailed in Configuring CPM Redundancy on page 121 to set up two CPM-8600s in sync with each other or to resynchronize the two CPM-8600s.

Installing the XOS Software Using the Crossbeam USB Installer (USBI)1. At the ROM> prompt, type boot usb.

2. When the menu appears, select either “Install XOS Release 9.x or Later” or “Install XOS Release 8.x or 7.x”, depending on the XOS version you want to install. The USBI boots the appropriate kernel version.

3. Enter the login and password provided, accept the license if required, and follow the on-screen instructions:

Select the XOS version to install.

Answer the hard disk configuration questions. (The USBI presents only valid hard disk options.)

Install XOS. Installing from the USBI takes 10 to 20 minutes, on average.

4. When the installation completes, select an option from the menu. Crossbeam recommends that you halt the CPM. When the system has halted, unseat the CPM, remove the USBI device, and then reseat the CPM to reboot it.

NOTE: After the system halts, you may see one or more error messages, and the Active LED on the CPM may blink. If you received the “System halted” message, you can ignore the errors and the blinking LED and safely reboot the CPM.

Adding Hard Drives to an APM-8600, APM-8650, or APM-9600 and Configuring RAID

To add drives and configure the APM-8600, APM-8650, or APM-9600 for RAID, complete the following steps:

1. RAID is enabled on a VAP group-basis, not on individual APMs. Ensure that the APMs that you are going to configure for RAID are all part of a VAP group that will use the RAID feature.

2. From the CLI, configure the desired RAID feature using one of the following settings:

non-RAID — Drives work independently, or device can work with only one drive.

CBS# configure vap-group vapgroupname no raid

RAID 0 — Drives work at maximum speed and efficiency with different data written to each drive. Both working drives are required to access the data.

CBS# configure vap-group vapgroupname raid 0

RAID 1 — Drives work as a single drive with the same data written to both drives (data is duplicated).

CBS# configure vap-group vapgroupname raid 1

3. Remove the APM and insert one or two disk drives, as required. If you need instructions for physically removing the modules and inserting the hard drives, refer to the document that is included with the disk drive(s).

Warning: Do not remove the USBI while the CPM is booted from the device. Do not leave the USBI in the CPM during normal operation. Either action can compromise the integrity of the USBI device.

Caution: Setting or changing the RAID settings on the VAP group deletes everything on the hard drives in that group.


4. After reinserting the APM, the next time that APM is used in the VAP group, the system detects the changes and reconfigures the drive(s) as needed for the new RAID setting.

5. Continue this process until you have added the needed hard drives to each of the APMs.

Change the Existing RAID Setting on a CPM-8600To change to a different RAID setting on a CPM-8600, follow the XOS reinstall process below.


2. If desired, back up the System’s Running Configuration using the CLI:



/tftpboot/.private/home/admin/file-name

4. Reboot the system from the CPM-8600 using the CLI command:

CBS# reload all

5. You must be ready to stop the boot process. From the console, as soon as the Spinning Pipe prompt displays, type:

<Esc><Ctrl-r>











NOTE: If you have two CPM-8600s, follow the CPM redundancy process as detailed in Configuring CPM Redundancy on page 121 to set up two CPM-8600s in synchronization with each other or to resynchronize the two CPM-8600s.










Change the Existing RAID Setting on the APM-8600s, APM-8650s, or APM-9600s in a VAP Group

To change the RAID setting on the APMs in a VAP group, complete the following steps:

1. From the CLI, configure the desired RAID feature setting to one of the following.

non-RAID — Drives work independently, or device can work with only one drive.

CBS# configure vap-group <VAP_group_name> no raid

RAID 0 — Drives work at maximum speed and efficiency with different data written to each drive. Both working drives are required to access the data.

CBS# configure vap-group <VAP_group_name> raid 0

RAID 1 — Drives work as a single drive with the same data written to both drives (data is duplicated).

CBS# configure vap-group <VAP_group_name> raid 1

2. (Change From one RAID Setting to Another) Reload the APMs so that the configuration takes effect. To minimize traffic interruption, reload slot-by-slot as follows:

CBS# reload module <slot#>

3. To make this configuration permanent, save the configuration.

NOTE: If you convert from RAID to non-RAID, and leave the second hard drive installed, be aware that the second drive is accessed as /mnt/aplocaldrive_2. Refer to Identifying Drives in an APM-8600, APM-8650, or APM-9600 with Non-RAID Configured on page 255.

Configuring Non-RAID on a CPM-8600 with Existing RAID 1

Use the following procedure to convert CPM-8600 modules back to non-RAID condition. You must perform these steps for each of the CPM-8600s. For APM-8600s, refer to Change the Existing RAID Setting on the APM-8600s, APM-8650s, or APM-9600s in a VAP Group on page 251.


Caution: Setting or changing the RAID settings on the VAP group deletes everything on the hard drives in that group.


To convert from RAID to non-RAID on a CPM-8600, follow these steps:


2. Back up the System’s Running Configuration (if desired) using the CLI:


3. FTP the configuration file to a workstation or server or transfer the configuration file to an external storage device. By default, the configuration file is stored in the following location:


4. Shut down or reboot the system from the CPM-8600 using one of the following CLI commands:

Use the shutdown command if you want to remove the second hard drive from the CPM-8600. This is recommended. However, if you do leave the second drive in, the install process will make the drive non-bootable (after warning you).

CBS# shutdown

Remove the module and remove the second hard drive. If you need physical installation instructions, refer to the more complete instructions in the APM-9600, APM-8650, APM-8600, CPM-9600, and CPM-8600 Hard Disk Replacement and Upgrade Guide. document.

Use the reload all command if you choose to leave the second drive in the CPM-8600.

CBS# reload all

5. When you reinsert the CPM-8600 (or issue the reload all command) the module reboots. You must stop the boot process so you can issue the install command. From the console, as soon as the Spinning Pipe prompt displays, type:

<Esc><Ctrl-r>











NOTE: If you have two CPM-8600s, follow the CPM redundancy process as detailed in Configuring CPM Redundancy on page 121 to set up two CPM-8600s in sync with each other or to resynchronize the two CPM-8600s.










5. If your system has a second CPM-8600, repeat the above steps to convert it to non-RAID.

Detect and Replace Failed Drives on an Existing RAID System

The following sections explain how to restore a working RAID pair when a hard disk drive fails.

IMPORTANT: To determine if an HDD is bad, and which drive is bad, refer to Using mdstat to Determine RAID-Related Drive Information on page 244.

In RAID 1 configurations, you can replace only the failed drive. In RAID 0 configurations, replace both drives.

Always replace a failed drive with a blank drive.

Replace Failed Drive on a CPM-8600 or CPM-9600 Configured for RAID 1IMPORTANT: You must insert a blank replacement drive. If you insert a drive pre-configured for RAID 1, the

system will not know which drive to sync to.

Complete the following steps to replace a failed RAID 1 drive on a CPM-8600 and reestablish a working RAID 1 pair:

1. Determine which drive on the CPM-8600 has failed by examining the file /proc/mdstat.

2. Log on to the console port of the CPM-8600 with a failed HDD as admin.

3. Back up the System’s Running Configuration (if desired) using the CLI:




5. Shut down the CPM-8600 with the failed drive using the following CLI command:



CBS# shutdown

6. Remove the CPM-8600 with the bad drive from the system.

7. Remove the failed drive from the CPM-8600 board by removing the four screws using a Phillips #1 screwdriver (or a small flat blade screwdriver if necessary).

8. Install a blank replacement drive.

9. Reinsert the CPM-8600 into the chassis.

10. To sync the drives on CPM-8600 with the replacement HDD, connect to the CPM-8600 using the console connection and log into XOS as root:



12. Issue the command:

/usr/os/bin/xos-raid-add

Replace Failed Drive on an APM-8600 with RAID 1NOTE: The good drive of the RAID 1 pair still contains your logging and other stored information. It is not

necessary to back up this drive unless you are unsure which drive is the bad one.

Complete the following steps to replace a failed RAID 1 drive on an APM-8600 and reestablish a working RAID 1 pair:

1. Pull out the module with the bad drive. Remove the failed drive (based on the information in mdstat), insert a blank, good replacement drive, and reinsert the APM-8600.

2. When the APM-8600 is next used in the VAP group, it will be configured according to the RAID setting for that VAP group and disk mirroring will occur.

Replace Failed Drive on an APM-9600 with RAIDThe APM-9600 supports 1 or 2 disk Serial Attached SCSI (SAS) drives. To remove a disk drive, remove the four mounting screws on the reverse side of the module, then slide the drive up and away from the handle side of the module.

Identifing a Failed Disk Drive on an APM-9600 Configured for RAID

Identify which APM-9600 disk drive has failed, by performing these steps.

1. Log in to the primary CPM on your chassis and execute these commands.

CBS# unix suPassword: <root_password>[root@<chassis_hostname> admin]# rsh <IP_address_of_APM-9600><VAP_group_member> (<chassis_hostname>): root# /crossbeam/bin/cbs_scsi_disk_info.pl

2. The output from the /crossbeam/bin/cbs_scsi_disk_info.pl script includes identifying information for the failed disk drive, including the serial number.

3. Record the serial number of the failed disk drive.


Replacing a Failed Disk Drive on an APM-9600 Configured for RAID

To replace the failed disk drive, perform these steps.

1. Remove the APM-9600 module from the chassis.

2. Use the serial number that you recorded to identify which disk drive to remove. The serial number of each disk drive is located on the label on the top of the disk drive.

3. Replace the disk drive with the matching serial number and reinsert the APM-9600 into the chassis.

Identifying Drives in an APM-8600, APM-8650, or APM-9600 with Non-RAID Configured

In a non-RAID configuration, if you have one or two hard drives in the APM-8600, depending on the slot they are inserted into, they are accessed according to the information in the following table.

Table 22. Drive Access Information

You can determine a drive’s slot position by examining the /proc/scsi/scsi file as follows:

1. Log on to the CLI and use the following command to see information about the APM-8600s.

CBS# show ap-vap-mappingModule Slot Status VAP IP Address VAP Group Index Master AP6 8 Up 1.1.2.101 v3 1 true

2. Use the VAP IP address information to log in to the VAP group from the Unix prompt and access the /proc/scsi/scsi file.

CBS# uni su [root@mycomputer admin]# rsh 1.1.2.101 Last login: Sat Apr 28 14:36:35 on ttyS0 v3_1 (mycomputer): root$ cat /proc/scsi/scsi Attached devices: Host: scsi0 Channel: 00 Id: 00 Lun: 00 Vendor: ATA Model: HTE721010G9SA00 Rev: MCZO Type: Direct-Access ANSI SCSI revision: 05 Host: scsi3 Channel: 00 Id: 00 Lun: 00 Vendor: ATA Model: HTE721010G9SA00 Rev: MCZO Type: Direct-Access ANSI SCSI revision: 05 fw_1 (10PercentCPM2): root$

The example above shows the /proc/scsi/scsi file with two disks installed. The scsi0 drive refers to the drive in the SATA1 slot; scsi3 refers to SATA2, the drive in the outside slot. Refer to the diagram below.

Position on the Board (see diagram below) Hard-coded Mounting Name Reference Name in /proc/scsi/scsi

SATA1 (inside slot) /mnt/aplocaldrive scsi0

SATA2 (outside slot) /mnt/aplocaldrive_2 scsi3


Figure 24. Disk Drive Locations on an APM-8600 or APM-8650

Figure 25. Disk Drive Locations on an APM-9600

��

��

Disk Drives


CSwatch Dynamic Display Tool

This chapter describes the Swatch tool, used to display dynamically-changing data related to the X-Series Platform operation. This chapter includes the following major topics:


Understanding the Swatch Tool on page 258

Starting and Stopping the Swatch Tool on page 259

Using Single Key Commands on page 259

Swatch Tool Configuration Guidelines on page 260

Performance Considerations on page 264

Using Existing Swatch Tool Configuration Files on page 264

Troubleshooting the Swatch Tool on page 265

OverviewThe XOS Open Security Appliance Swatch Dynamic Display Tool (Swatch) is a Linux tool designed to display dynamically-changing data on a terminal screen in a customized display format. The Swatch tool runs from the UNIX prompt and the XOS CLI.

Originally designed to format and display device packet statistics from Linux/proc files, the Swatch tool now allows you to easily spot changes in your system status and can display any type of data that can be obtained from an ASCII file or by executing any Linux command that writes to a standard output.

The Swatch tool supports the following data types:

String

32-bit integer

64-bit integer

64-bit floating point

The Swatch tool can also perform simple arithmetic operations on data, such as rate, min, max, sum, and diff. All data can be reset (resets min, max, rate) and counter data can be zeroed out.


Understanding the Swatch ToolThe Swatch tool is designed to read configuration files, determine the data (known as Swatch variables) to acquire, and then determine where to display them on your screen. Configuration files are simple text files and can be created and edited using any text editor. Refer to the TCL documentation for the appropriate TCL commands and Swatch Tool Configuration Guidelines on page 260 for Swatch-specific commands.

When reading the configuration file, the Swatch tool:

Parses the file using a TCL interpreter, allowing you to use any TCL commands. After parsing the configuration files, the Swatch tool displays it on the first screen.

Defines one or more acquisition files to obtain data from. The Swatch tool periodically reads all fields from acquisition files into Swatch variables in memory.

Defines one or more virtual screens that can be displayed. The Swatch tool periodically displays values of Swatch variables on a virtual screen.

Data from APMs can be displayed using the Swatch tool by using shell commands such as telnet or rsh. These commands allow you to log in to a remote slot and then execute a command on that slot.

Performance Monitoring with the Swatch ToolThe swatch scripts provided in XOS indicate problems at a high level. When swatch scripts indicate problems, further use of precise measurement tools such as network traffic analyzers is often necessary to determine the root cause problem.

As a best practice for validating configurations, baselining a performance test, or integrating an application into an X-Series Platform, the following subset of XOS CLI commands and swatch scripts should be run.

1. CBS# show flow distribution Shows the distribution of traffic from the NPM(s) to the APM(s).

2. CBS# show flow active Determines if traffic is passing through the X-Series Platform. Look for the word drop if a flow is dropped.

3. Swatch script 16. npmdevstats Shows the Rx and Tx traffic from the NPMs. The traffic should be roughly equivalent of total Rx and Tx traffic into the APMs.

4. Swatch script 1. apmdevstats Shows the Rx and Tx traffic for the APMs. The traffic should be roughly equivalent to the total Rx and Tx traffic into the NPMs.

5. CBS# show cpu Displays aggregate CPU usage.

6. CBS# show chassis Displays the status of installed modules.

Swatch Dynamic Display Tool 258

Starting and Stopping the Swatch ToolTo start the Swatch tool from the XOS CLI, enter swatch. When running Swatch from the CLI, you can only run existing configuration files, as described in Using Existing Swatch Tool Configuration Files on page 264.

To start the Swatch Tool from the Linux shell prompt on a CPM or APM, cd to /crossbeam/bin and enter the following command:

./swatch [<options>] [ <script> ... ]

Where the valid options are:

-D<Varname>=[<value>,...], assigns values to the TCL variable

--debug |-g <level>, set debugging level

-i, show script information

-g <n>, sets the debugging level to <n>. Where the value zero means display no debug information and the higher value levels display more information.

-x <n>, causes the Swatch tool to dump internal program objects up to level <n>, with a higher level value displaying more information.

-h | -- help, displays help for command usage and exits.

-v | --version, displays the Swatch tool program version and exits.

-p <interval>, log the screen data to files every <interval> seconds in background

-f <filename>, log the screen data to file in foreground

To stop the Swatch tool, press the q or Q key anytime while the Swatch tool is running.

Using Single Key CommandsThe following single-key commands can be used anytime while the Swatch tool is running:

<space-bar>, pauses/resumes screen updates.

s, causes a single screen update.

c (lowercase), clears all clear-able counters displayed on the current screen only.

C (uppercase), clears all clear-able counters on all screens, whether or not they display in current screen.

u (lowercase), restores the original values of the counters on the current screen.

U (uppercase), restores the original values of the counters on all screens.

r (lowercase), resets the data it on the current screen.

R (uppercase), resets the data it on all screens.

d, downloads the current screen to the /tmp directory, where the filename is the <screenname>.scr.

+/-, moves to the next or previous screen respectively.

00-99, moves to the designated screen number from 0 to 99.

h, displays a help page.

l, displays a screen list page.

q or Q, quits the Swatch tool program.


Swatch Tool Configuration GuidelinesThe following are guidelines to use when creating a Swatch tool configuration file.

Use the first few lines of the file to set default values for TCL variables, which can be overridden by using a -D option on the Swatch command line. You can use -Dline=x to limit the output display. For example, ./swatch -Dline=10 interrupts will display the interrupt stats from line 10 onwards.

Minimize the number of swacquire commands within a given configuration file.

Minimize the number of lines and columns displayed on a virtual screen.

Place a Title line at the top of each virtual screen.

Use unique virtual screen names to allow users to easily identify the screen.

Use the counter option on all swread commands to indicate if an item is a counter.

Use the once option on swread commands if the item value does not change.

Define only one or two different screen types per configuration file.

Creating a Swatch Tool Configuration FileTo create a Swatch tool configuration file, use the following commands:

swacquire

swread

swcalc

swaggr

swscreen

swdisplay

Using the swacquire CommandThe swacquire command specifies the file read periodically or the shell run to retrieve the command output.

swacquire Syntaxswacquire [file <filename>] [shell <shell>] [shellinit <prompt> <resp>] [shellcmd <prompt> <resp>] [period <poll-period>] [delim <delim-chars>]

Where:

file parameter is the existing files (including /proc files) or you can specify a file that saves the shell command output.

shell parameter designates the running commands that generate output.

shellinit parameter defines a one-time dialog. For example, use this parameter to specify a login-prompt/username or password-prompt/password.

shellcmd parameter defines a dialog at each poll period. For example, use this parameter to specify a shell prompt/command executed in a telnet session.

delim parameter specifies a set of characters used as a delimiter between fields.

The following example specifies that the file /proc/net/dev is read every 2 seconds:

swacquire file /proc/net/dev period 2


Display Position Syntax

The following information defines the position of a display or data item, when using this command.

line/field/column-spec [+|-]x[@y]

Where:

+|- parameter specifies the new position is relative to last position.

@y parameter specifies the new position is incremented by y for each variable instance.

In the following example, the starting line number is +4 from the current line number, and the line number is incremented by 2 for each variable instance referenced in the command.

line-spec is +4@2

Using the swread CommandThe swread command defines a data item at fixed positions within the swacquire file (or shell output).

swread Syntax:swread <varname> <line-spec> <field-spec> <fmt> [counter] [once]

Where:

<varname> is a Swatch tool variable name that holds values read from a file.

<line-spec> and <field-spec> define the line# and field# position within a file.

<fmt> defines how a data value is read. This is the same value as the scanf() function.

counter parameter indicates that the data item is a counter. Note, this value can be zeroed out.

once parameter indicates that a data item is read one-time only.

NOTE: All swread commands are relative to the last swacquire command

The following example defines an integer variable pktdrops at line 2; field 1 is read from the previous acquisition file with a format of %u. This variable is a counter.

swread pktdrops 2 1 %u counter

Using the swcalc CommandThe swcalc command defines the data item calculated from other data it.

swcalc Syntaxswcalc <varname> <operation> <operand> [<operand> ...]

Where:

<varname> is the result variable name. If this is an array, the number of array indices in the operands must match the number of array indices in the result.

<operation> can be rate, min, max, add, or sub.

<operand> is the name of Swatch tool variable used as a source for the operation. If this is an array, the number of array indices in the operands must match the number of array indices in the result.

The following example defines an array drop rate that is calculated by computing the rate of the values in the pktdrops array.

swcalc droprate(1:10) rate pktdrops(1:10)


Using the swaggr CommandThe swaggr command defines a data item aggregated from other data.

swaggr Syntaxswaggr <varname> <operation> <operand> [<operand> ...]

Where:

<varname> is the result variable name. This must be a scalar value.

<operation> is the sum or diff operation.

<operand> is the name of the Swatch tool variable used as the source for operation. This can also be an array.

The following example defines a scalar variable totdrops that is calculated by computing the sum of all values in the pktdrops array.

swaggr totdrops sum pktdrops(1:10)

Using the swscreen CommandThe swscreen command defines the virtual screen displayed.

swscreen Syntaxswscreen <screenname> [period <n>]

Where the period parameter defines the screen refresh rate in seconds. The default value is 1 second.

The following example defines a virtual screen named devpktcnt that when displayed, refreshes every 2 seconds.

swscreen devpktcnt 2

Using the swdisplay CommandThe swdisplay command defines the display item created at a fixed position on a virtual screen.

swdisplay Syntaxswdisplay <line-spec> <col-spec> <size> <fmt> [<varname>] [scale <scale-factor>]

Where:

<line-spec>, <col-spec> parameter defines the position of the display item on the virtual screen.

<size> is the total width of the display item.

<fmt> is the format used for writing a display item. This value is the same as the printf function.

<varname> is the name of the previously defined Swatch tool data variable.

scale parameter defines the scale factor applied to the data variable prior to output.

NOTE: All swdisplay commands are relative to last swscreen command.

The following example displays the values of the array pktdrops at column 15 on lines 4-13, with a width of 10 characters.

swdisplay 4@1 15 10 %u pktdrops(1:10)


swdisplay Position Syntax

The following information defines the position of a display or data item, when using this command.

line/field/column-spec [+|-]x[@y]

Where:

+|- parameter specifies the new position is relative to last position.

@y parameter specifies the new position is incremented by y for each variable instance.

In the following example, the starting line number is +4 from the current line number, and the line number is incremented by 2 for each variable instance referenced in the command.

line-spec is +4@2

Logging Screen Data to a FileYou can log screen data to files in the background and foreground for a swatch function.

Collect Screen Data in the BackgroundUse the flag -p with a swatch command to collect all screens data in the background. For example:

swatch -p interval <script_name>

The output files are written to /crossbeam/logs/ using the following filenaming scheme:

scriptname_scrID-MM_DD_YY:HH:MM:SS-pid.log

Use the command kill <pid> to terminate the background instance.

Collect Screen Data in the BackgroundUse the flag -f with a swatch command to display the logged screen in the foreground. For example:

swatch script_name -f filename

Using Predefined TCL Variables in a Swatch Tool Configuration FileThe following TCL variables can be referenced in a Swatch tool configuration file:

SLOT_IP(1)..SLOT_IP(14), contains the IP address of slot or the value 0.0.0.0 if no slot is present.

SLOT_NAME(1)…SLOT_NAME(14), contains the host name of the slot or unknown if the slot is not present.

Using Swatch Tool Variables in All CommandsThe following are Swatch tool variable guidelines and restrictions:

Variable names can be any alphanumeric string. For example, pktdrops.

Variables can be either scalar or arrays.

Variables are only visible in current configuration file, unless they are prefixed with global tag.

Array elements are specified as <varname>(<index>) where the index is a non-negative integer. The notation varname(<index1>:<index2>) is shorthand, which expands the command for each variable instance between varname(index1) and varname(index2) inclusive.


Using Existing Swatch Tool Configuration FilesThis section describes the existing Swatch tool configuration files that you can use:

apmdevstats.swc - displays packet statistics per APM slot (one screen for each selected network device). Use a Ddevs value of <dev1>, <dev2>, and so on to specify the desired devices. The default value is devs=sdp0.

apmdevstats_slot.swc - displays packet statistics per device slot (one screen for each selected APM slot). Use a Dslots value of <slot1>, <slot2>, and so on to specify a list of slots. Valid slot values are from 3 to 12.

apmfwstats.swc - displays firewall statistics per APM slot in a single screen display. Use a Dslots value of <slot1>, <slot2>, and so on to specify a list of slots. Valid slot values are from 3 to 12.

apmfwstats_slot.swc - displays firewall stats per device (one screen per selected APM slot). Use a Dslots value of <slot1>, <slot2>, and so on to specify a list of slots. Valid slot values are from 3 to 12.

apmsnmpstats.swc - displays IP, ICMP, TCP, and UDP statistics and rates for the APMs.

cbsinitdstats.swc - displays a list of Crossbeam daemons and their status.

fabricstats.swc - displays NPM statistics and rates for the switched data path fabric on the dataplane.

flowcalcstats.swc - displays information on flow calculation statistics.

flowsched.swc - displays which member of a VAP Group is eligible for the next flow assignment.

groupintstats.swc - displays statistics for group interface members.

health_cpubsy.swc - displays CPU activity for the APMs and CPMs that are installed.

health_cpumem.swc - displays CPU load, utilization, and memory information for each physical slot.

health_temp.swc - displays CPU and Board minimum, maximum, and chassis current temperature information.

moduleuptime.swc - displays the Linux up-time for each NPM and APM slot.

netifstats.swc - displays packet statistics for all Linux devices on a current slot. This file can be run on a CPM or VAP.

npmdevstats.swc - displays packet statistics for NPM physical interfaces. Use a Dslots value of <slot1>, <slot2>, and so on to specify a list of slots.

npmfragstats.swc - displays virtual defragmentation (vdf) statistics per NPM.

Performance ConsiderationsThe following are performance consideration to heed when using the Swatch tool.

Swatch Tool Child ProcessesThe Swatch tool creates many child processes on a local host. This can slow system performance. The following are examples of this:

There is one child process for computing all derived or calculated data it. For example, data it defined in swcalc or swaggr commands.

There is one child process for each swacquire command in the configuration files listed on startup. Note, if a swacquire command has a shell option, an additional child process is created to execute the shell command in.

NOTE: Some shell commands may cause TCP/UDP connections to remote hosts or cause new processes to be created on a remote host.


Single Shared-Memory Region and SemaphoreThe Swatch tool uses a single shared-memory region to communicate data between processes. Currently, the region is set to 400,000 bytes.

The Swatch tool also uses a single semaphore to provide atomic access to the shared memory region.

Troubleshooting the Swatch ToolThe following are some known problems that may result when using the Swatch Tool.

Data Not Being Displayed or UpdatedThe following bullets list some solutions when data is not being displayed or updated:

For swacquire file commands, make sure the file exists.

For swacquire shell commands, make sure that the shell command can be invoked manually.

Run the Swatch tool with a -g <n> option to create debug files. Note, higher debug levels give more debug information. Debug files are located in the swatch.xxxxx.log, where the xxxxx is the child process ID.

Data is Not Displayed in the Correct LocationThe following bullets list some solutions when data is not displayed in correct screen location:

Check the swdisplay line, column, and format options for the data item.

Run the Swatch tool with a -x <n> option to download program objects. Note, higher dump levels give more dump information.

Swatch Tool is Affecting CPM PerformanceThe following bullets provide some solutions when the Swatch tool is affecting performance of the CPM:

Reduce the number of Swatch tool programs being run simultaneously.

Reduce the number of startup configuration files listed on Swatch tool command line.

Reduce the polling rate for swacquire commands (period option).

Reduce the display refresh rate for swscreen commands (period option).

Swatch Tool is Affecting Performance of Remote APMs or NPMsThe following bullets provide possible solutions when the Swatch tool is affecting performance of remote APMs or NPMs:

Reduce the number of Swatch tool configurations that remotely access APMs or NPMs.

Reduce the polling rate for swacquire commands that remotely access APMs or NPMs.

NOTE: Some CPM programs, such as apmdevstats and npmdevstats, query data on remote slots through the control bus.



DCrossbeam SNMP MIB Reference

This chapter contains a reference to Crossbeam simple network management protocol (SNMP) management information base (MIB) files.

The main topics in this appendix are:


New Crossbeam MIB Entries in XOS V9.5. on page 269

Greenlight Element Manager (GEM) MIB Reference on page 269

XOS Alarms and SNMP Traps on page 275

Using a MIB browser on page 268

CROSSBEAM-SYSTEMS-MIB Reference on page 280

CBS-HARDWARE-MIB Reference on page 281

CBS-MODULE-RESOURCES-MIB Reference on page 288

CBS-VAPGROUP-MIB Reference on page 293

CBS-VRRP-MIB Reference on page 294

Standard MIBs on the Crossbeam X-Series Platform on page 296

OverviewThe following five Crossbeam SNMP MIB files are located in the /usr/share/snmp/mibs directory:

CROSSBEAM-SYSTEMS-MIB.txt

CBS-HARDWARE-MIB.txt

CBS-MODULE-RESOURCES-MIB.txt

CBS-VAPGROUP-MIB.txt

CBS-VRRP-MIB.txt

Each entry in the Crossbeam MIBs contain an object identifier (OID). The Crossbeam specific reference is 6848 as seen here:

.1.3.6.1.4.1.6848 iso(1) org(3) dod(6) internet(1) private(4) enterprises(1) crossbeamSystems(6848)

Entries in the Crossbeam MIBs are organized into branches with unique OIDs.

For example:

CBS-HARDWARE-MIB::cbsHardware .1.3.6.1.4.1.6848.2.1 iso(1) org(3) dod(6) internet(1) private(4) enterprises(1) crossbeamSystems(6848) cbsMgmt(2) cbsHardware(1)

The information in this appendix describes a tree view into Crossbeam MIBs with a reference to the complete OID for each MIB entry. For comprehensive details, view each MIB entry in a MIB browser.


Using a MIB browserThere are several free MIB browsers, such as snmptranslate, that display details for the Crossbeam MIBs. The MIB browser displays the following:

1. A tree-view of the MIBs and MIB entries

2. The name and OID for each entry

3. Syntax rules that include options. For example: up, down

4. Access, status, and a text description for each entry

If you do not have access to a MIB browser, access the control processing module (CPM) from a command prompt and display the MIB files in /usr/share/snmp/mibs for details related to each MIB entry.

Using HP OpenView Network Node ManagerCrossbeam provides a Crossbeam Systems Integration Package V1.0 for HP OpenView Network Node Manager 9.00i. The official Crossbeam integration package for this release is cbs_nnmi_v900i_integration.tar.gz. The integration package is available for download from the Crossbeam Support Portal at http://www.crossbeam.com/support/online-support/. A valid support account with username and password is required.

Crossbeam Systems Integration Package V1.0 for HP OpenView Network Node Manager 9.00i enables custom integration of Crossbeam X-Series Platforms into HPOV/NNMi v9.00. Once installed, it provides the following features:

Ability to launch the Crossbeam Greenlight Element Manager (GEM) in a new browser window from HP OpenView.

A node group representing all Crossbeam hardware making it easier to apply changes to only Crossbeam equipment. Access this node group from Inventory > Node Groups > Crossbeam Devices.

Integration of proprietary Crossbeam MIBs into HP OpenView / NNMi, enabling the custom polling of Crossbeam MIBs, and provides information in addition to standard MIBs.

Overall device status is indicated by icon color.

For information, see the Crossbeam Systems Integration Package V1.0 for HP OpenView Network Node Manager 9.00i Release Notes.

New Crossbeam MIB Entries in XOS V9.5Crossbeam added new MIB entries in the XOS V9.5 release. A summary of the new MIBs is available in Table 23 on page 269. Refer to the respective MIB reference section for specific entries, descriptions, and OID.

In addition to new MIBs to expand the data available about the Crossbeam chassis, Crossbeam added MIBs for all data available in the browser based views of the Greenlight Element Manager. A detailed listing of the new MIBs for each view of the Greenlight Element Manager is available in the Greenlight Element Manager (GEM) MIB Reference on page 269.

Most alarms raised on an the Crossbeam chassis also generate one or more SNMP traps. Refer to XOS Alarms and SNMP Traps on page 275 for a list of alarms and their associated SNMP traps.

Crossbeam SNMP MIB Reference 268



Table 23. New Crossbeam MIB Entries in XOS V9.5.

Greenlight Element Manager (GEM) MIB ReferenceThe table below contains a list of the MIBs for all the data presented in the GEM User Interface.

Table 24. GEM MIB Reference

Crossbeam MIB Description

CBS-HARDWARE-MIB New information includes:

Fan Tray compatibility and status

Firmware revision status

CP Redundancy state

Minor, Major, and Critical Alarms

CBS-MODULE-RESOURCES-MIB New information includes:

Flow Distribution Table

CBS-VRRP-MIB New information includes:

VRRP Failover Group Status (DBHA)

View / Component MIB MIB Object

System View

Hostname MIB-II sysName

Chassis Type CBS-HARDWARE-MIB cbsHwSystemDescription

Alarms See Alarms View

Hardware Properties

Revision CBS-HARDWARE-MIB cbsHwSystemBackplaneRevision

Serial Number CBS-HARDWARE-MIB cbsHwSystemSerialNumber

Part Number CBS-HARDWARE-MIB cbsHwSystemPartNumber

Chassis Temp CBS-HARDWARE-MIB cbsHwSystemChassisTemp

DBHA See DBHA View

XOS Version MIB-II sysDescr

Firmware Version See Firmware View

Modules Module Name CBS-HARDWARE-MIB cbsHwModuleDescription

Module Status CBS-HARDWARE-MIB cbsHwModuleStatus

VAP Groups Name CBS-VAPGROUP-MIB cbsVgVapGroupName

Application CBS-MODULE-RESOURCES-MIB

cbsMgmt.cbsModuleResources.cbsModuleApplicationTable.cbsModuleApplicationEntry.cbsModuleApplicationName

cbsMgmt.cbsModuleResources.cbsModuleApplicationTable.cbsModuleApplicationEntry.cbsModuleApplicationVersion

VAP count CBS-VAPGROUP-MIB cbsVgVapCount

Max load count CBS-VAPGROUP-MIB cbsVgMaxLoadCount

Fan Trays Status CBS-HARDWARE-MIB cbsHwSystemUpperFanTray

cbsHwSystemLowerFanTray

cbsFanStatus

Compatible CBS-HARDWARE-MIB cbsHwSystemStatus.cbsHwSystemUpperFanTrayCompatible

cbsHwSystemStatus.cbsHwSystemLowerFanTrayCompatible

Revision CBS-HARDWARE-MIB cbsHardware.cbsHwSystemStatus.cbsHwSystemUpperFanTrayRevision

cbsHardware.cbsHwSystemStatus.cbsHwSystemLowerFanTrayRevision

Serial Number CBS-HARDWARE-MIB cbsHwSystemStatus.cbsHwSystemUpperFanTraySerialNumber

cbsHwSystemStatus.cbsHwSystemLowerFanTraySerialNumber

Part Number CBS-HARDWARE-MIB cbsHwSystemStatus.cbsHwSystemUpperFanTrayPartNumber

cbsHwSystemStatus.cbsHwSystemLowerFanTrayPartNumber

Power Type CBS-HARDWARE-MIB cbsHwSystemPowerType

Power Supplies Power Supply CBS-HARDWARE-MIB cbsHwSystemStatus

Power Feed CBS-HARDWARE-MIB cbsHwSystemStatus

Firmware View

Slot CBS-HARDWARE-MIB cbsHwModuleID

Module Type CBS-HARDWARE-MIB cbsHwModuleDescription

Board Revision CBS-HARDWARE-MIB cbsHwModuleBoardRevision

Bootstrap Rev Current CBS-HARDWARE-MIB cbsHwModuleBootStrapRev

Required CBS-HARDWARE-MIB cbsHwModuleReqBootStrapRev

Bootloader Rev Current CBS-HARDWARE-MIB cbsHwModuleBootloaderRev

Required CBS-HARDWARE-MIB cbsHwModuleReqBootloaderRev

Ctrl FPGA revision

Current CBS-HARDWARE-MIB cbsHwModuleCtrlFpgaRev

Required CBS-HARDWARE-MIB cbsHwModuleReqCtrlFpgaRev

Focus FPGA revision

Current CBS-HARDWARE-MIB cbsHwModuleFocusFpgaRev

Required CBS-HARDWARE-MIB cbsHwModuleReqFocusFpgaRev

DBHA View

Failover Group ID CBS-VRRP-MIB crossbeamSystems.cbsMgmt.cbsVrrp.cbsVrrpFailoverGroupTable.cbsVrrpFailoverGroupEntry.cbsVrrpFailoverGroupID

Failover Group Name CBS-VRRP-MIB crossbeamSystems.cbsMgmt.cbsVrrp.cbsVrrpFailoverGroupTable.cbsVrrpFailoverGroupEntry.cbsVrrpFailoverGroupName



Status CBS-VRRP-MIB crossbeamSystems.cbsMgmt.cbsVrrp.cbsVrrpFailoverGroupTable.cbsVrrpFailoverGroupEntry.cbsVrrpStatus

Actual Priority CBS-VRRP-MIB crossbeamSystems.cbsMgmt..cbsVrrp.cbsVrrpFailoverGroupTable.cbsVrrpFailoverGroupEntry.cbsVrrpPriority

Configured Priority CBS-VRRP-MIB crossbeamSystems.cbsMgmt.cbsVrrp.cbsVrrpFailoverGroupTable.cbsVrrpFailoverGroupEntry.cbsVrrpAdminPriority

Master Priority CBS-VRRP-MIB crossbeamSystems.cbsMgmt.cbsVrrp.cbsVrrpFailoverGroupTable.cbsVrrpFailoverGroupEntry.cbsVrrpMasterPriority

NPM View



Hardware Properties

Revision CBS-HARDWARE-MIB cbsHwModuleBoardRevision

Serial Number CBS-HARDWARE-MIB cbsHwModuleSerialNumber

Part Number CBS-HARDWARE-MIB cbsHwModulePartNumber

Status CBS-HARDWARE-MIB cbsHwModuleStatus

Uptime CBS-MODULE-RESOURCES-MIB

cbsModuleUptime

Interfaces Label IF-MIB ifDescr

State IF-MIB ifOperStatus

BW IF-MIB ifSpeed

Data Rate(In) IF-MIB ifInOctets

Data Rate(Out) IF-MIB ifOutOctets

APM View



Vap Name CBS-VAPGROUP-MIB cbsVmVapName

Hardware Properties




Module Status CBS-HARDWARE-MIB cbsHwModuleStatus


cbsModuleUptime

Application CBS-MODULE-RESOURCES-MIB

cbsModuleAppName

cbsModuleAppVersion



App Status CBS-MODULE-RESOURCES-MIB

cbsModuleAppNewState

Avg CPU Util CBS-MODULE-RESOURCES-MIB

cbsModuleCPUUtil5

Avg CPU Load CBS-MODULE-RESOURCES-MIB

cbsModuleCPULoad5

Peak Core Util CBS-MODULE-RESOURCES-MIB

cbsModuleCpuCoreHiUtilPerc

Total Memory CBS-MODULE-RESOURCES-MIB

cbsModuleMemoryTotal

cbsModuleMemoryFree

cbsModuleMemoryBuffer

cbsModuleMemoryCached

Low Memory CBS-MODULE-RESOURCES-MIB

cbsModuleMemoryLoTotal

cbsModuleMemoryLoFree

CPM View



Hardware Properties




Module Status CBS-HARDWARE-MIB cbsMgmt.cbsHardware.cbsCPRedundancyState.cbsCP2RedundancyOperState or cbsCP1RedundancyOperState

XOS Version MIB-II sysDescr


cbsModuleUptime

Admin state CBS-HARDWARE-MIB crossbeamSystems.cbsMgmt.cbsHardware.cbsCPRedundancyState.cbsCPRedundancyAdminState

crossbeamSystems.cbsMgmt.cbsHardware.cbsCPRedundancyState.cbsCP1RedundancyAdminState

crossbeamSystems.cbsMgmt.cbsHardware.cbsCPRedundancyState.cbsCP2RedundancyAdminState

Sync state CBS-HARDWARE-MIB crossbeamSystems.cbsMgmt.cbsHardware.cbsCPRedundancyState.cbsCPRedundancyOperState

crossbeamSystems.cbsMgmt.cbsHardware.cbsCPRedundancyState.cbsCPRedundancySync

Avg CPU Util CBS-MODULE-RESOURCES-MIB

cbsModuleCPUUtil5



Avg CPU Load CBS-MODULE-RESOURCES-MIB

cbsModuleCPULoad5

Peak Core Util CBS-MODULE-RESOURCES-MIB

cbsModuleCpuCoreHiUtilPerc

Total Memory CBS-MODULE-RESOURCES-MIB

cbsModuleMemoryTotal

cbsModuleMemoryFree

cbsModuleMemoryBuffer

cbsModuleMemoryCached

Disk usage root CBS-MODULE-RESOURCES-MIB

cbsModuleDURoot

boot CBS-MODULE-RESOURCES-MIB

cbsModuleDUBoot

cbconfig CBS-MODULE-RESOURCES-MIB

cbsModuleDUCbconfig

tftpboot CBS-MODULE-RESOURCES-MIB

cbsModuleDUTftpboot

mgmt CBS-MODULE-RESOURCES-MIB

cbsModuleDUMgmt

RAID Status CBS-HARDWARE-MIB cbsHwModuleRAIDStatus

Mode CBS-HARDWARE-MIB cbsHwModuleRAIDMode

Flows View

Chassis

Utilization

Flow Table Partition Threshold

CBS-MODULE-RESOURCES-MIB

cbsFlowTablePartitionThreshold

NPM Flow Table Critical Alarm


cbsFlowTableCriticalAlarm

NPM Flow Table Utilization


cbsFlowTableUtilization

NPM Flow Table Utilization TCP Flow Entries


cbsFlowTableUtilizationTcpFlowEntries

NPM Flow Table Utilization UDP Flow Entries


cbsFlowTableUtilizationUdpFlowEntries

NPM Flow Table UtilizationI CMP Flow Entries


cbsFlowTableUtilizationIcmpFlowEntries

NPM Flow Table Utilization Other IP Flow Entries


cbsFlowTableUtilizationOtherIpFlowEntries



NPM Slot ID CBS-MODULE-RESOURCES-MIB

cbsNpmFlowDistTable.cbsNpmFlowDistEntry.cbsNpmSlotID

TCP Current Flows


cbsNpmFlowDistTable.cbsNpmFlowDistEntry.cbsNpmProtoTcpCurrentFlows

UDP Current Flows


cbsNpmFlowDistTable.cbsNpmFlowDistEntry.cbsNpmProtoTdpCurrentFlows

ICMP Current Flows


cbsNpmFlowDistTable.cbsNpmFlowDistEntry.cbsNpmProtoIcmpCurrentFlows

Other IP Current Flows


cbsNpmFlowDistTable.cbsNpmFlowDistEntry.cbsNpmProtoOtherIpCurrentFlows

New Flow Rate CBS-MODULE-RESOURCES-MIB

cbsNpmFlowDistTable.cbsNpmFlowDistEntry.cbsNpmNewFlowRate

Aged Flow Rate CBS-MODULE-RESOURCES-MIB

cbsNpmFlowDistTable.cbsNpmFlowDistEntry.cbsNpmAgedFlowRate

VAP VapName CBS-MODULE-RESOURCES-MIB

cbsVapFlowDistTable.cbsVapFlowDistEntry.cbsVapNmae

Current Flows CBS-MODULE-RESOURCES-MIB

cbsVapFlowDistTable.cbsVapFlowDistEntry.cbsVapCurrentFlows

New Flow Rate CBS-MODULE-RESOURCES-MIB

cbsVapFlowDistTable.cbsVapFlowDistEntry.cbsVapNewFlowRate

Aged Flow Rate CBS-MODULE-RESOURCES-MIB

cbsVapFlowDistTable.cbsVapFlowDistEntry.cbsVapAgedFlowRatge

Alarms View

ID CBS-HARDWARE-MIB crossbeamSystems.cbsMgmt.cbsHardware.cbsAlarmObjects.cbsActiveAlarmsTable.cbsActiveAlarmsEntry.cbsActiveAlarmId

Date CBS-HARDWARE-MIB crossbeamSystems.cbsMgmt.cbsHardware.cbsAlarmObjects.cbsActiveAlarmsTable.cbsActiveAlarmsEntry.cbsActiveAlarmDate

Severity CBS-HARDWARE-MIB crossbeamSystems.cbsMgmt.cbsHardware.cbsAlarmObjects.cbsActiveAlarmsTable.cbsActiveAlarmsEntry.cbsActiveAlarmSeverity

Source CBS-HARDWARE-MIB crossbeamSystems.cbsMgmt.cbsHardware.cbsAlarmObjects.cbsActiveAlarmsTable.cbsActiveAlarmsEntry.cbsActiveAlarmSource

Description CBS-HARDWARE-MIB crossbeamSystems.cbsMgmt.cbsHardware.cbsAlarmObjects.cbsActiveAlarmsTable.cbsActiveAlarmsEntry.cbsActiveAlarmDescription



XOS Alarms and SNMP TrapsXOS provides a number of alarms to help you monitor the state of the XOS system. The following table presents the alarms that can be raised by XOS and the associated SNMP traps.

This information is also available in the Greenlight Element Manager (GEM) online help.

Device CBS-HARDWARE-MIB crossbeamSystems.cbsMgmt.cbsHardware.cbsAlarmObjects.cbsActiveAlarmsTable.cbsActiveAlarmsEntry.cbsActiveAlarmDevice

Sensor CBS-HARDWARE-MIB crossbeamSystems.cbsMgmt.cbsHardware.cbsAlarmObjects.cbsActiveAlarmsTable.cbsActiveAlarmsEntry.cbsActiveAlarmSensor

Interface CBS-HARDWARE-MIB crossbeamSystems.cbsMgmt.cbsHardware.cbsAlarmObjects.cbsActiveAlarmsTable.cbsActiveAlarmsEntry.cbsActiveAlarmIface

Sub Interface CBS-HARDWARE-MIB crossbeamSystems.cbsMgmt.cbsHardware.cbsAlarmObjects.cbsActiveAlarmsTable.cbsActiveAlarmsEntry.cbsActiveAlarmSubIface

Disk CBS-HARDWARE-MIB crossbeamSystems.cbsMgmt.cbsHardware.cbsAlarmObjects.cbsActiveAlarmsTable.cbsActiveAlarmsEntry.cbsActiveAlarmDisk

Partition CBS-HARDWARE-MIB crossbeamSystems.cbsMgmt.cbsHardware.cbsAlarmObjects.cbsActiveAlarmsTable.cbsActiveAlarmsEntry.cbsActiveAlarmPartition

Active minor alarms number CBS-HARDWARE-MIB crossbeamSystems.cbsMgmt.cbsHardware.cbsAlarmObjects.cbsAlarmStats.cbsActiveMinorNumber

Active major alarms number CBS-HARDWARE-MIB crossbeamSystems.cbsMgmt.cbsHardware.cbsAlarmObjects.cbsAlarmStats.cbsActiveMajorNumber

Active critical alarms number CBS-HARDWARE-MIB crossbeamSystems.cbsMgmt.cbsHardware.cbsAlarmObjects.cbsAlarmStats.cbsActiveCriticalNumber

Overall Application

IpAddr MIB-II ipAdEntAddr


Alarm Name MIB SNMP Trap

applicationDown CBS-HARDWARE-MIB cbsModuleAppStateChange

batteryVoltageProblem CBS-HARDWARE-MIB cbsHwBatteryVCCNormal

cbsHwBatteryVCCOutOfRange


chassisTemperatureExceeded CBS-HARDWARE-MIB cbsHwChassisTempNormal

cbsHwChassisTempExceeded

componentTemperatureExceeded CBS-HARDWARE-MIB cbsHwFPGATempExceeded

cbsHwFPGATempNormal

cpuCoreUtilizationExceeded CBS-MODULE- RESOURCES-MIB

cbsModuleCpuCoreUtilExceeded

cbsModuleCpuCoreUtilNormal

cpuTempLimitExceeded CBS-HARDWARE-MIB cbsModuleTempExceeded

cbsModuleTempNormal

cbsModuleCpu2TempExceeded

cbsModuleCpu2TempNormal

cpuUptimeLimitExceeded

cpuUtilization15min

cpuUtilization1min

cpuUtilization5min

diagsFailure CBS-HARDWARE-MIB cbsNPMDiagFailure

diskConfigurationBad CBS-MODULE- RESOURCES-MIB

cbsCpmDiskCheck

diskDriveError

diskUtilizationBootExceeded CBS-MODULE- RESOURCES-MIB

cbsModuleDUBootHigh

cbsModuleDUBootNormal

diskUtilizationCbConfigExceeded CBS-MODULE- RESOURCES-MIB

cbsModuleDUCbconfigHigh

cbsModuleDUCbconfigNormal

diskUtilizationMgmtExceeded CBS-MODULE- RESOURCES-MIB

cbsModuleDUCbMgmtHigh

cbsModuleDUCbMgmtNormal

diskUtilizationRootExceeded CBS-MODULE- RESOURCES-MIB

cbsModuleDUCbRootHigh

cbsModuleDUCbRootNormal

diskUtilizationTftpBootExceeded CBS-MODULE- RESOURCES-MIB

cbsModuleDUTftpbootHigh

cbsModuleDUTftpbootNormal

diskUtilizationVarExceeded CBS-MODULE- RESOURCES-MIB

cbsModuleDUVarHigh

cbsModuleDUVarNormal

dualBoxHaLinkFailure CBS-HARDWARE-MIB cbsDbhaLinkDown

cbsDbhaLinkUp

exhaustAirTemperatureExceeded CBS-HARDWARE-MIB cbsHwExhaustTempExceeded

cbsHwExhaustTempNormal

fanFailure CBS-HARDWARE-MIB cbsHwFanFailed

cbsHwFanRecovered

fanTrayIncompatible



fanTrayMissing CBS-HARDWARE-MIB cbsHwLowerFanTrayInserted

cbsHwLowerFanTrayRemoved

cbsHwUpperFanTrayInserted

cbsHwUpperFanTrayRemoved

firmwareMismatch

flowTableFull CBS-HARDWARE-MIB cbsAFTUsageHigh

cbsAFTUsageNormal

flowTableMedianThresholdExceeded

CBS-MODULE- RESOURCES-MIB

cbsNpmFTTcpMedianExceeded

cbsNpmFTTcpMedianNormal

cbsNpmFTUdpMedianExceeded

cbsNpmFTUdpMedianNormal

cbsNpmFTIcmpMedianExceeded

cbsNpmFTIcmpMedianNormal

cbsNpmFTOtherIpMedianExceeded

cbsNpmFTOtherIpMedianNormal

flowTableThresholdExceeded CBS-MODULE- RESOURCES-MIB

cbsNpmFTTcpLimitExceeded

cbsNpmFTTcpLimitNormal

cbsNpmFTUdpLimitExceeded

cbsNpmFTUdpLimitNormal

cbsNpmFTIcmpLimitExceeded

cbsNpmFTIcmpLimitNorma

cbsNpmFTOtherIpLimitExceeded

cbsNpmFTOtherIpLimitNormal

fragPktRateExceeded CBS-MODULE- RESOURCES-MIB

cbsModuleFRHigh

cbsModuleFRNormal

freePageUtilization CBS-MODULE- RESOURCES-MIB

cbsModuleFreePageAvailableHigh

cbsModuleFreePageAvailableNorm

guestHealthProblem CBS-MODULE- RESOURCES-MIB

cbsApmGuestHealthCheck

intakeAirTemperatureExceeded CBS-MODULE- RESOURCES-MIB

cbsHwInTempExceeded

cbsHwInTempNormal

linkFailure IF-MIB linkDown

linkUp

memoryConfiguration CBS-MODULE- RESOURCES-MIB

cbsModuleSdramCheck

memoryMismatch

moduleFailure




moduleStateChange CBS-HARDWARE-MIB cbsHwModuleInserted

cbsHwModuleRemoved

cbsHwModuleStatusChanged

noRemoteBoxConfigured CBS-VRRP-MIB cbsVrrpTrapNoRemoteBoxConfigured

npmBuffersUsageHigh CBS-MODULE- RESOURCES-MIB

cbsModuleBUHigh

cbsModuleBUNormal

npmOutOfMemory CBS-MODULE- RESOURCES-MIB

cbsModuleMUHigh

cbsModuleMUNormal

ntpServiceFailure CBS-MODULE- RESOURCES-MIB

cbsModuleNtpdMonStateChange

onlyOneRemoteBoxPathDefined CBS-VRRP-MIB cbsVrrpTrapOnlyOnePathDefined

outOfService

powerFeedFailure CBS-HARDWARE-MIB cbsHwPowerFeedFailed

cbsHwPowerFeedRecovered

powerSupplyA_Bfailure CBS-HARDWARE-MIB cbsHwPowerSupplyFailed

cbsHwPowerSupplyRecovered

powerSupplyMissing CBS-HARDWARE-MIB cbsHwSystemACPwrBay1Failed

cbsHwSystemACPwrBay1Recovered

cbsHwSystemACPwrBay2Failed






raidStatusChange

resourceProtectionFlowThresholdExceeded

CBS-MODULE- RESOURCES-MIB

cbsNpmFlowTableUsageHigh

cbsNpmFlowTableUsageNormal

rswFail CBS-MODULE- RESOURCES-MIB

cbsModuleRSWStateChange

systemAlarm CBS-HARDWARE-MIB cbsHwSystemAlarmTrap

systemRestart

vapStateChange CBS-VAPGROUP-MIB cbsVapStatusChanged


voltageProblem CBS-HARDWARE-MIB cbsHw125VoltageNormal

cbsHw125VoltageOutOfRange

cbsHwModule1-2VNormal

cbsHwModule1-2VOutOfRange





cbsHw1-8VoltageNormal

cbsHw1-8VoltageOutOfRange





cbsHwCPUVoltageNormal

cbsHwCPUVoltageOutOfRange

cbsHwCPU2VoltageNormal

cbsHwCPU2VoltageOutOfRange

cbsHwFPGA1-8VoltageNormal

cbsHwFPGA1-8VoltageOutOfRange

cbsHwModuleETHSwVNormal

cbsHwModuleETHSwVOutOfRange

cbsHwModuleNP6EZDDRNormal

cbsHwModuleNP6EZDDROutOfRange

cbsHwModuleNP6EZCORENormal

cbsHwModuleNP6EZCOREOutOfRange

cbsHwModuleNP6EZHSTLNormacbsHwModuleNP6EZHSTLOutOfRange

cbsHwModuleNP6XBPRCNormal

cbsHwModuleNP6XBPRCOutOfRange

cbsHwModuleNP6OCTDDRNormal

cbsHwModuleNP6OCTDDROutOfRange

voltageProblem (cont’d) CBS-HARDWARE-MIB cbsHwModulePrismVNormal

cbsHwModulePrismVOutOfRange

cbsHwModuleSupplyFGNormal

cbsHwModuleSupplyFGOutOfRange

vrrpFailGroupPriorityChange CBS-VRRP-MIB cbsVrrpTrapFailGrPriorityChanged

vrrpFailGroupStatusChange CBS-VRRP-MIB cbsVrrpTrapFailGrStatusChanged



CROSSBEAM-SYSTEMS-MIB ReferenceThe CROSSBEAM-SYSTEMS-MIB includes high-level entries that describe Crossbeam C-Series and X-Series chassis and related components.

This section contains a tree view of the CROSSBEAM-SYSTEMS-MIB entries and traps with OID information. For comprehensive attributes and text descriptions, use a MIB browser or read the CROSSBEAM-SYSTEMS-MIB.txt file on your CPM at /usr/share/snmp/mibs.

Identifying the CROSSBEAM-SYSTEMS-MIB OIDsIdentify the OID for specific MIB entries by appending the ordered list number for the entry to the CROSSBEAM-SYSTEMS-MIB OID .1.3.6.1.4.1.6848.

The OID for Crossbeam specific product entries uses the prefix .1.3.6.1.4.1.6848.1

The OID for Crossbeam specific management entries uses the prefix .1.3.6.1.4.1.6848.2

The OID for Crossbeam specific MIBs use the prefix .1.3.6.1.4.1.6848.3

The OID for Crossbeam specific traps uses the trap prefix .1.3.6.1.4.1.6848.4

NOTE: Crossbeam originally created the CROSSBEAM-SYSTEMS-MIB for the X-Series X40 chassis. Therefore this MIB includes later X-Series chassis models such as the X45 and the X80 into a separate OID branch. However, all X-Series modules are only listed under the X40 chassis branch.

CROSSBEAM-SYSTEMS-MIB product entriesCROSSBEAM-SYSTEMS-MIB product OID: .1.3.6.1.4.1.6848.1

1. cbsX40

1. cbsX40Chassis

2. cbsX40Modules

1. cbsCPModules

1.cbsCPM4100

2.cbsCPM8100

3.cbsCPM8400

4.cbsCPM8600

vrrpRemoteBoxNoActivePath CBS-VRRP-MIB cbsVrrpTrapRemoteBoxNoActivePath

vrrpRemoteBoxNoSecondaryPath CBS-VRRP-MIB cbsVrrpTrapRemoteBoxNoSecondaryPath

vrrpRemoteBoxNoStandbyPath CBS-VRRP-MIB cbsVrrpTrapRemoteBoxNoStandbyPath

vrrpRemoteBoxPathSharedSingleIntf

CBS-VRRP-MIB cbsVrrpTrapRemoteBoxPathSharedIntf

vrrpRemoteBoxPathStatusChange

CBS-VRRP-MIB cbsVrrpTrapRemoteBoxPathStatusChanged

vrrpRemoteBoxRunLegacyXOS CBS-VRRP-MIB cbsVrrpTrapRemoteBoxRunLegacyXOS



5.cbsCPM9600

2. cbsAPModules

1.cbsAPM4100

2.cbsAPM4200

3.cbsAPM4250

4.cbsAPM8200

5.cbsAPM8400

6.cbsAPM8600

7.cbsAPM8650

8.cbsAPM9600

3. cbsNPModules

1.cbsNPM4100

2.cbsNPM4110

3.cbsNPM8200

4.cbsNPM8210

5.cbsNPM8600

6.cbsNPM8650

7.cbsNPM8620

2. cbsXSeries

1. cbsXChassis

1. cbsX45Chassis

2. cbsX80Chassis

3. cbsX60Chassis

4. cbsX20Chassis

5. cbsX30Chassis

3. cbsCSeries

1. cbsCChassis

1. cbsC30Chassis

2. cbsC10Chassis

1. cbsC30iChassis

2. cbsC2Chassis

3. cbsC6Chassis

4. cbsC12Chassis

5. cbsC25Chassis

CBS-HARDWARE-MIB ReferenceThe CBS-HARDWARE-MIB includes entries for Crossbeam specific hardware features. The features include system status, hardware components, temperature, voltage, power, and LED status.

This section contains a tree view of the CBS-HARDWARE-MIB entries and traps with OID information. For comprehensive attributes and text descriptions, use a MIB browser or read the CBS-HARDWARE-MIB.txt file on your CPM at /usr/share/snmp/mibs.


Identifying the CBS-HARDWARE-MIB OIDsIdentify the OID for specific MIB entries by appending the ordered list number for the entry to the CBS-HARDWARE-MIB management OID entries (.1.3.6.1.4.1.6848.2.1). The OID for Crossbeam hardware-specific traps uses the trap prefix (.1.3.6.1.4.1.6848.4.1)

For example:

The Crossbeam system serial number (cbsHwSystemSerialNumber) OID is .1.3.6.1.4.1.6848.2.1.1.3

The system status (cbsHwSystemAlarm) OID is .1.3.6.1.4.1.6848.2.1.2.6

The trap for a failed fan (cbsHwFanFailed) is .1.3.6.1.4.1.6848.4.1.1.9

CBS-HARDWARE-MIB entriesCBS-HARDWARE-MIB management OID: .1.3.6.1.4.1.6848.2.1

1. cbsHwSystem

1. cbsHwSystemProductID

2. cbsHwSystemDescription

3. cbsHwSystemSerialNumber

4. cbsHwSystemBackplaneRevision

5. cbsHwSystemPartNumber

6. cbsHwSystemIdentifier

2. cbsHwSystemStatus

1. cbsHwSystemRedundantPowerSupplyStatus

2. cbsHwSystemRedundantPowerFeedStatus

3. cbsHwSystemChassisTemp

4. cbsHwSystemUpperFanTray

5. cbsHwSystemLowerFanTray

6. cbsHwSystemAlarm

7. cbsHwSystemPowerType

8. cbsHwSystemRedundantACPowerSupplyState

9. cbsHwSystemRedundantACPowerFeedStatus

10. cbsHwSystemDCPowerFilterA

11. cbsHwSystemDCPowerFilterB

12. cbsHwSystemDCPowerFeedA

13. cbsHwSystemACPowerFeedB

14. cbsHwSystemACPwrBay1Presence




18. cbsHwSystemACPwrBay1Status





22. cbsHwSystemAftStatus

23. cbsHwSystemUpperFanTrayCompatible

24. cbsHwSystemUpperFanTrayRevision

25. cbsHwSystemUpperFanTraySerialNumber

26. cbsHwSystemUpperFanTrayPartNumber

27. cbsHwSystemLowerFanTrayCompatible

28. cbsHwSystemLowerFanTrayRevision

29. cbsHwSystemLowerFanTraySerialNumber

30. cbsHwSystemLowerFanTrayPartNumber

3. cbsFanStatusTable

1. cbsFanStatusEntry

1. cbsFanGroupIndex

2. cbsFanIndex

3. cbsFanStatus

4. cbsHwModuleTable

1. cbsHwModuleEntry

1. cbsHwModuleID

2. cbsHwModuleProductID

3. cbsHwModuleDescription

4. cbsHwModuleBoardRevision

5. cbsHwModuleSerialNumber

6. cbsHwModuleBootStrapRev

7. cbsHwModuleBootloaderRev

8. cbsHwModuleDiagnosticsRev

9. cbsHwModuleModuleSDRAMSize

10. cbsHwModuleRDRAMSize

11. cbsHwModuleDiskAPresent

12. cbsHwModuleDiskBPresent

13. cbsHwModuleDuartAPresent

14. cbsHwModuleDuartBPresent

15. cbsHwModuleAccelCard1Present

16. cbsHwModuleAccelCard2Present

17. cbsHwModulePartNumber

18. cbsHwModuleReqBootStrapRev

19. cbsHwModuleReqBootloaderRev

20. cbsHwModuleReqDiagnosticsRev

21. cbsHwModuleCtrlFpgaRev

22. cbsHwModuleReqCtrlFpgaRev

23. cbsHwModuleFocusFpgaRev


24. cbsHwModuleReqFocusFpgaRev

5. cbsHwComponentRevTable

1. cbsHwComponentRevEntry

1. cbsHwComponentIndex

2. cbsHwComponentDescription

3. cbsHwComponentRevision

6. cbsHwModuleStatusTable

1. cbsHwModuleStatusEntry

1. cbsHwModuleStatus

2. cbsHwModuleStatusActive

3. cbsHwModuleCpuTemp

4. cbsHwModuleFPGATemp

5. cbsHwModuleInTemp

6. cbsHwModuleInTempAlarm

7. cbsHwModuleExhaustTemp

8. cbsHwModuleExhaustTempAlarm

9. cbsHwModuleCPUVoltage

10. cbsHwModule1-8Voltage



13. cbsHwModuleControlLinkA

14. cbsHwModuleControlLinkB

15. cbsHwModuleActiveLED

16. cbsHwModuleStandbyLED

17. cbsHwModuleFailedLED

18. cbsHwModuleCpu2Temp

19. cbsHwModuleCPU2Voltage

20. cbsHwModuleFPGA1-8Voltage

21. cbsHwModule125Voltage

22. cbsHwModule1-5V

23. cbsHwModuleBatteryVCCVoltage

24. cbsHwModuleSupplyFGVoltage

25. cbsHwModuleSupplyETHSwVoltage

26. cbsHwModuleSupplyPrismVoltage

27. cbsHwModule1-2V

28. cbsHwModule12-0V

29. cbsHwModuleSupplyNP6OCTDDRVolt

30. cbsHwModuleSupplyNP6EZCOREVolt

31. cbsHwModuleSupplyNP6EZDDRVolt

32. cbsHwModuleSupplyNP6EZHSTLVolt


33. cbsHwModuleSupplyNP6XBPRCVolt

34. cbsHwModuleRAIDStatus

35. cbsHwModuleRAIDMode

7. cbsHwSdpStatusTable

1. cbsHwSdpStatusEntry

1. cbsHwSdpNpmSlot

2. cbsHwSdpRemoteSlot

3. cbsHwSdpNpmState

4. cbsHwSdpRemoteState

8. cbsCPRedundancyState

1. cbsCPRedundancyAdminState

2. cbsCP1RedundancyAdminState

3. cbsCP2RedundancyAdminState

4. cbsCPRedundancyOperState

5. cbsCP1RedundancyOperState

6. cbsCP2RedundancyOperState

7. cbsCPRedundancySync

9. cbsAlarmObjects

1. cbsActiveAlarmsTable

1. cbsActiveAlarmsEntry

1.cbsActiveAlarmIndex

2.cbsActiveAlarmId

3.cbsActiveAlarmDate

4.cbsActiveAlarmSeverity

5.cbsActiveAlarmSource

6.cbsActiveAlarmDescription

2. cbsAlarmStats

1. cbsActiveMinorNumber

2. cbsActiveMajorNumber

3. cbsActiveCriticalNumber

10. cbsTrapObjects

1. cbsTrapSeverity

CBS-HARDWARE-MIB Trap EntriesCBS-HARDWARE-MIB trap OID: .1.3.6.1.4.1.6848.4.1

1. cbsHwTraps

1. cbsHwPowerSupplyFailed

2. cbsHwPowerSupplyRecovered

3. cbsHwPowerFeedFailed

4. cbsHwPowerFeedRecovered


5. cbsHwUpperFanTrayInserted

6. cbsHwUpperFanTryRemoved

7. cbsHwLowerFAnTrayInserted

8. cbsHwLowerFanTrayRemoved

9. cbsHwFanFailed

10. cbsHwFanRecovered

11. cbsHwSystemAlarmTrap

12. cbsHwChassisTempExceeded

13. cbsHwChassisTempNormal

14. cbsHwModuleStatusChanged

15. cbsHwModuleInserted

16. cbsHwModuleRemoved

17. cbsHwModuleTempExceeded

18. cbsModuleTempNormal

19. cbsDbhaLinkUp

20. cbsDbhaLinkDown

21. cbsModuleCpu2TempExceeded

22. cbsModuleCpu2TempNormal

23. cbsAFTUsageHigh

24. cbsAFTUsageNormal

25. cbsHwSystemACPwrBay1Failed

26. cbsHwSystemACPwrBay1Recovered







33. cbsNPPMDiagFailure

34. cbsNPMPrcLink13Down

35. cbsNPMPrcLink13Up

36. cbsNPMPrcLink14Down

37. cbsNPMPrcLink14Up

38. cbsHwInTempExceeded

39. cbsHwInTempNormal

40. cbsHwExhaustTempExceeded

41. cbsHwExhaustTempNormal

42. cbsHwFPGATempExceeded

43. cbsHwFPGATempNormal

44. cbsHwCPUVoltageNormal


45. cbsHwCPUVoltageOutOfRange

46. cbsHw1-8VoltageNormal

47. cbsHw1-8VoltageOutOfRange





52. cbsHwCPU2VoltageNormal

53. cbsHwCPU2VoltageOutOfRange

54. cbsHwFPGA1-8VoltageNormal

55. cbsHwFPGA1-8VoltageOutOfRange

56. cbsHw125VoltageNormal

57. cbsHw125VoltageOutOfRange

58. cbsHwBatteryVCCNormal

59. cbsHwBatteryVCCOutOfRange

60. cbsHwModule1-5VNormal

61. cbsHwModule1-5VOutOfRange

62. cbsHwModuleSupplyFGNormal

63. cbsHwModuleSupplyFGOutOfRange





68. cbsHwModuleNP6OCTDDROutOfRange

69. cbsHwModuleNP6OCTDDRNormal

70. cbsHwModuleNP6EZCOREOutOfRange

71. cbsHwModuleNP6EZCORENormal

72. cbsHwModuleNP6EZDDROutOfRange

73. cbsHwModuleNP6EZDDRNormal

74. cbsHwModuleNP6EZHSTLOutOfRange

75. cbsHwModuleNP6EZHSTLNormal

76. cbsHwModuleNP6XBPRCOutOfRange

77. cbsHwModuleNP6XBPRCNormal

78. cbsHwModuleETHSwVOutOfRange

79. cbsHwModuleETHSwVNormal

80. cbsHwModulePrismVOutOfRange

81. cbsHwModulePrismVNormal


CBS-MODULE-RESOURCES-MIB ReferenceThe CBS-MODULE-RESOURCE-MIB includes entries that describe module features such as CPU usage, memory usage, and other module characteristics.

This section contains a tree view of the CBS-MODULE-RESOURCES-MIB entries and traps with OID information. For comprehensive attributes and text descriptions, use a MIB browser or read the CBS-MODULE-RESOURCES-MIB.txt file on your CPM at /usr/share/snmp/mibs.

Identifying the CBS-MODULE-RESOURCES-MIB OIDsIdentify the OID for specific MIB entries by appending the ordered list number for the entry to the CBS-MODULE-RESOURCES-MIB management OID (.1.3.6.1.4.1.6848.2.3). The OID for Crossbeam module-resource-specific traps uses the trap prefix (.1.3.6.1.4.1.6848.4.2)

For example:

The Crossbeam module CPU count (cbsModuleCPUCount) OID is .1.3.6.1.4.1.6848.2.3.1.1.2

The trap for module CPU load exceeded (cbsModuleCPULoadExceeded) OID is .1.3.6.1.4.1.6848.4.2.1

CBS-MODULE-RESOURCES-MIB entriesCBS-MODULE-RESOURCES-MIB management OID: .1.3.6.1.4.1.6848.2.3

1. cbsModuleCPULoadTable

1. cbsModuleCPULoadEntry

1. cbsModuleCPUSpeed

2. cbsModuleCPUCount

3. cbsModuleCPULoad1



6. cbsModuleCPUUtil1



2. cbsModuleMemoryTable

1. cbsModuleMemoryEntry

1. cbsModuleMemoryTotal

2. cbsModuleMemoryUsed

3. cbsModuleMemoryFree

3. cbsModuleSwapTable

1. cbsModuleSwapEntry

1. cbsModuleSwapTotal

2. cbsModuleSwapUsed

3. cbsModuleSwapFree

4. cbsModuleDUTable

1. cbsModuleDUEntry


1. cbsModuleDURoot

2. cbsModuleDUBoot

3. cbsModuleDUCbconfig

4. cbsModuleDUTftpboot

5. cbsModuleFreePageTable

1. cbsModuleFreePageEntry

1. cbsModuleFreePageAvailable

2. cbsModuleFreePageThreshold

3. cbsModuleFreePageSeverity

4. cbsModuleFreePageVapName

6. cbsModuleCPUAvgUtilTable

1. CbsModuleCPUAvgUtilEntry

1. cbsModuleCPUAvgUtilCore1User

2. cbsModuleCPUAvgUtilCore1Nice

3. cbsModuleCPUAvgUtilCore1Syst

4. cbsModuleCPUAvgUtilCore1Idle

5. cbsModuleCPUAvgUtilCore1IRQ

6. cbsModuleCPUAvgUtilCore1SoftIRQ

7. cbsModuleCPUAvgUtilCore1IOWait






13. cbsModuleCPUAvgUtilCore2SoftIRQ,













































7. CbsModuleSDPTable

1. CbsModuleSDPEntry

1. cbsModuleSDP0OutPkts

2. cbsModuleSDP0OutOctets

3. cbsModuleSDP0InPkts

4. cbsModuleSDP0InOctets






























8. cbsModuleUptimeTable

1. cbsModuleUptimeEntry

1. cbsModuleUptime

9. cbsModuleNPMFlowCountTable

1. CbsModuleNPMFlowCountEntry

1. CbsModuleNPMFlowCount

10. CbsModuleAppMonTable

1. CbsModuleAppMonEntry

1. cbsModuleAppName

2. cbsModuleAppVersion

3. cbsModuleAppRelease

4. cbsModuleAppCPMHostName

5. cbsModuleAppCPMIPAddress

6. cbsModuleAppVAPGroupName

7. cbsModuleAppVAPIndex

8. cbsModuleAppOldState

9. cbsModuleAppNewState

11. CbsModuleNtpdMonTable


1. CbsModuleNtpdMonEntry

1. cbsModuleNtpdCPMHostName

2. cbsModuleNtpdCPMIPAddress

3. cbsModuleNtpdState

CBS-MODULE-RESOURCES-MIB Trap EntriesCBS-MODULE-RESOURCES-MIB trap OID: .1.3.6.1.4.1.6848.4.2

1. cbsModuleCPULoadExceeded

2. cbsModuleCPULoadNormal

3. cbsModuleMemoryUsageExceeded

4. cbsModuleMemoryUsageNormal

5. cbsModuleCPUUtilExceeded

6. cbsModuleCPUUtilNormal

7. cbsModuleDURootHigh

8. cbsModuleDURootNormal

9. cbsModuleDUBootHigh

10. cbsModuleDUBootNormal

11. cbsModuleDUCbconfigHigh

12. cbsModuleDUCbconfigNormal

13. cbsModuleTftbootHigh

14. cbsModuleTftbootNormal

15. cbsModuleFreePageAvailableHigh

16. cbsModuleFreePageAvailableNormal

17. cbsModuleMUHigh

18. cbsModuleMUNormal

19. cbsModuleFRHigh

20. cbsModuleFRNormal

21. cbsModuleBUHigh

22. cbsModuleBUNormal

23. cbsModuleAppStateChange

24. cbsModuleNtpdMonStateChange

25. cbsModuleDUMgmtHigh

26. cbsModuleDUMgmtNormal

27. cbsModuleCpuCoreUtilExceeded

28. cbsModuleCpuCoreUtilNormal

29. cbsNpmFlowTableUsageHigh

30. cbsNpmFlowTableUsageNormal

31. cbsNpmFTUdpLimitExceeded

32. cbsNpmFTUdpLimitNormal

33. cbsNpmFTUdpMedianExceeded


34. cbsNpmFTUdpMedianNormal

35. cbsNpmFTTcpLimitExceeded

36. cbsNpmFTTcpLimitNormal

37. cbsNpmFTTcpMedianExceeded

38. cbsNpmFTTcpMedianNormal

39. cbsNpmFTIcmpLimitExceeded

40. cbsNpmFTIcmpLimitNormal

41. cbsNpmFTIcmpMedianExceeded

42. cbsNpmFTIcmpMedianNormal

43. cbsNpmFTOtherIpLimitExceeded

44. cbsNpmFTOtherIpLimitNormal

45. cbsNpmFTOtherIpMedianExceeded

46. cbsNpmFTOtherIpMedianNormal

47. cbsModuleSdramCheck

CBS-VAPGROUP-MIB ReferenceThe CBS-VAPGROUP-MIB includes entries that describe VAP features, configuration settings, and other details.

This section contains a tree view of the CBS-VAPGROUP-MIB entries and traps with OID information. For comprehensive attributes and text descriptions, use a MIB browser or read the CBS-VAPGROUP-MIB.txt file on your CPM at /usr/share/snmp/mibs.

Identifying the CBS-VAPGROUP-MIB OIDsIdentify the OID for specific MIB entries by appending the ordered list number for the entry to the CBS-VAPGROUP-MIB management OID (.1.3.6.1.4.1.6848.2.4). The OID for Crossbeam VAP group specific traps uses the trap prefix (.1.3.6.1.4.1.6848.4.3)

For example:

The VAP group name (cbsVgVapGroupName) OID is .1.3.6.1.4.1.6848.2.4.1.1.2

CBS-VAPGROUP-MIB entriesCBS-VAPGROUP-MIB management OID: .1.3.6.1.4.1.6848.2.3

1. cbsVapGroupTable

1. cbsVapGroupEntry

1. cbsVgVapGroupID

2. cbsVgVapGroupName

3. cbsVgLoadPriority

4. cbsVgPreemPriority

5. cbsVgApmList

6. cbsVgVapCount


7. cbsVgMaxLoadCount

8. cbsVgLBList

9. cbsVgBackUpMode

10. cbsVgIpForward

11. cbsVgRpFilter

12. cbsVgLogMartians

13. cbsVgReloadTimer

14. cbsVgRemoteBoxBackup

15. cbsVgDelayFlow

2. cbsVapMappingTable

1. cbsVapMappingEntry

1. cbsVmVapGroupID

2. cbsVmVapGroupName

3. cbsVmVapName

4. cbsVmVapIndex

5. cbsVmVapStatus

6. cbsVmSlotNumber

CBS-VAPGROUP-MIB Trap EntriesCBS-VAPGROUP-MIB Trap OID: .1.3.6.1.4.1.6848.4.3

1. cbsVapStatusChanged

CBS-VRRP-MIB ReferenceThe CBS-VRRP-MIB includes entries for virtual routing redundancy protocol (VRRP).

This section contains a tree view of the CBS-VAPGROUP-MIB entries and traps with OID information. For comprehensive attributes and text descriptions, use a MIB browser or read the CBS-VAPGROUP-MIB.txt file on your CPM at /usr/share/snmp/mibs.

Identifying the CBS-VRRP-MIB OIDsIdentify the OID for specific MIB entries by appending the ordered list number for the entry to the CBS-VRRP-MIB management OID (.1.3.6.1.4.1.6848.2.5). The OID for Crossbeam VRRP-specific traps uses the trap prefix (.1.3.6.1.4.1.6848.4.4)

For example:

The trap for a new master for VRRP (cbsVrrpTrapNewMaster) OID is .1.3.6.1.4.1.6848.4.4.1

CBS-VRRP-MIB entriesCBS-VRRP-MIB management OID: .1.3.6.1.4.1.6848.2.5

1. cbsVrrpOperations

1. cbsVrrpTrapNewMasterReason


2. cbsVrrpTrapNewMasterFailGrName

3. cbsVrrpTrapNewMasterVrId

4. cbsVrrpTrapNewMasterCirId

5. cbsVrrpTrapNewMasterCirName

6. cbsVrrpTrapProtoErrReason

7. cbsVrrpTrapFailGrOldStatus

8. cbsVrrpTrapFailGrNewStatus

9. cbsVrrpTrapFailGrStatusChgReason

10. cbsVrrpTrapFailGrOldPriority

11. cbsVrrpTrapFailGrNewPriority

12. cbsVrrpTrapFailGrPriorityChgReason

13. cbsVrrpTrapRemoteBoxPathOldStatus

14. cbsVrrpTrapRemoteBoxPathNewStatus

15. cbsVrrpRemBoxPathAddrs

2. cbsVrrpFailGrTable

1. cbsVrrpFailGrEntry

1. cbsVrrpFailGrID

2. cbsVrrpFailGrName

3. cbsVrrpFailGrStatus

4. cbsVrrpFailGrConfigPriority

5. cbsVrrpFailGrActualPriority

6. cbsVrrpFailGrEnabled

7. cbsVrrpFailGrPreemption

8. cbsVrrpFailGrLastChangeTime

9. cbsVrrpFailGrLastChangeReason

10. cbsVrrpFailGrMasterSysID

11. cbsVrrpFailGrMasterPriority

3. cbsVrrpFailGrCompTable

1. cbsVrrpFailGrCompEntry

1. cbsVrrpFailGroupID

2. cbsVrrpFailGroupName

3. cbsVrrpFailGrCompIndex

4. cbsVrrpFailGrCompType

5. cbsVrrpFailGrCompDescr

6. cbsVrrpFailGrCompDelta

4. cbsVrrpRemoteBoxPathTable

1. cbsVrrpRemoteBoxPathEntry

1. cbsVrrpRemoteBoxID

2. cbsVrrpRemoteBoxPathIndex

3. cbsVrrpRemoteBoxPathAddr


4. cbsVrrpRemoteBoxPathLocalIntf

5. cbsVrrpRemoteBoxPathLocalAddr

6. cbsVrrpRemoteBoxPathStatus

7. cbsVrrpRemoteBoxPathLastChangeTime

8. cbsVrrpRemoteBoxPathLinkQuality

CBS-VRRP-MIB Trap EntriesCBS-VRRP-MIB Trap OID: .1.3.6.1.4.1.6848.4.4

1. cbsVrrpTrapNewMaster

2. cbsVrrpTrapProtoError

3. cbsVrrpTrapFailGrStatusChanged

4. cbsVrrpTrapFailGrPriorityChanged

5. cbsVrrpTrapRemoteBoxPathStatusChanged

6. cbsVrrpTrapRemoteBoxNoActivePath

7. cbsVrrpTrapRemoteBoxNoSecondaryPath

8. cbsVrrpTrapRemoteBoxNoStandbyPath

9. cbsVrrpTrapRemoteBoxPathSharedIntf

10. cbsVrrpTrapNoRemoteBoxConfigured

11. cbsVrrpTrapOnlyOnePathDefined

12. cbsVrrpTrapRemoteBoxRunLegacyXOS

Standard MIBs on the Crossbeam X-Series PlatformIn addition to the Crossbeam proprietary MIBs, the X-Series Platform uses the following standard MIBs:

DISMAN-EVENT-MIB

IF-MIB

IP-FORWARD-MIB

IP-MIB

IPV6-MIB

NOTIFICATION-LOG-MIB

RFC1213-MIB

RMON-MIB

SNMPv2-MIB

TCP-MIB

UDP-MIB

To view specific MIB entries used by XOS, use an snmpwalk tool.

For information on configuring SNMP on XOS, including SNMP server setup, see Configuring Simple Network Management Protocol (SNMP) on page 163.


EMaintenance Mode

Using Maintentance ModeMaintenance mode can be applied to Application Processor Modules (APMs) and to Network Processor Modules (NPMs) for these purposes:

Upgrading the firmware on the module (APM and NPM)

Applying hotfixes or patches while upgrading an application (APM)

Replacing an existing APM or NPM module with a newer version

When peforming firmware upgrade or patch/hotfix operations, the process involves these steps:

Configuring the module to place it in maintenance mode

Performing the operation on the module

Enabling the module after the operation has been completed

When replacing an APM or NPM module with a newer version the process involves these steps:

Placing the module in maintenance mode

Removing the module

Inserting the newer version of APM or NPM


Maintenance Mode 298

Glossary

The following terms are used throughout the X-Series Platform documentation set.

3DES

Triple Data Encryption Standard. Provides a stronger form of DES encryption where the algorithm is applied three times in order to encrypt data.

ACL

Access Control List. Provides packet filtering through the permission or denial of packets based on certain IP criteria, such as IP address, port, or protocol.

APM

Application Processor Module. The XOS Application Processor system module that provides application processing, status monitoring, and standard and application specific logging. The APM contains one or more CPUs to host applications and network services while processing packets belonging to individual flows.

ARP

Address Resolution Protocol. An Internet protocol used to map an IP address to a MAC address.

BOTW

Bump-on-the-Wire. A device with two or more interfaces that are transparent to the adjacent Layer 3 devices.

cbsflowagentd

Flow Agent daemon that collects statistics and runs on each VAP.

cbsflowcalcd

Flow Calculator daemon that runs the flow scheduling chow file and executes on the CPM.

circuit

An abstract object representing a logical network interface (network service access point). A circuit can be mapped to either single or multiple logical lines. Attributes of a circuit include: a set of physical line or channel pairs, a Layer 2 encapsulation type, a Layer 2 address, and an IP address (optional).

CLI

Command Line Interface.

CM

Configuration manager/monitor.

core-intf

An interface which is attached to the core-facing networks.


CPM

Control Processor Module. The XOS system module that coordinates the actions of all other modules, enables management access to the platform, and supports access to a local disk containing configuration files and databases necessary to execute the applications which reside on the platform.

DES

Data Encryption Standard. A popular algorithm for encrypting data. It is a product cipher that operates on 64-bit blocks of data, using a 56-bit key.

device

OS concept representing either a physical or logical I/O port connected to the APM.

domain

A set of interconnected IP networks belonging to a unique address space. A domain is uniquely identified within the X-Series Platform by a 8-bit domain ID. IP flows must be unique within a given domain.

DSA

Digital Signature Algorithm.

ECC

Error Checking and Correcting. A collection of methods to detect errors in transmitted or stored data and a means to correct them.

edge interface

An interface that is attached to edge-facing networks (typically where subscribers are located).

edge server

A server that is physically located close to its end users designed to deliver faster, higher quality transmissions, typically in a local commercial ISP facility. The number of edge servers in a region depends on the number of users in the locale.

Element Management System

Graphical user interface for X-Series Platforms accessed via the web server.

Error Checking and Correcting (ECC)

A collection of methods for detecting and correcting errors in transmitted or stored data.

FCAPS

Faults, Configuration, Accounting, Performance, and Security. The general requirements of a network management system as defined by the International Organization for Standardization.

FIB

Forwarding Information Base. A set of IP data structures replacing a route table in Linux.

firewall

A set of software tools that protects a company's internal network from unwanted entry by unauthorized external users. The firewall works in conjunction with a router program to filter incoming network packets and reject those of unknown origin.

Glossary 300

flow

Specific stream of data traveling between two endpoints across a network. Specified by source IP, destination IP, source port, destination port and IP protocol type.

flow rule

A filter rule specifying how a packet is processed.

flow specific

A stream of data traveling between two endpoints across a network. Specified by source IP, destination IP, source port, destination port and IP protocol type.

flow table

A table maintained on the NPM that maps individual flows to their respective processors.

FPGA

Field Programmable Gate Array. A gate array where the logic network can be programmed into the device after its manufacture. An FPGA consists of an array of logic elements, either gates or lookup table RAMs, flip-flops, and programmable interconnect wiring.

FTP

File Transfer Protocol.

gateway

A Layer 3 devices with at least two logical interfaces, that uses a routing table to forward packets between interfaces. Note that a gateway may also act as a multi-homed host.

GBIC

Gigabit Interface Converter. A transceiver that converts electric currents (digital highs and lows) to optical signals, and optical signals to digital electric currents. The GBIC is typically employed in fiber optic and Ethernet systems as an interface for high-speed networking. The data transfer rate is one gigabit per second (1 Gbps) or more.

GEM

Greenlight Element Manager. A GUI that provides a view into the components and health of your X-Series Platform.

GLM

Gigabit Link Module.

GRUB

GRand Unified Bootloader.

hash

A crytographic operation where an entire message is run through a mathematical operation that results in a fixed-length string that is unique.

HTTP

Hypertext Transfer Protocol.


IDEA

International Data Encryption Algorithm. A conventional encryption algorithm, using block cipher, operating on 64-bit blocks with a 128 bit key.

IDS

Intrusion Detection System.

IOP

I/O Processor.

IP Address

Internet Protocol (IP). A numerical address that identifies senders and receivers of Internet data. The address accompanies data packets, identifying a particular network on the Internet and the specific device (such as a server) from which it originated.

IPS

Intrusion Prevention System.

In-Service Upgrade

ISU is an alternate method of upgrading XOS software while minimizing downtime. This feature has several requirements for successful completion of an ISU, including redundant CPMs, APMs, and NPMs. During ISU, the chassis is virtually split in two halves during which time only one half of the chassis will be responsible for forwarding traffic.

LACP

Link Aggregation Control Protocol.

load balancing

Distributing flows in real time amongst multiple APMs.

load table

A table that maps flow profiles to weighted lists of virtual processors.

logical interface

A channelized interface on a physical interface. A subdivision of a physical interface. Currently supported logical interface types are default and VLAN.

logical line

A combination of a physical line and a sub-line (channel). A logical line is uniquely identified by a physical line ID or channel ID pair.

MD5

Message Digest 5. A one-way function that takes a variable-length message and produces a fixed-length hash.

MAC address

Media Access Control (MAC). A hardware address that uniquely identifies each node of a network. In IEEE 802 networks, the Data Link Control (DLC) layer of the OSI Reference Model is divided into two sub-layers: the Logical Link Control (LLC) layer and the Media Access Control (MAC) layer. The MAC layer interfaces directly with the network media. Consequently, each different type of network media requires a different MAC layer.

Glossary 302

MIB

Management Information Base.

MS

Management Server.

multi-homed host

A Layer 3 device with at least two logical interfaces that generate packets but does not forward the packets.

NMS

Network Management System. Platform that manages one or more X-Series Platforms in a networked environment.

NPM

Network Processor Module. The XOS module responsible for network interface access (up to 10 Gb/sec full-duplex), flow classification, distribution of flows to APMs, and load balancing of the APMs.

PGP

Pretty Good Privacy. A high-security RSA public-key encryption application that enables files or messages to be exchanged with privacy and authentication.

physical interface

The physical hardware connector on the NPM or CPM representing a network interface port.

POS

Packet Over Sonet.

POST

Power On Self-Test.

PPP

Point to Point Protocol.

RAID

Redundant Array of Inexpensive/Independent Drives. A data storage scheme used to allow multiple drives to work as a single drive. RAID level 0 and level 1 are supported by Crossbeam Systems in our newer modules. RAID 0 writes data to whichever drive is currently free. This method is used for greater data speed efficiency (however, all drives in the RAID are needed to fully access all the data). RAID 1 writes identical data to all the drives in the RAID grouping. This method is used for greater data integrity.

RDRAM

Rambus Direct Random Access Memory.

RMON

Remote Network Monitoring.

RPM

Red Hat Package Manager.


SCP

Secure copy.

SMP

Symmetric Multi Processor.

SNMP

Simple Network Management Protocol. The Internet standard protocol developed to manage and monitor nodes on an IP network.

SSH

Secure Shell. A powerful authentication and encryption program replacing older and less secure tools like Telnet. SSH provides both authentication and encryption and is therefore the preferred method of network access. SSH allows a secure connection to be established between a client computer and a server host. The X-Series Platform provides SSH server, SSH client, and scp capability.

SSL

Secure Socket Layer.

Stateful Inspection (dynamic packet filtering)

A firewall architecture operating at the network layer. Unlike static packet filtering, which examines a packet based on the information in its header, stateful inspection examines not just the header information but also the contents of the packet up through the application layer in order to determine more about the packet than just information about its source and destination. A stateful inspection firewall also monitors the state of the connection and compiles the information in a state table. Because of this, filtering decisions are based not only on administrator-defined rules (as in static packet filtering) but also on context that has been established by prior packets that have passed through the firewall. As an added security measure against port scanning, stateful inspection firewalls close off ports until connection to the specific port is requested.

static route

A user-defined route that causes packets to move between a source and destination along a specific path.

sub-line

A multiplexed channel within a single line. Examples include: a DS0 channel within a T1/T3 serial interface, a ATM PVC, and a tagged VLAN. A sub-line is uniquely identified by a 32-bit channel ID.

SYSLOGD

System Logger Daemon.

Telnet

An administrator can enable Telnet as part of the boot dialogue, or by using a CLI command. Telnet comes disabled because traffic is not encrypted between the client and the X-Series Platform.

VAP

Virtual Application Processor. An application operating environment which can be run on an APM. A VAP consists of the OS, system software, and a set of applications which run concurrently.

VAP group

A virtual set of Application Processor Modules identically configured for load balancing and redundancy to process the same set of applications.

Glossary 304

VLAN

Virtual Local Area Network. A local area network with a definition that maps workstations on some other basis than geographic location (for example, by department, type of user, or primary application).

VND

Virtual Network Device. A Linux kernel object representing a logical network interface. A virtual network device is directly mapped to an NPM circuit.

VPN

Virtual Private Network. Consists of private lines, switching equipment and other networking equipment that are provided for the exclusive use of one customer. A VPN gives users a secure way to access resources over the Internet or other public or private networks using encryption, authentication, and tunneling.

VRRP

Virtual Router Redundancy Protocol. This protocol allows several routers on a multiaccess link to utilize the same virtual IP address. One router will be elected as a master with the other routers acting as backups in case of the failure of the master router. The protocol should also support the ability to load share traffic when both routers are up.

A Virtual Router in XOS is an IP address or a set of IP addresses that can be instantiated on a circuit for a subset of the VAP groups on which the circuit is configured, and active only on one of the X-Series Platforms participating in multi-system High Availability configuration.

XML

Extensible Markup Language. The universal format for structured documents and data on the Web as defined by a set of specifications and recommendations from the W3C.


Glossary 306

Documents

XOS 9.5.4.0 Xos Configuration Guide