VIA User's Guide - VMware Cloud Foundation 2 · PDF fileCloud Foundation Deployment Options 8 ... The VMware Cloud Foundation Overview and ... vRealize Suite into a natively integrated

VIA User's GuideVMware Cloud Foundation 2.3

VIA User's Guide

VMware, Inc. 2

You can find the most up-to-date technical documentation on the VMware website at:

https://docs.vmware.com/

If you have comments about this documentation, submit your feedback to

[email protected]

Copyright © 2015 – 2017 VMware, Inc. All rights reserved. Copyright and trademark information.

VMware, Inc.3401 Hillview Ave.Palo Alto, CA 94304www.vmware.com

https://docs.vmware.com/

mailto:[email protected]

http://pubs.vmware.com/copyright-trademark.html

Contents

About the VIA User's Guide 5

1 Overview 6

Cloud Foundation Deployment Options 7

Software Bundle 7

VIA Components 8

Components of a Physical Rack 9

2 Requirements for VIA 12

3 Setting up your Environment for Imaging 13

Rack Power 13

Network Cables 14

Rack Wiring 16

Rack Component Ports 26

Physical Servers 29

Network Switches 31

Laptop or Management Host 32

Virtual Machines 34

4 Pre-Imaging Checklist 36

5 Installing VIA 38

Installing VIA on a Laptop or Desktop 38

Installing VIA on a Management Host 39

6 Imaging Physical Racks 42

Image a Physical Rack 43

Resume Imaging 55

Image Additional Racks 57

7 Imaging Individual Devices 58

Image Individual Server 58

Image New Management Switch 60

8 Review Alarms and Notifications 61

9 Viewing the VIA Log File 62

VMware, Inc. 3

10 Viewing Results of an Imaging Run 63

View Imaging History 63

View Inventory 64

11 Guidance on Server OOB Port Management 66

OOB Ports Wiring to Management Switch 66

OOB Ports Configuration 66

12 Troubleshooting VIA 67

ESXi Server has Incorrect BIOS Settings 67

ESXi Server has Bad SD Card 67

Management Switch Boots into EFI Shell 68

A switch did not start imaging 68

One or More Servers Are Not Discovered 68

ServerImaging Failed at Kickstart Delivery Task 69

Server Imaging Failed at Check Host StatusTask 70

13 Cloud Foundation Glossary 71

VIA User's Guide

VMware, Inc. 4

About the VIA User's Guide

The VIA User's Guide provides information about how to install VIA, manage software bundles, andimage physical racks.

Intended AudienceThis information is intended for anyone who wants to install or upgrade VIA and image physical racks.The information is written for experienced Windows or Linux system administrators who are familiar withvirtual machine technology and datacenter operations.

Related PublicationsThe VMware Cloud Foundation Overview and Bring-Up Guide contains detailed information about theCloud Foundation product, its components, and the network topology of an Cloud Foundation installation.

The Administering VMware Cloud Foundation provides information about how to manage a VMwareCloud Foundation™ system, including managing the system's physical and logical resources, managingusers, configuring and deploying service offerings, and upgrading and monitoring the system.

VMware Technical Publications GlossaryVMware Technical Publications provides a glossary of terms that might be unfamiliar to you. Fordefinitions of terms as they are used in VMware technical documentation, go to http://www.vmware.com/support/pubs.

VMware, Inc. 5

http://www.vmware.com/support/pubs

Overview 1VMware Cloud Foundation is the unified SDDC platform that brings together vSphere, vSAN, NSX, andvRealize Suite into a natively integrated stack to deliver enterprise-ready cloud infrastructure for theprivate cloud.

The first step in deploying the Cloud Foundation solution is to image your hardware using VIA. VIA is avirtual appliance that installs VMware software on the physical racks in your datacenter using a collectionof HTTP services.

A physical rack consists of a management switch, two Top of Rack (ToR) switches, and 4 to 32 physicalservers. If you have multiple physical racks in your datacenter, the second rack must contain two inter-rack switches for inter-rack connectivity.

For imaging the rack, VMware provides the VIA OVF template and the Cloud Foundation softwarebundle. The software bundle contains the SDDC components.

The imaging infrastructure at the customer or partner site includes a laptop, desktop, or ESXi host(referred to as the management host) and a managed switch. The VIA OVA template is deployed on thelaptop or management host and the software bundle is uploaded on the VIA VM. The laptop ormanagement host is connected to the corporate network and to the private network used by the VIA VM.You use a browser (on the laptop) or jump VM (on the management host) to connect to the VIA VM andimage the physical rack.

Figure 1‑1. VIA Deployment

During imaging,

n The management, ToR, and inter-rack switches (if applicable) are configured.

n VMware ESXi is installed on each server in the physical rack.

n The Cloud Foundation software bundle, SDDC Manager Controller VM, and SDDC Manager UtilityVM, are installed on the primary host.

VIA runs an internal DHCP service to allocate private IP addresses to the switches and servers duringimaging. After imaging is complete, VIA compiles a manifest file that provides an inventory of the physicalrack components. The rack is now ready to be configured for Cloud Foundation, a process that is calledbring-up. For more information on bring-up, see VMware Cloud Foundation Overview and Bring-upGuide.

VMware, Inc. 6

This chapter includes the following topics:

n Cloud Foundation Deployment Options

n Software Bundle

n VIA Components

n Components of a Physical Rack

Cloud Foundation Deployment OptionsCloud Foundation provides flexibility in choosing on-premises deployment options.

Customers begin by sizing their Cloud Foundation deployment to determine the number of physicalservers in their rack and number of racks. Each rack requires a minimum of 4 servers.

Customers have two deployment options for Cloud Foundation

n Deploy the Cloud Foundation software on qualified vSAN ReadyNodes in your datacenter.

Customers can start with qualified hardware (qualified ReadyNodes and qualified switches) in theirdatacenter, wire it up, and deploy the Cloud Foundation software stack on the ready system. Forinformation on qualified hardware, see VMware Compatibility Guide.

n Purchase a fully integrated system that combines software and hardware from select VMwarepartners.

The partner works with a VMware representative to complete the Site Readiness document. Thistranslates into a bill of materials (BoM) consisting of both hardware and software components. Withthis BoM in hand, the partner assembles the rack and images it. The partner then ships the system,consisting of physical racks, servers, server sub-components, power distribution units, switchinginfrastructure and the Cloud Foundation software, to customers.

Deploying VMware Cloud Foundation on Qualified Hardware(http://link.brightcove.com/services/player/bcpid2296383276001?bctid=ref:video_deploy_cloud_foundation_hardware)

Software BundleThe software bundle is a collection of all the software, configuration files, utilities, and tools used by VIAto image a physical rack. It contains a manifest file that lists the contents of the bundle. The bundle isbased on a hardware bill-of-materials (BoM), that includes specific servers, switch models, and theircomponent level configurations.

In addition to the core bundle, you can upload 3rd party VIBs, and custom ESXi installer ISOs on VIA.

The core software bundle contains the following software:

n vSphere (vCenter Server and ESXi)

n NSX

n vSAN

VIA User's Guide

VMware, Inc. 7

https://www.vmware.com/resources/compatibility/vcl/cloudfoundation.php

http://link.brightcove.com/services/player/bcpid2296383276001?bctid=ref:video_deploy_cloud_foundation_hardware

http://link.brightcove.com/services/player/bcpid2296383276001?bctid=ref:video_deploy_cloud_foundation_hardware

n vRealize Log Insight

n SDDC Manager

n Platform Services Controller

n VMware Horizon

n App Volumes

n vRealize Operations

n vRealize Automation

See the VMware Cloud Foundation Release Notes for the software component versions.

VIA ComponentsVIA uses multiple components to track and perform the imaging process. This section describes thesecomponents, but you do not need to perform any configuration on them.

DatabaseVIA stores information about all activities during an imaging run in an MongoDb database. This includescurrent imaging information as well as the previous imaging status. All entities utilized by the imagingprocess are stored as an entry in the database. These entities include the software bundle, imagedcomponent, manifests, user information, and hardware information.

ServicesIn order to handle disparate requests that may be required to service its components, VIA deploysmultiple services. Each service has a specific goal, and is instantiated based on the state of the imagingactivity.

Switch ServiceA Switch Service is developed for each switch vendor by using an imaging developer kit. The developerkit provides the imaging orchestration engine, API, data models, and extension points to create imagingtasks (PRE, INTRA, POST, INVENTORY), custom controller, and automated integration tests. VIA loadsthese services on demand and discovers the switch type (management, ToR, inter-rack) supported bythese services.

ESXi ServiceThe ESXi Service images the servers in the physical rack. It uses the same developer kit as the SwitchService.

VIA User's Guide

VMware, Inc. 8

Core Platform ServiceCore Platform is the main VIA web service which supports external facing API, DHCP, bundlemanagement, imaging workflow orchestration, and UI. This service automatically loads the Switch andESXi services on demand based on the BOM and bundle and then orchestrates the imaging workflow.This service can image an entire rack as well as individual devices.

Components of a Physical RackVMware recommends that you use a white cabinet that is 19" wide with 42 Rack Units (RU) for thephysical rack. The cabinet must have a loading capacity of 2000 lbs and have adjustable levelling feetwith heavy duty casters and seismic bracing. Since switches do not cover the full shelves, the cabinetmust have a grill on one side for proper airflow.

Table 1‑1. Rack Components

Component Rack 1 Additional Racks

PDUs 4 4

Console serialswitch

1 1

Inter-rackswitches

NA 2 (Rack 2 only)

TOR Switches 2 2

Managementswitch

1 1

Servers Up to 32 Up to 32

n PDUs

Each physical rack must have 4 PDUs (2 primary and 2 standby) even if it contains less than 32servers. It is recommended that the primary PDUs be blue and the standby be red. The primaryPDUs must be placed on the rear left side and the standby PDUs must be placed on the rear rightside of the cabinet. The capacity requirements for each PDU are:

n 208 V

n 30 AMP

n 3 phase

n 60 Hz/50 H

The plug type needs to be determined based on the customer's environment.

n Console serial switch

Each physical rack contains a 16-port console serial switch. The console serial switch is connected toall the other switches in the rack and is used for troubleshooting.

n Inter-rack switches

VIA User's Guide

VMware, Inc. 9

Rack 2 in your Cloud Foundation system contains two 32 x 40 GE inter-rack switches. Theseswitches connect multiple racks by using uplinks from the Top of Rack switches.

Inter-rack switches must not be connected to a corporate network. They are only used for ToRconnectivity between physical racks.

n Top of Rack (ToR) switches

Each rack contains two 1RU 48-port 10 GE ToR switches with four 40 GE uplinks. Servers in eachrack are connected to both ToRs. The ToRs on rack 1 connect Cloud Foundation to the corporatenetwork.

n Management switch

Each rack contains a 1 GE management switch, which is used to access the physical switches. Themanagement switch is connected to the management ports of the ToR and inter-rack switches (onRack 2 only). The management switch is also connected to the data ports of the ToR switches. VIAuses the management port connection to the ToR switch for imaging and configuration of themanagement switch, and the data port connection for imaging and configuration of the other switchesand servers.

n Servers

A Cloud Foundation rack can contain 4 to 32 heterogeneous servers. Servers must be from the samevendor, but can be of different models and sizes with variable CPU, memory, storage size and type,and disk configurations (e.g. hybrid or all flash).

You can select the servers you want to use for the management domain as well as workloaddomains. This gives you the flexibility to use say hybrid servers for the management domain and allflash servers for workload domains.

Table 1‑2. Server Configuration for Cloud Foundation

Component Minimum Maximum

CPU perserver

Dual-socket, 8 cores per socket Dual-socket, no maximum on cores per socket

Memory perserver

256 GB 1.5 TB

Storage perserver

4 TB for capacity tier. Follow vSAN guidelines forcache tier sizing as described in VMware vSAN Designand Sizing Guide.

Each server to be used in the domain must contain atleast 3 capacity tier disks.

Note Cloud Foundation only supports vSAN RAIDcontrollers in pass-through mode.

8 disks per disk-group and 2 disk-groups per host.Follow vSAN guidelines for cache tier sizing asdescribed in VMware vSAN Design and SizingGuide.

NICs perserver

Two 10 GbE NICs and one 1 GbE BMC NIC Two 10 GbE NICs and one 1 GbE BMC NIC

VIA User's Guide

VMware, Inc. 10

https://storagehub.vmware.com/#!/vmware-vsan/vmware-r-vsan-tm-design-and-sizing-guide

https://storagehub.vmware.com/#!/vmware-vsan/vmware-r-vsan-tm-design-and-sizing-guide

https://storagehub.vmware.com/#!/vmware-vsan/vmware-r-virtual-san-tm-design-and-sizing-guide

https://storagehub.vmware.com/#!/vmware-vsan/vmware-r-virtual-san-tm-design-and-sizing-guide

Table 1‑2. Server Configuration for Cloud Foundation (Continued)

Component Minimum Maximum

Servers perrack

Four 1U or 2U servers

A minimum of 7 servers are required for VI and VDIworkload creation.

32 1U servers or 16 2U servers

Rack 1 8

Figure 1‑2. Example Physical Rack Configuration

42 U

Back

42 U

Front

IPMIDATA

Server 24

Server 1Server 2Server 3Server 4Server 5Server 6Server 7Server 8Server 9

Server 10Server 11Server 12Server 13Server 14Server 15Server 16Server 17Server 18Server 19Server 20Server 21Server 22Server 23

Inter-rack 2

Inter-rack 1

ToR 2

ToR 1

ManagementSwitch

Serial ConsoleSwitch

Server 25

Server 28

Server 26Server 27

Server 29Server 30Server 31Server 32

VIA User's Guide

VMware, Inc. 11

Requirements for VIA 2VIA requires the following infrastructure.

n You need a laptop/desktop or an ESXi host (called management host) to run the VIA VM.

The laptop need to be connected to the management switch of the rack being imaged. If there aremultiple racks to be imaged, this would mean physically moving the laptop for imaging each rack. Themanagement host is connected to all the management switches through an internal switch, so theconnection to the rack being imaged can be managed remotely.

n Desktop or laptop (Windows or Mac) with 4 GB memory and a multi core processor to access thejump VM. A Windows laptop must have Workstation 8 or later and a Mac should have Fusion 4 orlater installed on it. You also need a network adapter, a cable, and a minimum 4-port unmanagedswitch.

n Management host - a standalone VMware vSphere ESXi 6.0 or later server to host the Windowsjump VM. The management host must have at least two NICs, with one NIC connected to thecorporate network and one NIC connected to the private network.

n If you are using a management host for imaging, you need a jump VM to access VIA

n Supported 1GE private managed switch with RJ45 ports and Cat 5/5E cables and at least 3 ports.Private indicates that it is being used only by you. The managed switch provides the ability toconfigure, manage, and monitor your LAN, which gives you greater control over how data travels overthe network and who has access to it.

VMware, Inc. 12

Setting up your Environment forImaging 3You must inspect the components of the physical rack, verify cable connectivity, and validate BIOSsettings before beginning the imaging process.

If you are imaging your own rack, refer to the VMware Compatibility Guide to ensure that the hardware inyour datacenter is supported for Cloud Foundation.

If you are a partner imaging a rack for your customer, review the VMware Bill Of Materials (BOM) beforeyou set up your environment for imaging.


n Rack Power

n Network Cables

n Rack Wiring

n Rack Component Ports

n Physical Servers

n Network Switches

n Laptop or Management Host

n Virtual Machines

Rack PowerEnsure that rack power meets the following requirements.

n Verify that each device in the rack has a connection to each PDU.

n VMware recommends that you cable each server to the nearest power port so that the cable lengthcan be kept to a minimum. Length of power cables should be as follows.

n From the Physical Server: (9) .5m (3) 1m

n From the Top-of-Rack Switch: 1.5m

n From the Inter-rack Switch: .5m

It is common for power cables within a rack to be longer than required. However, if excess cabling isnot managed properly, it may create electromagnetic interference. Avoid bundling of excess cables asthis may lead to the cables being damaged due to bending.

VMware, Inc. 13

https://www.vmware.com/resources/compatibility/vcl/cloudfoundation.php

n The power connector from the PDU must match the power connector in the Site ReadinessAssessment.

n Power cables must be seated properly from each device to the PDU.

n The cables connect the primary PDUs to the other components must be blue and the cables from thesecondary PDUs must be red.

n Power cables should not be in an area where there is a risk of touching sharp edges, excessive heat,or subject to pinching between sliding rails.

Network CablesProper management of network cables promotes the elimination of crosstalk and interference, coolerperformance, improved maintenance, and easier upgrades. Incorrect cable management may result indamage or failure, which may lead to data transmission errors, performance issues, or system downtime.This section contains cable color and management recommendations. You can adapt therecommendations to suit your environment.

Regardless of the number of servers in each rack, cables must be in place for 32 servers. Ideally, dataand power cables must be at opposite ends of the physical rack. If they are aggregated in a bundle or runparallel to each other, induction may introduce electromagnetic interference.

Cable ColorsUsing specific colors for cables from each device makes for easier troubleshooting.

n All cables from the management switch (except those going to the ToRs): yellow

n Management switch ports 49 and 50 going to the ToRs: black

n ToR 1 cables to servers: blue

n ToR 2 cables to servers: red

n ToR 1 and ToR 2 connections to inter-rack switches: orange

n Console serial switch connections: grey

Cable Type and LengthThe Telecommunications Industry Association (TIA) and the Electronic Industries association (EIA)structured cabling standards define how to design, build, and manage cabling systems. The specificationis TIA/EIA-568-A. When used for 10/100/1000BASE-T Category 6 (Cat 6) cable length can be up to 100meters (328 ft). This distance includes up to 90 meters (295 ft) of horizontal cabling between the patchpanel and the wall jack, and up to 10 meters (33 ft) of patch cabling. When used for 10GBASE-T, Cat 6cable length is reduced to 55 meters (180 ft) assuming minimal exposure to crosstalk. Category 6A (Cat6A) does not have this limitation and can run at the same distances as 10/100/1000BASE-T.

VIA User's Guide

VMware, Inc. 14

Cable Bend RadiusModifying the geometry of a cable can impair data transmission and affect performance. When a cable istied or tightly looped, the pairs within the cable jacket can be separated impacting the integrity of thecable. Therefore, bend radius should be considered when verifying cable management.

n The minimum bend radius of a twisted pair patch cable is 4x the external cable diameter, and theminimum bend radius of an LC-type fiber optic cable is 0.8" (~2cm) and SC-type fiber optic cable is 1"(~3cm).

n Where articulated arms or rail slides are used, there must be sufficient slack in the cable to allowoperation.

n No creases in the sheathing should be visible on any cable.

Cable RoutingImproperly routed cables can contribute to thermal issues, make field replaceable units difficult to access,or impact performance.

Cable ties can damage cables due to excessive over tightening or by violating the bend radius of a cable.Cable ties also increase service time when an add, move, or change request is received. Cables shouldbe bundled with Velcro straps where possible to avoid damage, simplify addition or removal of cables,and reduce service times.

n Use velcro straps instead of cable ties.

n Network cables should not be in an area where there is a chance of contacting sharp edges,excessive heat, or subject to pinching between sliding rails.

n Cables must be free of tension. Where articulated arms or rail slides are used, there must besufficient slack in the cable to prevent the cables from being stressed.

n Forced air cooling is recommended to draw cool air from the front of the rack and push warm air outthe back.

n Ventilation slots, power supplies, and rear fans must be clear of cable obstructions.

n Field replaceable units such as power supplies must be clear of any cable obstructions that mayprevent access for service.

Cable LabelingPartners must label the cables in their datacenter. Properly labeled cables reduce troubleshooting timesince it is easier to trace and validate connections.

Cable TestingCable testing ensures that the installed cabling links provide the transmission capability to support thedata communication required.

VIA User's Guide

VMware, Inc. 15

Several tools are available for copper testing. Tests fall into three categories: Verification, Qualification,and Certification. Verification tools are used to perform basic continuity, cable length, and openconnection verification. Qualification tools can provide information that details the cable capabilities, e.g.supports 10GBase-T. Certification tools determine whether the cable meets TIA standards such asTIA-568-B.

Options for testing SFP+ and QSFP+ cables are limited. Because handheld cable testers are notavailable, many network administrators typically reserve ports between two adjacent switches, thenconnect a suspect cable between active ports to determine if the cable is functional.

n Review the test print out to confirm that the cables passed the test.

n Cables from the physical server 10G interface to the ToR switches must be tested prior to installation.They must be seated properly.

n Each 10G interface must be connected to a separate ToR switch.

n Inter-switch SFP+ and QSFP+ cables must be tested prior to installation.

n Each 40G QSFP+ cable from the ToR switch must be connected to a separate inter-rack switch.

n There must be two 40G QSFP+ cable connections between each ToR switch and inter-rack switch.

n Inter-switch SFP+ and QSFP+ cables must be seated properly.

Rack WiringDownload VCF Wiremap from the Product Downloads page and connect the wires in your physical rackaccording to the wiremap. This section contains the logical views of the wiremaps.

VIA User's Guide

VMware, Inc. 16

High Level Wiring for Rack with Dell Management SwitchFigure 3‑1. Wiremap for rack 1 with Cisco ToR Switches and Dell Management Switch

1

1

Inter-rack 2on Rack 2

ToR 2

Managementswitch

ToR 1

For port connections, see Rack Component Ports tables.

Management port at the back



Corporate network


Corporate network

2

Server 1

Server 2 Server 12 Server 24

39

Server 1


Server 32

Server 32

Managementport

Jump host

Private managed switch

Server 13

Server 13

41

32

31

32

31

50

49

50

49

48

484240

2

Inter-rack 1on Rack 2

49

48 42

41

44

43

50

VIA User's Guide

VMware, Inc. 17

Figure 3‑2. Wiremap for rack 2 with Cisco ToR Switches and Dell Management Switch

3

Inter-rack 2

ToR 2

Management switch

Inter-rack 1

ToR 1






Corporate network

Corporate network

31

31

32

32

Server 1


39

48

Server 1


Server 32

Server 32

48

Management port

Jump host


Server 13

Server 13 39

40

40

41

41

42

42

3

4

4

49

48 42

41

44

43

50

VIA User's Guide

VMware, Inc. 18

High Level Wiring for Rack with Quanta Management SwitchFigure 3‑3. Wiremap for rack 1 with Cisco ToR Switches and Quanta Management Switch

ToR 2

Management switch

ToR 1


Management port

Corporate network

Management port

1

2

Management port

Server 1

Server 2 Server 12 Server 24 Server 32

Server 32

39

Management port

Server 1


Management port

Jump host


Corporate network

Server 13

Server 13

41

32

31

32

31

50

49

50

49

48

484240

12

Inter-rack 2 on Rack 2

Inter-rack1 on Rack 2

49

4842

41

44

43

50

VIA User's Guide

VMware, Inc. 19

Figure 3‑4. Wiremap for rack 2 with Cisco ToR Switches and Quanta Management Switch

Inter-rack 2

ToR 2

Management switch

Inter-rack 1

ToR 1


Management port

Corporate network

Management port

3

4

Management port

Server1


39

48

Management port

Server 1


Server 32

Server 32

Management port

Jump host


Corporate network

Server 13

Server 13

41

42

32

31

32

31

50

49

50

49

40

48

3

4

49

48 42

41

44

43

50

High level Wiring for Additional RacksRack 2 in the integrated system powered by Cloud Foundation must include two inter-rack switches forinter-rack connectivity. The inter-rack switches are connected during the physical environment inspection,but must be disconnected before imaging the rack.

Additional physical racks do not contain inter-rack switches. ToR switches in the additional physical racksare connected to the two inter-rack switches in rack 2.

VIA User's Guide

VMware, Inc. 20

Detailed Wiring for Rack ComponentsThis section shows the wiring for each component in a physical rack. These are example sketches. Portplacement and numbering on components in your environment may be slightly different.

Figure 3‑5. Management SwitchManagement switch

front view

Management switchback view

01 03 05 07 09 11 1302 04 06 08 10 12 14

1516

17 19 21 23 25 27 2918 20 22 24 26 28 30

3132

33 35 37 39 41 43 45 49

50

51

5234 36 38 40 42 44 464748

Management port

ToR 1 port 48

ToR 2 port 48


ToR 2 management port

Inter-rack 2 management port

Inter-rack 1 management port


ToR 1 management port

VIA User's Guide

VMware, Inc. 21

Figure 3‑6. Inter-rack Switch 2Inter-rack 2front view

Rack 1 ToR 1 port 50
















Management portInter-rack 2back view

01 03 05 07 09 11 13 15 17 19 21 23 25 27 29 31

02 04 06 08 10 12 14 16 18 20 22 24 26 28 30 32

VIA User's Guide

VMware, Inc. 22

Figure 3‑7. Inter-rack Switch 1

Inter-rack 1front view

















Management portInter-rack 1back view

01 03 05 07 09 11 13 15 17 19 21 23 25 27 29 31

02 04 06 08 10 12 14 16 18 20 22 24 26 28 30 32

VIA User's Guide

VMware, Inc. 23

Figure 3‑8. ToR 2

ToR 2front view

ToR 2back view

01 03 05 07 09 11 1302 04 06 08 10 12 14

1516

17 19 21 23 25 27 2918 20 22 24 26 28 30

3132

33 35 37 39 41 43 45 49

50

51

52

53

5434 36 38 40 42 44 4647

Server 1

48

Server 2

Server 4

Server 6

Server 8

Server 10

Server 12

Server 14

Server 16

Server 3

Server 5

Server 7

Server 9

Server 11

Server 13

Server 15 Server 17

Management port

Server 18

Server 20

Server 22

Server 24

Server 26

Server 28

Server 30

Server 19

Server 21

Server 23

Server 25

Server 27

Server 29

Server 31

Server 32

ToR 1 port 39

ToR 1 port 40

ToR 1 port 42

ToR 1 port 41

Corporate network

reservedCorporate network

Corporate network

Inter-rack 1port 2

Inter-rack 2port 2

Mgmtswitch port 50

Corporate network

Management switch port 42

VIA User's Guide

VMware, Inc. 24

Figure 3‑9. ToR 1

ToR 1front view

ToR 1back view

01

01 03 05 07 09 11 1302 04 06 08 10 12 14

1516

17 19 21 23 25 27 2918 20 22 24 26 28 30

3132

33 35 37 39 41 43 45 49

50

51

52

53

5434 36 38 40 42 44 464748

Server 1

Server 2

Server 4

Server 6

Server 8

Server 10

Server 12

Server 14

Server 16

Management port

Server 3

Server 5

Server 7

Server 9

Server 11

Server 13

Server 15 Server 17

Server 18

Server 20

Server 22

Server 24

Server 26

Server 28

Server 30

Server 32

Server 19

Server 21

Server 23

Server 25

Server 27

Server 29

Server 31

ToR 2 port 39

ToR 2 port 40

ToR 2 port 42

ToR 2 port 41

Corporate network

Corporate network

Corporate network

Inter-rack 1port 1

Inter-rack 2port 1

Mgmtswitch port 49

Corporate network

Management switch port 41

reserved

VIA User's Guide

VMware, Inc. 25

Figure 3‑10. Servers

Server 1

ToR 1 port 1 ToR 2 port 1






Server 2

Server 3

Server 4

Server 5

Server 6


Server 32

Rack Component PortsRefer to the tables below for port connectivity information using Cisco 9372PX as the illustrative example.Connections in your environment may vary based on the actual switches being used.

Console Serial Switch

Port Number Connects To

1 Management switch console port

2 ToR 1 console port7

3 ToR 2 console port

4 Inter-rack 1 console port

5 Inter-rack 2 console port

6 PDU 1

7 PDU 2

8 PDU 3

9 PDU 4

10 - 16 Not connected

VIA User's Guide

VMware, Inc. 26

Inter-rack 2 (Rack 2 only)

Port Number Speed Connects To

1 40 Gbps Rack 1 ToR 1 port 50
















Inter-rack 1 (Rack 2 only)















VIA User's Guide

VMware, Inc. 27





ToR 2 (e.g. Cisco 9372PX)


1 - 32 10 Gbps node 1 - node 32

where port 1 connects to node 1, port 2 connects to node 2, and so on

33 - 38 NA Not connected

39 - 42 10 Gbps ToR 1 ports 39 - 42

43 - 46 10 Gbps Corporate network as required (see note below table)

47 Blank

48 1Gbps Management switch port 50

49 40 Gbps Inter-rack 1 port 2



Management 1 Gbps Management switch port 42

Note Depending on the switches in your environment, connect two 40 Gbps ports or multiple 10 Gbpsports to your corporate network.

ToR 1 (e.g. Cisco 9372PX)


1 - 32 10 Gbps Node 1 - node 32

where port 1 connects to node 1, port 2 connects to node 2, and so on


39 - 42 10 Gbps ToR 2 ports 39 - 42


47 Blank

48 1Gbps Management switch port 49



VIA User's Guide

VMware, Inc. 28



Management 1 Gbps Management switch port 41

Note Depending on the switches in your environment, connect two 40 Gbps ports or multiple 10 Gbpsports to your corporate network.

Management Switch


1 - 32 1 Gbps Ports 1 - 32 are blank.

You can connect these ports to the BMC ports on servers. This is optional and is notrequired for imaging. See Chapter 11 Guidance on Server OOB Port Management.


41 1Gbps ToR 1 management port

42 1Gbps ToR 2 management port

43 1Gbps Inter-rack 1 management port

44 1Gbps Inter-rack 2 management port

45-47 NA Not connected

48 1Gbps Private managed switch

49 10 Gbps ToR 1 port 48

50 10 Gbps ToR 2 port 48

51-52 NA Not connected

Management port Private managed switch

Note PDU ports are not reflected in the table above.

Physical ServersThis section lists the recommended Rack Unit (RU) location of each device.

Hardware DevicesTable 3‑1. Physical Device Location in Rack 1 and Rack 3 - Rack n

RU Location Device

42 Console serial switch

41 Blank

40 Management switch

39 Blank

VIA User's Guide

VMware, Inc. 29

Table 3‑1. Physical Device Location in Rack 1 and Rack 3 - Rack n (Continued)

RU Location Device

38 ToR 2

37 Blank

36 ToR 1

33-35 Blank

1-32 Nodes 1-32

Table 3‑2. Physical Device Location in Rack 2

RU Location Device

42 Console serial switch

41 Management switch

40 Inter-rack 2

39 Blank

38 Inter-rack 1

37 Blank

36 ToR 2

35 Blank

34 ToR1

33 Blank

1-32 Nodes 1-32

PowerAll servers must have redundant power supplies and each power supply must be connected to a separaterack PDU.

AirflowInstall the servers to allow front-to-back airflow.

Firmware SettingsEnsure that the firmware settings are set correctly as per the BoM.

BIOS SettingsThe BIOS settings for each server in the physical rack must match the values given below. It is a goodpractice to set the values in the order specified below to avoid any imaging failures.

VIA User's Guide

VMware, Inc. 30

If BIOS changes are needed to set the boot device, disable all NICs on each server except for themanagement port and dual port card. If the server can boot from the connected NIC cards, you do notneed to disable the additional NICs.

For information on how to set the required BIOS values, refer to the vendor documentation.

Setting Value

CPU and Performance Settings Set per ESXi recommendations. See VMware vSphere documentation.

Onboard SATA controller

Required only for servers with aSATADOM or M2 as boot disk

AHCI mode

Boot Order Network First

Second in Boot Order Boot Disk

Boot mode Legacy or UEFI

Storage Controller Passthrough Mode

Reboot the servers after changing the BIOS settings to ensure that the changes are in place.

Power Cycle RequirementsVIA does not reboot servers during the imaging process. You must manually reboot the servers when theVIA UI displays a notification asking you to do so.

You can assign OOB addresses to the servers in your rack. For more information, see Chapter 11Guidance on Server OOB Port Management. Or you can manage the IP assignment in other ways.

Network SwitchesVerify that SFP+ connectors are supported by the switches in your environment.

Powern All switches must have redundant power supplies.

n Each power supply must be connected to a separate rack PDU.

AirflowSwitches must be installed to allow front-to-back airflow.

Switch ModeEnsure that the mode for each switch is set such that it is ready to accept a new image and configuration.

VIA User's Guide

VMware, Inc. 31

Table 3‑3. Required Mode for Supported Vendors

Vendor Mode

Cisco ToR and inter-rack switches POAP

Recommended NXOS version is 7.0.3.I7.1.

Arista ToR and inter-rack switches ZTP

Dell management switch ONIE

Recommended version is 3.24.1.2.

Laptop or Management HostYou need a laptop or management host where you install VIA.

LaptopYou need a desktop or laptop (Windows or Mac) with 4 GB memory and a multi core processor to accessthe jump VM. A Windows laptop must have Workstation 8 or later and a Mac should have Fusion 4 orlater installed on it. You also need a network adapter, a cable, and a 4-port unmanaged switch.

Management HostIf you are using a management host to image the rack, you need a standalone VMware vSphere ESXi 6.0or later server to host the Windows jump VM. The management host must have at least two NICs, withone NIC connected to the corporate network and one NIC connected to the private network. You alsoneed a 24-port private managed switch.

Table 3‑4. VLAN Configuration of the Private Managed Switch

Port Access Ports

1,2,3,4 VLAN 2000

5,6,7,8 VLAN 2001

9.10.11.12 VLAN 2002

13,14,15,16 VLAN 2003

VIA User's Guide

VMware, Inc. 32

Table 3‑4. VLAN Configuration of the Private Managed Switch (Continued)

Port Access Ports

17,18,19,20 VLAN 2004

21,22,23,24 VLAN 2005

Figure 3‑11. Management Host Connection

Private Managed SwitchIf this is a multi-rack scenario and the private switch is being shared between racks, configure a privateVLAN. For example, create two VLANs in a dual rack environment - VLAN 101 and VLAN 102. VLAN 101is for rack 1 and VLAN 102 is for rack 2. Port 48 and the management port from the imaging managementswitch in rack 1 are connected to ports 2 and 3 on the private switch which is configured for VLAN 101.Port 48 and the management port from the imaging management switch in rack 2 are connected to ports4 and 5 on the private switch which is configured for VLAN 102. The imaging management host isconnected to Port 1 which is configured for both VLAN 101 and VLAN 102.

VIA User's Guide

VMware, Inc. 33

A print out of the VLAN configuration on the imaging management switch should look like this:

interface Vlan 1

!untagged GigabitEthernet 0/0-1,6-47

!untagged TenGigabitEthernet 0/48-49

!untagged Port-channel 1-2

!

interface Vlan 2001

no ip address

tagged TenGigabitEthernet 0/48-49

untagged GigabitEthernet 0/2-3

no shutdown

!

interface Vlan 2002

no ip address

tagged TenGigabitEthernet 0/48-49

untagged GigabitEthernet 0/4-5

no shutdown

Management Host SettingsConfigure the following settings on the imaging management host:

n Install ESXi on the local disk. For the supported version, see the VMware Cloud Foundation ReleaseNotes.

n Enable the Allow virtual machines to start and stop automatically with the system option.

n Assign the IP address 10.1.0.200 to the vmk0 management network.

n Set the NTP server to 0.vmware.pool.ntp.org.

It is important to ensure that the time on the management host is set correctly.

n Enable SSH on the managament host.

In a multi-rack scenario, configure an additional vSphere Standard Switch (vDS) for each additional rack.In a dual rack scenario, vSwitch1 should use vmnic1 and should be configured with two Virtual MachinePort Groups (VIA1 and VIA2). The VIA1 port group should be tagged to use VLAN101, and the VIA2 portgroup should be tagged to use VLAN102. vmnic1 should be connected to the private switch on a port withVLAN101 and VLAN102 visible.

Virtual MachinesThe VIA VM runs on the laptop or management host. A jump VM runs on the management host.

If you have multiple physical racks in your environment, you have the following options:

n Image the racks sequentially - image rack 1 first followed by the remaining racks one at a time.

n Image the racks in parallel by configuring a VIA VM per physical rack.

VIA User's Guide

VMware, Inc. 34

Hardware ConfigurationTable 3‑5. Jump VM Hardware Configuration

Virtual Hardware Value

Memory 4 GB

vCPU 1 virtual socket, 2 cores per socket

Video card 1 display

SSCI Controller 0 LSI Logic SAS

bus sharing: none

Hard disk 120 GB, Thin Provision

CD/DVD Client device

Floppy drive Removed

Network adapters 2 VMXNET3 vNICs

Operating system Microsoft Windows 7 64-bit or Win2K12

Virtual Machine version Hardware version 8

Navigate to Options > Advanced >General

Disable logging

keyboard.typematicMinDelay = "2000000"

Software ConfigurationPerform the following tasks to prepare the jump VM.

n Install a Windows operating system on the VM. This is the version tested with VIA. Other versionsmay work as well.

n Install VMware Tools.

n Install the latest Windows patches.

n Enables Windows update using the VMware OS Optimization Tool.

n Install the following applications:

n Firefox or Chrome web browsers

n PuTTy

n WinSCP

n vSphere Web Client

n Java Runtime Environment

n If internet access is not available from the Access Virtual Machine, download the executables andbinaries for the applications on the VM.

n Verify that Remote Desktop Connection is enabled on the Access Virtual Machine.

VIA User's Guide

VMware, Inc. 35

Pre-Imaging Checklist 4You must complete this checklist before beginning the imaging process. It is important that each item inthe checklist is set to the specified value, otherwise imaging may fail. You may want to print this checklistand checkmark each row as you verify the setting.

For more information on any item in this table, see Chapter 3 Setting up your Environment for Imaging.

Table 4‑1. Pre-Imaging Checklist

Setting Verified

Review the Bill of Materials (BOM) and verify that there are no discrepancies between the BOM and thehardware being used. If there is a discrepancy, contact VMware Support.

Review the VMware Compatibility Guide and verify you have the supported server models, disk andstorage adapter models, and switch models.

Verify that the physical racks are wired according to the wiremap. See Rack Wiring.

Confirm that the inter-rack switches are not connected to the ToR switches.

The VIA VM must be connected only to the private network - there should be no external networkconnections in place before imaging.

Validate that BIOS Settings for all components are correct. See BIOS Settings.

Ensure that all servers are powered off.

Note This is a new requirement for Cloud Foundation 2.3. All servers must be powered off, otherwiseimaging will fail.

Ensure that switch versions are supported. See Network Switches.

Verify that firmware settings are set correctly as per the BoM.

Connect each device in the rack to both PDUs.

Verify that the cable bend radius is proportionate to the external diameter. See Network Cables.

Verify that cables are properly routed and labeled.

Test cables to ensure that installed cabling links provide the transmission capability to support therequired data communication.

Verify that each server has redundant power supplies and that each power supply is connected to aseparate rack PDU.

Ensure that servers and switches have the same airflow direction.

Verify that switches have redundant power supplies and each power supply is connected to a separatepower strip.

VMware, Inc. 36

http://vmware.com/go/cloudfoundation-vcg

Table 4‑1. Pre-Imaging Checklist (Continued)

Setting Verified

If you are using a management host to image your system, ensure that:n A supported version of ESXi is installed on the management host. For the supported version, see

the VMware Cloud Foundation Release Notes.n The Allow virtual machines to start and stop automatically with the system option is enabled.n IP address 10.1.0.200 is assigned to the vmk0 kernel interface.n SSH is enabled on the management host.n The access VM, VIA VM, and jump VM meet the required hardware configuration. See Virtual

Machines.n The required software has been installed on the VMs. See Virtual Machines.

Ensure that there are no DHCP or TFTP servers on the network that VIA will connect to for imaging therack. VIA uses DHCP to assign IP addresses to the rack components, and any external DHCP serverscan cause imaging to fail. Disconnect the uplink cables on the TORs to limit exposure.

VIA User's Guide

VMware, Inc. 37

Installing VIA 5You can install VIA on a desktop, laptop, or an ESXi host, also referred to as the management host. Alaptop or desktop is recommended when you are imaging a single rack. A management host is bettersuited for an environment where you have several physical racks in your datacenter.


n Installing VIA on a Laptop or Desktop

n Installing VIA on a Management Host

Installing VIA on a Laptop or DesktopVIA is a virtual appliance. After you install the VIA VM on your laptop, you copy the software bundle to theVIA VM. You can then access the VIA user interface through a browser on the laptop.

Prerequisites

n Ensure that you have the infrastructure for VIA available and that you have set up your physicalenvironment as described in Chapter 3 Setting up your Environment for Imaging.

n Download the VIA OVF file, Cloud Foundation software bundle, and the md5sum file on the laptop ordesktop.

Procedure

1 Connect one port of the network adapter to your laptop and one port to the unmanaged switch.

2 Connect two ports of the unmanaged switch as follows:

n one port to the ethernet management port of the management switch

n one port to port 48 of the management switch

3 Connect the .iso image to the VM’s CDROM drive.

Follow the wizard prompts to specify where to save the OVF file and accept the license agreement.

4 Configure time settings on the laptop.

5 Upload the Cloud Foundation software bundle on to the VIA VM by pointing the CD /DVD drive to thebundle ISO. Ensure that the CD/DVD drive is connected.

VMware, Inc. 38

6 Configure network settings on the laptop.

a Connect one NIC on the laptop to the corporate network and the other to the unmanaged switch.

b Manually assign IP address 192.168.100.190 to the interface facing the unmanaged switch on thelaptop.

This allows for separation of network traffic between the corporate network and the private networkthat is established between the physical rack and VIA. It also helps ensure that the DHCP servicewhich is part of is VIA is confined to the private network between the physical rack and VIA.

7 For the browser on the laptop that will be used to access VIA, make the following selections.

n In Network Connection, disable the proxy.

n Select Auto-detect proxy settings for this network so that the browser detects the proxysettings for your network.

8 Power on the VIA VM.

9 Ensure that you can ping the VIA VM (IP address is 192.168.100.2) from the laptop.

What to do next

Open a web browser on the laptop and type the following URL to connect to VIA:

http://192.168.100.2:8080/via/

Installing VIA on a Management Host

Prerequisites

n Ensure that you have the infrastructure for VIA available and that you have set up your physicalenvironment as described in Chapter 3 Setting up your Environment for Imaging.

n Download the VIA OVF file, Cloud Foundation software bundle, and the md5sum file on your local filesystem.

Procedure

1 Deploy the VIA OVF file on the management host.

a Login to the vSphere Web Client on the management host.

b Right-click the management host and click Deploy OVF Template.

c In source location, select Local file. Click Browse and select the VIA OVF from your local filesystem.

d Click Next.

e Review the OVF file details and click Next.

f Accept the OVF license agreements and click Next.

g Specify a name and location for the OVF and click Next.

VIA User's Guide

VMware, Inc. 39

h Select a resource and click Next.

i Select the disk format to store the VIA disks and the datastore to store the deployed OVFtemplate and click Next.

j On the Setup networks page, connect VIA to the private switch connected to rack 1.

k Review the deployment details and click Finish.

2 Copy the Cloud Foundation bundle to the management host.

a On the management host, create a single datastore named datastore1.

b In datastore1, create a folder named ISO bundle and copy the Cloud Foundation bundle file tothis folder.

3 Configure time settings on the management host.

a In the vSphere Web Client, navigate to the management host in the vSphere inventory.

b Select Manage and then select Settings.

c Under System, select Time configuration and click Edit.

d Select Manually configure the date and time on this host.

e Set the time and date manually.

f Click OK.

4 Upload the software bundle on to the VIA VM.

a Right-click the VIA VM and select Edit Settings.

b Click the Hardware tab and select the CD/DVD drive.

c Select the Connected check box to connect the CD.

d Select Connect at power on so that the CD-ROM drive is connected when the virtual machinestarts.

e Select Datastore ISO under Device Type.

f Click Select, browse to the ISO Bundle folder in datastore1 on the management host, and selectthe bundle.

g Click OK.

5 Create a VM on the management host to serve as the jump VM.

Connect one NIC on the jump VM to the network and the other to the private managed switch.

The jump VM must have a static IP address. The IP range 192.168.100.151 to 192.168.100.199 isusually available for the jump VM.

This allows for separation of network traffic between the datacenter network and the private networkthat is established between the physical rack and VIA. It also helps ensure that the DHCP servicewhich is part of is VIA is confined to the private network between the physical rack and VIA.

6 Copy the md5sum file on the jump VM.

VIA User's Guide

VMware, Inc. 40

7 For the browser on the jump VM that will be used to access VIA, make the following selections.

n In Network Connection, disable the proxy.

n Select Auto-detect proxy settings for this network so that the browser detects the proxysettings for your network.

8 Ensure that you can ping the VIA VM (IP address is 192.168.100.2) from the jump VM.

If you cannot ping the VIA VM, check the route on the jump VM.

9 Power on the VIA VM.

What to do next

Open a web browser and type the following URL to connect to VIA:

http://192.168.100.2:8080/via/

VIA User's Guide

VMware, Inc. 41

Imaging Physical Racks 6When you image a physical rack, the software in the Cloud Foundation bundle is loaded onto the physicalrack.

In a multi-rack environment, you can either image all racks in parallel, or image rack 1 first followed by theother racks one at a time. To image multiple racks in parallel, you need a vSphere Distributed Switch andVIA VM for each rack.

Figure 6‑1. VIA Setup for Parallel Imaging of Multiple Physical Racks

VMware, Inc. 42


n Image a Physical Rack

n Resume Imaging

n Image Additional Racks

Image a Physical RackVIA images the rack components in a pre-determined order, which is determined by the availability ofnetwork route to the different components of the rack. All switches are imaged first. This enables VIA toaccess the servers through the switches for imaging. The imaging order is as follows.

1 Management switch

The management switch is the main access gateway through which the Cloud Foundationmanagement data is routed. The management ports of the ToR and inter-rack switches areconnected to the management switch. The data ports of the ToR switches are also connected to themanagement switch. VIA is connected to the rack through a designated port on the managementswitch. It is therefore required that the management switch is the first component imaged by VIA inorder to obtain access to the other components of the rack.

2 ToR switches and Inter-rack switches (inter-rack switches are on rack 2 only)

ToR and inter-rack switches are imaged in parallel.

ToR switches provide connectivity to servers in each rack out to inter-rack switches. The first pair ofToR switches provide connectivity to your datacenter network. Inter-rack switches inter-connectmultiple racks enabling a scale out architecture for the datacenter. They create an stretched L2backplane between racks.

3 Servers

Servers must be rebooted after the switches are imaged. They are then imaged in parallel.

For each component that is being imaged, the following tasks are performed.

1 Discovery

Rack components are discovered using the DHCP service running on the VIA VM. The DHCPService uses the device type information to identify the device being discovered. Apart from thedevice type information, the DHCP service also uses hardware vendor specific strings to determinewhether the switch being imaged is a management, ToR, or inter-rack switch.

The first component to be discovered is the management switch. The DHCP service hands out a pre-determined IP address for the management switch followed by a PXE image specific to themanagement switch.

After the management switch is imaged, the ToR and inter-rack switches are discovered and imaged.The ToR switch enables discovery of the data network of the servers which is used to receive theinstallation image delivered by the DHCP service.

VIA User's Guide

VMware, Inc. 43

2 Image installation

Image installation refers to installing software on the components to make them operational. Thesoftware depends on the component type - an Operating System for switches and a Hypervisor forservers.

3 Configuration

This step in the imaging process ensures that the components of the rack work like a homogenoussystem. Configuration of each rack component is different. If any configuration step fails for themanagement, ToR, or inter-rack switches, the imaging run stops at that point. You must resolve theissue before you can proceed with imaging. If a configuration step fails for a server, you can skip thatserver and continue imaging the remaining servers.

1 Upload Software Bundle

The software bundle ISO file contains the software bits and scripts to be imaged on the physicalrack. You can upload multiple bundles at a time and activate the bundle that is to be used forimaging.

2 Add Server VIBs

If you have servers in your datacenter that need VIBs not included in the core bundle, you canupload these additional VIBs on VIA.

3 Add Custom ESXi Installer ISO

You can choose to image the servers in your rack with an ISO different from the one included in thecore software bundle. The ISO must be of the same version that is included with theCloud Foundation software bundle. For more information, see the Cloud Foundation Release Notes.

4 Specify Imaging Details

At the Details step of an imaging run, you provide a name and description for the imaging run as wellcomponent and port information for the rack.

5 Monitor Imaging

In the Monitor Imaging step of the imaging workflow, you can see the imaging status on all devicesin your physical rack. You must reboot the servers after the switches are imaged.

6 Verify Inventory

In the Verify step of the imaging workflow, the system collects inventory information for each devicein the rack.

7 Post Imaging Checks

In the final step of the imaging workflow, VIA creates a rack inventory file.

8 Download Inventory File

You must download and save the inventory file for each imaged rack or host. This file is requiredduring rack bring-up and while adding a host. For the first rack, the inventory file is copied over tothe SDDC Manager Controller VM during imaging. However, you must still download the file andsave it to an accessible location in case the file on is corrupted for some reason. For additionalracks, you must upload this file manually while adding the rack to your system.

VIA User's Guide

VMware, Inc. 44

Upload Software BundleThe software bundle ISO file contains the software bits and scripts to be imaged on the physical rack. Youcan upload multiple bundles at a time and activate the bundle that is to be used for imaging.

The bundle contains the following software:

n SDDC Manager

n vSphere (Platform Services Controller, vCenter Server and ESXi)

n vSAN

n NSX

n vRealize Suite (vRealize Log Insight, vRealize Operations, and vRealize Automation

n VMware Horizon

n App Volumes

You can upload additional VIBs based on server models and can also provide an ESXi ISO different fromthe one included in the core bundle to image the servers in your racks.

Prerequisites

n Download the Cloud Foundation software bundle and the md5sum file on your laptop, desktop, orjump VM.

n If you are re-purposing hosts in your datacenter, backup the data on the hosts. They are wiped cleanduring imaging.

n Ensure that the KVM console for servers is closed.

Procedure

1 In a browser window on the jump VM, type http://192.168.100.2:8080/via.

VIA User's Guide

VMware, Inc. 45

2 Click Bundle. Ensure that you are in the Core Bundle tab.

3 In the Bundle Location area, click Refresh.

Wait for the message CD mounted successfully to be displayed.

4 In the Bundle Hash area, click Browse, navigate to the directory that contains the MD5SUM file,select the file, and click Open.

5 Click Upload Bundle.

The bundle upload can take several minutes. After the upload is complete, the message Bundleuploaded successfully is displayed in the Upload Bundle area.

6 In the Bundle Info area, select the bundle in Available Versions and click Activate Bundle.

The selected bundle is now the active bundle for imaging and is ready to be used. Active bundledetails are displayed next to Active Bundle.

The bundle is copied to the /mnt/cdrom/ directory on the VIA VM.

Add Server VIBsIf you have servers in your datacenter that need VIBs not included in the core bundle, you can uploadthese additional VIBs on VIA.

Prerequisites

Download the 3rd party VIB you want to add to the software bundle on the jump VM where you arerunning VIA.

VIA User's Guide

VMware, Inc. 46

Procedure

1 In the Modify VIBs tab, select the vendor or click Add Vendor.

2 For a new vendor, type the vendor name and click Add.

3 Select the server model or click Add Model.

4 For a new model, type the model name.

The model name you enter must match the model or product name on the server's BIOS settings.This ensures that the correct VIB is loaded on the server.

The available VIBs are displayed.

5 To add 3rd party VIBs, click Browse next to the Select VIB field, select the VIB and click UploadVIB.

Note Only files with a .vib extension are accepted.

6 In Available VIBs, select the VIB to be added to the bundle.

7 Click Upload VIB.

Add Custom ESXi Installer ISOYou can choose to image the servers in your rack with an ISO different from the one included in the coresoftware bundle. The ISO must be of the same version that is included with the Cloud Foundationsoftware bundle. For more information, see the Cloud Foundation Release Notes.

Prerequisites

Download the ISO you want to use for imaging on the jump VM where you are running VIA.

VIA User's Guide

VMware, Inc. 47

Procedure

1 In the Bundle tab, click Modify ISOs.

u Active

2 Select the vendor name.

The ISO in the core bundle is displayed.

3 In Select ISO to Add, click Browse, select the ISO to add and click OK..

4 Type the MD5 or SHA-1 checksum for the ISO.

You can either download the MD5 checksum from the location where you downloaded the ISO, orgenerate it via another tool.

5 Click Upload ISO.

6 In the Available ISOs section, click the Active radio button next the newly uploaded ISO.

Specify Imaging DetailsAt the Details step of an imaging run, you provide a name and description for the imaging run as wellcomponent and port information for the rack.

Prerequisites

n Software bundle must have been uploaded and activated.

n All servers are powered off.

n You must be able to power on the servers one at a time.

n All items in the pre-imaging checklist have been completed.

VIA User's Guide

VMware, Inc. 48

Procedure

1 In the VIA user interface, click Imaging.

2 (Optional) Type a name and description for the imaging run.

It is recommended that you add the rack serial number or other rack identification details in the Nameor Description field. The name and description is recorded in the imaging details JSON file afterimaging is complete, which helps identify the rack to which the imaging details file belongs.

3 In Deployment Type, select Cloud Foundation Full Deployment.

4 In Rack Type, select First Rack if you are imaging rack 1. For additional racks, select AdditionalRack.

5 For the management switch and ToR switches, select the vendor and model.

The IP address for each switch is displayed.

VIA User's Guide

VMware, Inc. 49

6 (Optional) Type the MAC address of each switch.

7 If the physical rack contains inter-rack switches (rack 2 only), type the number of inter-rack switchesin the Number field in the Inter-rack Switch section.

8 Select the vendor and model number of the inter-rack switches.

9 In the Server section, type the number of servers in the physical rack.

10 Select the vendor for each server.

11 You can specify the server where the SDDC Manager VMs will be deployed. This server is referred toas ss the primary host. Either specify the port number on the ToR switch that the server is connectedto, or the MAC address of the server.

If you do not type the port number or MAC address, the SDDC Manager VMs are deployed on thefirst server to be rebooted.

12 Click in any section to view the VIA properties file.

The VIA properties file displays rack specification values from the activated software bundle. Ifrequired, edit the properties as appropriate. For example, delete the ToR ports that you are not usingto speed up the imaging process.

13 Click Save.

14 Click Start Imaging. The wiring diagram for the rack is displayed.

Review the wiring diagram to check if your rack is wired according to the displayed wiremap. See Rack Wiring.

Click Confirm if it is accurate. If you need to make any wiring changes, click Cancel.

What to do next

Once imaging starts, notifications (errors, warnings, and information) are displayed in the top right corner

of the VIA window. Click the icon to review the messages so that you can take the appropriatesteps to complete the imaging successfully.

Monitor ImagingIn the Monitor Imaging step of the imaging workflow, you can see the imaging status on all devices in yourphysical rack. You must reboot the servers after the switches are imaged.

VIA User's Guide

VMware, Inc. 50

Procedure

1 Click the Imaging > Imaging tab.

The run details, rack details, and imaging status for switches and servers in the rack are displayed.The devices in the physical rack are displayed in the order in which they will be imaged. IP addressesfor each device are displayed as well.

For information on next steps if a device fails to be imaged, see Resume Imaging.

2 Note the Run ID.

3 After the switches are imaged, the UI displays a notification to reboot the servers. Reboot the serverswithin 2 hours. If the seession times out before the reboot is completed, click Resume to continueimaging.

Once server imaging begins, the IP address for the primary ESXi host is displayed. Note down thisaddress since you may need to log in to this host.

VIA User's Guide

VMware, Inc. 51

4 Click a device to see information about the imaging tasks completed and in-progress tasks.

It can take approximately 95 minutes for rack 1 to be imaged. During imaging, the password of allrack components except SDDC Manager is set to EvoSddc!2016. The SDDC Manager password isset to vmware1234. Note that the ToR and inter-rack switches are named as follows:

n ToR switch connected to port 41 of the management switch is named TOR20

n ToR switch connected to port 42 of the management switch is named TOR21

n Inter-rack switch connected to port 43 of the management switch is named INTER-RACK30

n Inter-rack switch connected to port 44 of the management switch is named INTER-RACK31

For information on next steps if a device fails to be imaged, see Resume Imaging.

Verify InventoryIn the Verify step of the imaging workflow, the system collects inventory information for each device in therack.

VIA User's Guide

VMware, Inc. 52

Procedure

1 Click the Imaging > Verify

The status of inventory collection on each device in the rack is displayed.

2 Click each component to see its inventory information. You can expand a component to see more

details.

Post Imaging ChecksIn the final step of the imaging workflow, VIA creates a rack inventory file.

VIA User's Guide

VMware, Inc. 53

Procedure

1 Click the Imaging > Finish tab.

Post imaging tasks are displayed.

2 If a task is not completed successfully, click Rerun.

3 After each displayed task has an icon next to it, click Complete.

The rack inventory file is created for the customer. This file includes the SDDC Manager passwordgenerated during imaging. The imaged rack is now ready to be shipped to the customer.

4 Power down the primary rack.

It is important to power down the rack even if you are deploying Cloud Foundation on a ready systemso that the management switch is rebooted.

Download Inventory FileYou must download and save the inventory file for each imaged rack or host. This file is required duringrack bring-up and while adding a host. For the first rack, the inventory file is copied over to the SDDCManager Controller VM during imaging. However, you must still download the file and save it to anaccessible location in case the file on is corrupted for some reason. For additional racks, you must uploadthis file manually while adding the rack to your system.

Procedure

1 Click the Inventory tab.

2 Select the Run ID.

VIA User's Guide

VMware, Inc. 54

3 Click Download.

The file is download on the jump VM. Each inventory file is identified by the ImagingID. The name anddescription in the file match the details you specified in the Imaging tab.

4 Copy the downloaded file to an accessible location.

5 Rename the file such that the file name indicates the rack or host that the inventory file belongs to.

Resume ImagingA device may fail to be imaged because of possible hardware faults or mis-configuration, or networkissues. You can take a number of actions that can help in continuing with the imaging run.

Fix Issues During the Monitor Imaging StepDuring the monitor step in the imaging workflow, you can identify imaging failures by looking at theprogress bar on the components in the Imaging > Imaging tab. An icon indicates that it has beenimaged successfully. An icon indicates that one or more imaging tasks on that devices failed.

1 Click the component to display the imaging task list for that device. Then do one of the following:

n Click Retry to re-start imaging on that device .

n Click Remove to remove that device from the VIA UI and database and then click Yes to confirm.The removed device is grayed out and it is not imaged. Ensure that you remove this device fromthe physical rack before shipping it to the customer. To add a removed device back to the VIA UI,click the device and click Add to Inventory. The device is added back to the VIA UI anddatabase.

If the primary ESXi server fails to be imaged, you cannot resume imaging by removing it. It ismandatory for the primary server to be imaged.

2 If you need to resolve a hardware issue before re-trying imaging on that device, close the task listdialog box. In the Imaging > Imaging window, click Stop.

VIA User's Guide

VMware, Inc. 55

3 If you are able to resolve the hardware problem, click Resume. Imaging is resumed from the statewhere it had stopped. If you need additional time to resolve the hardware issue or there are otherhardware problems, click Abort. The imaging run is discarded.

If you are unable to resolve the hardware issues, contact VMware Support.

4 Click Next to proceed to the next step in the workflow.

Fix Issues During the Verify Imaging StepDuring the verify step in the imaging workflow, you can identify imaging failures by looking at the progressbar on the components in the Imaging > Verify tab. An icon indicates that inventory information hasbeen collected successfully. An icon indicates that the tasks on that device failed.

1 Click the component to display the verification task list for that device.

2 Click Retry

3 Once the device displays an icon, click Next.

VIA User's Guide

VMware, Inc. 56

Fix Issues During the Finish Imaging StepDuring the finish step in the imaging workflow, failed post-imaging tasks are displayed with an icon.

1 Click Rerun to run the failed task again.

2 After all tasks display an icon, click Complete.

Opening an Aborted RunIf you had accidentally aborted an imaging run, you can re-open it.

1 In the VIA user interface, click History.

2 In the Select Run ID drop-down, select the run ID you want to open.

3 Click Reopen.

The selected run is opened in the state it was at the time the run had been aborted.

Image Additional RacksFollow this procedure for each additional rack if you are imaging racks incrementally in a multi-rackenvironment.

Procedure

1 Disconnect port 48 of the management switch on rack1 from the private managed switch.

2 Connect port 48 of the management switch on the next rack to the private managed switch.

3 Follow Image a Physical Rack.

VIA User's Guide

VMware, Inc. 57

Imaging Individual Devices 7You can image a server or management switch as an individual device.


n Image Individual Server

n Image New Management Switch

Image Individual ServerIf a server fails to be imaged, you can image that server as an individual device rather than re-imaging thecomplete rack. Or you can image a new or replacement server before adding it to a rack.

Prerequisites

n Mount the host in the appropriate slot in the physical rack. For a replacement host, mount it in thesame slot as the previous host and wire it according to the same wiring connections.

n If VIA is installed on a laptop, connect the NIC port on the laptop to port 48 of the management switchon which the host is being imaged. If VIA is installed on a management host, connect the host to aprivate managed switch that is connected to port 48 of the management switch on which the host isbeing imaged.

n VIA must have access to the inband network only.

n Software bundle must have been uploaded. See Upload Software Bundle.

n BIOS settings must have been set on the server to be imaged. See BIOS Settings.

n Server must be in PXE boot mode.

n Power off the server before imaging.

Procedure


Ensure that you are in the Details tab.


3 In Deployment Type, select Cloud Foundation Individual Deployment.

4 In Device Type, select ESXi_SERVER.

VMware, Inc. 58

5 In Rack Type, select Primary Rack if you are imaging a server in rack 1. For a server in an additionalracks, select Add-On Rack.

6 Select the vendor and model number of the server. The IP address of the server is displayed.

7 If you are imaging a host for a multi-rack Cloud Foundation deployment, you must type the MACaddress of the host being imaged. This ensures that VIA images the correct host.

8 Click Start Imaging.

9 Power on the server.

The server is discovered and the imaging process begins.

10 Open the KVM console to the server.

11 Change the boot device for the next boot to PXE Mode.

a Ina web browser, open the Integrated Dell Remote Access Controller page and login with defaultcredentials (root/calvin).

b In the Properties tab, click Launch in the Virtual Console Preview.

The console opens in a new window.

c Open the keyboard.

d In the Properties tab, click Power Cycle System (cold boot) in the Quick Launch Tasks

e In the console window, press F11 to access the Boot Manager.

F11 = Boot Manager is highlighted.

f Select One-shot BIOS Boot Method

g Select PXE Boot.

Do not power cycle or reset the server.

12 Monitor the ESXi installation on the console.

The server is rebooted after ESXi installation is complete. Ensure that ESXi is booting from theinstalled copy of ESXi and not from the network.

After the server boots from the installed copy of ESXi, VIA continues imaging the server.

13 Download the inventory file. You need this file when adding an imaged host to a Cloud Foundationrack.

a Click the Inventory tab.

b Select the Run ID.

c Click Download.

The file is download on the jump VM.

d Copy the downloaded file to an accessible location.

14 Disconnect VIA and shutdown the laptop or management host.

VIA User's Guide

VMware, Inc. 59

Image New Management SwitchImaging the new management switch with VIA installs the necessary software on the switch.

Prerequisites

n Management switch must be connected to the laptop or management host where VIA is installed.

n If VIA is installed on a laptop, the NIC port on the laptop must be connected to port 48 of themanagement switch.

n If VIA is installed on a management host, the management host must be connected to a privatemanaged switch that is connected to port 48 of the management switch.

n Identify the Cloud Foundation version in your environment and ensure that the appropriate bundleand md5sum file is uploaded on VIA.

Note Do not connect the management switch to any host before or during imaging.

Procedure


Ensure that you are in the Details tab.


3 In Deployment Type, select Cloud Foundation Individual Deployment.

4 In Device Type, select MGMT_SWITCH.

5 Select the vendor and model number of the switch. The IP address is displayed.

6 Click Start Imaging.

Imaging fails at collect BMC-IP information task. This is expected behavior.

7 Disconnect the switch from the laptop or management host.

VIA User's Guide

VMware, Inc. 60

Review Alarms and Notifications 8After the imaging run is completed, you must review all alarms and notifications by clicking the bell icon atthe top right corner of the VIA window.

VMware, Inc. 61

Viewing the VIA Log File 9The log file displays information for all VIA services.

Procedure

u On the left navigation bar in the VIA user interface, click Logs.

A consolidated log of VIA services is displayed sorted by the time stamp. A maximum of 500 entriesis displayed at a time.

You can filter the logs by typing a search string and clicking Submit. For example, you can search foractivities on the primary ESXi server.

To display the complete log file, click Open a new window.

The Auto Refresh option is selected by default where the log file automatically scrolls to display themost current information.

VMware, Inc. 62

Viewing Results of an ImagingRun 10You can view the imaging history for an imaged rack or the status of individual devices on an imagedrack.

This chapter includes the following topics:n View Imaging History

n View Inventory

View Imaging HistoryYou can view the status of an imaging run by specifying its run ID if the rack state has not changed sincethat run was completed. If you imaged multiple racks using the same VIA VM, you can view the imaginghistory of each rack by specifying its run ID.

Prerequisites

Verify that an imaging run is not in progress.

Procedure

1 In the VIA user interface, click History.

VMware, Inc. 63

2 In Select Run ID, select the run ID for which you want to view the imaging history. You can only viewthe history for a run if the state of the rack has not changed since the run was completed.

Imaging history appears for all devices that are imaged during the specified run.

3 To view details for a device, click the expand icon next to the device.

4 To reopen a previous run, select the run ID and click Reopen.

You can continue imaging an aborted run by reopening it.

View InventoryThe Inventory page displays a consolidated report of the rack inventory. You can view device details byexpanding the appropriate device.

Prerequisites

Verify that an imaging run is not in progress.

Procedure

1 In the VIA user interface, click Inventory.

VIA User's Guide

VMware, Inc. 64

2 In Select Run ID, select run ID.

The device inventory for the selected imaging run is displayed.

3 To view details for a device, click the expand icon next to the device.

4 To download the rack inventory click Download Manifest and specify the directory where the file is tobe saved.

The device inventory is saved as a JSON file.

VIA User's Guide

VMware, Inc. 65

Guidance on Server OOB PortManagement 11Cloud Foundation does not manage OOB ports for 2.3 and later releases. You can manage the OOBports per your needs. You must make a note of IP addresses assigned to servers and OOB passwords.


n OOB Ports Wiring to Management Switch

n OOB Ports Configuration

OOB Ports Wiring to Management SwitchThis section describes ways in which you can wire the OOB ports to the management switch.

n Connect server OOB ports to an external OOB management switch. This is the recommendedmethod.

n Connect server OOB ports to the management switch in the rack and assign static IP addresses tothem. If you choose to use the 192.168.0.0/22 network, ensure that the IP addresses are within therange of 192.168.1.2 to 192.168.3.253. You cannot use IP address outside this range.

OOB Ports ConfigurationAfter imaging is complete, add a route to your jump VM to connect to the OOB ports on the servers.

Procedure

u Add a route on the laptop/desktop or jump VM where you have deployed VIA to allow access to thephysical servers. Here is an example on how to add a route on a Windows jump VM for a 192.168.0.0network.

route add 192.168.0.0 mask 255.255.255.0 192.168.100.1 if 16

where 16 is the ID for rack 1. To find the interface number on a Windows laptop or desktop , followthe steps below.

a In a command window, type the command netsh.

b Type the command int ipv4 show interfaces.

VMware, Inc. 66

Troubleshooting VIA 12This chapter includes the following topics:

n ESXi Server has Incorrect BIOS Settings

n ESXi Server has Bad SD Card

n Management Switch Boots into EFI Shell

n A switch did not start imaging

n One or More Servers Are Not Discovered

n Server Imaging Failed at Kickstart Delivery Task

n Server Imaging Failed at Check Host Status Task

ESXi Server has Incorrect BIOS Settings

Problem

Host failed to be imaged with the message Post install reboot ESXi task failed.

Cause

ESXi server has incorrect BIOS settings.

Solution

1 Check the ESXi server console.

2 If the console displays a gray screen with the message Unable to find boot device, check that theBIOS setting is SATADOM for Quanta servers, and SD card or SATADOM for Dell servers.

3 Fix the hardware problem.

4 On the Imaging tab, click the host that displayed the red icon and click Retry.

ESXi Server has Bad SD Card

Problem

Device failed to be imaged with the message Kickstart image not delivered..

VMware, Inc. 67

Cause

ESXi server has bad SD card.

Solution

1 Replace the SD flash card in the ESXi server.

2 On the Imaging tab, click the host that displayed the red icon and click Retry.

Management Switch Boots into EFI Shell

Problem

After rebooting, the management switch boots into EFI shell instead of ONIE mode.

Note This issue affects only management switches running the Cumulus OS.

Cause

The switch was not in ONIE mode and after rebooting, it boots into an EFI shell.

Solution

1 Connect to the management swtich with a console cable.

2 Press DEL to change the boot order.

3 Select P0.

4 Select Save changes.

5 Select Save changes and restart.

6 To wipe the switch login as cumulus, type sudo cl-img-select -k.

A switch did not start imagingImaging did not begin on a switch.

Problem

Cause

The switch mode was not set correctly prior to imaging.

Solution

u Using the serial console port, check the switch mode and ensure it matches the mode specified in Network Switches.

One or More Servers Are Not DiscoveredOne or more servers are not discovered by VIA.

VIA User's Guide

VMware, Inc. 68

Problem

When servers are not discovered, the corresponding UI objects on the Imaging > Imagingtab do not turngreen.

Cause

There are several reasons due to which servers might not be discovered:

n Server was not powered on.

n Server was powered on, but is stuck in BIO mode due to a hardware problem.

n Server was not properly connected to the ToR switches.

n Server was powered on, but is not trying to PXEBoot from the appropriate network card for VIA tosend the ESXi image.

To identify the servers which are not discovered in VIA, you must search through the VIA log file.

Solution

1 In the VIA user interface, click Logs.

2 Type one of the following search strings. All search strings must be entered without quotes.

n Identifying undiscovered servers by their assigned BMC IP addresses

Type IPMI BMC IP Address ="

All discovered servers display the corresponding BMC IP address programmed on those servers.The BMC IP of the servers which were not discovered do not appear here. Log in to the KVMconsole of the undiscovered BMC consoles for further troubleshooting.

n Identifying undiscovered servers based on server port connection to the ToR switches

Type getToRPortNum

Displays discovered servers with their ToR port connectivity. Use the process of elimination toidentify the servers whose corresponding port numbers are not displayed in the search.

n Identify undiscovered servers based on server serial number (Service Tag)

Type Key = ServiceTag

Displays service tag for all discovered servers. Servers not displayed here were not discovered.

n Identify undiscovered servers based on server MAC address

Type Mac address of vmknic =

Displays the MAC address of all discovered servers. Servers whose MAC addresses are notdisplayed were not discovered.

ServerImaging Failed at Kickstart Delivery Task

VIA User's Guide

VMware, Inc. 69

Problem

Server imaging failed at task Kickstart delivery and wait for Host up.

Cause

Th kickstart delivery task fails if the server failed to request the kickstart script after pulling the ESXi imagefrom VIA. This is usually due to network issues or due to an ESXi installer cras.

Solution

u Check the server console to see if the server is actually booted into ESXi and shows the expected IPaddress.

Server Imaging Failed at Check Host StatusTask

Problem

Server imaging failed at task Check host status.

Cause

This issue occurs becuase VIA is unable to log in to the server right after ESXi has been installed on it.

Solution

1 Click the alarm icon on the top right corner of the window and look for the following alarm:

Host 192.168.100.XX is different than the one with Mac Addr aa:bb:cc:dd:ee:ff

discovered through DHCP

This alarm indicates that all servers were powered off before imaging

Identify the servers that were not powered off, power them down, and retry imaging on those servers.

2 Search for the following strings in the log file.

Esxi imaging failed on host or Maximum retries exceeded

Log in to the server console for additional troubleshooting.

VIA User's Guide

VMware, Inc. 70

Cloud Foundation Glossary 13Term Description

add rack Configure an additional rack for a Cloud Foundation system.

additional rack Additional racks (added after the first rack) to a Cloud Foundation system.

bring-up Initial configuration of a newly deployedCloud Foundation system. .

Cloud Foundation system. Set of physical racks managed as a unit by a single SDDC Manager.

first rack First (primary) rack in the Cloud Foundation system. The management domain is deployed onthis rack.

Hardware Management System(HMS)

Manages hosts and switches in the Cloud Foundation system.

host An imaged server.

imaging During imaging, SDDC software is pre-configured on a physical rack.

integrated system System that combines hardware and software. Can be purchased from select VMware partners.The partner images the rack before sending it to the customer site.

inter-rack switches Connects individual ToR switches with each other to provide connectivity across racks. Theseswitches are required only when you have more than one rack in your Cloud Foundation system,and are placed on the second rack.

Lifecycle Manager (LCM) Automates patching and upgrading of the software stack.

management domain Cluster of physical hosts (first fours hosts in the physical rack) that house the managementcomponent VMs

management host Standalone ESXi server to host the Windows jump VM used for imaging.

primary host Host on which VIA deploys the SDDC Manager VMs during imaging, and which bootstraps thefirst rack during bring-up.

SDDC Manager Software component that provisions, manages, and monitors the logical and physical resourcesof a Cloud Foundation system.

SDDC ManagerController VM Contains the SDDC Managerservices and a shell from which command line tools can be run.This VM exposes the SDDC Manager UI.

SDDC Manager Utility VM Contains the LCM depot, backup repository containing NSX Manager and host backups, and2nd DNS instance.

server Bare metal server in a physical rack. After imaging, it is referred to as a host.

Top of Rack (ToR) switch Connects servers within a rack through 10Gbps links to the NICs on each server. ACloud Foundation rack contains two ToR switches connected to each other.

VMware, Inc. 71

Term Description

unassigned host Host in the capacity pool that does not belong to a workload domain.

workload domain A policy based resource container with specific availability and performance attributes andcombining vSphere, vSAN and NSX into single a consumable entity. A workload domain can becreated, expanded, and deleted as part of the SDDC lifecycle operations.

VIA User's Guide

VMware, Inc. 72

Documents

VIA User's Guide - VMware Cloud Foundation 2 · PDF fileCloud Foundation Deployment Options 8 ... The VMware Cloud Foundation Overview and ... vRealize Suite into a natively integrated