Basic ESXi Networking

About vmnics, vSwitches, management and virtual machine networks

In the vSphere client you can see the network diagram for your ESXi host by clicking

Networking on the Configuration tab:

After you installed ESXi, before you could connect to it with the vSphere client, you had to

nominate a physical network adapter (a “vmnic” in the diagram) which would be the ESXi

Management Network. You also had to assign an IP address to the ESXi host's Management

Network interface, either through DHCP or manually through the console.

The network diagram above shows that I've assigned the Management Network to vmnic0

and gave it an IP address of 192.168.111.30. (192.168.111.0/24 is my home LAN. Yours will

probably be different.) Whatever subnet you've chosen, the VMkernel Port in the diagram is

your Management Network and that’s what your vSphere client is now talking to.

ESXi will name the first physical NIC it finds “vmnic0″. If vmnic0 is your management

interface, ESXi will have automatically attached a virtual switch, vSwitch0, to that interface.

In addition to the VMkernel port, ESXi will also attach a Virtual Machine Port Group to the

vSwitch. In the diagram above it's labeled as "Virtual Machine Network". The VM Port group

is where Virtual Machines can be attached to this virtual network.

In summary, in the above diagram, vSwitch0 has both a VM Port Group (Virtual Machine

Network) and a VMkernel Port (Management Network) attached.

Creating the LAN

In a small network it is quite common to use the Virtual Machine Port Group on vSwitch0 to

provide the LAN interface for your pfSense. That allows you to access the LAN side of your

pfSense virtual machine and manage your ESXi host, with the vSphere client, from a single

PC. Of course, the virtual machine (e.g., pfSense) and the ESXi management interface must

have different IP addresses.

COMMENT: I must say here that I always separate the ESXi Management network from

other networks. I won’t go into the detail but there are some very good reasons for doing this.

Without using VLANs, though, separation would mean that you need an additional NIC on the

ESXi host just for ESXi management. What’s more, you would also need another NIC in your

vSphere client PC to connect to the management NIC in your ESXi host. If you want to follow

that path and you have enough NICs, simply delete the Virtual Machine Port Group by

clicking the Properties link above vmnic0, highlight the VM Port Group and click Remove.

Assuming you are just working with just two NICs in your ESXi host, rename the VM Port

Group from "Virtual Machine Network" to something a bit more meaningful. Click the

Properties… link for vmnic0:

Highlight the Virtual Machine Network and click the Edit button.

Change the Network label to “LAN” and click OK then Close.

You’ll see how this makes life a little easier when we assign virtual network interfaces to

pfSense.

Creating the WAN

As we’re not going to deal with VLANs here, you will need a second physical NIC in your

ESXi host. This will be your WAN interface.

HINT: If you have multiple physical interfaces in your ESXi host, it can be a bit of a struggle

to work out which one has been identified as vmnic1, vmnic2 and so on. If you wrote down the

MAC address of each NIC and the slot it occupied when you put it in the machine, you’re in

business – just look at the Network Adapters screen under the Configuration tab to match up

the MAC addresses (new to ESXi 5). However, you probably didn’t think do that so the easiest

way match physical NICs to vmnics is to plug a PC or switch into them, one at a time. The

speed and duplex on the Networking or Network Adapters screens should change as the

interface comes up. Because VMware didn’t provide a Refresh link on the Network Adapters

screen, you may have to refresh by navigating to somewhere else then going back.

Now we need to link the second physical NIC (vmnic1), to a new vSwitch. Click the Add

Networking link at the top right of the Networking screen and the following dialog will

appear.

We are adding a Virtual Machine network so select that option and click Next.

We want this NIC to be attached to a new vSwitch so select Create a virtual switch and check

vmnic1. Click Next.

As we did with the LAN, let’s give this VM Port Group a more meaningful name of

“WAN”. Click Next.

The next dialog simply lets you check that everything looks OK. Click Finish.

Your networking diagram should now look like this:

Now we can configure a new virtual machine on which pfSense will be installed.

Configuring the Virtual Machine

Right click your ESXi host in the left pane of the vSphere client and select New virtual

machine…

Configuration

In the Configuration window, I always like to take the Custom option. (I’ve never really

trusted what someone else thinks is “typical”). Click Next

Name and Location

In the Name and Location window, let’s give the virtual machine a meaningful name like

“pfSense” and click Next

Storage

Now we need to decide where disk storage will be allocated to hold the virtual machine's

configuration and operating files. (This is not necessarily the same location as the file system

for pfSense, as you'll see later. ) There are two datastores on this server – a small 80GB drive

on which ESXi is installed and a 500GB disc which is just for virtual machine

storage. Highlight a datastore from your list and click Next.

Virtual Machine Version

WARNING

Note that for ESXi 5.5 the default virtual machine version is 10. However, if you select

version 10 with the FREE version of ESXi 5.5, you will not be able to edit the virtual

machine settings through the vSphere client. Use virtual machine version 8.

Here is where you tell ESXi the virtual machine version you want to use for pfSense. Note

the warning above. Select version 8 and Click Next.

Guest operating System

If you’re really new to pfSense you may not have noticed that it’s built on the FreeBSD

operating system, not Linux. Select Other and chose FreeBSD (32-bit) or FreeBSD (64-

bit).

Probably the most noticeable difference between the 32 and 64-bit versions is that counters on

the 32-bit version, such as the number of bytes sent and received, will go back to zero after

about 4GB. On the 64-bit version it will take a lot longer to reach the limit.

That said, the 32-bit version is by far the most widely deployed and tested. As you will see a

few steps later, the amount of memory needed to run pfSense is unlikely to require a 64-bit

operating system. I would recommend that you take the FreeBSD (32-bit) option but,

whatever you choose, it must match pfSense ISO image you're going to install. Click Next.

CPUs

To get started, a single-socket, single-core configuration will do for now. You can always

change this and other virtual machine settings later if you want. Click Next

Memory

Depending on the number and type of packages you intend to install, a basic pfSense VM

should run very, very comfortably in 512MB. A lot of simple, non-virtual installations run on

old PCs with 256MB and less.

If you’re really squeezed for physical RAM on your host - perhaps because you intend to run

lots of other virtual machines - you could cut back the allocation to the pfSense VM to, say,

384MB. If you intend to run lots of memory-hungry packages give it more.

To change the memory allocation to one of the sizes shown on the scale of the memory

"thermometer", click that value on the scale. Click Next

Network

Remember that your two virtual networks were renamed to LAN and WAN. This is where

we attach those networks to our pfSense virtual machine.

Select the number of virtual NICs you want to assign to pfSense. In this case it will be

2. Now, using the drop-down lists assign your virtual machine’s NIC 1 to the LAN network.

Assign NIC 2 to WAN. (This is why we gave the virtual machine port groups these names –

much easier to recognize.)

Note that for each NIC you can also select an Adapter type. Different adapter types may

give better or worse performance (and some may not work at all) but you’ll have to study that

elsewhere. To get started, choose the dependable E1000 type for each adapter. Make sure

that Connect at Power On is checked and click Next.

SCSI Controller

An emulation of an LSI Logic SCSI controller is offered on this system and, as far as I know,

the recommendation is based on the operating system of the virtual machine you intend to

install. Accept the default and click Next.

Select a Disk

This is where the operating system will build its file system.

Choose Create a new virtual disk and click Next.

Create a Disk

Here you can see that I’ve given pfSense's virtual disk a capacity of 8GB but I have quite a lot

to spare and 8GB isn’t really that much these days.

Under Location, keep the virtual machine’s hard disk with the virtual machine itself. If you

want to learn more about this set of options, read the help.

Click Next.

Advanced Options

Like it says – these options do not normally need to be changed. Next.

Ready to Complete

Now you'll see a summary of what you've configured for this virtual machine.

Before you finish, check the box Edit virtual machine settings before completion. The

label on the Finish button will change to Continue. This will allow you to set up the boot CD

from which we’ll install pfSense. Click Continue.

Editing the Virtual Machine's Properties

In the Virtual Machine Properties dialog:

If you have a CD/DVD in your ESXi host, select the CD/DVD drive and check

Connect at power on. This change allows you to pop the pfSense CD/DVD into the

host’s drive and start installing a soon as you power on the virtual machine.

If you want to install from the CD/DVD drive in your vSphere Client PC, select the

Client Device option. As the contents of the CD will be read across your network, this

will be a bit slower than using a drive in your ESXi host. In addition, Connect at

power on is not available.

If you want to install from an ISO image stored in an ESXi datastore, that is also an

option but it won’t be covered here.

Click Finish.

Installing pfSense

Booting your VM from CD/DVD

Option 1: Installing from the ESXi host’s CD/DVD drive

If you chose the Host Device option in the Virtual Machine Properties above, slip the

pfSense CD into your ESXi host’s drive.

In the left-hand pane of the vSphere client window, right-click your new pfSense

virtual machine. You will see a number of things you can do with the VM, including

Power > Power on. Select that or highlight your virtual machine and click the green

arrowhead in the toolbar.

Now click the Console tab and you should see the virtual machine booting from the

CD.

Skip to Installing pfSense.

Option 2: Installing from your client’s CD/DVD drive

If you chose to use the drive in your client PC, put the CD into its drive. Remember

that Connect at power on was not a choice if using the client’s CD/DVD drive, so

you will need to do a little bit of extra work to connect it after you power on the virtual

machine.

In the left-hand pane of the vSphere client window, right-click your new pfSense

virtual machine. Here you will see a number of things you can do. Select Power >

Power on. Alternatively, just highlight your virtual machine and click the green

arrowhead in the toolbar.

Now, with your virtual machine highlighted, click the Console tab.

Because the CD drive is not attached to the virtual machine yet, you may see it trying

to boot from the network or it may be showing an Operating system not found or some

other error. Don’t worry about this.

At this point (and only after you have powered up the virtual machine) you can attach

your virtual machine to your client PC’s CD/DVD drive. Click on the toolbar icon that

looks like a CD with a spanner. CD/DVD Drive 1 will be offered in the menu and

you’ll see the choices available. Select Connect to D: (or whatever drive letter

represents your client PC's CD/DVD drive).

Right click your virtual machine in the left pane of the Sphere client and select Guest

> Send Ctl+Alt+Del. This will reboot your virtual machine without disconnecting the

CD/DVD drive.

In the Console tab you should now see pfSense booting from the CD.

Installing pfSense

If everything has gone well you will soon see the pfSense boot menu.

What follows is very much a standard pfSense installation procedure. However, it's

included here to save you jumping around between documents.

NOTE that to enter information through the virtual machine's console you must click inside

the console window. To release the cursor, press Ctl+Alt.

You can allow the timer to expire and boot pfSense from the “Live CD”.

When you see the following console message:

Type “i” to launch the pfSense installer.

The next few screens are the standard pfSense install screens and are fairly self explanatory.

Take the highlighted choice in each of the following screenshots:

At this point the pfSense virtual machine will reboot and you should remove the CD from

the drive.

Interface Assignment

Next up, the pfSense boot menu returns.

As pfSense is already installed on the virtual disk, just allow the timer to expire.

Once pfSense has booted you will see the message: Network interface mismatch – Running

interface assignment option. This just means that you haven’t yet told pfSense which virtual

network interface is LAN and which is WAN.

First of all, though, as you don’t need to assign VLANs, type n and press return.

The order that the virtual NICs were assigned to pfSense when you set up the virtual machine

is important here. ESXi presents those network interfaces to pfSense in sequence. That is,

your pfSense virtual machine sees NIC 1 (LAN) as em0, NIC 2 (WAN) as em1, etc.

Note that the MAC addresses assigned to the virtual NICs and seen by pfSense are also

virtual. They are not the MAC addresses of the physical NICs. If you've forgotten which

network (LAN or WAN) was assigned to which virtual NIC, right-click the virtual machine

and choose Edit settings. You can see the NIC to Network association in there.

So go ahead and enter the WAN interface name, "em1", and press return.

Now enter the LAN interface name, "em0", and press return.

As we don’t have any OPT interfaces, yet, press return.

Lastly, check that you’ve got the interface assignments around the right way, enter "y" and

press return.

After a short interval pfSense will reconfigure itself, restart and you should be presented with

the main pfSense screen, above.

If you had your modem connected to the physical WAN port of your ESXi host, you'll see

that the WAN interface has received an IP address, via DHCP, from your ISP or your modem.

Other types of WAN connections and configurations are beyond the scope of this article.

You'll need to go elsewhere to read up on those.

The LAN interface has its installation default IP address of 192.168.1.1. If want to use another

network address and/or subnet you’ll need to start reading the pfSense documentation.

Remember that my LAN, as shown at the beginning, was 192.168.111.0/24

Adding a DMZ

Having a WAN and a LAN is fine but perhaps you would like to add another virtual machine

to your virtual network – maybe a mail server or a web server. After all, that is likely to be

one of the reasons you decided to use ESXi in the first place – as an alternative to running

multiple physical machines.

You probably want to make these servers accessible from the Internet but, at the same time,

protect them by putting them behind your pfSense firewall. That way you can control all

access to them from both the LAN and the WAN.

Another interesting aspect of virtualization is that you don't have to stop at one DMZ.

Because the DMZ network can be completely virtual, you don't need any additional physical

NICs. You could, for example, attach a virtual mail server to one DMZ and a virtual web

server to another. Then, by connecting them through pfSense with virtual NICs, you can fully

control all access between the DMZs. In addition, if one server is compromised, access to any

of the others will be just that little but harder.

That's not to say that a DMZ can't also be connected to a real physical network as well. You

may want to connect a game console or video/music server behind pfSense but not have it

directly connected to your LAN. All you need to do is connect a physical NIC to the DMZ.

Creating the DMZ network

Go to your vSphere client and highlight your ESXi host. Click the Configuration tab and the

Networking link. You will see your ESXi network diagram.

Click the Add Networking link near the top right of the Network pane.

We want to add a new virtual machine network, so select that option and click Next.

Chose the option to Create a vSphere standard switch. We aren’t going to need a physical NIC

– it’s just going to be virtual - so just make sure that, if you have more physical NICs in your

ESXi host, none of them are selected, then click Next

As with the LAN and WAN, give the new network a name. “DMZ” would be good. Click

Next.

Click Finish.

Now your Networking diagram will look like this - just a vSwitch and a Virtual Machine Port

group called “DMZ” with no physical NICs attached.

The next step is to connect the pfSense to this new DMZ network. Right-click your pfSense

virtual machine and select Edit Settings. Click the Add button.

Choose Ethernet adapter and click Next.

As you did for LAN and WAN, choose the E1000 type of virtual network adapter. Select

DMZ from the drop-down list of available networks and choose Connect at power on. Click

Next.

Now your network diagram should look like the above.

Note that you can do all of this while the pfSense virtual machine is still running. To make

pfSense aware of the changes, though, you will need to restart it and go through assigning the

interfaces again.

Now you can attach additional virtual machines to the DMZ network.

Installing VMware Tools

There are a number of benefits to installing the VMware tools, including better memory

management, as well as improved network and disk performance. I can't vouch for those

benefits but I find the most useful feature is the ability to shutdown or reboot a virtual

machine without needing to log in to it directly. I use this to have all my VMs and the ESXi

host gracefully shutdown in the event of a power outage that might exhaust the UPS battery ...

but that's another story.

The VMware Tools have been made available as a pfSense package, which makes the install

very quick and easy.

Log in to the pfSense Web GUI and click System > Packages.

From the Available Packages list, look for the Open-VM-Tools package and click the +

button on the right to install the package. Confirm that you want to install the package.

There is really nothing to configure with this package, it should just work.

Alternately, the official VMware tools can be used, but it is a much more manual process. See

here: VMware Tools

A note about time synchronization - NTP

Vmware Tools will allow you to have the clock of the pfSense virtual machine synchronized

with the clock of the ESXi host. You would need to have the ESXi host use NTP to maintain

its own clock for that to work but you also need to disable NTP on the pfSense virtual

machine.

Unfortunately, using the host to control the clock of the pfSense virtual machine has been

known to cause a problem with the time appearing, to pfSense, to run backwards.

My recommendation is to have pfSense and the ESXi host maintain their clocks

independently.

You should never use a virtual machine as a time source for the ESXi host.