Vmware Guide May 2010

Sponsored by VMware

A Guide to Virtualization Hardening Guides

A SANS Whitepaper – May 2010 Written by Dave Shackleford

Network Security and Access Controls

User and Group Security

Logging and Auditing

Guest/Host Interaction Controls

Management Server Controls

Additional ESX and ESXi Controls

SANS Analyst Program 1 A Guide to Virtualization Hardening Guides

Introduction

As virtualization adoption grows, most security teams are already developing internal policies and processes that define how virtual infrastructure should be enabled and maintained. How-ever, virtualization is a complex technology with many facets and numerous types of controls that can be implemented to secure virtual assets and their host machines.

VMware has released several guides explaining how to securely configure VMware’s Virtual Infrastructure 3 and the newest version of VMware’s enterprise solution, vSphere.1 Several other well-known guides have been released from organizations such as the Defense Informa-tion Systems Agency (DISA)2 and Center for Internet Security (CIS).3 With so many guides to turn to, many security analysts are left wondering what guidelines to apply and how to apply them in accordance with particular organizational needs.

With its latest “vSphere Security Hardening Guide,” VMware has broken its configuration guid-ance into more tactical, actionable categories and scenarios to align more closely with other sources of guidance. The guide breaks down controls for a variety of network environments. For example, it includes recommendation levels for systems in general enterprise environ-ments, demilitarized zones (DMZs), and Specialized Security Limited Functionality (SSLF) for high-security environments. As of this writing, both the DISA and CIS guides apply to VMware’s Virtual Infrastructure 3 (and ESX Server 3.5.x), which were released before vSphere. Still, we can apply most of the controls in the DISA and CIS guides to vSphere as well as to Infrastructures using 3/ESX 3.5.x.

For defense and military organizations with high-security requirements, controls with Special-ized SLF settings might be more applicable, whereas Enterprise Settings might be more rea-sonable for general business use. For example, security teams that have relied on the DISA STIG may evaluate vSphere hardening guidelines’ SSLF controls as complements to their existing base set of configuration options.

To help organizations use these guides effectively, SANS created a guide to the guides that includes key configuration and system security settings for VMware ESX and vSphere/Virtual Infrastructure as detailed in the VMware vSphere, DISA and CIS guides. The intent is to help organizations steer themselves toward the guidance areas that make the most sense for their needs. This paper is not a comprehensive replacement for these guidelines, which should be consulted in-depth and should be checked often for updates.

This document covers key control areas organizations need to consider and dis-cusses applicability under the categories of network security and access, user and group security, logging and auditing, guest/host interactions, management server and additional ESX and ESXi controls.

1 http://communities.vmware.com/docs/DOC-12306 2 http://iase.disa.mil/stigs/stig/esx_server_stig_v1r1_final.pdf 3 www.cisecurity.org/tools2/vm/CIS_VMware_ESX_Server_3.5_Benchmark_v1.2.0.pdf


Network Security and Access Controls

Access controls and proper network segmentation are key requirements in many compliance mandates, ranging from the Payment Card Industry Data Security Standard (PCI DSS) to the Federal Information Security Management Act (FISMA) and other federal regulations. Creating network boundaries and enforcing traffic flow among distinct network segments is a security best practice that should be considered fundamental to securing both physical and virtual environments.

Specific threat vectors include exposure of traffic to attackers and sniffers (as was the case at Heartland Systems), as well as common network-based attacks such as spoofing and man-in-the-middle. In this section, we cover virtual network components within VMware vSphere and a number of security settings impacting traffic flows among the ESX host and virtual guests, multiple virtual guests, and guests with the physical network. In addition, we also cover the ESX server’s built-in firewall configuration options.

4 www.scmagazineus.com/electronic-payments-hurricane-gonzalez/article/158017


Control

Isolate VMotion tra!c (to protect con!dentiality of virtual network tra"c)

Prevent MAC address spoo"ng in a virtual environment (to prevent spoo!ng and man-in-the-middle attacks)

Con"gure the ESX Firewall for High Security (to prevent abuse of unnecessary ports and services)

Manage network access control/segmentation (to protect powerful control data)

VMware vSphere 4.0 Hardening Guide

NAR02: Ensure VMotion tra!c is isolated

VMware has guidance for both physical NIC separation and vSwitch and port group-based separation in typical enterprise and more security-conscious environments, respectively.

NCN03: Ensure the “MAC Address Change” policy is set to “Reject”

NCN04: Ensure the “Forged Transmits” Policy is set to “Reject”

VMware recommends implementing these controls for all environments unless clustering, vShield Zones, or other partner products are needed.

CON01: Ensure ESX Firewall is con"gured to High Security

NAR04: Strictly control access to Management network

DISA ESX Server STIG V1r1

ESX0030: Dedicated physical NIC for VMotion tra!c

ESX0040: Dedicated virtual switch and VLAN for VMotion

The DISA STIG mandates a separate physical NIC for VMotion tra"c.

ESX0250: Con"gure the MAC Address Change to “Reject” on all virtual switches

ESX0260: Set “Forged Transmit” to “Reject” on all virtual switches

DISA makes exceptions for clustering, legacy applications, and licensing issues, if documented.

ESX0320: Con"gure the ESX "rewall at the High Security level

ESX0130: The Service Console and VMs should be on separate VLANs or network segments

CIS ESX Benchmark v1.2.0

1.1.1 Do Not Use the Management Network for the Virtual Machine Network

CIS only suggests a separate VLAN and port group for VMotion tra"c.

1.5.1 Protect Against MAC Address Spoo"ng, Forged Transmits, and Promiscuous mode

CIS treats this as a Level 1 control, indicating it is a best practice control with minimal impact that should be implemented if possible.

1.5.2 Con"gure the Firewall to Allow Only Authorized Tra!c

1.1.1 Do Not Use the Management Network for the Virtual Machine Network

Recommendations

Vmotion tra"c is in cleartext and should be protected from other tra"c and access, usually by segmentation via a separate vSwitch or port group.

Virtual or physical separation of this tra"c should be implemented in all environments, regardless of security level or compliance requirements.

In highest-security or compliance environments, a separate physical NIC is recommended.

Setting MAC Address Change and Forged Transmits to Reject can adversely a#ect production systems such as Microsoft Clustering and vShield Zones.

These are important controls, but may break functionality. If availability is a primary concern, consider avoiding these controls. If integrity of the environment and data con!dentiality are more important, then implement this control.

Both VMware and DISA recommend High Security, while CIS is more general. Unless additional ports and services are needed, this should be set for all environments. ESXi does not currently have a built-in !rewall, but it does have a local reverse proxy that drops tra"c on unrecognized ports by default.

By default, the High Security setting only allows ports needed for virtualization operations inbound and outbound to the ESX server. Many organizations need additional ports opened for other forms of tra"c. It is strongly recommended they open those ports inbound and outbound explicitly instead of changing the !rewall security level to Medium (all outbound permitted) or Low (all tra"c allowed).

All three guides are straightforward in this guidance: Because the management network contains sensitive data and management interfaces could potentially expose powerful control and administration capabilities, they should be separated from other network areas.

VMware Con!guration Guidance

mkretzschmar

Highlight


User and Group SecurityLocal users and groups on ESX platforms need to be managed properly to ensure local security and compliance. Controlling users and groups (and requiring strong passwords) is a compo-nent of most major compliance mandates. PCI DSS Section 8, for example, details numerous requirements for user accounts and password complexity, length, and history. In this section, we look at groups, authorization and authentication control areas discussed in our leading guidelines.

Control

Use Directory Services for User and Group Authentication (to protect resources from unauthorized access)

Implement sudo for privilege control (to prevent abuse by privileged insiders and escalation attacks)

Restrict root access via SSH (to prevent brute force access)


COP01: Use a Directory Service for Authentication

COA05: Limit access to the su command COA06: Con"gure and use sudo to control administrative access

COA03: Ensure root access via SSH is disabled


The DISA guide has no speci!c recommendations for this control.




The CIS guide has no speci!c recommendations for this control.

1.3.2 Implement SUDO

1.3.1 SSH Access

Recommendations

For organizations with large numbers of users accessing the ESX Service Console, authentication via Active Directory, LDAP or NIS can be con!gured. This is much easier to maintain over time but will only apply to certain types of organizations. This control will simplify management of users and groups on ESX systems.

Sudo should be implemented to control who has access to privileged !les and commands (e.g., su. Internal policy should dictate what type and level of sudo control is implemented). All organizations should implement this control to restrict use of the root account and provide detailed logging and audit trails for use by audit and compliance teams.

This is a well-established security best practice and is easily enabled on all ESX servers with a single con!guration entry. All organizations should implement this control to prevent unauthorized remote access to ESX systems via a compromised superuser account.

Following audit trails for any root access is di"cult to associate with an individual administrator, so audit is not enough. In general, direct root logins should be disallowed.


mkretzschmar

Highlight

mkretzschmar

Highlight

mkretzschmar

Highlight


Logging and AuditingLogging and auditing are critical functions on all virtualization platforms in order to meet com-pliance mandates as well as for system monitoring, troubleshooting and forensics follow-up. In a virtualized environment, logging and auditing policies need to pay particular attention to the ESX server, virtual machines, and any management components such as vCenter server. Although not addressed specifically below, the vSphere guide also includes a number of recommendations for VMware’s more “lightweight” hypervisor architecture, ESXi. Many of the recommendations are similar, with the primary differences being the syntax and commands that enable the controls.

Control

Record critical logs (to meet regulatory requirements for monitoring access to critical data, follow-up and legal cases)

Con"gure NTP for time synchronization for logs (for forensics and incident response)

Maintain "le system integrity (for incident response and regulatory compliance)


COL01: Con"gure syslog logging:/var/log/vmkernel

/var/log/secure

/var/log/messages

/var/log/vmware/*log.

/var/log/vmware/vpx/vpxa.log

COL02: Con"gure NTP time synchronization

COH03: Establish and Maintain File System Integrity


ESX0410: ESX Server should record major critical log "les:/var/log/vmkernel

/var/log/vmkwarning

/var/log/vmksummary.html

/var/log/vmware/hostd.log

<path to VM>/vmware.log

/var/log/vmware/vpx/vpxa.log

/var/log/vmware/webAccess

/var/log/messages

/var/log/secure

ESX0440: Syslog should send logs to a remote server

3.3.1.6 Time Synchronization

DISA mandates hashing authentication between NTP peers to prevent tampering, which should be followed if possible. This is important, but not crucial.

3.3.1.4 File System Integrity


1.4.2 Review Logs

1.4.3 Con"gure syslogd to Send Logs to a Remote LogHost

1.2.4 Con"guring NTP

1.9 File/Directory Permissions and Ownership

Recommendations

Security best practices dictate sending log !les to a remote log server. All three guides provide the same guidance to protect against modi!cation (integrity), local read access (con!dentiality) and backup (availability) to a remote system.

The DISA guide provides the most granular guidance on which logs to maintain: Follow its recommendations in most cases, regardless of security level.

Within system log !les like /var/log/messages, multiple levels and types of logs can be retained per policy. All organizations will want to preserve some logs for troubleshooting, incident analysis, forensics and speci!ed compliance mandates.

The CIS guide provides the most granular detail on con!guring Network Time Protocol (NTP), which should be applied to all systems regardless of security level.

Maintaining consistent time on all ESX systems is important for log management and correlation, incident handling and forensic timelines, and operational activities (e.g., running scheduled scripts).

The ESX !le system has a number of critical !les that should be monitored for changes and accidental deletion or corruption. By creating a catalog of !le permissions and hashes representing !le state, administrators can monitor and maintain these !les to ensure they aren’t accessed or modi!ed without permission.


mkretzschmar

Highlight

mkretzschmar

Highlight

Guest/Host Interaction ControlsVirtual Machines (VMs) have calls and interactions directly with their hosts that have been proven vulnerable. Organizations should limit and control the Guest-Host interactions to prevent poten-tially serious security breaches. By restricting some of these capabilities, organizations can better protect the confidentiality of data passing into and out of guest VMs, as well as the availability of disk space on the ESX host in one particular case. Virtual machines running on VMware platforms can be configured with numerous options that dictate communication and interaction with the underlying ESX or ESXi hosts, as well as with each other. These settings are primarily found in each virtual machine’s .vmx configuration file.

Control

Disable Copy/Paste to Remote Console (to prevent copy/paste data from being placed on clipboard between VM and the local machine)

Disable unnecessary devices within virtual machines (to minimize threat landscape)

Prevent connection and removal of devices from virtual machines (to prevent data leakage, malware on external drives, and other security threats)


VMX03: Disable Copy/Paste to Remote Console

VMX10: Ensure Unauthorized Devices are Not Connected

VMX11: Prevent Unauthorized Removal, Connection and Modi"cation of Devices


ESX0970: Disable the VMware Tools clipboard capabilities [copy and paste] for all virtual machines.


ESX1000: Disable the following con"guration tools:

DISA guide provides a long list of controls and tools in their document.


1.8.3 Disable Cut and Paste


1.8.1 Remove Guest Control of Hardware Devices

Recommendations

By disabling copy/paste, data cannot be placed on the clipboard between the VM and local machine. This should be applied to all VMs, if possible, with the following con!guration options:

isolation.tools.copy.disable=TRUE

isolation.tools.paste.disable=TRUE

isolation.tools.setGUIOptions.enable=FALSE

isolation.tools.dnd.disable=TRUE

This control protects both con!dentiality (of data) and integrity (of VMs and ESX hosts). Availability of clipboard data is negatively impacted if copy/paste is disabled.

This control, which disables CD-ROM, $oppy, and other devices, is recommended, particularly for high-security environments.

Preventing VM users from enabling and disabling devices like DVD and USB drives should be considered a fundamental con!guration setting to prevent leakage of intellectual property. All three guides provide similar guidance. In most cases, individual VM users need to be restricted more than administrators.



mkretzschmar

Highlight

mkretzschmar

Highlight


Management Server ControlsProtecting management systems is akin to controlling the interaction of privileged users and accounts on most operating systems: This interaction is necessary for maintenance and opera-tion and should be carefully monitored and protected.

The security of a vCenter system is paramount to the overall security posture of the virtualized environment. Compromise could potentially lead to complete attacker ownership of ESX and ESXi hosts, virtual machines, and virtual networks. In most large deployments, ESX systems and virtual machines are managed by vCenter servers (formerly called VirtualCenter in the older VMware product line). Installed on Windows systems, with database components and other important applications, vCenter server controls have several components to consider.

Control

Control local vCenter server Administrator access (to prevent administrator abuse of over-privileged Windows systems)

Avoid using default self-signed certi"cates for SSL communication with vCenter (to prevent potential man-in-the-middle attacks)

Restrict network access to vCenter (to prevent exposure from untrusted networks or systems within the environment)

Protect the Windows installation on a vCenter host platform


VSH05: Install vCenter Server using a Service Account instead of a built-in Windows account

VSC01: Do not use default self-signed certi"cates

VSC05: Restrict network access to vCenter Server system VSC06: Block access to ports not being used by vCenter

VSH03: Provide Windows system protection on the vCenter Server host


ESX0710: Create a dedicated VirtualCenter administrator within the Windows Administrator Group on the Windows Server for managing the VirtualCenter environment



The DISA guide has no speci!c recommendations for this control, although general VirtualCenter installation guidelines are discussed.



1.3 System Access, Authentication, Authorization, and User Accounts



Recommendations

Creating one or more dedicated users on local Windows servers running vCenter is critical because the built-in Windows service account has more privilege than necessary. This is a requirement for DISA and is suggested by VMware. This is a general best practice that should be implemented for all organizations running vCenter on Windows platforms.

Trusted certi!cates (from an internal Certi!cate Authority or a third party) assure users that the systems and services they’re interacting with are approved and trustworthy. Attackers often try to trick users into accepting self-signed or bogus certi!cates that they have created in the interest of intercepting data. The VMware guide has the best description of this control, which should be implemented if at all possible.

The VMware guide recommends using local or network !rewalls to restrict network access to vCenter and blocking local ports that are unnecessary for operation. This is a good practice to follow, if possible, and will rely entirely on existing controls. vCenter servers should be placed on a management network segment that is isolated from all production tra"c and systems. Only a limited number of users and workstations should be able to access this network at all.

This control describes essential security best practices (e.g., hardening and patching Windows systems and using antivirus tools, when possible). Consider this control mandatory. The VMware guide has the most detail. When possible, leverage existing patching/con!guration management processes and controls.



Additional ESX and ESXi ControlsFollowing are several other controls to consider that are specific to VMware virtualization environments.

Control

Do not manage ESX or ESXi like a Red Hat host

Do not apply Red Hat patches


COM03: Do Not Manage the Service Console as a Red Hat Linux Host

COM01: Do not apply Red Hat patches to the Service Console





Appendix C: CIS Red Hat Enterprise Linux 5 Benchmark

1.2.1 Keep System Patched

Recommendations

Despite similarities, ESX and ESXi are very di#erent from Red Hat Linux, and need to be managed as unique platforms.

There is confusion on this point among the guides. The VMware guide speci!cally states that ESX should not be managed as a Red Hat system, whereas the CIS guide suggests setting con!guration options in ESX as you would for Red Hat (Appendix C). The general security community suggests following VMware’s guidance here.

All guidance available speci!cally states that Red Hat patches should not be applied to ESX or ESXi systems. These patches can seriously impact the integrity of the system and should be avoided.



Conclusion

Organizations need to decide which controls are most applicable to them. The decision depends largely on their business and the nature and location of the systems themselves. For general business and education environments, the VMware and CIS guides provide a good mix of best practices guidance in most areas. For more sensitive military and defense environments, the DISA guide’s more stringent controls and evaluation criteria are a more suitable choice. With the latest vSphere guide, VMware has bridged this gap somewhat by differentiating some controls on exposure level and security needs (Enterprise, DMZ, and SSLF settings). The CIS and DISA guides offer more technical implementation guidance, so all of these guides are useful.

Numerous vendor tools are available that can integrate with VMware’s VMsafe program, which exposes VMware Application Programming Interfaces (APIs) to security and configuration management vendors to allow third-party functionality to be integrated with vCenter and other components.5 For example, shortly after the latest VMware “vSphere Hardening Guide” was released, William Lam produced a Perl script that could automate ESX system assessment and produce a variety of detailed reports on configuration settings within the guide.6

Because more organizations are moving applications and services to the cloud and leveraging virtualization technologies for both private and public cloud infrastructure, the need to ensure consistent, maintainable configuration standards is paramount. The Center for Internet Secu-rity, the Defense Information Systems Agency, and VMware guides provide a wide selection of controls from which to choose for the foreseeable future.

5 www.vmware.com/technical-resources/security/vmsafe.html 6 http://communities.vmware.com/docs/DOC-11901


About the Author

Senior SANS Analyst, Dave Shackleford, is director of security assessments and risk & compliance at Sword & Shield Enterprise Security, a SANS instructor and GIAC technical director. He has consulted with hundreds of organizations in the areas of regulatory compliance, security, and network architecture and engi-neering. He has worked as chief security officer for Configuresoft, chief technol-ogy officer for the Center for Internet Security, and as a security architect, analyst, and manager for several Fortune 500 companies.


SANS would like to thank this paper’s sponsor:

Documents

Vmware Guide May 2010