603D: The UltimateXenDesktop 5.x Troubleshooting Workshop

Fred DonovanSenior Support Product Manager

May 2012

Rene AlfonsoArchitect, Technical Support

Next Generation Support: TaaS

Personal vDisk Troubleshooting

XenDesktop 5.x Database

VDA High Availability Mode

XenDesktop Troubleshooting

Brief Architectural Review

Session Topics

Lab

Lab

Lab

Lab

Brief Architectural Review XenDesktop

The XD DDC Core Architecture

XenDesktop 4.xController Services

Licensing

LicenseServer

Workstation(VM or blade PC)

Workstation(VM or blade PC)

VDA(VM or blade PC)

CGP SvcPortICA Drivers

PortICA Svc ICA

CGP Client(via AG)

1494

2598

DomainController

LDAP

DataStore

ADO

Pool Manager

VM HostWeb Svc

XML Service

HTTP

WI80, 443

DDCServers

IMA

2512

WCF

8080 4.x80 5.x

Controller Service

Desktop Service

XenDesktop 5.xController Services

Machine Creation

AD Identity

MachineIdentity

Host

Configuration

Broker

Broker Services

Broker Service

IMAXMLSSL CDS Pool Management

XenDesktop 4:

XenDesktop 5:

443 80

443 80

Database Management XML components Hosting

Management PowerShell SDK VDA Management Licensing

• IMA Service is replaced by various .NET services communicating over Windows Communication Foundation (WCF)

• New relational SQL database

• No controller to controller communication each controller talks to the database

New Desktop Delivery Controller (XD5.x)

SQLDatabase

DDC1

DDC2

Database

Controller#2

Controller

#1

• Controllers no longer hold state information; it is now located in the database

• No Local Host Cache

• Hypervisor connection maintained by one controller(Get-BrokerHypervisorConnection)

Controller 5.x Roles & States

High-level Service-Oriented Architecture

Desktop Delivery Controller

Broker Service

Machine CreationService

AD Identity Service

Machine Identity ServiceHost Service

Configuration Service

SQL Server

Desktop Studio Desktop Director

PowerShell WCFSDK available for automation

Virtual Desktop

WinRM(WMI)

VDA Registration (CTX126704)

Allows Broker to control VDA

Power Management

Open TCP Ports

3 Strikes before VDA is placed in Maintenance Mode(MaxFailedRegistrationsAllowed)

After a period of time (20 Min) unsuccessfully registered VDA shutdown(MaxRegistrationDelayMin)

VDA Registry Based Registration

VDA initiates communication over WCF and provides its SID

Broker Verifies Computer AD Object

Broker then communicates directly to VDABroker verifies the VDA indatabase and sendsinformation

Registration Success Results

VDA

Controller

VDA initiates communication over WCF and provides its SID

Broker verifies computer AD object

Broker then communicates directly to VDABroker verifies the VDA in database and sends configuration information

Registration completes successfully

• Registry based VDA registration from ListofDDCs registry key on the

VDA

• Uses WCF/ Connection Brokering Protocol

•Validates VDA, test call-back and writes state into database

• VDA broker service role handles launch sequence, status updates

and session control

Broker service and VDA registration (XD5.x)

Registration Types:

Soft – sets up minimal ping support to maintain heartbeatHard – fully configures the VDA with desktop group membership

Under The Hood: VDA Registration (Active Directory)

VDA

VDA DDC

AD

WCF

DDC Server

Broker Service

Service starts and looks up the farm OU in the registry

Service queries DC for all Service ConnectionPoint objects (DDCs) in that OU

DDC service receives connection (Kerberos prevents man-in-the-middle)

OS retrieves Kerberos ticket for DDC

Service looks up VDA computer account

Service selects one DDC at random, looks up DDC computer account and initiates a connection through WCF

DDC checks that computer account (SID) is valid in AD

Controller initiates WCF connection to VDA (using peer IP address and Kerberos ticket)

Connection succeeds and VDA is marked registered

Controller validates that caller is member of “Controllers” group, and sets configuration

LDAP

Desktop Service

CDF Tracing andCDF Control

What is CDF tracing?

Introduction to Citrix Diagnostic Facility Tracing

• A unified debug tracing system integrated into various Citrix products

• It is high performance tracing, supported by the Microsoft OS (ETW)

• It allows various components, such as DLL’s, EXE’s and drivers to be traced

• A single architecture that an Engineer can leverage to troubleshoot problems

XenDesktop, XenApp, Receiver

• A GUI based CDF trace capture and analysis tool was developed

• Expert Control Feature

Introduction to CDFControl (CTX111961)

• Expert Control finds the messages that are *meaningful*

• Known issues are automatically flagged

• Knowledge Base articles are automatically launched

Expert Control

• Can be run on both the DDC and VDA to troubleshoot registration issues

• Used to collect data related to basic components

• Will verify if the components are working correctly

• Verify Domain Membership

• Network Interfaces

• WCF Endpoints

• Services

• DNS lookup

• Time difference between machine and Domain Controller

XDPing Tool (CTX123278)

• Check Domain Membership /Time Synchronization

• Check value under HKLM\Software\Citrix\VirtualDesktopAgent\ListOfDDCs

• Check Citrix Desktop Service warnings and errors•Look under Error Details for the exact condition

• Get-BrokerDesktop -MachineName DOMAIN\MACHINE•LastDeregistrationTime LastDeregistrationReason VALUES

• See CTX117248 for details

Troubleshooting Tips for VDA Registration Issues

Lab #1 VDA Registration Troubleshooting

URL http://training.citrixsynergy.net/

Class Code: <Provided by Instructor>Requirements Latest Citrix ICA Client

Lab Environment

Push button easy data collection system

Citrix Scout / XD Collector (CTX130147)

• Makes data collection and upload push button easy

• Integrates data collected by Scout with the Citrix Tools as a Service

(TaaS) backend

• Simplifies data collection & analysis

• VDA User Experience Monitoring •Diagnostics•Configuration•Performance

• Remote VDA Monitoring 2.x

• Available for XD5.x XA 6.x

HDX Monitor 2.x (CTX123058)

Lab #2HDX Monitor

• XenDesktop 5 built on PowerShell

• Desktop Studio PowerShell Tab

• Citrix Developer Network

•http://www.citrix.com/cdn

•Code Share

•SDK Documentation

XD PowerShell SDK

PowerShell for troubleshooting

• Add-PSSnapin citrix*• Get-BrokerServiceStatus

Returns OK if broker service is responding to requests

• Get-BrokerDBConnectionDisplays SQL connection information

• Get-ConnectionLog *New in XD 5.6Displays user connection history both success and failures

Checking Broker State

Lab #3PowerShell

XenDesktop 5.x VDA High Availability Mode

SQLDatabase

Controller1

Controller2

VDA Registration

ICA File

ICA ConnectionDedicated Virtual Desktop

ICA Ports14942598

• VDA can operate in high availability mode if all controllers fail in a XenDesktop 5 site

• Dedicated Desktops Only

• Registry enabled – HighAvailability REG_DWORD 1

• Configurable time period (default 5 minutes)

• Must register with controller within 30 days

• Prompt for password EnforceAutoLogon REG_DWORD 0

XenDesktop 5 HA Mode (CTX127392)

• Citrix Desktop Service• Event ID 1014 controller communication failure• Event ID 1012 controller communication success

• CDF Trace Event (on VDA)• CdsWorkerAgent:2:1:Register FAILURE: HighAvailabilityActive = True,

InHighAvailabilityMode = True, _firstRegistrationAttemptTime = xx/xx/xxxx xx:xx:xx, HighAvailabilityRegistrationTimout = 00:05:00

• ICA File

• http://support.citrix.com/article/ctx127392

HA Mode Demo / Lab

Lab #4VDA HA Mode

XenDesktop 5.xDatabase Overview

• Microsoft SQL Server 2008 R2 (SP1 supported)

• Microsoft SQL Server 2008 w/SP1 or later

• Microsoft SQL Server 2008 R2 Express Edition(on media, SP1 supported)

• Microsoft SQL Server 2012 (see CTX132438)

• Supported Databases For Citrix Products CTX114501

Support Database System Requirements

• It’s important to consider DB sizing during initial setup of an environment

• Sizing depends on several factors

• # of registered VDAs• # of connected sessions• Connection rate• # of managed desktops• # of provisioned desktops

Database sizing information (1 of 2)

• A formula from the test teams has produced the following data - CTX127939

Database sizing information (2 of 2)

Unmanaged Desktops 5,000 10,000 20,000

Per Worker (KB) 14,500 29,000 58,000

Per Session (KB) 25,500 51,000 102,000

Per Connection (KB) 840 1,680 3,360

Approx. Total (MB) 40 80 160

Provisioned Desktops Using MCS

5,000 10,000 20,000

Per Worker (KB) 14,500 29,000 58,000

Per Session (KB) 25,500 51,000 102,000

Per Connection (KB) 840 1,680 3,360

Per AD Account (KB) 9,000 18,000 36,000

Per MCS machine (KB) 9,700 19,400 38,800

Approx. Total (MB) 59 117 233

• Schemas are a type of container used to organize database objects and permissions

• Multiple SQL schemas exist in the database

ADIdentitySchema DesktopUpdateManagerSchemaChb_Config (broker) HostingUnitServiceSchemaChb_State (broker) MachinePersonalitySchemaConfigurationSchema

• Schemas map to the Windows services on the Controller

Database schemas

How controllers connect to the DB

Controller1

Controller2

SQL Server Instance

SQL Logindomain\Broker2$

XD5 Database

Tables + SPs

SQL Logindomain\Broker1$

DB Userdomain\Broker1$

DB Userdomain\Broker2$

DB Roles

• SysAdmin rights needed for special administrative operations

• Each database user maps to database roles with specific rights to corresponding objects -

Database Security (CTX127998)

• ADIdentitySchema_ROLE

• Chr_Broker

• Chr_Controller

• ConfigurationSchema_ROLE

• DesktopUpdateManagerSchema_ROLE

• HostingUnitServiceSchema_ROLE

• MachinePersonalityServiceSchema_ROLE

•Live session state held in database, including registration info

•One transaction per desktop for health ‘ping’

• Default period 30 seconds per desktop

•Small # of transactions per desktop launch

•Automatic database clean-up (purge stale records)

•Controller health heartbeat• Loss of database access = dead broker

• No heartbeat other brokers will take over responsibilities

Database Activity

Database Maintenanceand Troubleshooting

Monitoring SQL Connections

BrokerService

Citrix.ADIdentity.SdkWcfEndPoint

Citrix.Configuration.SdkWcfEndPoint

Citrix.Host.SdkWcfEndPoint

Citrix.Identity.SdkWcfEndPoint

Citrix.MachineCreation.SdkWcfEndPoint

Controller SQL Connections

SQL Activity Monitor

Right Click SQL Server Activity Monitor

• Database Avg Transaction Time

• Database Connected

• Database Transaction/sec

• Database Transaction Error/sec

• Brokered Sessions

Performance Counters

• A transaction log is the recording of all changes performed against a database

• Recovery model options

• Simple – Reclaims log space but can only recover to the last backup (default option for XD5 database)

• Full – Can recover to any point in time but requires the most disk space

Managing the transaction log (1/2)

• Average transaction log sizing formula

• A VDA with no activity generates approximately 62KB of transaction log data per hour

• # of VDAs X 24 Hours X Approx. 62KB of data• 100 VDAs X 24 Hours X 62KB = 145MB

• Set the log to a fixed size and backup the log when it reaches 50%.

• Place the log on a separate physical disk

• http://support.citrix.com/article/CTX126916

Managing the transaction log (2/2)

• Database Integrity

• Index Maintenance

• Update Statistics

• Database Backups

SQL Maintenance Plans

XDDBDiag (CTX128075)

• Overview

○ Command line tool

○ Performs database consistency check on data

○ Collects diagnostic Information

Site Information

VDA Information

Current Connections / Connection Log

Hypervisor Connections

Policy Information

Desktop Groups

Controller Information

XDDBDiag (CTX128075)

• Usage

○ XDDBDiag 2.1 Citrix Support

○ To connect to a local SQL Express database use [local] switch

○XDDBDiag [local]

○To pass database connection settings via command line

○Command line Parameters [windows] [SQL server] [SQL Database]

○New SQLStats

– Database Backup History–SQL Connections–SQL Version

Lab #5General Database Troubleshooting

XenDesktop 5.6Personal vDisk Troubleshooting

Standard Pooled Desktop

User Environment

User Data User Settings

User Installed

Applications

Corporate Installed

Applications

Profile ManagementProfile Management

OSCommon Base Image

Citrix Personal vDiskCitrix Personal vDisk

Pooled Desktop with Personal vDisk

A personal vDisk is created for each provisioned virtual machine. During initial session launch that VM is assigned to a single user.

User personalization (settings and installed applications) are stored in the personal vDisk that is merged with the base VM image during session launch.

52

Citrix Personal vDisk

On base VMas the P: Drive

On Hypervisoras <vm_name>_pvdisk

Drive Location

53

Drivers and Components

Drivers

Ivm.sys

Ivmvhd.sys

Ivmboot.sys

Ivmpnp.sys

CtxPvd.exe

CtxPvdSvc.exe

PvDWMI.dll

VhdTool.exe

Components

These drivers load at system boot time and are responsible for applying the virtualization logic

These components handle the initial inventory and image updates as well as VHD management

and performance statistics

54

Personal vDisk Process Flow

Inventory Process of

Base VM(Initial or Image

Update)

Pooled Desktop Start-Up

Virtual Logic Application

PersonalizedDesktop

55

• Desktop Director includes helpdesk-facing PvD metrics and support◦ % of application area in use / total size◦ % of user profile area in use / total size◦ PVD reset

• PVD reset allows the helpdesk to reset the application area while leaving the user’s data intact:◦ Similar to “reset to factory default”◦ Useful when user installed applications are causing

problems with the desktop image

Desktop Director and Personal vDisk

PvD maintains logs in the base of the volume attached to the VM

• Run ctxpvd.exe –log to create a single PVDLOGS folder created under the %temp% directory

Review IvmSupervisor.log …

• Located in the root of the P: drive

• Windows event logs may also be helpful

Log Files

Lab #6Personal vDiskTroubleshooting

TaaS Preview

Critical Tool Matrix for XenDesktop CTX129149

PowerShell SDK CTX127254

See CTX107572 For a Complete List of All Citrix Support Tools

XenDesktop 5.x Database Sizing and Mirroring Best Practices CTX127939

How To Move an Existing XenDesktop 5.x Database CTX128365

PvD FAQ CTX131553

Session Take Away

Launch your browser and type http://training.citrixsynergy.net

Your session code is:

“session code”

Lab Environment Login