Customizing the PowerPivot Management DashboardSQL Server Technical Article

Writer: Ankur Goyal

Technical Reviewers: Artur Pop, Dave Wickert, John Hancock, and Lee Graber

Published: May 2010

Applies to: SQL Server 2008 R2

Summary: The purpose of PowerPivot for Microsoft® SharePoint® is to enable users to share their PowerPivot workbooks in a secure and scalable environment. This white paper focuses on the features that Microsoft SQL Server® 2008 R2 provides to help IT users to manage and understand the self-service business intelligence (BI) activity. It describes key features of the PowerPivot Management Dashboard that is included with PowerPivot for SharePoint. The PowerPivot Management Dashboard is built using PowerPivot, SharePoint Web Part pages, and Microsoft Excel® Services technologies. The primary focus of the white paper is to describe how IT users can extend the capabilities of the management dashboard by creating their own custom reports and dashboards.

Copyright

The information contained in this document represents the current view of Microsoft Corporation on the issues discussed as of the date of publication. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information presented after the date of publication.

This white paper is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED, OR STATUTORY, AS TO THE INFORMATION IN THIS DOCUMENT.

Complying with all applicable copyright laws is the responsibility of the user. Without limiting the rights under copyright, no part of this document may be reproduced, stored in, or introduced into a retrieval system, or transmitted in any form or by any means (electronic, mechanical, photocopying, recording, or otherwise), or for any purpose, without the express written permission of Microsoft Corporation.

Microsoft may have patents, patent applications, trademarks, copyrights, or other intellectual property rights covering subject matter in this document. Except as expressly provided in any written license agreement from Microsoft, the furnishing of this document does not give you any license to these patents, trademarks, copyrights, or other intellectual property.

Unless otherwise noted, the example companies, organizations, products, domain names, e-mail addresses, logos, people, places, and events depicted herein are fictitious, and no association with any real company, organization, product, domain name, e-mail address, logo, person, place, or event is intended or should be inferred.

© 2010 Microsoft Corporation. All rights reserved.

Microsoft, Excel, Silverlight, SharePoint, and SQL Server are trademarks of the Microsoft group of companies.

All other trademarks are property of their respective owners.

ContentsIntroduction.................................................................................................................................................4

Usage Collection..........................................................................................................................................5

PowerPivot Management Dashboard Data.................................................................................................9

Dashboard Cube Architecture...............................................................................................................10

Creating Custom Reports...........................................................................................................................12

Infrastructure........................................................................................................................................14

Popular Workbooks...............................................................................................................................18

Data Sources..........................................................................................................................................22

Publishing Custom Reports to SharePoint.................................................................................................23

Extending the Existing Dashboard.........................................................................................................23

Creating a New Dashboard....................................................................................................................25

Conclusion.................................................................................................................................................30

Acknowledgements...................................................................................................................................31

Appendix...................................................................................................................................................31

IntroductionPowerPivot for SharePoint provides visibility for IT into the activities of end users who are building self-service Business Intelligence (BI) applications. The management experience for PowerPivot for SharePoint is integrated with SharePoint Central Administration. The IT experience for PowerPivot starts with PowerPivot Management Dashboard. The PowerPivot Management Dashboard helps IT not only to find the mission-critical PowerPivot workbooks but also to understand how data sources are used across the company and it helps IT to make mission-critical decisions. This white paper provides a detailed overview of the PowerPivot Management Dashboard, and it also demonstrates techniques for building custom reports on top of the collected data. You can access the dashboard page from the SharePoint Central Administration site (under General Application Settings, click PowerPivot, and then click Management Dashboard). The starting point of the PowerPivot Management Dashboard is shown in Figure 1. The dashboard has seven different Web Parts, and each Web Part shows results\charts on collected usage information.

Figure 1: PowerPivot Management Dashboard page

Web Part # 1: This Web Part displays information about infrastructure; it shows the CPU and memory usage for the PowerPivot Analysis Services service. It also contains a histogram of overall query response for the PowerPivot service application.

Web Part#2: This Web Part contains links to the other pages in Central Administration.

Web Part#3: This Web Part is a Silverlight® control that displays a bubble chart. The chart has axes that represent the number of users and the number of queries sent to a workbook. A sliding bar indicates the date. As you move the pointer over each bubble, the name of the workbook it corresponds to and the number of users that are connected are displayed, along with the number of queries sent to the workbook. Also, as you move the date sliding bar, it shows animation on bubble size which represents how the size of the workbook is growing over time.

Web Part#4: This Web Part is a list view that shows how often a data source is used. You can see how many users are connected to PowerPivot data sources, how many queries were sent, and the size of the data source.

Web Part#5: This Web Part shows a report on the status of data refresh jobs, including jobs that failed to run.

Web Part#6: This Web Part lists data sources, with their respective PowerPivot workbooks, that failed on Data Refresh.

Web Part#7: This Web Part links to other reports and a data connection file (.odc) that in turn links to PowerPivot management data. To extend the dashboard by including custom usage reports, you can add other workbooks to this Web Part.

Usage CollectionAlthough a detailed discussion of PowerPivot usage infrastructure and collection process is beyond the scope of this white paper, this section provides a brief introduction to the usage collection process. This section starts by providing an overview of the workflow, and then it shows you how to perform the individual steps. For a link to troubleshooting tips, see the appendix.

The usage infrastructure allows us to record how the PowerPivot system and workbooks are being used. It is similar to SharePoint ULS logging and based on Event Tracing for Windows (ETW). The SharePoint Usage system generates usage files and collects data from various SharePoint events. Usage files are binary files that contain data in the serialized format. Usage data files (.usage) are collected in the SharePoint log directory (system drive->\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14\LOGS). By default, .usage files are created every 30 minutes. As shown in Figure 2, PowerPivot system service (also known as the MidTier service) uses the SharePoint ETW process to generate .usage files.

Figure 2: Shows the workflow of usage data collection

There are two SharePoint timer jobs (abbreviated to TJ in the figure), which must run in order to move usage data from .usage files to the PowerPivot database, and a PowerPivot health timer job, which collects health information about the application server where PowerPivot system service is running.

To manage configuration and collection settings, start at the Central Administration site, where PowerPivot-related events are also listed, as shown in Figure 3. Click Monitoring, click Reporting, and then click Configure usage and health data collection.

MidTier

Service

XYZ.usageFile

WSS_Logging

Database

PowerPivot

Database

PowerPivot

UsageWorkbo

ok

PowerPivot Health TJ

SP Usage Import TJ

SP APIETW

SP UsageProcessing TJ

PowerPivot Processing TJ

Figure 3: Configure usage and health collection page

By default, only four PowerPivot related events are selected.

Event DescriptionLoads Occurs when a PowerPivot workbook is

loaded as an Analysis Services database.Connections Occurs when a connection is opened to a

PowerPivot database.Requests Logs statistics about requests made to a

PowerPivot database.Unloads Occurs when a PowerPivot database is

unloaded.

None of these events is required; if you clear a check box, usage data that corresponds to that event is not collected. If you do choose to clear one or more check boxes, a set of health rules generates a message alerting you that certain kind of usage information won’t be collected.

The timer jobs are located on the Central Administration site. To view them, click Monitoring, click Review job definitions, and then click View All.

Figure 4: Timer jobs

Usage Data Import timer job: As shown in Figure 2, this timer job moves .usage files from the log directory to the centralized SharePoint logging database. By default, it runs every 30 minutes. To change the setting of the timer job, on the Central Administration site, click Monitoring, click Timer Jobs, and then click Review job definitions.

Figure 5: SharePoint Usage Data Import timer job schedule page

Usage Data Processing timer job: This timer job moves usage data from the SharePoint logging database to the PowerPivot database. By default, it runs once every 24 hours, between 1:00 AM and 3:00 AM.

Figure 6: SharePoint Usage Data Processing timer job schedule page

PowerPivot Health Statistics Collector Timer Job: This timer job collects health information such as CPU and memory usage from every application server in the farm where the PowerPivot system service (the Analysis service) is running. It then moves this data into the PowerPivot database directly. By default, this timer job runs every 15 minutes.

Figure 7: PowerPivot Usage Health collector timer job schedule page

Finally, usage data is collected into the PowerPivot database in the usage tables for the respective PowerPivot service application. Figure 8 shows the location of the tables.

Figure 8: Usage tables under the PowerPivot service application database

PowerPivot Management Dashboard DataThe underlying data for the PowerPivot Management Dashboard is provided as a PowerPivot workbook and stored on the Central Administration site. After usage data is collected into the PowerPivot database, the PowerPivot Management Dashboard Processing timer job updates the workbook on the Central Administration site. By default, the Dashboard Processing timer job runs every 24 hours, in the middle of the night, after the Usage Data Processing timer job completes. After the Dashboard Processing timer job is finished running, it processes (using Full process) usage data into the PowerPivot cube, which is stored at the Central Administration site. To view the cube, click Site Actions, click View All Site Content, and then under Document Libraries, open the PowerPivot Management folder.

Figure 9: PowerPivot Management folder location

Figure 10: An individual PowerPivot Management data folder

As shown in Figure 10, the folder name is a GUID that serves as an internal identifier for the respective PowerPivot Service Application. This folder contains a PowerPivot Management Data workbook. This workbook contains the PowerPivot cube that has all the usage data, some of which is displayed on different Web Parts on the dashboard page. This folder contains 22 different folders; each folder represents a specific language that PowerPivot Management Dashboard supports. Inside each folder are locale-specific Excel workbooks that are been built using the PowerPivot management data cube; for example, the 1033 folder contains workbooks

in English, and the 1041 folder contains workbooks in Japanese. For a list that maps locale IDs to languages, see Appendix 2.

Note: PowerPivot Management Data workbooks contain just the data to be used on the report, not the actual report. The actual reports are in the locale-specific folders.

Dashboard Cube ArchitectureAfter you run the dashboard processing timer job, you can navigate the actual cube data by opening the PowerPivot Management Data workbook using the PowerPivot client. If you go to PowerPivot window, you will notice that this workbook contains twelve tables, which contain the usage data.

Figure 11: PowerPivot Management data in PowerPivot client

You can browse these tables to see the existing data. If you go to the Design tab in the PowerPivot window, you can see the existing relationships by clicking Manage Relationships.

Figure 12: PowerPivot Management data relationships

You can also open the PowerPivot field list to see different calculated measures that are defined in the Loads, Connections, Requests, and Health tables.

Figure 13: PowerPivot field list showing custom calculated measures

Creating Custom ReportsThis section describes how to create reports that use the existing dashboard data. This section also explains the three major categories of reports on the dashboard (Infrastructure, Popular workbooks, and Data sources) and includes examples of custom reports in these categories, based on the existing data. The dashboard includes a data connection file (.odc) that links to the PowerPivot management data.

Figure 14: Reports Web Part on PowerPivot Management Dashboard page

The Reports Web Part on the dashboard is linked to a regular SharePoint document library. You can use the PowerPivot Management Data file to create reports with the data in Excel. However, you must first edit the connection string in the .odc file.

Important: Before you open the .odc file, save the file on your desktop, and then right-click it to edit it in Notepad. Locate the “<odc:ConnectionString>” tag, and then remove the “Embedded Data = False” property from the connection string.

Figure 15: PowerPivot Management Data ODC file in Notepad

Remove the following highlighted property on the connection string:

Provider=MSOLAP.4;Integrated Security=SSPI;Persist Security Info=True;Initial Catalog=;Data Source=http://<servername>/PowerPivot Management/<guid>/PowerPivot Management

Data.xlsx;MDX Compatibility=1;Safety Options=2;MDX Missing Member Mode=Error;Edit Mode=0;Embedded Data=False

After you make the change, save the .odc file back on the Central Administration site under the specific locale folder. After you refresh the dashboard page, you will see an updated .odc file under the Reports Web Part. Double-click the file to start Excel. In Excel, you can now see measures and dimensions that you can use to build custom PivotTable reports and charts.

Figure 16: PowerPivot Excel client connected with PowerPivot Management Data ODC file

The following section describes how to create each type of custom report.

InfrastructureThe usage data in this category contains information about CPU\Memory usage on the application servers where PowerPivot system service is running and information about the workbooks that have been loaded into the memory or unloaded from the memory. The three major fact tables in this category are Loads, Unloads, and Health. These fact tables are related to other dimension tables such as Servers, Documents, Time, and Date.

The Loads fact table stores information about the PowerPivot workbooks that have been used in the SharePoint farm. This table contains information such as the application server (where Analysis Services is running) on which the sandbox is loaded, the size of the workbook, the size

of the sandbox when it is loaded onto the Analysis Server, and user login information. Inside the PowerPivot Management Data workbook, the Loads fact table is related to other dimension tables (as shown in Figure 17).

Figure 17: Relationship diagram for the Loads table

Time and Date are regular time and date dimension tables. The Date table contains fields for 32 different languages that are supported in SharePoint. It also has a field for Today, which is the last date the data was collected in the cube.

Figure 18: Table schema for Servers, Users, Date, and Time tables

The Servers table contains the machine names and Analysis Services instance names of all the application servers where PowerPivot system service is running in the SharePoint farm. The

Loads

Time Date

Servers

UsersDocuments

DocumentVersions

Users table contains information such as user login, name, email, and job title. For every load event, an entry is logged in the Loads and DataSources table, even in the case of two different load events with the same DatabaseID. These two tables have a many-to-many relationship, and the relationship is built using two intermediate tables: Documents and DocumentVersions. The Documents table contains information about every PowerPivot workbook that a user uploads to a SharePoint site. DocumentVersions, another dimension table in the PowerPivot management data cube, contains all the unique DatabaseID values that result from loading a workbook into the memory, making connection to a workbook, sending requests, or unloading a workbook. The DocumentVersions table is a view in the relational database.

Figure 19: Relationship diagram among Loads, Documents, DocumentVersions, and DataSources tables

The Unloads fact table is related to the same dimension tables as the Loads table is in the PowerPivot management data cube (as shown in Figure 20).

Figure 20: Relationship diagram for the Unloads table

Unloads

Time Date

Servers

UsersDocuments

DocumentVersions

The Health fact table contains CPU and Memory usage information of all the application servers where the PowerPivot system service is running in the SharePoint farm. This table is connected to the Time, Date, and Servers dimension tables (as shown in Figure 21).

Figure 21: Relationship diagram for the Health table

The next example shows how to create other reports on top of the collected data. When you double-click the .odc file, Excel opens, displaying a blank sheet on which you can create a PivotTable. On the right-hand side of the window, the PivotTable field list is displayed, where all the measures and dimensions are listed from the PowerPivot management data cube.

Here is an example in this category.

Problem statement: As an IT administrator, I want to know how my application servers are doing in terms of number of loads, connections, and requests in the SharePoint farm.

Solution: Build a custom report using PowerPivot in Excel. Figure 22 shows a screen shot of such a report. It lists application servers where PowerPivot system service is running and shows the total number of loads, connections, and requests that have been made to each of the servers. It also lists the average CPU and Memory usage on each server. Furthermore, you can slice the data with day of the week or with a particular date to see usage on the server for that given day.

Figure 22: Custom report

To generate this report, select MachineName and AnalysisServiceInstance from the Servers dimension table on the Row Labels. You can modify the PivotTable options so that result can be displayed in a tabular format as shown in Figure 22.

HealthTime

Date

Servers

Figure 23: Servers table under PowerPivot field list

To show the required counts, select respective measures from the PivotTable field list. Select the Count of Connections measure from the Connections table; for loads, select the Count of Loads from the Load table; for requests, select the Count of Requests from Requests table; and then select CPU\Memory from Health table (as shown in Figure 24).

Figure 24: PowerPivot field list with selected dimensions and measures

After you add all the dimensions and measures that contain the relevant data, you get a report in the form of a PivotTable. To insert slicers that enable you to view data by date or day of the week, under PivotTable Tools, click Options, and then click Insert Slicer.

Popular WorkbooksThe usage data in this category contains information about the connections that have been made to the PowerPivot workbooks and the requests that have been sent to these workbooks.

The two major fact tables in this category are Connections and Requests. These fact tables are related to other dimension tables such as Servers, Documents, Time, and Date.

The Connections fact table stores information about every single connection that has been made to sandboxes in the farm. This table contains information such as user login, client IP address, type of connection (through ECS, AMO, ADO MD, and so forth), and application server ID on which connection has been made. The Connections fact table is related to other dimension tables (as shown in Figures 25 and 26).

Figure 25: Relationship diagram for Connections table

Figure 26: Relationship diagram for Connections table

Connections

Time Date

Servers

UsersDocuments

DocumentVersions

For connection events, similar to loads and unloads, an entry is logged in the Connections and DataSources tables, because a user can connect to an existing workbook or can connect to a new workbook and, therefore, connection events don’t necessarily have an associated load event. The Connections table is related to the DataSources table using the Documents and DocumentVersions tables.

Figure 27: Relationship diagram among the Connections, Documents, DocumentVersions, and DataSources tables

The Requests fact table stores information about every single request that has been made to sandboxes in the farm. This table contains not only the information about the user and the application server on which the request has been made, but also the information about the type of queries that were issued during the request. For PowerPivot service applications, the query requests are divided in these five buckets: Trivial, Quick, Expected, Long, and Exceeded. Settings for these buckets can be changed on the PowerPivot service application setting page at the SharePoint Central Administration site. The Requests fact table doesn’t store the actual MDX query, but rather the number of queries under different buckets that were sent against sandboxes. This fact table is also connected to other dimension tables like Connections, Loads, and Unloads (as shown in Figure 28).

Figure 28: Relationship diagram for Requests table

Here is an example in this category.

Problem statement: As an IT administrator, I want to know which workbooks in the SharePoint farm have the most connections and requests made.

Solution: Build a custom report using PowerPivot in Excel. Figure 29 shows a screen shot of such a report. It lists the workbook URL and shows the connection and request counts, respectively. Furthermore, you can also slice the data on a given day of the week.

Figure 29: Custom report

To generate this report, select Url from the Documents dimension on the Row Labels. Select DataSource Connections Count and DataSource Requests Count on Values in the PowerPivot field list (as shown in Figure 30).

Requests

Time Date

Servers

UsersDocuments

DocumentVersions

Figure 30: PowerPivot field list with selected dimensions and measures

After you add all the required dimensions and measures, you get a report in the form of a PivotTable. Now, under PivotTable Tools, you can add a slicer to show day of the week on the report.

Data SourcesThe usage data in this category contains information about the data sources that have been used by PowerPivot workbooks. The DataSources fact table stores every single data source that has been used by any active PowerPivot workbook in the farm. This fact table is connected to the Documents and DocumentVersions dimension tables, as shown in Figure 31.

Figure 31: Relationship diagram for DataSources table

Here is an example in this category.

Problem statement: As an IT administrator, I want to know the authors who have created their workbooks on a given data source and how many times their workbooks have been accessed in the SharePoint farm.

DataSources

Documents

DocumentVersions

Solution: Build a custom report using PowerPivot in Excel. Figure 32 shows a screen shot of such a report. It lists the data source, catalog information, the author's name, and the number of times that data source has been accessed.

Figure 32: Custom report

To generate this report, select Source and Catalog from the DataSources and AuthorFullName from the Documents dimension respectively. Then, select DataSource Count as a measure on the Values, as shown in Figure 33.

Figure 33: PowerPivot field list with selected dimensions and measures

Publishing Custom Reports to SharePointAfter you have created custom reports on management dashboard data, you can either publish them to the existing PowerPivot Management Dashboard or you can create a new dashboard page on the SharePoint Central Administration site. This section walks you through both methods.

Extending the Existing DashboardThere are two ways to add your Excel workbooks into the existing dashboard: from the dashboard itself or from the library. From the PowerPivot Management Dashboard page, add your Excel workbooks under Reports Web Part, as shown in Figure 34.

Figure 34: Add a document from Reports Web Part

From the document library where PowerPivot management data files are stored (click Site Actions, click View All Site Content, and under Document Libraries, open the PowerPivot Management folder, and then click on the GUID folder), add the files under the specific locale folder. Figure 35 shows this method for the English language.

Figure 35: PowerPivot Management data folders

After you add the files, you can edit the PowerPivot Management Dashboard page to show your reports. To edit the dashboard page, on the Page menu, click Edit Page, as shown in Figure 36. The existing dashboard page becomes editable, and you can add a new Excel Web Access (EWA) Web Part or modify any existing Web Part. The following section describes in detail how you can add your own EWA Web Part.

Figure 36: Edit dashboard page link

Finally, after you are done editing the dashboard page, click Stop Editing to save your changes.

Figure 37: Stop Editing button

Creating a New DashboardBefore you create a new dashboard page, you may want to save your custom reports in a document library on Central Administration site. To create a new document library, go to all site content (under Site Actions, click View All Site Content) and create a new document library, as shown in Figure 38.

Figure 38: Creating new document library at Central Administration

After you create a new document library, you can upload all of your files to it. Before you create a new dashboard page that contains EWA Web Parts, make sure that the SharePoint Server Enterprise Site Collection features are enabled. On the Site Settings page, under Site Collection Administration, click Site collection features, and under Site Actions, click Manage site features, as shown in Figure 39.

Figure 39: Site settings page at Central Administration

For site collection, ensure that the Active button appears next to the SharePoint Server Enterprise Site Collection features setting.

Figure 40: Site collection features settings page at Central Administration

Now you can create a new dashboard page by creating a custom Web Part Page. Click Site Actions and then click More Options. In the Create page, click Web Part Page (as shown in Figure 41).

Figure 41: Creating a new Web Part Page

Enter a name for the page, select a layout of your dashboard, and enter the location (your document library) where you want to save this Web Part Page (as shown in Figure 42).

Figure 42: New Web Part Page

The next page displays all the blank Web Parts. To add your own custom reports, click Add a Web Part.

Figure 43: Web Part edit page

To add an Excel Web Access (EWA) Web Part, under Categories, click Business Data and then under Web Parts, click Excel Web Access Web Part (as shown in Figure 44).

Figure 44: Adding an EWA Web Part

After you add the EWA Web Part, you can click on the tool pane link to open edit the properties of the Web Part.

Figure 45: Linking an EWA Web Part to a workbook

In the tool pane, select the workbook that you want to link with the Web Part. You can also modify other Web Part properties such as height, width, and description.

Figure 46: Customizing Web Part properties

Note: when you add a new Web Part, you will see a PowerPivot category that lists all the existing Web Parts from the PowerPivot Management Dashboard. Currently, there is a bug in the system that prevents the addition of these existing Web Parts to a new page. However, in future updates we hope to provide this functionality so that administrators can leverage PowerPivot Web Parts.

ConclusionPowerPivot for SharePoint provides many features in terms of underlying data and technology; the PowerPivot Management Dashboard is just the tip of this iceberg. I hope the information that we provided in this white paper will open new horizons for IT users.

For more information:

http://powerpivot.com/: PowerPivot official site

http://blogs.msdn.com/powerpivot/: PowerPivot official blog site

Did this paper help you? Please give us your feedback. Tell us on a scale of 1 (poor) to 5 (excellent), how would you rate this paper and why have you given it this rating? For example:

Are you rating it high due to having good examples, excellent screen shots, clear writing, or another reason?

Are you rating it low due to poor examples, fuzzy screen shots, or unclear writing?

This feedback will help us improve the quality of white papers we release.

Send feedback.

AcknowledgementsI would like to acknowledge the SQL Server Analysis Services team for giving me the inspiration and support to accomplish this task. I would like to thank John Hancock, Artur Pop, Dave Wickert, Lee Graber, and Adrian Dumitrascu for their tremendous support and guidance; without their help it would not have been possible for me to achieve this task.

Appendix1. Diagnostic guide on Usage and the PowerPivot Management Dashboard:

http://powerpivotgeek.com/troubleshooting/diagnostic-guide-for-usage-and-the-powerpivot-management-dashboard/

2. List of locale IDs with descriptions: http://msdn.microsoft.com/en-us/goglobal/bb964664.aspx