Keeping It Tidy- How to Warehouse your SharePoint Data

John WhiteChief Technology OfficerUnlimitedViz Inchttp://whitepages.unlimitedviz.com@diverdown1964

How to Warehouse Your SharePoint Data

Keeping It Tidy

2 | SharePointalooza – Branson, MO 2014

A Few Reminders

Download the attendee packet at http://bit.ly/SPAloozaAttendee

Attend the “Rock Star” Sessions at the end of each day for fun, raffle prizes, wrist bands for concert access, and your chance to win a Surface Pro 3

Attend Nintex’s Brown Bag lunch Friday and Saturday (lunch provided for first 100 people)

Tweet about the event using #SharePointalooza Thank our sponsors Have a great time!

http://bit.ly/SPAloozaAttendee


Thank you, sponsors!Rock Star Sponsors

Performer Sponsors

Giveaway Sponsors

Soloist Sponsors


The Bands

What better way to unwind after a long day of working out your brain than with some great live music at the amazing outdoor stage at Branson Landing! The bands will be playing both Friday and Saturday night from 6:30 pm to 10 pm.

John WhiteCTO/Co-Founder of UnlimitedVizSharePoint Server MVP, SQL Server v-TS

[email protected]://whitepages.unlimitedviz.com@diverdown1964


Agenda

SharePoint Data Storage Data Warehousing Moving SharePoint Data Surfacing Warehoused Data in SharePoint Q&A

The Problem

SharePoint Data Storage


Excel Data Storage

List based data Powerful Features Massive adoption Good usability

NOT a database Not relational Capacity limits Not authoritative

Excellent BI Client Surface the data.. DON’T store it

SharePoint Content Databases


SharePoint Content Databases (2010)

~160 fields


SharePoint Capacity Boundaries

Boundaries and Limits - http://technet.microsoft.com/en-us/library/cc262787

200 GB per content DB (general usage scenarios - or up to 4 TB)

List view threshold – 5,000 simultaneous items

List view lookup threshold – 8 join operations per query

Items per content database – 60 million

http://technet.microsoft.com/en-us/library/cc262787


WorkflowHistory List

Demo

Data Warehousing

ONE DOES NOT SIMPLY

RETURN HIS RAW DATA FROM THE DATABASE


BI Architecture 101

Data Marts

Extract, Transform, and Load (ETL)

Middleware Server(s)

DataWarehouse

Storage Design and Visualization

Data Cubes and Tabular

Models

E

T

L

Reporting Server(s)

BI and Designer Clients

Source data


EE

Microsoft enterprise (classic) BI

SQL Server DBSQL Server Integration Services (SSIS)

SharePoint (with)• Excel Services• PowerPivot for

SharePoint• SSRS SharePoint Mode• PerformancePoint

SQL Server DB

Storage Design and Visualization

SQL Server Analysis Services

Multidimensional and Tabular modes

L

SQL Server Reporting Services (SSRS)

ExcelSQL Data ToolsReport Builder3rd party tools

ETL

E

T

Source data


Team BI and SharePoint Dashboards

Power Pivot Worksheets• Pivot Tables and

Charts• Power View

Data Marts and other

Data Cubes and Tabular

Models

Standard Worksheets• Pivot Tables and

Charts

PerformancePoint Reports• Analytic Charts and Grids• Decomposition trees

SQL Server Reporting Services Reports• Standard• Power View

PerformancePoint Scorecards and KPIs


Why Warehouse?

Optimize for End Goal Multidimensional Flattened

Usability Reduced Load on Source Systems Security Requirements Licensing Requirements SharePoint (Performance)

Moving SharePoint Data


ETL Tools

SQL Server Integration Services Comes with ALL Editions of SQL Server (except Express)

SQL Server Data Tools – BI Edition http://go.microsoft.com/fwlink/?LinkID=320744 (For VS2010) http://www.microsoft.com/en-ca/download/details.aspx?id=36843 (For VS2012) http://www.microsoft.com/en-us/download/details.aspx?id=42313 (For VS2013)

SharePoint List Source and Destination (CodePlex Project) http://sqlsrvintegrationsrv.codeplex.com/releases/view/17652 (Download) http://

dataqueen.unlimitedviz.com/2011/06/how-to-use-a-sharepoint-list-as-a-data-source-in-your-ssis-package/ (How to)

OData Source for Microsoft SQL Server 2012 (Official) http://www.microsoft.com/en-us/download/details.aspx?id=42280 (Download) http://whitepages.unlimitedviz.com/2014/03/using-the-odata-source-connector-with-sharepoint-on

line-authentication/ (How to)

SharePoint Server 2013 Client Components SDK - http://www.microsoft.com/en-us/download/details.aspx?id=35585 – Required for Office 365 authentication

http://go.microsoft.com/fwlink/?LinkID=320744

http://go.microsoft.com/fwlink/?LinkID=320744

http://www.microsoft.com/en-ca/download/details.aspx?id=36843

http://www.microsoft.com/en-ca/download/details.aspx?id=36843

http://www.microsoft.com/en-us/download/details.aspx?id=42313


http://sqlsrvintegrationsrv.codeplex.com/releases/view/17652

http://sqlsrvintegrationsrv.codeplex.com/releases/view/17652

http://dataqueen.unlimitedviz.com/2011/06/how-to-use-a-sharepoint-list-as-a-data-source-in-your-ssis-package/





http://whitepages.unlimitedviz.com/2014/03/using-the-odata-source-connector-with-sharepoint-online-authentication/







Moving SharePoint Data

Demo

Accessing Warehoused Data


Data Consumption

Business Connectivity Services External List All SharePoint Editions (yes..Foundation)

SQL Server Reporting Services Advanced Reporting All Editions of SQL Server (even Express!) ..except Express doesn’t do SP Integrated Mode

Excel/Power Pivot Excel Services requires Enterprise SharePoint PowerPivot for SharePoint requires SQL Server Standard or better


Accessing Warehoused Data

URL to report: http://home.nautilusdemo.com/sites/nmarine/finance/_layouts/15/ReportServer/RSViewerPage.aspx?rv:RelativeReportUrl=/sites/nmarine/finance/SSRSReports/SPAlooza.rdl&rv:ParamMode=Hidden&rp:ItemID=itemid, Our Workflow History

Demo


Considerations

Transformations Mashups Scheduling Data latency


In Conclusion

Querying SharePoint lists is bad for everyone Tools exist to move relevant data Warehousing applies to SharePoint too Easy to surface data in SharePoint


Q&A

[email protected]@diverdown1964whitepages.unlimitedviz.com

mailto:[email protected]