View
462
Download
1
Category
Preview:
DESCRIPTION
BIML is an XML-based language that allows us to completely model a BI solution . It's particularly interesting for the automatic creation of ETL processes , for which it can be used free of charge via the BIDS Helper, a free tool that should be known to all those who develop BI solutions with the platform Microsoft. In this session we will learn the basics and some advanced trick , how to use it and how it can help to significantly reduce the development time of an ETL solution and at the same time increase the quality
Citation preview
#sqlsatParma
#sqlsat355November 22nd, 2014
BI Markup Language - BI to the next level
Davide Mauri
SolidQ – Founder & Mentor
@mauridb
www.solidq.com
dmauri@solidq.com
#sqlsatParma
#sqlsat355November 22nd, 2014
Sponsors
#sqlsatParma
#sqlsat355November 22nd, 2014
Organizers
#sqlsatParma
#sqlsat355November 22nd, 2014
Davide Mauri
20 Years of experience on the SQL Server Platform Specialized in Data Solution Architecture, Database Design,
Performance Tuning, Business Intelligence
Projects, Consulting, Mentoring & Training Regular Speaker @ SQL Server events
Microsoft SQL Server MVP
President of UGISS (Italian SQL Server UG)
Mentor @ SolidQ
Find me here: Blog: http://sqlblog.com/blogs/davide_mauri/default.aspx
Twitter: @mauridb
#sqlsatParma
#sqlsat355November 22nd, 2014
Agenda
BI, DWH & ETL today
BIML – What’s that?
Tools
BIDS Helper
MIST
BIML Kick Off
Basic Concepts
Automation With “Embedded” C#
BIML & PowerShell
#sqlsatParma
#sqlsat355November 22nd, 2014
Building a BI/DWH/ETL in 2014
Is still a (almost) manual process
A *lot* of repetitive low-value work
No (or very few) standard tools available
#sqlsatParma
#sqlsat355November 22nd, 2014
How it should be
Semi-automatic process
“develop by intent”
Define the mapping logic from a semantic perspective
Source to Dimensions / Measures
(Metadata anyone?)
Design the model and let the tool build it for you
CREATE DIMENSION Customer
FROM SourceCustomerTable
MAP USING CustomerMetadata
ALTER DIMENSION Customers
ADD ATTRIBUTE LoyaltyLevel
AS TYPE 1
CREATE FACT Orders
FROM SourceOrdersTable
MAP USING OrdersMetadata
ALTER FACT Orders
ADD DIMENSION Customer
#sqlsatParma
#sqlsat355November 22nd, 2014
The perfect BI process & architecture
Iterative!
#sqlsatParma
#sqlsat355November 22nd, 2014
Design Pattern
“A general reusable solution
to a commonly occurring
problem within a given
context”
#sqlsatParma
#sqlsat355November 22nd, 2014
No Monkey Work!
Let the people think and let
the machines do the
«monkey» work.
#sqlsatParma
#sqlsat355November 22nd, 2014
BIML – What’s that?
Business Intelligence Markup Language
XML-Based
BI Object-Generator
SQL Server
Integration Services
Analysis Services
#sqlsatParma
#sqlsat355November 22nd, 2014
BIML – What’s that?
Allows a declarative approach to define BI objects
Favor automation
Human-readable
Compatible with SCC system
Allows merge of files
Born from “Project Vulcan”
Still available on CodePlex
#sqlsatParma
#sqlsat355November 22nd, 2014
BIML – What’s that?
Once objects are generated, BIML is not needed
anymore
No additional runtime/components required
Support for SQL Server 2008 & 2012 objects
2014 is coming very soon
Anyway generated packages are automatically converted
into 2014 packages by Visual Studio 2013, without problems
#sqlsatParma
#sqlsat355November 22nd, 2014
BIML – What’s that?
BIMLScript is based on the T4 templating engine.
Allows the generation of BIML using
C#
VB.NET
BIML + BIMLScript = HTML + ASP Classic
#sqlsatParma
#sqlsat355November 22nd, 2014
Tools
BIDS Helper
http://bidshelper.codeplex.com/
Visual Studio Plugin
Free
Source Code Available
Not all BIML features are supported
“Limited” to SSIS generation only
#sqlsatParma
#sqlsat355November 22nd, 2014
Tools
MIST
Full-Featured BIML IDE
Buy or Rent
Made by BIML creator: Varigence
#sqlsatParma
#sqlsat355November 22nd, 2014
DEMO
First Contact with BIML
#sqlsatParma
#sqlsat355November 22nd, 2014
Basic Concepts
A single BIML file can generate more than one SSIS
Package
All SSIS object and related properties are exposed via
BIML elements
Additional specific attribute: ConstraintMode
Linear or Parallel
#sqlsatParma
#sqlsat355November 22nd, 2014
Basic Concepts
Parallel Constraint Mode
“Inputs” connected to “Outputs”
SSIS Variables: “dot” notation
#sqlsatParma
#sqlsat355November 22nd, 2014
Basic Concepts
Control Flow and Data Flow 100% supported
In Data Flow it’s mandatory to define input and outputs
Since they aren’t constraints, but data streams
#sqlsatParma
#sqlsat355November 22nd, 2014
DEMO
BIML is your friend!
#sqlsatParma
#sqlsat355November 22nd, 2014
Automation with BIMLScript
BIML Script (T4) Directives: <#@ e #>
#sqlsatParma
#sqlsat355November 22nd, 2014
Automation with BIMLScript
BIML Script Tags: <# e #>
Allows usage of C# / VB.Net within BIML
The “=” symbol allows “in-line” code injection
#sqlsatParma
#sqlsat355November 22nd, 2014
Automation with BIMLScript
Metadata is needed in order to define what to generate
Metadata can be extracted by
BIML Engine (Hadron)
T-SQL Queries
#sqlsatParma
#sqlsat355November 22nd, 2014
Automation with BIMLScript
Metadata via BIML Engine (Hadron)
#sqlsatParma
#sqlsat355November 22nd, 2014
Automation with BIMLScript
Metadata via SQL Query
#sqlsatParma
#sqlsat355November 22nd, 2014
DEMO
BIMLScript RULES!
#sqlsatParma
#sqlsat355November 22nd, 2014
BIML & PowerShell
What if you don’t like C#...
Or if BIMLScript looks to complex?
Especially without MIST?
One option is to generate BIML using PowerShell
Is just XML after all….
#sqlsatParma
#sqlsat355November 22nd, 2014
DEMO
BIML & PowerShell
#sqlsatParma
#sqlsat355November 22nd, 2014
Conclusions
Use BIML!!!!
It helps to
Make better use of your time
Create your own ETL patterns
http://www.slideshare.net/davidemauri/automating-dwh-patterns-
through-metadata
Support Refactoring & Enables Agility
Favor use of metadata
#sqlsatParma
#sqlsat355November 22nd, 2014
References
BIML Script
http://bimlscript.com/
Stairway to BIML
http://www.sqlservercentral.com/stairway/100550/
Automating DWH Patterns Through Metadata
http://www.slideshare.net/davidemauri/automating-dwh-
patterns-through-metadata
#sqlsatParma
#sqlsat355November 22nd, 2014
Q&A
Questions?
#sqlsatParma
#sqlsat355November 22nd, 2014
THANKS!
#sqlsatParma
#sqlsat355
Recommended