SQL Server Summit 2005
SQL Server 2005 What is new for the DBA?
Cheryl Stepney3/31/05
Microsoft SQL Server Summit 2005
Minneapolis
What is new for the DBA?What is new for the DBA?
• High availability for enterprise applications• Key security and performance features• Focus on self manageability and optimization
• Integration with Visual Studio and .NET• Native XML technology • Interoperability via Web Services
• Real time decision making: reporting, data mining• Rich client and SharePoint solutions• Comprehensive ETL solution: up to 600% faster
Availability and security advancements
• Database mirroring, online operations for increased availability
• Data security and privacy: encryption, enhanced auditing
Performance and scalability
• Scales as the business grows: partitioning, snapshots
• From mobile devices to 64-bit data centers
Continued focus on manageability
• Integrated management toolset for relational and OLAP
• Expanded self tuning
This sessio
n
Topics for the DBA at the Summit
• Security – 9:40 am Sogeti• Service Broker – 2:15 pm Born
– Developer Track
• Reporting Services – 1:15 pmMagenic – Developer Track
• High Availability/Scalability – 1:15 pm Unisys• This Session – 2:15 pm
Microsoft
Agenda: SQL Server 2005 Internals and More
• SQL Packaging – Editions clarified • Integrated Data Management• Minimizing the Maintenance Window• Deployment Tools• Management and Performance Tools• New Datatypes• SQL and CLR Internals• Notification Services• Full-Text Engine
Introducing SQL Server 2005 PackagingNew for SQL Server 2005
Note: All higher editions include same functionality as the edition below it.
ExpressExpress WorkgroupWorkgroup StandardStandard EnterpriseEnterpriseFastest way for
developers to learn, build & deploy simple
data driven applications
Easiest to use & most affordable database solution for smaller
departments & growing businesses
Complete data management &
analysis platform for medium businesses
and large departments
Fully integrated data management and
analysis platform for business critical
enterprise applications
2 CPU2 CPU
3 GB RAM3 GB RAM
Management Management StudioStudio
Import/ExportImport/Export
Limited Replication Limited Replication PublishingPublishing
Back-up Log-Back-up Log-shippingshipping
4 CPU4 CPU
Unlimited RAMUnlimited RAM(64-bit)(64-bit)
Database MirroringDatabase Mirroring
OLAP ServerOLAP Server
Reporting ServerReporting Server
New Integration New Integration ServicesServices
Data MiningData Mining
Full Replication & Full Replication & SSB PublishingSSB Publishing
Unlimited Scale Unlimited Scale + Partitioning+ Partitioning
Adv. DB mirroring, Adv. DB mirroring, Complete online & Complete online &
parallel operations, parallel operations,
DB snapshotDB snapshot
Advanced Analysis Advanced Analysis Tools including full Tools including full
OLAP & Data OLAP & Data MiningMining
Report Builder + Report Builder + Customized & High Customized & High
Scale ReportingScale Reporting
Adv SSISAdv SSIS
1 CPU1 CPU1 GB RAM1 GB RAM
4GB DB Size4GB DB Size
Simple Simple Management ToolManagement Tool
Report Wizard & Report Wizard & Report ControlsReport Controls
Replication & Replication & SSB ClientSSB Client
Integration ServicesIntegration Services
What’s New for the DBA SQL Server 2005 Management Tools
• Computer Manager • Management Studio• Profiler and Replay• Database Tuning Advisor• SQL Agent• SQLCMD• Express Manager
SQL Computer Manager
• Combines the functionality of the following SQL Server 2000 tools:– Server Network Utility– Client Network Utility– Services Manager
• Ability to set service properties for:– SQL Server – SQL Server Agent – Analysis Server – Report Server – Microsoft Search – Microsoft Distributed Transaction Coordinator (MS DTC) – Full-Text Search
New Management Tools
• SQL Server Management Studio– New Rich GUI– Integrated Authoring, Management– Support all SQL Server components
• Management Objects (SMO)– Managed code, new design patterns– Improved Scalability, Scripting
• SQLiMail Replacement– SMTP Support, Cluster Support, No Outlook require
Management Studio
SQL Server Management Studio
Query Analyzer
Enterprise Manager
OLAP Analysis Manager
SQL Mobile
Report Services Notification Server
Integration Services
SQL Server Management Studio
• Enhanced Query Authoring – T-SQL– Analysis Services – MDX and XML/A
• Dynamic Help• SQLCMD Mode• Improved display for XML results• Integrated execution plan graphing tool for • Enhanced Project Management • Script Projects• Enhanced Templates
Database Management Infrastructure
• Monitoring & Performance Tuning– Dynamic Management Views– Catalog Views– Default Trace– Event Infrastructure Enhancements
• Problem Resolution– Dedicated Admin Connection (DAC)– Enhanced Mini Dumps– Plan Guides & Plan Forcing
• Very Large Databases– Partitioning– OnLine Index Operations– Database Snapshots– Page Checksums/Restore
Database Problem Resolution
• Dedicated Admin Connection– Connect even to a “hung” server – Uses reserved scheduler, port & resources
• Enhanced Mini-Dumps– All interesting internal structures now in mini-dump– No more full dumps!
• Plan Stability– Plan Guides
• controlling optimization with query hints – even when you can’t change the application
– Plan Forcing• Provide a full XML Showplan as a query hint
• Database Tuning Advisor– Time Bound Tuning
– What-If Analysis
Manageability - Diagnostics
• Dynamic Management Views– Server Transparency– “sys.memory_clerks” – how much memory is each component
consuming ?– “sys.schedulers” – Is a scheduler hung ?
• SQL Trace– Query data from live traces
• SQL Profiler– Deadlock and ShowPlan Visualization
Dynamic Management Views & Functions
• Server Level– dm_exec_*
• Execution of user code and associated connections
– dm_os_*• Memory, locking &
scheduling
– dm_tran_*• Transactions & isolation
– dm_io_*• Input/Output on network and
disks
– dm_db_*• Databases and database
objects
• Component Level– dm_repl_*
• Replication
– dm_broker_*• SQL Service Broker
– dm_fts_*• Full Text Search
– dm_qn_*• Query Notifications
– dm_clr_*• Common Language
Runtime
Data Base Encryption• Certificates used by
– SQL Server to create SSL connections– SQL Service Broker
• Asymmetric Keys– consists of a private key and the
corresponding public key– Each key can decrypt data encrypted by
the other– Asymmetric encryption and decryption
are relatively resource-intensive, but they provide a higher level of security than symmetric encryption
– An asymmetric key can be used to encrypt a symmetric key for storage in a database
• Symmetric Keys– A single key that is used for both
encryption and decryption – Encryption and decryption with a
symmetric key is fast– suitable for routine use with sensitive
data in the database
The certificates created by SQL Server comply with the X.509v3 certificate standard.
SQL Server 2005 Administration Enhancements• Dynamic AWE Memory Management
• ATTACH_REBUILD_LOG • Instant File Initialization • Programming: New System Management Objects (SMO)
API – extends and supersedes the Distributed Management Objects
(DMO) object model – provides classes for configuring and managing instances of the
Database Engine
• Programming: New SQL Server WMI Provider – The WMI Provider is an instance provider
Transact-SQL New Constructs
• Improved Error Handling – use familiar TRY-CATCH logic to trap and handle error conditions
• New Metadata Views – System catalog views provide information about the objects
• Data Definition Language (DDL) Triggers – DDL triggers can be used to audit or regulate changes to a database
schema• Event Notifications
– Send information about Transact-SQL data definition language (DDL) statements and SQL Trace events to a service broker service
– Can be used to log and review changes or activity occurring on the database
• Queue Processing Extensions – The INSERT, UPDATE, and DELETE now support an OUTPUT
clause that return data based on the rows modified by the statement– READPAST can now be specified on UPDATE and DELETE
statements
Transact-SQL Enhancements• Bulk Operations on OPENROWSET
– OPENROWSET now supports a BULK operation– Controlled using new table hints & BULK_BATCHSIZE and FORMATFILE
• TOP Enhancements – The TOP operator has been enhanced to take any numeric expression
• Distributed Query – The EXECUTE statement now supports an AT LinkedServer clause that
specifies executing a stored procedure on a linked server– Distributed queries also support the new large object data types
nvarchar(max), varchar(max), and varbinary(max)• TABLESAMPLE
– New TABLESAMPLE clause limits the number of rows processed by a query– Unlike TOP, which returns only the first rows from a result set
• New CASCADE Integrity Constraints – REFERENCES clause now supports the SET NULL and SET DEFAULT
cascading referential integrity actions • Overflow Data Can Exceed Page Size
Transact-SQL New Constructs
• Unified Large Object Programming Model – Now supports a MAX length specification for the shorter data types – When MAX is specified, the data types can store the same size strings
as text, ntext, and image values (up to 2 GB), • but are processed in the same way as when they store shorter strings
• Recursive Queries – Stores a Common Table Expressions (CTE) is a results set – that is stored temporarily during the execution of a SELECT, INSERT,
UPDATE, or DELETE statement
• New APPLY Operator – Invokes a table-valued expression (function or subquery) for each row
returned by an outer table reference
• New PIVOT and UNPIVOT Operators – Rotate result sets, where columns become rows and rows become
columns
• Query Notifications – Allow an application to cache a result set and request the Database
Engine to notify it if any of the underlying data is modified by another application
Index Enhancements
• Disabling Indexes – Indexes can be disabled to aid in administration and diagnostics – When rebuilding a non-clustered index
• disabling the index before rebuilding the index significantly reduces the amount of disk space required
• Reorganizing and Rebuilding Indexes – SQL Server 2005 introduces the ALTER INDEX statement that can
reorganize or rebuild indexes, or set index options
• Online index maintenance– Create, Rebuild, Reorganize, Drop, Add or drop constraint
• Fully parallel– Indexes can be rebuilt online, pages containing LOB data can be
compacted, and partitioned indexes can be reorganized or rebuilt on a single partition basis
• Online/offline are both supported
New XML Support
• New XML Data Type • XQuery Support
– based on working drafts of the W3C XQuery language definitions• XML Data Manipulation Language (XML DML)
– The current W3C definition of XQuery does not include the ability to modify XML documents
– Microsoft has extended the XQuery implementation in SQL Server 2005 to include the ability to insert, update, and delete XML documents or fragments
• Transact-SQL Enhancements: FOR XML Enhancements – can nest FOR XML statements to create a hierarchy of XML
documents • Indexes on XML Type Columns • Bulk Copy Format Files in XML
– an XML schema that can be used to specify bulk copy format files as XML documents
New Utilities
• New sqlcmd Utility – The sqlcmd utility replaces the osql and isql utilities – sqlcmd uses the OLE DB API to communicate with an instance of the SQL
Server Database Engine • while the earlier utilities used the ODBC or DB-Library APIs.
– sqlcmd also introduces a richer set of commands that allow it to operate better in application scripts,
• such as Visual Basic for Applications (VBA) scripts
• New sqlwb Utility – The sqlwb utility can be used to launch the new SQL Server
Management Studio from the command prompt
• New Profiler90 Utility – The Profiler90 utility can be used to launch the SQL Profiler tool
from the command prompt
• New dta Utility – used to launch the Database Tuning Advisor from the command
prompt
Notification Services
Rule Types • The operation of the generator is
controlled by the Transact-SQL rules defined for the application– Event chronicle rules store or
update the history of events in chronicle tables defined by the application developer
– Subscription event rules generate notifications for event-triggered subscriptions
– Subscription scheduled rules generate notifications for scheduled subscriptions.
Full Text Search
Snapshot Isolation
• Increased data availability for read applications– Allows non-blocking consistent reads in an OLTP environment– Copy on write mechanism– Writers don’t block readers– Readers don’t block writers
• Scenarios– Read-mostly database with relatively few writes – Reporting– Migration from Oracle to SQL Server
Database Snapshots
• Snapshot of a database at a point in time– Created instantly
– Read only
• Does not require a complete copy of the data– Shares unchanged pages of the database
– Requires extra storage only for changed pages
– Uses a “copy-on-write” mechanism
• Recover from User, Application or DBA error– Rewind database to Database Snapshot
Other replication changes
• Stand alone replication monitor• DDL on published objects supported seamlessly while
replicating• Merge performance and scalability improved 2-4x• New security architecture• New data types supported
– XML, CLR, varchar/varbinary(max) etc.
• New .Net API (RMO) for setup, admin and synchronization
Partitioning
• Range Partitioning breaks a single object into multiple
manageable pieces
• Partitioning is transparent to the application
• Partitioning applies to
– Tables
– Indexes
– Indexed views
• The row is the unit of partitioning
• All partitions reside in a single database
Partition Manageability
• Common operations– Split, merge, add, and drop partitions– Bulk data loading, Index maintenance
– Switch data instantly between• Table and partition of another table• Two partitions of different tables
TimeTime
Sliding WindowSliding Window
ArchivedArchived
HotRead-OnlyNear-Line Read-MostlyObsolete
SQL Server Data Access API Layer• SQL Native Client replaces Microsoft Data Access Components (MDAC)
– a stand alone data access API that is used for both OLE DB and ODBC – It combines the SQL OLE DB Provider and the SQL ODBC Driver into one native
dynamic link library (DLL) while also providing new functionality that is separate and distinct from MDAC
– create new applications to take advantage of new SQL Server features such as Multiple Active Result Sets (MARS), User-Defined Types (UDT), and XML data type support
• SqlClient Namespace and SQL Native Client- Multiple Active Result Sets (MARS)
– introduces the ability for multiple statements to return result sets at the same time on the same connection
• SqlClient Namespace and SQL Native Client: New Data Types – A new xml data type that supports storing XML documents and fragments in databases– ability of the char, nchar, varchar, nvarchar, binary, and varbinary data types to support
strings longer than 8,000 bytes
• Web Access: Native HTTP SOAP Access – Instances of the Database Engine can be configured with HTTP endpoints, or addresses
to which SOAP requests can be sent from applications– NO Need for IIS
HTTPS SynchronizationMerge Replication
-InternetUrl https://myserver.mydomain.com
Publisher/DistributorPublisher/Distributor
FirewallFirewall
IISIISSubscriberSubscriber
Replisapi.dllReplisapi.dll
•No VPN or port required for SQL Server•Eases deployment of mobile applications
•Off by default•Secure by default
•Only authenticated users can send request to web server•Requires HTTPS •Wizard turns off anonymous access
Programmability Enhancements - .Net
• Common Language Runtime Integration – The Microsoft .NET Framework Common Language Runtime V2.0 (CLR) is
now hosted in the SQL Server Database Engine– .NET languages such as C# and VB.NET– User-defined types and aggregates can also be written in .NET languages
• User-defined Aggregates – User-defined aggregate functions written in a .NET language extend the
basic aggregate functions, such as SUM, MIN, and MAX– An aggregate function is one that processes several data values and returns
a single value
User-defined Data Types – CLR user-defined data types (UDTs) allow you to implement database data
types in any of the languages supported by the .NET CLR– CLR UDTs are integrated with the Database Engine type system and can be
used in all the contexts where the built-in types are used
• Stored Procedures, Triggers, and User-defined Functions – Database procedural objects can be written in .NET languages to
incorporate more complex logic than is supported by the Transact-SQL language
Common Language Runtime Internals
SQL EngineSQL Engine
WindowsWindows
SQL OSSQL OS
CLRCLR
HostingHostingLayerLayer
• CLR goes through SQL OS for• Memory• Threads/fibers• Synchronization
• SQL-CLR host co-ordinates• Assembly Loading• Security management• App domains• Escalation policy for exceptions
CLR Memory Management
• All memory allocation from CLR through SQL Server• Most of CLR memory (GC Heap) comes from multi-page
memory allocator (outside Buffer-pool or “MemToLeave”)– Max server memory setting does not cover CLR memory
• On memory pressure– SQL informs CLR in order to initiate a GC
• DMVs for monitoring CLR memory usage– Sys.dm_os_memory_clerks– Sys.dm_os_memory_objects
• Perf counters: # GCs, amount of memory allocated• Memory clerks of interest: SQLCLR, SQLCLRASSEMBLY
CPU: threads / synchronization
• All managed “threads” are mapped to tasks managed by the SQL Server scheduler– Co-operative scheduling– Exception: GC threads
• SQL Scheduler detects non-yielding tasks (managed or T-SQL) and “punishes” them– Force the task to stop, put back in the queue and miss some turns
• Callback to SQL Server on P/Invoke – (when calling native code)– Switched to pre-emptive mode
• CLR lock requests go through SQL Server lock manager– Unified deadlock detection between locks held by CLR and native
SQL
Application Domain Management
• App-domains– CLR mechanism for code isolation, code unloading
• SQL Server uses two kinds of app-domains – DDL-time: temporary use during assembly DDL for verification– Execution time: actual code execution
• Granularity– One per assembly owner in each database– All assemblies owned by same user are in same app domain
• Cross-app domain mgd → mgd calls not allowed• DMVs of interest:
Sys.dm_clr_appdomains
Summary: DBA Productivity
• Reliable and secure
• Ease-of-use and TCO leader for database management
• Achieve high availability without added complexity, cost
• Scales from mobile device to the datacenter
SQL Server Summit 2005