Embed Size (px)
Citation preview
Microsoft Sync Framework:Advances in v2
Lev NovikArchitectMicrosoft Corporation
TL30
Microsoft Sync Framework overview
New in-the-box Sync Providers Live Mesh!, SQL Data Services, more
Writing Sync Provider with Provider Framework Easy!
Using Filtering or “my device doesn’t want all these”
Using Constraint Conflict support or “This just won’t do”
Agenda
Why Sync?
Service
Thin Client RichClient
Sync!
Why Sync?
Service
Sync!
Client App Client AppSync!
Sync!
All stores are different Some are very capable, most are not
Conflicts are difficult to detect And even harder to manage afterwards
Misidentification of conflicts bad experience Duplicates, over-prompting, data loss
Inefficient change enumeration makes sync too expensive
Sync loops are a fact of life Few people set out to get into one, many end up in one
80% solution much simpler than 100% Corner cases are everywhere!
Why use a Framework?Sync is hard
Home PC
SmartPhone
Server
Settings
Files
Settings
Files
We strike for the balance of power and flexibility
Powerful: solve hard sync problems Conflict detection and resolution Unreliable networks, i.e. interruptions/partial syncs Loops, metadata loss, etc Easy-to-use and performant
Flexible Arbitrary stores Arbitrary protocols Arbitrary data types Arbitrary topologies
Why Microsoft Sync Framework?Solving hard problems exactly once
Makes it easy for you to sync participating endpoints
Makes it easy for your endpoint to participate
Comes with built-in endpoints: V1: File System, Relational Databases V2: SQL Data Services, Live Mesh, ADO.NET Data Services and more
Provides integrated tools and user experiences Sync Services for ADO.NET SyncToy Project Codename “Anchorage”
What does Microsoft Sync Framework do?
LiveMesh SmugMug
Sync
StoreSyncable
Builds on the success of SyncToy v2 Ecosystem Builder for Sync Framework
End-user configuration tool: Set up endpoints, communities, sync rules Run sync, see progress, past issues. Fix them.
Customizable for different content types Now: photos, contacts, videos Tomorrow: widgets?
Start your own ecosystem, or plug into one! Define content types Write matching endpoints and config Uis See them come together in arbitrary combinations
Starts out with support for Live Framework (Live Mesh, Live Contacts) SmugMug Photobucket File Systems
Project Codename "Anchorage"
Using Microsoft Sync FrameworkHow synchronization happens
Sync Provider
Sync Application
Sync Provider
Sync Orchestrator
DataStore
Data Store
changes
changes changes
changes
Write Sync Applications to synchronize stores Using built-in, other people’s or your own
providers
Write Providers for your stores and apps Using the Framework’s Sync Runtime Choose your balance of performance vs.
complexity
Using Microsoft Sync FrameworkWhat do I do?
Writing a Sync Application
SyncOrchestrator orch = new SyncOrchestrator();
orch.LocalProvider = new FileSyncProvider(guid1, “c:\\temp\\myfolder”);
orch.RemoteProvider =
new LiveMeshProvider(guid2,
new NetworkCreditial(username, password),
“MyMeshFolder”);
orch.Synchronize();
Not too hard!
Live ServicesSynchronizing Life
Sync Framework and Live FrameworkPartnership
Cloud LOE
Client LOE
Client LOE
App
App
App Store
? Live
Fram
ew
or
kPro
vid
er
Syn
c Fr
am
ew
orkApp
Sync Runtim
e
Sync
Pro
vid
er
Live ServicesSynchronizing Life
Sync Framework and Live Framework Partnership
Cloud LOE
Client LOE
Client LOE
App
App
App Store ?
Syn
c Fr
am
ew
ork
Web App
Sync Runtim
e
Browser
Sync
Pro
vid
er
Live
Fram
ew
or
kPro
vid
er
Store responsibilities: Track changes Store metadata
Provider responsibility: expose store’s capabilities Detect local changes Enumerate changes when requested Apply changes and record metadata
Some stores have no such abilities, e.g. FAT No place to store metadata No ability to track changes
What is a provider to do?
Writing ProvidersResponsibilties
Two flavors: full-enumeration and anchor-based
Full-enumeration provider (think FAT) must Enumerate all objects in the scope
Find properties that tell objects apart
Find properties that tell if the object changed
Be able to Create, Delete, and Update objects
Anchor-based providers (think NTFS) must Do all of the above Enumerate all objects that changed since some point in the past
Introducing Provider FrameworkThe easiest way to write simple providers
How Provider Framework worksUnder the covers: Enumeration
Sync Orchestrator
ProviderFramework
with Runtime
Simple Sync Provider
DataStore Meta-data
Store
GetChangeBatch
Enumerate all objects
Here’s one:Id=‘foo’, LMT=5pm
What was it last time?NewUpdatedSame
Update metadata
Bring metadata up-
to-date
Enumeratechanges
Metadata is up-to-date!
All done!
What’s missing?
Record deletes
…
How Provider Framework worksUnder the covers: Applying changes
Sync Orchestrator
ProviderFramework
with Runtime
Simple Sync Provider
DataStoreMeta-data
Store
Enumerate all objects
Here’s one:Id=‘foo’, LMT=5pm
What was it last time?
NewUpdatedSame
Update metadata
Bring metadata up-
to-date
Metadata is up-to-date!
All done!
What’s missing?
Record deletes
…
ProcessChangeBatch
Get versions
Update item id=‘foo’LMT was 1pm
New data is ‘bar’
New LMT=8pm
Check LMTand write
Updatemetadata
Writing a Full-Enumeration Provider Presenter Name
TitleGroup
demo
Write a provider with very little work
Good for Stores that do not have sophisticated capabilities to expose Stores with not enough data to worry about performance
No functionality tradeoffs! Supports efficient peer-to-peer
No extra data on the wire! Supports all conflict scenarios Supports granular change units, metadata loss recovery, etc
Available (native and managed) in Sync Framework CTP1
Provider Framework: SummaryEasy!
Storing per-item “cookies” avoids most race conditions
Anchor-based enumeration efficiency
When tombstones cannot be maintained Keeping item counts helps detect if there are any deletes
Writing better providers
Filtering Support for column filtering Filter negotiation “Forgetting” and “remembering” items Performance improvements
Conflicts Constraint conflict resolution Improved conflict logging support
New capabilities in Sync Framework v2
Often, an endpoint doesn’t want all the objects “Give me only emails from John right now” “Give me only appointments within 3 weeks from today”
Sync Framework (v1) supports filtering See FilteredChangeBatch, etc
But how do providers know what filter to use? In V2 (CTP1), we add interfaces for this negotiation
Source: ISupportFilteredSync to indicate willingness to filter Destination: IRequestFilteredSync to indicate desire to filter
Supported by Provider Framework
Why Filter ObjectsOr "I don't want all of these!"
Replicas that “forget” items over time “I don’t want to have this object anymore”
“But I may change my mind in the future!”
Hard: sometimes replicas forget own changes!
Limited support available in CTP1 Provider Framework
Highly efficient filtered knowledge representation
More advanced filtering conceptsComing in CTP2
Not all stores mean the same thing when they say “Contact” Providers need to be able to convert to/from common format But sometimes this conversion is lossy (e.g. some properties are lost)
How can we maintain correctness?
Why Filter Columns?Or "my device cannot store this property!"
First: JohnLast: Doe
Phone: 555-1212Spouse: Jane
First: JohnLast: Doe
Phone: 555-1212
First: JohnLast: Doe
Phone: 555-1212Spouse: ???
Sync Sync
PC #1 PC #2
Sync?
Phone
Spouse: Jane
Sync Framework supports change units Divide an object into groups of properties Track those groups separately Updates to different change units do not conflict! Unchanged change units are not sent!
Also supported by Provider Framework:
Review: Change UnitsGranularity at minimal cost
ContactFirst Name
Last Name
Street Address
City
Phone Number
Area code
item.Add(new ItemField(1, typeof(UInt32), name.Hash);
item.Add(new ItemField(2, typeof(UInt32), address.Hash);
item.Add(new ItemField(3, typeof(UInt32), phone.Hash);
items.Add(item);
Provider responsibilities Implement ISupportFilteredSync to indicate ability to project
Implement IRequestFilteredSync to indicate desire to project
Enumerate according to the filter
Sync Framework takes care of Recording correct knowledge Not re-sending the same changes, but yes getting the remaining ones!
A glimpse into some of the tricky bits Deleting a partial object: is it a conflict?
Using Column Filtering
Real-life scenarios have constraint conflicts Can’t have two files with the same name Brooklyn, NY is not a city
Sync Framework supports Reporting, logging, and resolving such conflicts Easily rearranging change application order to avoid
them
Provider Framework Makes using these capabilities easier
Constraint ConflictsOr “that just won’t do”
What if two replicas create “the same” object independently? Files with the same name, contacts for the same
person
Sync Framework identifies objects by Global ID Guaranteed unique, but those two objects will have
different ones
Providers detect and report violations
To resolve, key question is: keep both or merge?
Identity collisionsMost common constraint conflicts
Identity collisionsMost common constraint conflicts
Contact
ID Guid1
Creation A15
Update A15
Name John Doe
Phone 555-6789
Machine A Machine B
Contact
ID Guid2
Creation B10
Update B10
Name John Doe
Phone 555-1234
Contact
ID Guid1
Creation A15
Update A15
Name John Doe
Phone 555-6789
ProcessChangeBatch
Conflict Resolution
None
Keep Both
Merge
John A Doe
B25
Contact
ID Guid2
Creation B10
Update B25
Name John A Doe
Phone 555-1234
Identity collisionsMost common constraint conflicts
Contact
ID Guid1
Creation A15
Update A15
Name John Doe
Phone 555-6789
Machine A Machine B
Contact
ID Guid2
Creation B10
Update B10
Name John Doe
Phone 555-1234
Contact
ID Guid1
Creation A15
Update A15
Name John Doe
Phone 555-6789
ProcessChangeBatch
Conflict Resolution Keep Both
Merge
B25
555-6789
Tombstone
ID Guid1
Creation A15
Deletion B27
None
Identity collisionsMost common constraint conflicts
Contact
ID Guid1
Creation A15
Update A15
Name John Doe
Phone 555-6789
Machine A Machine B
Contact
ID Guid2
Creation B10
Update B10
Name John Doe
Phone 555-6789
ProcessChangeBatch
B25
Tombstone
ID Guid1
Creation A15
Deletion B27
Contact
ID Guid2
Creation B10
Update B25
Name John Doe
Phone 555-6789
Tombstone
ID Guid1
Creation A15
Deletion B27
Adding Name Collision support Presenter Name
TitleGroup
demo
Sync Framework v1 supports deferred conflict resolution Providers return TRANSFER_AND_DEFER as conflict resolution But providers need to do a fair amount of work
In V2, we are greatly simplify things
Providers implement IConflictLogAccess Direct store and retrieve conflict records (CRUD)
Sync Framework takes care of Not over-enumerating logged conflicts Sending logged conflicts to interested parties Cleaning up logged conflicts when resolved
Or superseded by newer conflicts
Conflict LoggingOr “I’ll deal with that later”
Sync Framework v2 is the easiest and most efficient way to synchronize your data with Live Mesh, File systems, Databases, ADO.NET Data Services,
SQL Data Services, and more
We have made it even easier to write providers for your own store, using Provider Framework
We support and simplify many advanced scenarios, such as Filtering Conflict Logging Constraint Conflicts
Summary
Resources
Sync Developer Centerhttp://msdn.microsoft.com/sync SDK (including documentation) Whitepapers Samples (including several end-to-end)
Sync Bloghttp://blogs.msdn.com/sync Announcements Tips and Tricks
SQL Labswww.sqlserviceslabs.com Cutting edge prototypes and pre-release functionality
Come visit us Discuss Sync in more detail at our Booth “Ask the Experts” session
Evals & Recordings
Please fill
out your
evaluation for
this session at:
This session will be available as a recording at:
www.microsoftpdc.com
Please use the microphones provided
Q&A
© 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market
conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
