ClientDataSet in Detail1

8/14/2019 ClientDataSet in Detail1

1/6

A ClientDataSet in Every Database ApplicationBy: Cary Jensen Abstract: This article is the first in an extended series designed to explore the ClientDataSet. Thebasic behavior of the ClientDataSet is described, and an argument is made for the extensive use of ClientDataSets in most all database applications.

The ClientDataSet is a component that holds data in an in-memory table. Until recently, it was onlyavailable in the Enterprise editions of Delphi and C++ Builder. Now, however, it is available in theprofessional editions of these products, as well as Kylix. This article is the first in an extended seriesdesigned to explore the capabilities and features of the ClientDataSet.

I have been playing with an idea for a while, and I wanted the title of this article to reflect this (withmy apologies to Herbert Hoover for the pathetic turn of his political promise of "two chickens in everypot and a car in every garage"). In short, I believe that a very strong argument can be made forincluding one ClientDataSet and a corresponding DataSetProvider for each TDataSet used in anapplication. Doing so provides your user interface and runtime code with a consistent set of features(filters, ranges, searches, and so forth) regardless of the data access technology being employed.

Actually I have two goals in this first of many articles detailing the ClientDataSet. The first is to setforth the reasons why I believe that ClientDataSets should play a primary role in most databaseapplications. The second goal, and the one that I hope you find useful whether or not you accept myarguments, is to provide a general introduction to the nature and features of the ClientDataSet.

It's this second goal that I will address first. Specifically, in order for my arguments to make sense, itis essential to first provide an overview of the ClientDataSet, and how it interacts with aDataSetProvider. This discussion will also serve as a primer for many of the technique-specific articlesthat will follow in this series. After this introduction I will return to my first premise, explaining indetail how you can improve your applications through the thoughtful use of ClientDataSets.

Introduction to the ClientDataSetThe ClientDataSet has been around for a while: Since Delphi 3 to be precise. But up until recently it

has only been available in the Client/Server or Enterprise editions of Delphi and C++ Builder. In theseeditions the ClientDataSet was intended to hold data in a DataSnap (formerly called MIDAS) clientapplication. While many Enterprise edition developers did make extensive use of the ClientDataSet'sfeatures in non-DataSnap application, that this component did not exist in the Profession editionproducts made recommending its widespread employment unrealistic.

With Borland's introduction of dbExpress, which first appeared in Kylix 1.0, the ClientDataSet, and itscompanion, the DataSetProvider, are now part of the Borland's Professional Edition RAD (rapidapplication development) products, including Delphi 6, Kylix 2, and C++ Builder 6. Now all BorlandRAD developers have access to this powerful and flexible component (I'm not counting the Personal orOpen edition developers in this group, since those versions do not have the database-relatedcomponents in the first place).

With this in mind, let's now take a closer look at how the ClientDataSet works.

The ClientDataSet is a TDataSet descendant that holds data in memory in a table-like structureconsisting of rows (records) and columns (fields). Using the methods of the TDataSet class, adeveloper can navigate, sort, search, filter , and edit the data held in memory. Because these

operations are performed on data stored in memory, they are very fast. For example, on a testmachine with 512 MB of RAM running an 850 MHz Pentium 3, an index was build on an integer fieldcontaining random numbers of a 100,000 record table in just under one-half second. Once built, thisindex can be used to perform near instantaneous searches and set ranges on this indexed field.

The ClientDataSet actually contains two data stores. The first, named Data, contains the current view

ientDataSet in Every Database Application http://dn.codegear.com/print/28876

6 20.11.2008 17:31


2/6

of the data in memory, including all changes to that data since it was loaded. For example, if a recordwas deleted from the dataset, that record is absent from Data. Likewise, records added to theClientDataSet are visible in Data.

The second store, named Delta, represents the change log, and contains a record of those changesthat have been made to Data. Specifically, for each record that was inserted or deleted from Data,there resides a corresponding record in Delta. For modified records it is slightly different. The changelog contains two records for each record modified in Data. One of these is a duplicate of the recordthat was originally modified. The second contains the field-by-field changes made to the original

record.The change log serves two purposes. First, the information in the change log can be used to restoreedits made to Data, so long as those changes have not yet been resolved to the underlying datasource. By default, this change log is always maintained, meaning that in most applications theClientDataSet is always caching updates.

The second role that the change log plays only applies to a ClientDataSet that is used in conjunctionwith a DataSetProvider. In this role, the change log provides sufficient detail to permit themechanisms supported by the DataSetProvider to apply the logged changes to the dataset from whichthe data was loaded. This process begins when you explicitly call the ClientDataSet s ApplyUpdatesmethod.

When a ClientDataSet is used to read and write data directly from a file, a DataSetProvider is notused. In those cases, the change log is stored in this file each time you invoke the ClientDataSet sSaveToFile method, and restored each time you call LoadFromFile (or if you open and close theClientDataSet when the FileName contains the name of the file). The change log is only cleared in thisscenario when you invoke MergeChangeLog or ClearChanges (this second method causes the changesto be lost).

There are quite a few differences between how you use a ClientDataSet depending on whether or nota DataSetProvider is employed. The following discussion focuses exclusively on the situation where aClientDataSet points to a DataSetProvider with its ProviderName property. Using a ClientDataSetdirectly with files will be discussed in detail in a future article.

How a ClientDataSet and a DataSetProvider InteractIn order to use a ClientDataSet effectively you must understand how a ClientDataSet interacts with aDataSetProvider. To illustrate this interaction I have created a Delphi project namedCDSLoadBehaviorDemo. The main form for this project is shown in the following figure. While I willdescribe what this project does, it is best if you download this project from Code Central and run it.That way you can observe first-hand the interaction.

Here is the basic setup. The ClientDataSet points to a DataSetProvider through its ProviderNameproperty, and the DataSetProvider refers to a TDataSet descendant through its DataSet property.When you set the ClientDataSet s Active property to True or invoke its Open method, the


6 20.11.2008 17:31


3/6

ClientDataSet makes a data packet request from the DataSetProvider. This provider then opens thedataset to which it points, goes to the first record, and then scans through the records until it reachesthe end of the file. With each record it encounters the DataSetProvider encodes the data into a variantarray. This variant array is sometimes referred to as the data packet . When the DataSetProvider isdone scanning the records, it closes the dataset to which it points, and then passes the data packet tothe ClientDataSet.

You can see this behavior in the CDSLoadBehaviorDemo project. The DBGrid on the right-hand side of the main form is connected to a data source that points to a TTable from which the DataSetProvider

gets its data. When you select ClientDataSet | Load from this project's main menu, you will literallysee the TTable's data being scanned in this DBGrid. Once the DataSetProvider gets to the last recordof the TTable, the TTable is closed and this DBGrid appears empty again, as shown in the followingfigure.

Whether or not the scanning of the TTable is visible in the CDSLoadBehaviorDemo project isconfigurable. Visible scanning is the default in this project, but because this visible scanning requiresso many screen repaints, the ClientDataSet takes quite a bit of time to load the not quite 1000 recordsof the Items.db table (the table pointed to by the TTable). If you select View | View Table Loading touncheck this menu option, and select ClientDataSet | Load (if data is already loaded, you must firstselect ClientDataSet | Unload), you will notice that these records load almost instantly. The actual loadtime of a ClientDataSet depends on how much data is loaded.

Returning to a description of the ClientDataSet/DataSetProvider interaction, upon receiving thevariant array, the ClientDataSet unpacks this data into memory. The structure of this dataset is basedon metadata that the DataSetProvider encodes in the variant array. Even though the dataset to whichthe DataSetProvider pointed may contain one or more indexes, the data packet contains no indexinformation. If you want indexes on the ClientDataSet, you must define or create them. ClientDataSetindexes can be defined at runtime using the IndexDefs property, and this topic will be discussed atlength in a future article.

The ClientDataSet now behaves just l ike most any other opened TDataSet descendant. Its data can benavigated, filtered, edited, indexed, and so forth. As pointed out earlier, any edits made to theClientDataSet will affect the contents of both the Data and Delta properties. In essence, these changesare cached, and are lost if the ClientDataSet is closed without specifically telling it save the changes.Changes are saved by invoking the ClientDataSet's ApplyChanges method.

Applying Changes to the Underlying Data SourceWhen you invoke ApplyChanges, the ClientDataSet passes Delta to the DataSetProvider. How the

DataSetProvider applies the changes depends on how you have configured it. By default, theDataSetProvider will create an instance of the TSQLResolver class, and this class will generate SQLstatements that will be executed against the underlying data source. Specifically, the SQLResolver willgenerate one SQL statement for each deleted, inserted, and modified record in the change log. Boththe UpdateMode property of the DataSetProvider, as well as the ProviderFlags property of the TFields


6 20.11.2008 17:31


4/6

for the provider's dataset, dictate exactly how this SQL statement is formed. Configuring theseproperties will be discussed in a future article.

If the dataset to which the DataSetProvider points is an editable dataset, you can alternatively set theprovider's ResolveToDataSet property to True. With this configuration, a SQLResolver is not used.Instead, the DataSetProvider will edit the dataset to which it points directly. For example, theDataSetProvider will locate and delete each record marked for deletion in the change log, and locateand change each record marked modified in the change log.

If you download the CDSLoadBehaviorDemo project, you can see this for yourself. From your designer,select DataSetProvider1 and set its ResolveToDataSet property to True. Next, run the project and loadthe ClientDataSet. After making several changes to the data, select File | ApplyUpdates. Depending onthe speed of your computer, you may or may not actually see the DBGrid become active as the TTableis edited. However, on most systems you will notice the DBNavigator buttons become active briefly asa result of the editing process. (If your computer is too fast, and you cannot see the DBGrid or theDBNavigator become active, you can assign an event handler to the AfterPost or AfterDelete eventhandlers of Table1, and issue a MessageBeep or ShowMessage call. That way you will prove to yourself that Table1 is being edited directly.)

There is a third option, which involves assigning an event handler to the DataSetProvider'sBeforeUpdateRecord event handler. This event handler will then be invoked once for each record inthe change log. You use this event handler to apply the changes in the change log programmatically,providing you with complete control over the resolution process. Writing BeforeUpdateRecord eventhandlers can be an involved process, and will be discussed in a future article.

When you invoke ApplyUpdates, you pass a single integer parameter. You use this parameter toidentify your level or tolerance for resolution failures. If you cannot tolerate any failures to resolvechanges to the underlying data source, pass the value 0 (zero). In this situation the DataSetProviderstarts a transaction prior to applying updates. If even a single error is encountered, the transaction isrolled back, the change log remains unchanged, and the offending record is identified to theClientDataSet (by triggering its OnReconcileError event handler, if one has been assigned).

If you pass a positive integer when calling ApplyChanges, the transaction will be rolled back only if thespecified number of errors is exceeded. If fewer than the specified number of errors is encountered,the transaction is committed and the failed records are returned to the ClientDataSet. Furthermore,the applied records are removed from the change log, leaving only the changes that could not beapplied.

If the number of failures exceeds the specified number, the transaction is rolled back, the change logis unchanged, and the records that could not be resolved are identified to the ClientDataSet asdescribed earlier.

You can also pass a value of 1 when invoking ApplyUpdates. In this situation no transaction isstarted. Any records that can be applied are removed from the change log. Those whose resolution failwill remain in the change log, and are identified to the ClientDataSet through its OnReconcileErrorevent handler.

That's basically how it works, although there are a number of variations that I have not considered.

For example, it is possible to limit how many records the ClientDataSet gets from the DataSetProviderusing the ClientDataSet's PacketRecords and FetchOnDemand properties. Similarly, you can passadditional information back and forth between the ClientDataSet and the DataSetProvider using anumber of provided event handlers. Future articles in this series will describe how and when to usethese properties.

Using ClientDataSets Nearly EverywhereNow that we've overviewed the basic workings of the ClientDataSet and DataSetProvider components,let's return to the premise that I laid out at the beginning of this article. As I mentioned in theintroduction, a strong argument can be made for using a ClientDataSet/DataSetProvider combinationanytime data needs to be modified programmatically or displayed using data-aware controls.

There are three basic benefits to using ClientDataSet and DataSetProvider components for all dataaccess.

The combination provides a consistent set of data access features, regardless of which dataaccess mechanism you are using.

1.


6 20.11.2008 17:31


5/6

Their use provides a layer of abstraction in the data access layer, making future changes to thedata access mechanism easier to implement.

2.

For local file-base systems (Paradox or dBase tables, for example), the ClientDataSet can greatlyreduce table and index corruption.

3.

Let's consider each of these points separately.

A Consistent, Rich Feature Set

The ClientDataSet provides your applications with a consistent and powerful set of featuresindependent of the data access mechanism you are using. Among these features are an editable resultset, on-the-fly indexes, nested dataset, ranges, filters, cloneable cursors, aggregate fields, group stateinformation, and much, much more. Specifically, even if the data access mechanism that you are usingdoes not support a particular feature, such as aggregate fields or cloneable cursors, you have access tothem through the ClientDataSet.

A Layer of Abstraction

In addition to the features supported by ClientDataSet, the ClientDataSet/DataSetProvidercombination serves as a layer of abstraction between your application and the data access mechanism.If at a later time you find that you must change the data access mechanism you are using, such as

switching from using the Borland Database Engine (BDE) to dbExpress, or from ADO to InterBaseExpress, your user interface features and programmatic control of data can remain largely unchanged.You simply need to hook the DataSetProvider to the new data access components, and provide anynecessary adjustment to your DataSetProvider properties and event handlers.

Some people don't like the fact that a ClientDataSet holds changes in cache until you callApplyUpdates. Fortunately, for those applications that need changes to be applied immediately youcan make a call to ApplyUpdates from the AfterPost and AfterDelete event handlers of theClientDataSet.

Reduced Corruption

For developers who are still using local file-based databases, such as Paradox or dBase, there is yet

another very powerful argument. Hooking a ClientDataSet/DataSetProvider pair to a TTable canreduce the likelihood of table or index corruption to near zero.

Table and index corruption occurs when something goes wrong while accessing the underlying table.Since a TTable component has an open file handle on the underlying table so long as the TTable isactive, this corruption happens all too often in many applications. When the data is extracted from aTTable to a ClientDataSet, however, the TTable is active for only very short periods of time; duringloading and resolution, to be precise (assuming that you set the TTable's Active property to False,leaving the activation entirely up to the DataSetProvider). As a result, in most applications, accessinga TTable's data using a ClientDataSet/DataSetProvider combination reduces the amount of time that afile handle is opened on the table to less than a fraction of one percent compared to what happenswhen a TTable is used alone.

But It's Not for Every ApplicationWhile these arguments are compelling, I must also admit that this approach is not appropriate forevery application. That a ClientDataSet loads all of its data into memory makes its use much moredifficult when you are working with large amounts of data. There are work-arounds that you can use if you point a ClientDataSet to, say, a multi-million record data source, but doing so sometimes requiresa fair amount of coding, thereby complicating the application.

For most applications, however, the combination of features provided by the ClientDataSet outweighthe disadvantages. But even if you do not accept this argument, I think that you will find manysituations where the use of a ClientDataSet enhances your application's features, and simplifies yourefforts.

About the AuthorCary Jensen is President of Jensen Data Systems, Inc., a Texas-based training and consulting companythat won the 2002 Delphi Informant Magazine Readers Choice award for Best Training. He is theauthor and presenter for Delphi Developer Days ( www.DelphiDeveloperDays.com ), an information-


6 20.11.2008 17:31


6/6

packed Delphi (TM) seminar series that tours North America and Europe. Cary is also an award-winning, best-selling co-author of eighteen books, including Building Kylix Applications (2001,Osborne/McGraw-Hill), Oracle JDeveloper (1999, Oracle Press), JBuilder Essentials (1998,Osborne/McGraw-Hill), and Delphi In Depth (1996, Osborne/McGraw-Hill). For information aboutonsite training and consulting you can contact Cary at [email protected] , or visit hisWeb site at www.JensenDataSystems.com .

Click here for a listing of upcoming seminars, workshops, and conferences where Cary Jensen ispresenting.

Copyright ) 2002 Cary Jensen, Jensen Data Systems, Inc.ALL RIGHTS RESERVED. NO PART OF THIS DOCUMENT CAN BE COPIED IN ANY FORM WITHOUT THEEXPRESS, WRITTEN CONSENT OF THE AUTHOR.

Published on: 7/15/2002 5:19:25 PM

Server Response from: BDN9A

Copyright 1994 - 2008 Embarcadero Technologies, Inc. All rights reserved.


Documents

ClientDataSet in Detail1