Client Data Set in Detail12

8/14/2019 Client Data Set in Detail12

1/10

Creative Solutions Using ClientDataSet

By: Eric Whipple

Abstract: ClientDataSets can be used for much more than displaying rows and columns from a database. See how they solve applications issues fromselecting options to process, progress messages, crea ting audit trails for data changes and more.

Creative Solutions

Using ClientDataSets by

Martin Rudy

ClientDataSets can be used for much more than displaying rows and columns from a database . They can be used to solve applications issues from selecting optionsto process, progress messages, creating audit trails for data changes and more.

This session shows techniques where ClientDataSets can be used for a variety of application solutions where data is to be created, stored, displayed, and used for internal processing that users never see. The intent of the session is to expand developer usage of ClientDataSets beyond the standard row/column usage.

Contents

The major topics covered are:

File selection and progress messagesCreating master lookup tablesCreating a record-copy routineCustom audit trail using change logUsing xml format for CDS developmentUsing ClientDataSet as an internal data structureStoring error codes and messages during development

File selection and progress messages

Contents

ClientDataSets (CDS) provide an easy-to-use data struc ture that is handy for many application tasks. The first example shown is how to use a CDS to display a list of files to selection for processing and then display progress messages as each file is processed. Figure 1 shows an example of the sample form from the Filelist project.

Figure 1: File list selection and progress messages

The concept here is to retrieve a list of files to be imported from a spec ified directory. The Get File List button retrieves a list of files and initially selects each filename retrieved. The checkmark in the Select column indicates the file is to be selected. Users can remove any of the files form the list by changing the select columnto N before selecting the Import Files button.

For each file imported, the Process Message column is updated indicating the progress of the import process and any problems with the import. In this example, thefile Order1404.xml had a problem with the file indicating there is an issue with the xml formatting. As each file is processed, the grid is updated with the progresswhen the import starts and after finishing the import the final result.

NOTE: In this example the components that ship with Delphi were used. Third-party grids provide a checkbox option for the Select column and a multi-line cell for the Process Message column giving an improved display for messages.

The structure of the CDS contains only three fields: SelectRcd, FileName and P rocessMsg. Records are initially inserted into the CDS by retrieving all files in aspecific directory and setting the SelectRcd field to Y on Post. A hidden TFileListBox is used to easily get the files in the directory. The code to load the data isshown below.

procedure TfrmFileListExample.pbGetFileListClick(Sender: TObject); var

I: Integer;

tive Solutions Using ClientDataSet http://conferences.codegear.com/print/3222

10 20.11.2008 17:2


2/10

CurCursor: TCursor; begin

CurCursor := Screen.Cursor;Screen.Cursor := crHourGlass;

trycdsFileList.Close;cdsFileList.CreateDataSet;cdsFileList.Open;cdsFileList.LogChanges := cbxLogChanges.Checked;

// put list of all available files in grid for i:=0 to FileListBox.Items.Count - 1 do begin

cdsFileList.Append;cdsFileList.FieldByName( 'FileName' ).AsString :=

FileListBox.Items.Strings[i]; { Ensure the SelectRcd field is set last because there is an

OnChange event which posts the record. This event will ensureat runtime the record is not left in edit when user changes therecord selection by clicking on the checkbox.

This also means the Post method below is not necessary.}cdsFileList.FieldByName( 'SelectRcd' ).AsString := 'Y' ;

// cdsFileList.Post; // THIS IS NOT REQUIRED, DONE IN OnChange // EVENT FOR SelectRcd Field // See AfterOpen event below end ;

cdsFileList.First; finally

Screen.Cursor := CurCursor; end ;end ;

In this example, the Select column is the only field that the user can modify. When the Select column value is changed, the record in automatically posted. In thedemo there is no real processing of the file, there is only a simulation of processing and displaying the progress. Normally a checkbox c ontrol would be used but theDBGrid does not support that feature. The demo project has modified the grid to include an OnDblClick event which toggles the value between Y and N.

The concept of this example is basic but using a CDS makes the creation of the selection and UI very easy to implement. After the import processing is complete,saving the CDS in an XML format c reates a file with the contents shown in Figure 2:

Figure 2: CDS contents as an XML file after getting file list import simulation

There are two main sections of the xml file: data structure and ac tual data. The METADATA section defines the fields that are in the file (also called data packe t).The field name, data type and width are included. The second sect ion, ROWDATA, contains the values for each field and the RowState value. The RowState showsthat status of the data row. Table 1 shows the values and their meaning. Any row can have a combination of RowState values to indicate multiple changes made to arow.

RowState

Value

Description

1 Original record


10 20.11.2008 17:2


3/10

2 Deleted record

4 Inserted record

8 Updated record

64 Detail updates

Table 1: RowState values and Description

By default, a CDS will create a change log for every modification made to the data. Each insert, delete and update is logged. If logging of the changes is notnecessary, you can set the CDS LogChanges property to False. In the Filelist project, if the Log Changes checkbox is unchecked, the contents of the XML file after getting the file list and simulating the import is shown in Figure 3.

Figure 3: CDS contents with no logging

You can also achieve both the ability to log changes and before saving merge all changes to each row into a single record. This is done using the MergeChangeLogmethod. Executing the MergeChangeLog method before saving the file creates the same output if no logging was performed.

Creating master lookup tables

Contents

Lookup tables provide an exce llent way to ensure valid values are entered into fields and UI support for combo and list boxes to select from. Using the built-indata-aware components that support this feature requires separate datasets for each component and sometimes DataSource components are required.

For some application requirements, creating a table that is essentially a grouping of tables can eliminate this need for multiple tables in the da tabase. This nextexample, named MstrLookup, demonstrates how you can have a master lookup table and how to use CDS components and the cloning feature to support multiplelookup datasets without individual tables for ea ch type of lookup.

The first step is to create a table in the database tha t contains the lookup values for the various types of groupings. The design used here is to have a field that groupsthe data, a field for the lookup code, and a field for a description. The following is an example of the create statement for a table named MasterLookup.

CREATE TABLE [MasterLookup] ([LookupGroup] [char] (10) NOT NULL ,[LookupCode] [char] (10) NOT NULL ,[LookupDesc] [varchar] (65) NOT NULL )

There are three fields in the table. LookupGroup is used as the grouping or table name. The lookup code is in the L ookupCode field. LookupDesc field contains thedescription for the code value. If the lookup table has codes that are self-explanatory, the code and description fields will be the same.

In this example, the ORDERS table is used from. There are two fields that can use this lookup feature: ShipVia and PaymentMethod.

The general technique is to retrieve the values from the MasterLookup table into a CDS. A separate CDS is placed in the client data module for each lookup table.The rows for each lookup CDS are set in the OnCreate of the data module. The code to create the two tables for ShipVia and PaymentMethod is shown below.

procedure TForm1.GetLkupData; begin { Get lookup data into CDS } with cdsMasterLookup do begin { Ensure MasterLookup is open }

Open;Filter := 'LookupGroup = ''ShipVia''' ;


10 20.11.2008 17:2


4/10


5/10

primary key. You can also add an additional parameter which specifies the fields to copy or the fields not to copy making the duplication process generic but specificto the fields to include or exclude.

Custom audit trail using change log

Contents

A change log is maintained by the CDS for each insert, update, and de lete. The CDS Delta property contains all records in the change log. A separate record is addedto the log for each insert and delete. When an existing record is modified, two records are entered in the log. The first record, with a status of usUnmodified ,contains all field values for the record before any modification was made. The second record, with a status of usModified , contains only the field values that havechanged. All non-modified fields are null in the second record. The CDS Delta property is what the provider receives as the DataSet property in the OnUpdateData

event.In the demo project, the third tab displays the contents of the change log. Figure 4 shows the log after performing an edit on one row, an insert, and a delete. Note,the UpdateStatus field does not exist in the data, it is a calculated field used to display the status of each record.

Figure 4: Change log display

The last two records are for a deleted a nd inserted record. On an insert, any field where data is entered is placed in the change log. For deleted records, all theoriginal field values are placed in the log.

The first tow records are matching pair. The first record of the pair contains all the values of the record before any changes were made. The second record, with anUpdateStatus of Modified, contains the values for every field in the record that changed. In this example, the values of both Addr1 and Addr2 fields have beenmodified.

Displaying the change log requires an extra CDS in the application and a small amount of code. The code that is used in the demo client is as follows: procedure TfrmMain.PageControl1Change(Sender: TObject); begin if PageControl1.ActivePage = tbsDelta then try

cdsCustDelta.Close;cdsCustDelta.Data := cdsCustomer.Delta;cdsCustDelta.Open;

exceptMessageDlg( 'No delta records exist.' ,mtWarning,[mbOK],0);

end ;end ;

The CDS cdsCustomer contains the data from the provider. The CDS for showing the change log is named cdsCustDelta. When the third tab is selected, the Delta property of cdsCustomer is assigned to the Data property of c dsCustDelta. The try except block is used to display a simple message when there are no modificationsto the data.

The value for the ca lculated UpdateStatus field is assigned using the CDS UpdateStatus method. Table 2 lists the four re turn values for UpdateStatus and a

description.

UpdateStatusValue

Description

usModified Modifications made to record

usInserted Re cord has been inserted

usDeleted Record has been deleted

usUnModified Original record

Table 2: RowState values and Description

The OnCalcFields for the CDS is shown below.

procedure TdmMain.cdsCustomerDeltaCalcFields(DataSet: TDataSet);


10 20.11.2008 17:2


6/10

begin with DataSet do begin case UpdateStatus of

usModified : FieldByName( 'UpdateStatus' ).AsString := 'M' ;usInserted : FieldByName( 'UpdateStatus' ).AsString := 'I' ;usDeleted : FieldByName( 'UpdateStatus' ).AsString := 'D' ;usUnModified : FieldByName( 'UpdateStatus' ).AsString := 'U' ;

end ; end ;end ;

Using the CopyDataSetRcd function described in the previous section along with the CDS logging feature, you can create a custom audit trail process in your applications. The project CDS_Audit is used for this example.

The basic concept he re is to copy the change log before the ApplyUpdates is executed then save the changes to an existing table. The following are two codesnippets used in the example.

procedure TForm4.cdsCustBeforeApplyUpdates(Sender: TObject; var OwnerData: OleVariant); begin { Save Delta }

CDSLog.Data := cdsCust.Delta;end ;

procedure TForm4.pbShowChangeLogClick(Sender: TObject); begin

CDSLog.Data := cdsCust.Delta;

Screen.Cursor := crHourGlass; try

tblCustAudit.Open;CDSLog.First;

while not CDSLog.Eof do begin try

CopyDataSetRcd(CDSLog,tblCustAudit);tblCustAudit.FieldByName( 'ModType' ).AsString :=

CDSLog.FieldByName( 'UpdateStatus' ).AsString;tblCustAudit.FieldByName( 'ModDate' ).AsDateTime := Date;tblCustAudit.Post;

except if tblCustAudit.State = dsBrowse then

tblCustAudit.Cancel; raise ; end ;

CDSLog.Next; end ; finally

Screen.Cursor := crDefault; end ;end ;

procedure TForm4.CDSLogCalcFields(DataSet: TDataSet);

begin with DataSet do begin case UpdateStatus of

usModified : FieldByName( 'UpdateStatus' ).AsString := 'M' ;usInserted : FieldByName( 'UpdateStatus' ).AsString := 'I' ;usDeleted : FieldByName( 'UpdateStatus' ).AsString := 'D' ;usUnModified : FieldByName( 'UpdateStatus' ).AsString := 'U' ;

end ; end ;end ;

The first procedure is used by a DSP to automatically copy the change log to an existing CDS before the updates are applied. This keeps a copy of the changes before they are applied to the database.

The second procedure is used to actually save the change log to a table. Each record in the change log is copied to the audit table. Additionally the type of modification and date of modification is added to the audit table record. The UpdateStatus field is a calculated field added to the log dataset. It is assigned using theUpdateStatus method of a CDS. The third procedure above show how the calculated UpdateStatus field is generated.

Using xml format for CDS development

Contents

Incremental design is part of many application development cycles. ClientDataSets provide an easy tool for structure modifications during the prototyping and proof-of-concept phases for da taset design. Using the XML format, fields can easily be added, deleted or modified. Data can easily be added or changed without having touse a database backend.

You can sta rt designing a data set with a new CDS. One of the options is to use is the Fields Editor to add new fields. After adding the fields, you right-mouse click on the CDS and select Crea te DataSet from the spee d menu. This creates the in-memory dataset which is static in the form or data module.

To start getting data into the CDS, a small application is needed. This is basically a grid, DataSource and a Button containing a SaveToFile call using the XMLformat as follows:

CDS.SaveToFile('CDS.XML',dfXML);

The first parameter specifies the file name with optional full path. The second parameter indicates the saved format is to be XML. After entering a few rec ords, theoutput is as shown in Figure 5.


10 20.11.2008 17:2


7/10

Figure 5: Saved CDS data in XML

The change log is always generated by default. You can change this default by adding to the OnCreate for the form a line setting the LogChanges property of theCDS to False or use the MergeChangeLog method to combine the data and change log before saving. The latter is used here to support undo of changes during dataentry.

Saving the structure now a llows both fields and data to be added. The XML file can but updated to include new fields and records, any existing data can be modified provided it follows the metadata definition.

There are two keys to getting this technique to work for you: 1) need to know what is required for the data type values; and 2) need to know if there are anyformatting issues for non-text data types. Figure 6 shows lists some of the commonly used data types, the value used in the XML metada ta and sample on how toformat the data.

Figure 6: Common data types as shown in XML metadata

This technique is demonstrated further in the next section.

Using ClientDataSet as an internal data structure

Contents

This section picks up from the previous topics and expands on the usage of ClientDataSets in the development phase. The example used is an application that was a prototype for a Sunday School game for the Books of the Bible where users would test their knowledge of putting the books in the correct order and correc t sections

on a bookshelf.

Part of the example is how to use ClientDataSets during the development of the concept and as internal data for drag/drop information to ensure an order sequenceas new UI items are added to the display dynamically and randomly at runtime.

Below is the prototype form for the example. Two grids are used: one to c reate the data used in the random creation of the books to display and second shows theinternal data stored as the user placed the new book on the shelf in the correct position. The two white rectangles represent the separate shelves in a bookcase.


10 20.11.2008 17:2


8/10

When the Create Book button is clicked, a new book is created and placed next to the top shelf. The width of the component was based on the BookWidth field inthe first CDS. The user then drags the book to the appropriate place and, based on the data in the internal CDS, a comparison is made if the drop of the book was inthe correct sequential location.

Figure 7: Application using CDS for application prototype

NOTE: The original prototype was done with Delphi 7 and third-party components. These components did not exist with Delphi 9 so modifications were made to usewhat shipped with Delphi. A TMemo control replaced the third-party component used to represent the book and the book width had to be exc luded.

Two ClientDataSets where used to create the prototype. The first CDS was used to store the books. It started with only the BookNo and BookName fields. The datafor the books was loaded using the technique described in the previous section. To assist the drag and drop processing and development, an internal data structurewas required. The second CDS was created for this purpose primarily to use a DBGrid for data display of the assigned values. This made debugging the processeasier because values where displayed as they were assigned. The Fields Editor was used to create the fields for the second dataset and CreateDataSet was added tothe form s OnCreate.

During the prototype creation, fields where added to both ClientDataSets as needed using the technique described in the previous section. For example, theObjectName field was added to the second CDS. This value is the Name property for each book added. It consists of the text Book and the book BookNo numeric

value converted to a two character text value. The ObjectName value is used with a call to FindComponent during the insertion process of a new book. Another example is the BookWidth property in the first CDS. This was used to provide data to set the width of the component representing the book when it was c reated.This gave a visual representation of the relative size of the book when the third-party control was used.

In the prototype, all books are listed in the upper-right grid. The final version is to randomly generate which book to be placed which is not in the prototype version.Generating a book to be placed is done by either clicking the Create Book button or a double-click on the book grid. The currently selected record in the grid will bethe book created to place in the shelf. This is placed to the left of the first shelf. The following code is used to generate a book.

procedure TForm1.pbCreateBookClick(Sender: TObject); var

bk: TMemo; begin { Generate a new book object for selected book }

bk := TMemo.Create(self);bk.Parent := self;bk.Tag := cdsBibleBooks.FieldByName( 'BookNo' ).AsInteger;

if bk.Tag < 10 thenbk.Name := 'Book0' + IntToStr(bk.Tag)

elsebk.Name := 'Book' + IntToStr(bk.Tag);

bk.Lines[0] := (cdsBibleBooks.FieldByName( 'BookName' ).AsString);bk.Left := 2;bk.Top := 5;bk.Height := 160;bk.Width := 14;bk.Alignment := taCenter;bk.DragMode := dmAutomatic;bk.Color := clBlue;bk.Font.Color := clWhite;bk.Font.Name := 'Courier New' ;

end ;

The list of books is stored in the CDS named cdsBibleBooks. The current record in the CDS is used to set property values of the TMemo instance c reated. The Tag property is assigned the BookNo field which is a unique value. The Name property is assigned to a unique value using the BookNo field. Some of the TMemo properties assigned are to get the vertical text display. Automatic drag mode is used to simplify the prototype creation for drag-and-drop.

When a user drags the new book to a location to place in on the se lf, the following code is executed:

procedure TForm1.Shape1DragDrop(Sender, Source: TObject; X, Y: Integer); var

L,T,S: Integer; begin with Sender as TShape do begin

L := Left;T := Top;S := Tag;


10 20.11.2008 17:2


9/10

end ; if Source is TMemo then with Source as TMemo do begin { Look to see if book dropped in correct location } if BookLocOK(S,Tag,X+L) then begin if cdsBkShlf.RecordCount > 0 then

TMemo(Source).Left := X + L else

TMemo(Source).Left := 8;TMemo(Source).Top := 4 + T;

{ Locate BookName, if found, update position.If not found, insert into table.

} if cdsBkShlf.Locate( 'BookName' ,TMemo(Source).Text,[]) then begin

cdsBkShlf.Edit;cdsBkShlf.FieldByName( 'ShelfNo' ).AsInteger := S;cdsBkShlf.FieldByName( 'ShelfLeftPos' ).AsInteger := X;

end else begin

cdsBkShlf.Append;cdsBkShlf.FieldByName( 'ShelfNo' ).AsInteger := S;cdsBkShlf.FieldByName( 'BookNo' ).AsInteger := Tag;cdsBkShlf.FieldByName( 'ShelfLeftPos' ).AsInteger := X;cdsBkShlf.FieldByName( 'BookName' ).AsString := TMemo(Source).Text;cdsBkShlf.FieldByName( 'ObjectName' ).AsString := Name;

end ;cdsBkShlf.Post;ResuffleBooks(S);

end else

MessageDlg( 'Book location incorrect.' ,mtWarning,[mbOK],0);

end ;end ;

After checking if the Sender is a TMemo, there is a check to see if the book is dropped in the correc t position. The call to BookLocOK indicates if the position was.If the result is True, the book is placed in the correct position otherwise a simple message is displayed indicating the position is incorrect. For each new book added,there is a check if the book already has been placed. In this prototype, there is no check for the correct shelf the book is to be placed in. Therefore, a book that wasoriginally placed in the first shelf can be moved to the second shelf and the ShelfNo and ShelfLeftPos fields need to be upda ted. If the book is newly added, a newrecord is appended to the CDS.

The data stored in cdsBkShlf is only used during the running of the application. The grid showing the contents is for development purposes only. It provides instantfeedback on the values assigned and is a tool that ca n be adjusted during the development process to find flaws in the basic design. Other data structures that aremore efficient can be used, but in a prototype, fast-paced development cycle, a CDS and DBGrid can expedite the process.

Storing error codes and messages during development

Contents

Some applications have a need to support error and message codes with associated text for the messages. ClientDataSets can be used to create a repository for storing both the codes and text for each message. The XML format is used in this example which allows for easy insertion and modification of error codes anddescriptions. This XML file is loaded at runtime and functions are shown which use this data. Figure 8 shows the basic structure of the XML file. This data can then

be moved to the shipping database when appropriate. A technique is also shown how to store this data statically in the application as a CDS thus allowing the samefunctions to be used when the XML was loaded at startup.

Figure 8: Sample structure for error codes and messages with sample data

A CDS is loaded with the error c ode and message data when the data module is created using the following:

cdsErrMsg.LoadFromFile('ErrMsg.XML');

The function in the following code retrieves the text for the specified error code.


10 20.11.2008 17:2


10/10

function TdmMain.GetErrMsg(ErrCd: Integer): String; begin if cdsErrMsg.Locate( 'ErrorCode' ,ErrCd,[]) then

result := cdsErrMsg.FieldByName( 'ErrorMessage' ).AsString else

result := 'Invalid Error Code' ;end;

This function can be used in any validation process where the business rule maps to a specific error code. The following is an example of using the function in theBeforePost event of the CDS.

procedure TfrmValidation.ClientDataSet1BeforePost(DataSet: TDataSet); begin { Ensure required fields are entered }

with DataSet do begin if FieldByName( 'Customer' ).AsString = '' then raise Exception.Create(dmMain.GetErrMsg(101)); if FieldByName( 'Contact_First' ).AsString = '' then raise Exception.Create(dmMain.GetErrMsg(102)); if FieldByName( 'Contact_Last' ).AsString = '' then raise Exception.Create(dmMain.GetErrMsg(103)); if (FieldByName( 'Address_Line1' ).AsString = '' ) and

(FieldByName( 'Address_Line2' ).AsString '' ) then raise Exception.Create(dmMain.GetErrMsg(104)); end ;end ;

As additional business rules are added, requiring the increase in error codes, new entries can easily be inserted into the XML file. This can be done using any texteditor or by creating a small application that loads the existing XML file, supports modifications to the data and saves the error codes and messages back to theXML file. This process continues during the development phase.

When time comes to ship the application, most likely you will not want to ship an XML file that is loaded on startup. Two options can be used to replace the loading

of the XML data. The data can be moved to a table in the database and re trieved as the application opens or the data can be made static in the exe itself.

Moving data to the a pplication s database requires a little bit of extra effort. You can write a small application that inserts the data from the CDS XML file by either directly inserting into the table or you can use a DataSetProvider, a second CDS and have the records inserted into the database using ApplyUpdates. Another modification is required at startup to re trieve the data into the CDS, replacing the LoadFromFile call that previously loaded the XML file.

The second option is to load the CDS at design-time and make the data pa rt of the exe. You right-click on the CDS and select the Load from MyBase table menuoption and choose the saved XML file. When the project is saved, the data becomes stored in the form/data module containing the CDS and becomes static for thereleased exe. Provided you don t change the Active property of the CDS, the error codes and messages exist for each exe created. When a change is required, theXML file is updated and you re-load at design time the new data.

Using the technique of an XML file provides another option during development. The developers can use the file for coding and the documentation department canreview the error messages and update per their standards. Adding another field to the CDS structure for notes allows both developers and documenters the ability to

provide further information on the e rror and further deta il about potential cause and correction if needed. This information can be added into the finaldocumentation or provide online information for the final product.

SummaryClientDataSets, they are not just for multi-tier applications any more. They can be used in applications based on Paradox or Access data, Inte rBase, Oracle, SQLServer, and any other database, for XML based applications and any combination of local storage to multi-tier. Any time an application needs some type of da tastructure to store da ta where the data needs to be viewed and manipulated, ClientDataSets provide an exce llent solution.

Published on: 10/11/2004 2:19:55 PM

Server Response from: BDN9A

Copyright 1994 - 2008 Embarcadero Technologies, Inc. All rights reserved.


Documents

Client Data Set in Detail12