Optimal Design of Distributed Databases

L.P. Daswin Pasantha De Silva and Manjula DissanayakeDepartment of Computing

Informatics Institute of TechnologyColombo, Sri Lanka

Abstract- The physical expansion of enterprises with thecurrent trends in globalization has had a positive influence on thetechnologies of distributed systems. At the forefront of thistechnological revolution are distributed database systems. For adistributed database to be at optimal performance and thusprovide an efficient service it needs to be designed appropriately.The significance of the perfect design is only emphasized by themultiple dimensions required in generating a design. The purposeof this paper is to suggest an approach to generate optimaldesigns for such distributed database systems and to develop aprototype to demonstrate the said approach. The approachemphasizes on the accuracy of inputs as it largely determines thequality of the final solution. Hence the extraction of networkinformation, a key dimension, is automated to ensure precision.The global schema is fragmented considering data requirementsas well as connectivity of each site. Allocation of fragments istreated as a combinatorial optimization problem and assigned toa memetic algorithm. An estimation of distribution algorithmcomplements the search effort of this memetic algorithm. Siteoptions for replication server environments are investigatedbased on a shortest path algorithm. Usability of the system in anobject oriented development environment, through conditionalobject-relational mapping, is also explored. The prototype wasdeveloped using an evolutionary prototyping approach. It wasevaluated by several experts in the relevant fields of application.The results of which, confirmed the practicality of the suggestedapproach.

Keywords-Distributed Database Design, Memetic Agorithms,Object-Relational Mapping, Bandwidth Measurement

I. INTRODUCTION

Data are ubiquitous just as they are indispensable. Allinformation systems regardless of the application area, userskill level, language or country of use require data tosuccessfully complete its purpose. The creation of the relationalmodel made way for the database paradigm as a suitablestorage and access mechanism for data. Expansion ofenterprises on the grounds of globalization prompted thehitherto centralized database technology to be geographicallydistributed along with the disseminated enterprise operations,thus leading to the distributed database technology.

A distributed database is a collection of multiple, logicallyinterrelated databases distributed over a computer network [1].As accentuated by the definition it is the fusion of two

conceptually different technologies, database systems whichrealize centralization and networking which realizedecentralization. The product of this combination in essencecan be stated as the best of both worlds. The complication thatis to harness the benefits of both the parent technologiesmentioned above ensures the importance of the design stage forany successful distributed database system. Thus, a gooddesign although intricate, is mandatory. Various factors, bothtechnical and non technical, need to be considered in theproduction of a good design. Ranging from the global schemato the disk capacity of a server, much information needs to beextracted from the distributed environment.

The key concerns when devising a distribution solution arefragmentation, replication and allocation. Fragmentation ariseswith the need to access different data of a single relation, in aglobal schema, by more than one site in the distributedenvironment. In such instances the relation is broken intoseveral disjoint sets. Replication is the storage of copies ofrelations at different sites as per the access frequencies.Replication can be extended to fragments as well as completeschemas. Allocation is the task of identifying the best locationsfor replicas, fragments and relations so that the performance, ofthe environment as a whole, is maximized and the costsincurred are minimized.

Documented academic research on this problem domainalthough available, is individually inadequate. Identification ofonly a subset of the problem domain, unrealistic assumptionson the behavior of the distributed environment, homogeneousapplication and over simplification of certain constraints aresome notable limitations. The focus of this paper is a completeand integrated solution for optimal design of distributeddatabases.

An overview of the sections to follow; Section 2 furtherexplores distributed database design along with existingresearch and techniques. Section 3 goes into detail on thesuggested methodology while section 4 describes theimplementation of the prototype and the features offered.Future enhancements are discussed in Section 5.

II. EXISTING RESEARCHAND TECHNIQUESAs accentuated in [1], there exist two key strategies for

fragmentation. Horizontal fragmentation; where division isbased on tuples and vertical; where division is based on

1-4244-0555-6/06/$20.00 (©2006 IEEE ICIA 2006Page 1

attributes. A hybrid of the two is generally used. Afragmentation scheme is required to conform to the threecorrectness rules of completeness, reconstruction anddisjointness [1]. Replication provides a greater fault toleranceand an increased availability. All replicas are required toconform to the mutual consistency rule where all copies of datashould be identical at any given time. Replication servers takethe data replication concept one step further by replicating thecomplete global database at several sites. Allocation is entirelyto do with the optimal location of data. Optimality can bedefined with respect to two measures, minimal cost andincreased performance [I].

From the existent literature, one of the notablemethodologies suggested is the iterative method for distributeddatabase design [2]. It attempts to integrate data allocationalong with query optimization. An iterative heuristic algorithmis performed on an initial, access based allocation model;where each repetition alternates between allocationoptimization and query optimization. Allocation optimizationis carried out by a search from convergence algorithm while thequery optimization methodology manipulates the relations inascending order (based on attribute density) till the relations arelinked in a low cost serial transmission pattern. A primaryshortcoming in this attempt is it addresses only part of thecomplete problem as it disregards the importance offragmentation, replication and the interdependence between thecore design issues. The inclusion of query optimization as adesign issue is inappropriate as tangible data on requiredqueries is unavailable or roughly estimated during the designstage. A more sustainable query optimization can be achievedin the post implementation phase. Further, the presentation ofthe solution exhibits a tight connectedness to the sampleproblem described- casting doubt to its practical applicability.

Another noteworthy effort is the tree structured transactionmodel method as proposed by Mukkumala et al [3]. Themethodology is developed upon the assumption that alltransactions can be broken down into sub-transactions whichconsist ofrelational database operations on the data items in thedatabases. Each transaction takes the structure of a tree whereeach leaf node is a fragment or relation and each internal nodea relational operator. Fragmentation takes into account the totalcommunication flow between a pair of data items in thedatabase per unit time. The heuristic used for allocating thefragments is based on the location of execution of each sub-transaction and communication costs.

This solution gives due prominence to the three maindesign issues and even identifies the inter-dependence betweenthem. However it over relies on the tree structured transactionalmodel which becomes unrealistic for transactions accessingsingle relations and transactions involving non-booleanoperations. The assumption that the network comprises ofequal bandwidths casts doubt on the practicality factor. Thepaper suggests the usage of a third party transaction compilerand optimizer, which in the pragmatic scenario will bring aboutfurther inconsistencies.

The design methodology proposed by Huang and Chen [4],suggests two heuristic algorithms to solve the allocation ofdata. It assumes that the global schema is already fragmented.

The first heuristic starts off with significant replication byallocating all sites with all fragments required. Then it removesfragments from the initial allocation on the basis ofminimizingcommunication cost. Finally all un-allocated fragments areassigned to the sites that incur the least costs. The secondalgorithm follows a similar suit though the removal offragments is based on how much data of a fragment copy willbe updated at each site.

In both the heuristics the individual site constraints aredisregarded, assuming all sites maintain unlimited or uniformcapacities. The assumption of fully connected environmentsalong with the execution times of the heuristics when applied toa large schema, further questions the practicality of theproposed method.

III. DESIGNING A SOLUTION

Taking into account the weaknesses as well as the strengthsof the existing methodologies, the focus was on designing anintegrated solution which is both practical and applicable toany distributed environment. Consideration was also given tothe generally overlooked area of integration with existingindustrial paradigms and practices, of which the design of adistributed database would be part of. From a broad perspectivethe design task can be broken down into four interrelatedoperations. The four areas are parameter extraction,fragmentation, replication and allocation.

A. Parameter ExtractionThe decisions on the information required from the external

environment were based on the findings of a survey on thecurrent design concerns of database administrators as well asthe facts emphasized in research material. From the industrialpoint of view the key concern next to access requirements ofeach site, were the capacities of the interconnecting network.Along with network information the main parameters to beextracted are site information, schema information and dataaccess requirements.

Thus, network information was included as one of the keyparameters. However, extraction of the actual bandwidth of alink between two sites required an additional effort as the facevalue of such a link varies with the time of the day andintermediate congestion. With special attention to single packetprobing and packet pair probing techniques, several othermeasurement methods [5] were evaluated. Ultimately, wedecided to utilize the Internet Control Message Protocol(ICMP- RFC 792), the connection management protocol of theIP suite. The reasons being, the ability to measure single aswell as multiple links, the availability of the end result at theoriginating site and its mandatory implementation in the IPmodule. The method devised was to create a number of(environment specific) time zones (e.g. peak, off-peak) andsend several packets of a reasonable, predetermined size duringeach time zone. The time zones provide a clearer picture of thebehavior of the network. Moreover, the effect of cross trafficwas addressed based on the assumption that it can only everincrease the delay seen by a single packet. Therefore takinginto account suggestions by Downey [6], the method devised

Page 2

was extended to send several packets at random time intervalsand average the results obtained.

Site information, a parameter overlooked in industrialpractices, was identified important as a supplementary alongwith network considerations and also as a key concern for dataaccess requirements. It is composed of the computingcapacities of each site within the environment. Thestraightforward values of processing, main memory andpersistent storage available for database operations willconstitute this computing capacity.

Under schema information the highlight is to obtain therelational structure of the data. Mandatory as an input, it is thisschema that will be fragmented, replicated and allocated. In thecase of a relational model the schema information is composedof the relations and the respective attributes. The attributes areessential for a fragmented distributed solution. With theintention of enhancing usability (in the context of an objectoriented design/development environment), extraction ofschema information was extended to object-relational modelsand object models. The requirement was to draw up aconnection between the object oriented paradigms and thedistributed database design task. An initial study of commercialsolutions uncovered several options in Hibernate, SQL Mapsand Java Data Objects (JDO). These, however, focused onobject relational mapping at an implementation level. The lackof design level object-relational mapping tools, led us toexplore options in academic research. The study conducted byRumbaugh et al [7] was selected as the design level mappingmethodology as it warranted scenario independent applicabilityas well as a close relationship with the UML. To counter theloss of precision due to the generality and vagueness inherentto all object models we included the option to present theoutcome of the mapping process for the user's perusal.Assimilation to contemporary software environments wasanother key benefit gained by the inclusion of the mappingmethodology.

Acquisition of access requirements of each of the sitesrequired special attention as the precision of this informationhas a direct and vital correlation to the quality of the finalsolution. The parameters thus needed are read access and writeaccess, with the frequency given as an additional option to bedecided by the user. Frequency as an option, varying fromhourly to weekly, preserves the dynamic property of data.Update frequency is given more weight out of the two. Bothparameters are required on per relation basis for each sitewithin the distributed environment. Emphasis should be madeon the precision of the parameters as it would largely determinethe quality of the final solution.

B. Fragmentation and ReplicationFrom what was delineated above, the mandatory categories

for the fragmentation process are schema, network and dataaccess information. The viable options for verticalfragmentation, as emphasized in Section II, include exhaustivesearch, incremental search and usage of an attribute affinitymatrix [8]. The latter, although popular, was rejected as itrestricts the view of affinity to a pair of attributes and limits the

suitability of the fragments generated to a subset of the siteswithin the distributed environment

What we suggest is a fragmentation process based on theimportance of each attribute for each site. Importance being avalue composed of the overall connectivity of the site requiringthe attribute as well as the access frequency. The algorithm isas follows:

Inputs:LCM: Link capacities matrix- Contains thelink capacities of the interconnections betweenthe sites requiring Relation R, where theintersection (ij) represent the capacity of thelink connecting site I and site J.

SRM: Site-data requirements matrix- Datarequirements for both reads and updates per siteon an attribute basis for Relation R

Outputs:Vertical Fragments, F, for Relation R

Process:Generate connectedness matrix, CM from LCM

n

CM.0 LCMIJj=o

n

CM d = Count(LCM )j=o

Identify IMP, importance of each attribute a ofRelation R for all sites

n

IMP( a )= [(Reads for a i from site S)j=o

+(Updates to a i from site S ** 2) ] CM jo

Sort IMP values in desc order, IMP desc

Find difference between adjacent elements inIMP desc as IMP diff,

Average (Z IMP diff) as IMP avg

Group adjacent attributes in IMP desc withcorresponding IMP diff < IMP avgThese groups form the vertical fragments, F,

Integrity Check: Assign primary keyattribute(s) to groups not uniquely identifiable

Algorithm 3.1: Vertical Fragmentation Process

Page 3

The consideration of site connectivity ensures that localdata requirements are not completely ignored when generatingpartitions. Horizontal fragments, where relational conventionsare concerned, is required to conform to the three correctnessrules [1] of completeness, minimalism and disjointedness. Thedatabase designer's intuition is mandatory to enforce theminimalism property as it requires qualitative knowledge of thepredicates involved. To overcome this obstacle and provide aviable set of horizontal fragments we suggest a process whichtakes into account the generated vertical fragments and theirrelative values of importance[ 12].

Where replication is concerned, the requirement is toidentify the best suited fragments or relations for duplicationalong with the locations. The replication process we suggestwill identify the variance of the access requirements of the finalfragments and relations and use this as the basis in determiningthe need for replicas. The location of each replica will bedecided by a shortest path algorithm whose weights will bebased on the connectivity of the distributed environment. Asimilar algorithm is suggested for schema replication purposesas well. Being the widely used variety of replication, theschema replication algorithm will suggest the best locations forbackup functions as well as real time replication servers [12].

C. AllocationAllocation is the design task that requires the most effort as

it has the greater impact on the quality of the final solution andhence the operational efficiency. The process suggestedrequires all of the parameters detailed above to optimallydistribute the fragments generated. Tackling it as acombinatorial optimization problem we propose a solutionbased on a memetic algorithm. Memetic algorithms are the'evolved' counterpart of evolutionary algorithms (EA) [9].Memetic algorithms (MA) attempt to mimic cultural evolutionas opposed to evolutionary algorithms where biologicalevolution was the key inspiration [10]. Although bothtechniques are population based global optimization methods,memetic algorithms go one step further to recognize theimportance of correlated landscapes during the task ofoptimization. The chromosome structure too is improvised,with real coded values replacing the binary strings. This isadvantageous when engaging optimization problems withvariables in continuous domains.

The key tasks in devising the allocation methodology wereidentification of a suitable chromosome structure, modificationof the evolutionary cycle to suit the problem domain and thecreation of an appropriate objective function. From memeticalgorithm view point, the multiple dimensions are the sitesalong with their capacities, the fragments and replicas withread/write frequencies and the bandwidths of the inter-connecting links. It is mandatory to incorporate all theseparameters to the fitness function which will evaluate thesuitability of each solution. The structure of a chromosomeshould be similar if not the same to the structure of a typicalsolution to the problem under investigation. [11] The skeletalof a chromosome in isolation should represent theconfiguration of the solution. Falling in line with this definitiona diagrammatic representation of the chromosome structureadopted is as shown in Figure 3.1.

N - Number of SitesM - Number of FragmentsK - Fragments per site i

I nZSite 1(k)=M

Structure:

Site 1 ite ...SiteN

Fragment 1

Fragment 2

|Fragment k |F

Figure 3.1: Chromosome Structure

A re-visit into the allocation techniques used by the existingresearch proved worthwhile in identifying a potentialevaluation mechanism to be incorporated to the MA cycle. Theraw parameters (network, site and data access information)needed to be bound to avoid conflicts which may occur due tolarge discrepancies between them. We opted for normalizationrather than a mapping process, as it would preserve thecorrelations while causing the least modifications. To ensuresurvival of the more realistic solutions and fair evaluation ofthe chromosomes, a fitness function that simulates anoperational distributed database environment was reckonedmost suitable. For the allocation model represented by thechromosome the performance of all retrieval and updaterequests on each relation by each site would be calculated [12].

With respect to application of the MA cycle, initiation isperformed in a quasi random mode where the initial populationwill be derived from the set of all solutions based on a numberof conditions specific to the distributed environment [13]. Inthe population optimization phase, a search function revalidateseach chromosome. Chromosomes with a majority of zerorelevance genes are replaced with solutions from an additionalpool of initial solutions. The aforesaid fitness function will beapplied in the evaluation phase. The output fitness value will bea performance index for the configuration depicted by thechromosome. The initial decision was to utilize a hybrid of thecompetitive-cooperative method and the conventional EAselection techniques during the interaction phase. However,due to the inter-dependency of the individual genes in eachchromosome, the generic crossover and mutation techniqueswere inefficient and gave rise to the scenario commonly knownas the 'linkage problem'. Most EAs overcome this byconstructing special crossover and mutation types whichincorporate problem-specific knowledge. It was identified thatthis option would restrict the search space as well as the multi-point search strategy of the EA. This would degrade the MA toa highly specific heuristic algorithm. [9]. Therefore, the multi-

Page 4

parent crossover operator as defined in Estimation ofDistribution Algorithms (EDA) was used as the selectionmechanism along with a domain-specific mutationoperator[14]. The multi-parent crossover operator builds anexplicit probabilistic model of distribution of fitter individualsfor each generation and samples from this model. We selectedthe Gaussian distribution model (Algorithm 3.2).

Gaussian Distribution of x; fg(x)

1 2 2

fg(x) (r2g2). (xe a)2L20

n set of values

a mean of n

e exponential

v = Standard deviation of n

EGDA:

n set of fitness values of individuals

in current generation

0 threshold for current generation

distribution

Start

For each n calculate fg(x).

if ( fg(x) > 0 )

sample from current generation

end if

end for

End

Algorithm 3.2: Estimation of Gaussian DistributionAlgorithm

IV. IMPLEMENTATION OF A PROTOTYPE

The prototype was developed using Sun's Java (Version1.4.2), due to its ease of use, and wide array of open sourceunits that together, provide a good foundation for RapidApplication Development while generating optimized, high-performance code. Implementation of the parameter extractionmodule required an additional effort for extracting network andschema information. The object model is required to be in

XML format (most contemporary UML modeling tools providethis option), upon which an XSL transformation is performedto extract the class and attribute values.

The implemented prototype possessed three high level,benchmark properties; adaptability- the solution is capable ofhandling any distributed environment ranging from fullyconnected to replication server environments, extensibility- themodular design which allows unrestricted addition of newfunctionality to the system with no effect to existing modules,flexibility- solution generation from partial or incompleteinformation. Moreover, on a technical note, the prototypedisplayed the following features.

* User friendly graphical interface to capture multipledimensions that are a prerequisite for the optimumdesign of a distributed database

* Management of criteria required for the generation ofan optimum distributed database.

* Generation of feasible optimal design solutions suitedto the distributed database environment and businessrequirements.

* Ability to suggest enhancements to the distributeddatabase environment based on the optimizationscheme required and customize the generated model.

* Provide integration with contemporary paradigms andan array of options for the user to generate thedistributed model.

* Representation of the generated models through meanswhich can be easily understood.

V. FUTURE DEVELOPMENTSVarious opportunities of further enhancing the developed

system were noted during the completion of the project. Someof these are modifications to overcome certain shortcomingsand some require detail re-design to incorporate theenhancement at a modular level. Given below are the authors'key recommendations for enhancements to the solution.

* Incorporation of object and object-relational datamodels.

The solution addresses only the relational data model fordistribution as it is the most sought after storage paradigm inthe current context. However, if object data modeling gainspopularity in the future, the solution should be able to distributesuch models as well. It would be necessary to re-design thecore modules to facilitate this.

* Include foreign key relationships, relation joins andtransactional information

This feature should incorporate information on foreign keyrelationships and relation joins in deciding both fragments aswell as allocation. The data requirements of each site will bedecided upon by the transactional data. Even if the informationis fairly accurate it will generate a more accurate designsolution as additional correlations within the inputs can befound.

Page 5

* Increase accuracy ofnetwork information

Similar to the previous enhancement this would be targetedat affecting the accuracy of the ultimate design solution.Accuracy can be increased by collecting data on off peak/ peakhours and also by complementing the ICMP technique withanother measurement method.

* Generation of a system specific probabilistic model

Where the estimation of distribution algorithm (EDA) isconcerned, the system makes use of a predefined probabilisticmodel -the Gaussian distribution model- to determine thelandscape of the next generation. This could be made moreeffective by adding a probabilistic model customized to suiteach design problem.

* Enhancements to the object-relational mapping module

A vast area for research, the object-relational mappingfunction can be further improved to incorporate identificationof relationships as well as additional rules which will increasethe accuracy of the mapping function.

VI. CONCLUSIONFrom a broad perspective, the methodologies proposed

hitherto for distributed database design lack the essentialfeatures of adaptability, flexibility and extensibility. Thepreliminary constraints and assumptions, upon which designsare generated, and the lack of information of the externalenvironment are some of the key causes for these limitations.What we have suggested in this paper is a complete, integratedsolution that is capable of generating precise, pragmatic andoptimal design solutions for distributed database environments.A solution comprising of accurate identification of decisiveparameters in the external environment, extraction techniquesfocusing on precision, in detail fragmentation and replicationbased on relative importance, association with contemporaryobject paradigms, a memetic approach to data allocationassisted by Gaussian estimations of distribution and an array ofoptions at each stage to sustain the quality of the solution frominitiation to completion. Collectively, these features present acomplete and integrated methodology for the generation ofoptimal designs for distributed database environments.

REFERENCES[1] Ozsu, M.T. and Valduriez, P. (1999) Principles of Distributed Database

Systems, 2nd ed., Pearson Education.[2] Blankinship, R., Hevner, A.R. and BingYao, S. (1998) 'An Iterative

Method for Distributed Database Design', Proceedings of theInternational Conference on Very Large Data Bases.

[3] Mukkumala, R., Bruell, S.C., Schultz, R.K. (2000) 'Design of PartiallyReplicated Distributed Database Systems', ACM Transactions 2000

[4] Huang, Y. and Chen, J. (2001) 'Fragment Allocation in DistributedDatabase Design', Journal of Information Science and Engineering,Vol. 17, p 491-506

[5] NLANR Measurement and Network Analysis Group. [Online] Reviewof Bandwidth EstimationTechniques.<http://moat.nlanr.net/Papers/bandwidth-review.html>

[6] Downey, A.B. (1999) 'Using pathchar to Estimate Internet LinkCharacteristics', Proceedings ofACM SIGCOMM 1999

[7] Rumbaugh, J.E., Premerlani, J., and Blaha, M.R. (1998) 'RelationalDatabase Design using an Object Oriented Methodology', Journal of theAssociation for Computing Machinery, Vol. 31, No. 4, pp 414-427

[8] Connolly, T. and Begg, C. (2003) Database Systems, 3rd ed., USA:Pearson Education.

[9] Muihlenbein, H., Mahnig, T. and Rodriguez, A. 0. (1999). 'Schemata,distributions, and graphical models in evolutionary optimization',Journal of Heuristics, Vol. 5, pp. 215-247.

[10] Moscato P. (1995) 'Towards Memetic Algorithms', Caltech JournalConcurrent Computation, Vol. 79.

[I1] Holland, J.H. (1975) Adaptation in Natural and Artificial Systems , 1stEdition., USA: University of Michigan Press.

[12] De Silva, L.P.D.P. and Dissanayake, M (2005), SmartFragment -

Optimal Design of Distributed Databases, Final Year Project Thesis,Informatics Institute of Technology, Sri Lanka.

[13] IlliGAL - IlliGAL Technical Reports [Online]. Illinois GeneticAlgorithms Laboratory.<http://www.illigal.ge.uiuc.edu/techreps.php3>[2005, Feb. 28]

[14] Pelikan, M., Goldberg, D.E., and Cantui-paz, E. (2000) 'LinkageProblem, Distribution Estimation and Bayesian Networks', Journal ofEvolutionary Computation Vol. 4, pp 311-340.

Page 6