Kiener Journal of Cheminformatics 2013, 5:48http://www.jcheminf.com/content/5/1/48

SOFTWARE Open Access

Molecule database framework: a framework forcreating database applications with chemicalstructure search capabilityJoos Kiener

Abstract

Background: Research in organic chemistry generates samples of novel chemicals together with their propertiesand other related data. The involved scientists must be able to store this data and search it by chemical structure.There are commercial solutions for common needs like chemical registration systems or electronic lab notebooks.However for specific requirements of in-house databases and processes no such solutions exist. Another issue isthat commercial solutions have the risk of vendor lock-in and may require an expensive license of a proprietaryrelational database management system. To speed up and simplify the development for applications that requirechemical structure search capabilities, I have developed Molecule Database Framework. The framework abstractsthe storing and searching of chemical structures into method calls. Therefore software developers do not requireextensive knowledge about chemistry and the underlying database cartridge. This decreases applicationdevelopment time.

Results: Molecule Database Framework is written in Java and I created it by integrating existing free andopen-source tools and frameworks. The core functionality includes:

� Support for multi-component compounds (mixtures)� Import and export of SD-files� Optional security (authorization)

For chemical structure searching Molecule Database Framework leverages the capabilities of the BingoCartridge for PostgreSQL and provides type-safe searching, caching, transactions and optional method levelsecurity. Molecule Database Framework supports multi-component chemical compounds (mixtures).Furthermore the design of entity classes and the reasoning behind it are explained. By means of a simpleweb application I describe how the framework could be used. I then benchmarked this example applicationto create some basic performance expectations for chemical structure searches and import and export ofSD-files.

Conclusions: By using a simple web application it was shown that Molecule Database Framework successfullyabstracts chemical structure searches and SD-File import and export to simple method calls. The frameworkoffers good search performance on a standard laptop without any database tuning. This is also due to thefact that chemical structure searches are paged and cached. Molecule Database Framework is available fordownload on the projects web page on bitbucket: https://bitbucket.org/kienerj/moleculedatabaseframework.

Keywords: Chemical structure search, Database, Framework, Open-source

Correspondence: joos.kiener@gmail.comGivaudan Schweiz AG, Fragrances S&T, Überlandstrasse 138, 8600 Dübendorf,Switzerland

© 2013 Kiener; licensee Chemistry Central LtdCommons Attribution License (http://creativereproduction in any medium, provided the orwaiver (http://creativecommons.org/publicdootherwise stated.

. This is an Open Access article distributed under the terms of the Creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, andiginal work is properly cited. The Creative Commons Public Domain Dedicationmain/zero/1.0/) applies to the data made available in this article, unless

Table 1 Main software components

Component Supplier

PostgreSQL http://www.postgresql.org/

Bingo PostgreSQL Cartridge GGA Software Services

Indigo Toolkit GGA Software Services

Spring Pivotal (owned by VMware and EMC)

Hibernate JBoss (Red Hat)

QueryDSL Mysema

Kiener Journal of Cheminformatics 2013, 5:48 Page 2 of 13http://www.jcheminf.com/content/5/1/48

BackgroundIn organic chemistry research the chemists synthesizenovel compounds that exhibit the desired properties andperform better than existing compounds. The commercialobjective is to sell these new and superior compounds dir-ectly or in new products containing them. During theprocess from the initial creation of a new compound untilit is commercialized, a lot of data related to it is generatedsuch as physiochemical properties, analytical- and toxico-logical data. All this data needs to be stored and scientistsmust be able to search it by chemical structure.The chemist performs a chemical reaction and enters

the relevant chemical structures, the exact procedureand outcome into his/her electronic lab notebook (ELN)so that other chemists can find it and repeat the reactionif necessary. After this new compound is synthesized, anew entry for it is generated in the chemical registrationsystem. Unfortunately there is not much scientific litera-ture concerning how and what such registration systemsdo [1]. The main purpose is to assign each compound aunique in-house reference number that then can be usedto cross-reference it. There are many commercial soft-ware solutions for ELNs and chemical registrations suchas those from PerkinElmer (CambridgeSoft), Accelrys,ChemAxon or Dotmatics to name a few. However thesesolutions may be costly especially if they also requirea commercial relational database management system(RDBMS). An exception is ChemAxon CompoundRegistration [2] which runs on MySQL. Another issueis that they are closed-source and if such systems areused in the context of scientific experiments, the ex-periment is not fully reproducible by other scientistsunless they also have access to a license for the samesystem. Because these systems need to be highly configur-able, they are also very complex. So configuring and ad-ministrating such a system requires a lot of very specificexpertise. Consequentially it can be advantageous to buildyour own solution especially for organisations that alreadyhave an in-house software development department. Acustom solution does not have to be highly configurableas it can be tailored to your needs. This includes adminis-trative interfaces for user management and specific taskslike informing all users about new features. Such tools cangreatly reduce administrative overhead.Another common requirement is in-house compound

databases and processes for which no commercial solu-tions are available. These databases might not store in-formation on novel compounds but are still important inthe discovery process. For example, an organisation wishesto gather and store information on commercially availableproducts from competitors or other suppliers such aspurity, pricing and other properties. With PerkinElmer’sBioSAR application you can search for a chemical struc-ture in any database table in oracle that was indexed by

their cartridge. However it offers no easy solution for dataentry. Either a trained expert needs to enter the data or acustom web interface for easy data entry must be devel-oped. Dotmatics Browser application is very similar toBioSAR in this regard. In general there seems to be a needto create custom chemical structure searchable databaseapplications [3,4].As a solution for these issues Molecule Database

Framework (MDF) was created. MDF can be used tocreate chemical structure search enabled database appli-cations. One of the advantages compared to directlyusing a chemical database cartridge is, that all code toconnect, access and query the database is provided byMDF. It abstracts the storing and querying of chemicalstructures into method calls. The software developersusing MDF do not require extensive knowledge of eitherchemistry or the underlying database cartridge.MDF has been designed with a flexible domain model.

An application can have many types of chemical com-pounds like a registration compound and an inventorycompound which can have different properties. Samplesof these different types of chemical compounds can beassociated with a container identified by a unique bar-code. Such containers can then be searched by chemicalstructure.The design and functionality of MDF makes it possible

to build many different types of systems like registra-tion systems, inventory systems and simple compounddatabases or all of these together in one single integratedsolution.

ImplementationOpen-Source software componentsMolecule Database Framework was developed by inte-grating free, open-source software components (Table 1).These components are backed by a commercial organ-isation offering services or are wide-spread and have alarge base of contributors like PostgreSQL. This meansall these components are well maintained and should befuture proof.PostgreSQL is the RDBMS used for MDF. It is Open-

Source and available for different platforms (Linux, BSD,Solaris, Windows, OS X).

Figure 1 A schematic representation of the 4 software layers inMDF. The 4 layers are: the database using PostgreSQL (blue), ObjectRelational Mapping (ORM) provided by Hibernate using JPA (red),the data access layer implemented with Spring Data JPA (green)and a service layer for transaction support and security (purple).

Kiener Journal of Cheminformatics 2013, 5:48 Page 3 of 13http://www.jcheminf.com/content/5/1/48

Bingo PostgreSQL Cartridge is an extension to Post-greSQL that enables chemical structure search withinPostgreSQL. Bingo offers functionality that not all com-mercial cartridges have like advanced tautomer searchand resonance substructure search [5]. It is also availablefor Oracle and SQL Server. Bingo is responsible for thechemical structure searches in MDF.The Indigo Toolkit is a chemistry toolkit written in C++

with application programming interfaces (API) for variousother languages like Python, Java or .Net. Indigo is usedfor format conversion, import and export of SD-Files andfor generating chemical structure depictions includinghighlighted substructures.The Spring Framework is a framework for creating a

diverse range of Java applications. It can be seen as analternative to Java EE. Its core module is an inversionof control container for injecting dependencies atruntime. For example this means that the behaviour of anapplication can be changed by using a different implemen-tation of an interface. Which implementation to use canbe configured in an Extensible Markup Language (XML)file and does not require any code changes. To activatesuch a configuration change, the application must berestarted.MDF also uses the Spring Data JPA and Spring Security

modules. Spring Data JPA is for fast creation of JavaPersistence API (JPA) based data access layers. SpringSecurity is used for method level security and security-aware database queries (results are limited to content thata user is allowed to see on the database level).

Software layersMDF consists of the 4 layers: the database using Post-greSQL, Object Relational Mapping (ORM) provided byHibernate using JPA, the data access layer implementedwith Spring Data JPA and a service layer for transactionsupport and security (Figure 1).

Entity classesMDF uses JPA with Hibernate as its implementation. Allentity classes are therefore annotated with JPA annota-tions. The entity classes are shown as an UML Class dia-gram in Figure 2.

Base entityBaseEntity is an abstract class that manages metadatalike creation and last modified date and who created andmodified the entity. It contains the abstract getId()method to retrieve the unique database id. BaseEntityis further responsible for the optimistic locking feature ofJPA. It contains a column annotated with @Version. If anentity is updated, the numeric version column is incremen-ted by 1. Update statements always contain the version in

the where-clause and if a concurrent update occurs, theydo not match and the concurrent modification is detected.

Chemical structureChemicalStructure entity holds the chemical structuredata (SMILES [6] or molfile) and the structure key (InChikey).A ChemicalStructure is unique and immutable andmanaged by MDF. Users operate on ChemicalCompoundsand not on ChemicalStructures directly. Unique meansthat when a new ChemicalCompound is saved, the frame-work checks if the referenced ChemicalStructuresalready exist and if yes, reuses them. The existence checkis done by using the standard InChikey which has aunique constraint in the database. Immutable means thatif a ChemicalCompound is updated and one of theChemicalStructures was changed, MDF will auto-matically check if the updated ChemicalStructurealready exists and will use it or create a new ChemicalStructure. The old one will remain unchanged.While the Bingo cartridge offers a molecular weight

search without explicitly storing it in the database, thiscannot yet be combined with a chemical structure search.Therefore MDF automatically calculates the molecularweight of each ChemicalStructure and saves it to thedatabase. This allows structure searches within a specifiedmolecular weight range.

Chemical compound compositionChemicalCompoundComposition links togetherChemicalStructure and ChemicalCompound and de-fines the relative occurrence of the ChemicalStructurewithin the ChemicalCompound.

Chemical compoundThe ChemicalCompound entity is an abstract modelof a chemical compound. A ChemicalCompound is a

Figure 2 UML class diagram. The image shows the UML class diagram of the entity classes of MDF. Fields and methods are omitted. The focusis on the relationships between the entity classes.

Kiener Journal of Cheminformatics 2013, 5:48 Page 4 of 13http://www.jcheminf.com/content/5/1/48

“descriptive entity”. It is not a concrete object that physicallyexists. It consists of ChemicalCompoundCompositions.A ChemicalCompound is therefore a mixture if it con-sists of multiple ChemicalCompoundCompositions(see Figure 2).The class contains some basic fields like compound

Name and cas. ChemicalCompounds can also be as-sociated with Containables. Developers using MDFmust create concrete implementations of this abstractclass. An application can have multiple implementationsof ChemicalCompound and each implementation isstored and searched separately (Table per Concrete class

Inheritance). Note that due to better usability it wasdecided to make CAS-Number column nullable and it isnot unique. This behavior can be changed by manuallycreating the according constraints in the database.

ContainableA Containable is a set of a ChemicalCompoundsthat have a common source. For example they were pro-duced in the same way. In a Chemical Registration Systemthis would be a Batch and in an Inventory System a Lot.The important part is that ChemicalCompound andContainable are generic classes and must form a pair:

Kiener Journal of Cheminformatics 2013, 5:48 Page 5 of 13http://www.jcheminf.com/content/5/1/48

Chemical compound containerA ChemicalCompoundContainer holds exactly oneContainable of any type. An application should only haveone implementation of ChemicalCompoundContainer.A ChemicalCompoundContainer instance repre-sents a concrete physically available object containinga ChemicalCompound linked by a Containable.ChemicalCompoundContainer has a barcode fieldwhich is unique and not nullable. The barcode henceuniquely identifies a physically available sample of aChemicalCompound.

Role and userRole and User are only relevant if MDF is used withSpring Security. ChemicalCompound and Containablehold a reference to their read-role. This is used to filterChemicalCompoundContainers in the database basedon the current users’ privileges. Example:An application has two ChemicalCompound-

Implementations, DefaultCompound and SecretCompound. Current user has the privilege to readDefaultCompounds but not for reading (viewing)SecretCompounds. So if this user searches forChemicalCompoundContainers, only ChemicalCompoundContainers that contain a DefaultCompound must be returned by the search. To achieve that, thequery’s WHERE-clause is extended and the filter based onthe Role is added automatically. The main advantage ofdoing this filtering in the database compared to filtering theresults within the applications is that you get pageableresults which would not be easily possible (if at all) withapplication-side filtering.

Data access layerSpring data JPAThe data access layer was created with Spring Data JPA.Spring Data JPA is a module for fast creation of data

access layers. It provides methods for create, read, up-date and delete (CRUD) operations. The developer doesnot need to write any code for this.When using Spring Data you create a new interface

that extends from generic interfaces provided by SpringData JPA and represents the repository for an entity.There are different kinds of repository interfaces butthe repositories in MDF all extend JpaRepository.JpaRepository provides CRUD-methods and basicretrieval methods for your entity. All these methods areimplemented automatically by Spring Data. The interfaceitself can remain completely empty.Repositories in MDF also extend QueryDslPredicate

Executor. This adds findOne(predicate) andfindAll(predicate) methods. Predicates are basic-ally type-safe WHERE-Clauses provided by QueryDSL.Repositories can also contain custom find-methods. The

convention is that the methods starts with “findBy” and isthen followed by a path. An example would be the methodT findByCas(String cas) in ChemicalCompoundRepository. You only need to write the method signatureinto your interface. The implementation is created automat-ically by Spring Data.Another option is to annotate a repository interface

method with @Query. The value of the annotation iseither a JPQL Query or native SQL.

Chemical structure searchThe main challenge was how to make Bingo Cartridgemethods usable as Spring Data Repository methods.This involved two steps: First the Bingo Cartridgemethods were exposed to Hibernate by extendingPostgreSQL82Dialect and then were used by creat-ing custom Repository implementations.Spring Data JPA allows you to create method implementa-

tions and to use them in your repository interface. In MDF ageneric interface for chemical structure search was created.

As a last step the methods from ChemicalStructureSearchRepository must be implemented. This can bedone in a custom repository implementation. The convention is to name the implementation the same as the inter-face and add “Impl” at the end of the name.

A repository offering chemical structure search extends above interface.

Kiener Journal of Cheminformatics 2013, 5:48 Page 6 of 13http://www.jcheminf.com/content/5/1/48

Spring Data JPA will merge the interface ChemicalCompoundRepository and its implementation Chem-icalCompoundRepositoryImpl together under theinterfaces type.See Additional file 1 for the full source code of

ChemicalCompoundRepositoryImpl and AbstractStructureSearchRepositoryImpl which it extends.As can be deduced from above source code, the chem-

ical structure search methods support paging. Paging ofsearch results has one main benefit: only the requested

data needs to be loaded from the database. This can havea huge impact on performance especially if the structuresare stored as molfiles. Paging requires that the total num-ber of records is known. The need to count the total hitsof a query for each page will greatly reduce performance.MDF solves this issue by caching the count of a query.Hence the count is only determined when the query is ex-ecuted the first time. Subsequent requests for the nextpage use the cached value. In the event that relevant datais edited or added, the cache is cleared.

Kiener Journal of Cheminformatics 2013, 5:48 Page 7 of 13http://www.jcheminf.com/content/5/1/48

MDF contains a Spring data repository for each entity(ChemicalCompound, Containable, and ChemicalCompoundContainer) with methods for chemicalstructure searching. A developer must extend the providedrepositories for each of his entity implementations andmay add his own custom search methods to them.

Service layerThe service layer is responsible for transaction manage-ment and security. It also offers methods for importingand exporting SD-Files. Transactions and security are im-plemented declaratively by using according annotationsfrom Spring framework. Security is entirely optional. Ifyou do not need it, MDF remains fully operational andevery user can see everything.

Service for ChemicalStructure entityThis service manages entities of type ChemicalStruc-ture. This service is provided by the framework and shouldbe used as-is. Any access of ChemicalStructures shouldbe through this service. ChemicalStructureServicecontains the logic that makes ChemicalStructuresimmutable. For convenience ChemicalStructureService has a method that allows changing an exist-ing ChemicalStructure. This will then affect allChemicalCompoundCompositions that containthe changed ChemicalStructure.

Services for ChemicalCompound and ContainableThe services for ChemicalCompound and Containableare very similar. They consist of an interface whichcontains the security annotations and an implementa-tion containing the @Transactional annotations fordeclarative transactions. They also offer methods withoptional loading of lazy collections.When an entity is saved, the service automatically sets

the correct ChemicalStructures by either selectingan existing one from the database or creating a new one.Before the entity is passed to Hibernate, the method

preSave(entity) is executed. This method is empty inthe provided abstract services like ChemicalCompoundServiceImpl but can be overridden in subclasses. For ex-ample one could set all non-nullable properties to a default ora sequence value if it is null.For every ChemicalCompound and Containable

implementation the application uses, a service interface andan implementation must be created. The services must imple-ment the getRepository(), checkUniqueness()and getExistingCompound() methods.

Service for ChemicalCompoundContainerThe main difference to services for ChemicalCompoundand Containable is that an application should only have1 implementation of ChemicalCompoundContainer

and hence only one such service. The service can be used asprovided or in some cases must be extended. When theChemicalCompoundContainer implementation addsadditional unique constraints, checkUniqueness() andgetExistingContainer() must be overridden.

Results and discussionFunctionalityBelow is a listing of the core functionality of MDF:

� Chemical structure search: Full, Sub, SMARTS,Similarity, Formula

� Chemical structure searches can be combined withproperty searches

� Chemical structure searches are paged and cached� Support for multi-component compounds

(mixtures)� 3 chemical structure searchable entities:

ChemicalCompound, Containable andChemicalCompoundContainer

� Import and Export of SD-Files for above 3 entities� Transactional database access� Optional security (authorization)

With the design and functionality of MDF it is possibleto build many different types of system such as registra-tion systems, inventory systems or just a simple com-pound database. While you could also create your ownELN, there also exists the free Indigo ELN. This ELN wascreate by GGA Software Services and is used at Pfizer [7].In contrast to MolDB5R [3] and to MyMolDB [4],

MDF is not a fully functional standalone web applicationwith chemical structure search. As the name implies it’sa framework to simplify creating such an application.MDF may also be used to create local or client-serverdesktop applications. MDF is targeted at software devel-opers and not intended for use by scientists themselves.However MDF features are very robust. Chemical structuresearching is done in the database and not the applicationcode. Hence you can search by chemical structure andother properties at the same time, the results can be sortedby multiple properties and can be paged (SQL OFFSET andLIMIT clauses). Note that if you do the chemical structuresearch in application code, any query will require at leasttwo trips to the database, namely the structure search andsubsequently the filtering by other properties, sorting and/or limiting. Both need to happen in the same transaction. Itwas not determined if MolDB5R [3] and MyMolDB [4] ac-tually do this in the same transaction.In MDF the chemical compounds can be associated with a

containable, which in registration systems would be a batchor in an inventory system a lot. A specific physically availablesample in a bar-coded bottle can then be associated with acontainer. These containers can also be searched by chemical

Kiener Journal of Cheminformatics 2013, 5:48 Page 8 of 13http://www.jcheminf.com/content/5/1/48

structure. This is the foundation for creating an inventorysystem. Developers may add as many additional propertiesas they want to each of the entities and all of them aresearchable together with the chemical structure.All data access in MDF is transactional to prevent data

inconsistencies. MDF can be configured to use a data-base connection pool. When querying a RDBMS creat-ing a connection often takes more time than the queryitself and hence if you already have open connections re-sponse times can be reduced.For similarity searching MDF exposed the algorithms

provided by the Bingo Cartridge which are Tanimoto,Tversky and Euclidean metric for substructures.MDF is ready to be used with Spring security. Security is

optional. MDF offers method level security (authorization).It does not offer any authentication features.

Mixture handlingMDF supports multi-component chemical compounds.Searching by substructure will return all compoundsthat have at least one component (chemical structure)matching the query structure. This is important becausereaction products that may be entered into a chemicalregistration system are almost always mixtures unlessextensive purification is done.

Figure 3 Search page of the example web application using MDF. A uand/or properties like compound name or CAS number.

If an entry in an imported SD-File consists of multipledisconnected structures it is assumed that this entry is amixture and each structure is stored as a separate chemicalstructure.

Structure normalizationBy default MDF stores the chemical structures as theyare submitted. MDF does not do any standardization/normalization of chemical structures. It is up to the de-veloper using MDF to ensure that chemical structuresare correctly normalized prior to saving them to thedatabase. It is currently suggested that developers imple-ment such a feature by overriding the preSave() methodof ChemicalCompoundServiceImpl. This method iscalled before any chemical compound is created or updated.Within this method the chemical compound and all the chem-ical structures it consists of can be freely manipulated as de-sired. Note that every compound being saved will be processedby this method.

Salts, solvates and solutionsMDF current version 1.0.1 has no special handling forsalts, solvates or solutions. MDF will store separate com-ponents in a chemical structure file as a separate chemical

ser can search the compound database by chemical substructure

Kiener Journal of Cheminformatics 2013, 5:48 Page 9 of 13http://www.jcheminf.com/content/5/1/48

structure. Therefore saving a salt like [NH+]1 = CC =CC =C1.[Br-] will be represented as a mixture of the twoions without any percentages set. An exact structuresearch for either ion would return this salt. If the salt has acharge greater than 1 and multiple ions associated to itlike [NH+]1 = CC= [NH+]C =C1.[Br-].[Br-] the salt willbe stored as a mixture of [NH+]1 = CC = [NH+]C = C1and [Br-] without any percentages set. If the chemical struc-ture is a single ion it will be stored and searchable like anyother chemical structure. If this behaviour is unsuitable in aspecific case, developers may implement salt and solvatehandling functionality in the preSave() method.Some commercial systems also appear to have no way

of handling solutions. It is recommended to create thecompound as if it were pure and add the solution infor-mation as separate fields on the compound level.

Example web applicationA simple web application using MDF was created. The webapplication makes use of Spring MVC. The application doesnot make use of the security integration and it does not usethe entities Containable and ChemicalCompoundContainer. It only uses ChemicalCompound entity. The

Figure 4 Result page of a substructure search. The results are displayedchemical structure images have the matching substructure highlighted in rfor it.

application is a compound database for multi-componentcompounds. It has a page for importing the chemicalstructures in an SD-file into the compound database.The database can be searched by substructure and prop-erties. It uses JSME [8] for drawing the chemical struc-tures (Figure 3). The search results page displays thesearch hits in a tabular and paged fashion. When asubstructure search is done, the substructure will behighlighted in the search results (Figure 4). The hits of asearch can be exported as an SD-file. The search resultscontain a link to a single compound view. The propertiesof the compound may be edited and compositions canbe added, edited and deleted (Figures 5, 6). When edit-ing a compound or a composition the application dealswith concurrent modifications transparently and conflictresolution dialog is shown on which the user can selectwhich values to use for each property and then save thatnew version.

PerformanceMDF has one main performance issue when handling mix-tures. If an application uses mixtures, i.e. compounds withmultiple components, a chemical structure query will

in a paged table generated by the JQuery plugin datatables [9]. Theed. Clicking in the chemical structure image will display the SMILES

Figure 5 Individual compound view. This web page displays a single compound. The compound can be edited or deleted by clicking on theaccording link within the page. There is a tab displaying all contained chemical structures and a tab for each individual composition thecompound consists of.

Figure 6 Single composition. Shows the same page as Figure 5 but instead of the Combined Structure tab, the tab of the first composition isselected. The composition can be edited by clicking on the according link within the page.

Kiener Journal of Cheminformatics 2013, 5:48 Page 10 of 13http://www.jcheminf.com/content/5/1/48

Table 2 Substructure search performance

SMILES First page (s) Second page (s) Hits

OCC1CCN1 0.5 0.5 21

CCCC1CCOC1 0.6 0.5 815

CCNCCOCC 2.1 0.5 31802

CCCCCC 4.8 0.5 92788

c1ccccc1 19 0.5 420299

Table 4 Substructure search with distinct query andadditional filter ‘ZINC34%’ on compound name

SMILES First page (s) Second page (s) Hits

OCC1CCN1 1.2 - 0

CCCC1CCOC1 1.3 - 1

CCNCCOCC 1.0 0.7 49

CCCCCC 1.2 0.6 198

c1ccccc1 4.2 0.6 981

Kiener Journal of Cheminformatics 2013, 5:48 Page 11 of 13http://www.jcheminf.com/content/5/1/48

return one row for every component in a compoundmatching the query. This is undesirable because the endusers want to see each compound that matches the queryonly once. The solution to the problem is to use a distinctquery and this is where the performance issue happens. Ifyou perform a distinct query, the whole database needs tobe searched regardless of limit clause which greatly in-creases execution time. Note that sorting has the same ef-fect. So sorting can have a huge performance penalty tooand when paging you should always sort to get a predict-able outcome. To make this even worse Bingo Cartridgefor PostgreSQL does not yet have a proper implementationfor cost estimation and the cost for using the chemicalstructure index is hard coded and underestimated. Thismisleads the PostgreSQL query planer to always use a fullindex scan on structure search index even when the queryhas an additional where-clause that greatly limits the amountof results. In these cases it would, for example, be faster touse the index for the CAS number and use the Bingo match-sub function for filtering. The matchsub function does sub-structure matching without index. This is of course slowerthan with an index but if it only has to be done for a smallnumber of structures it is a lot faster than the full index scan.To fix the cost estimation issue, MDF does some internal cal-culations to explicitly decide whether the structure index orthe matchsub function is used. This can improve perform-ance by an order of magnitude. Note that the supplier of theBingo Cartridge is aware of this issue and the timeline for thefix was end of 2013. The main issue of distinct queries andsorting however is inherent to how relational databases workand cannot be solved except with a better substructuresearch index or better hardware. MDF also offers asetting to disable distinct queries application-wide forsingle-component compound databases.

Table 3 Substructure search performance with distinctresults

SMILES First page (s) Second page (s) Hits

OCC1CCN1 0.6 0.4 21

CCCC1CCOC1 2.7 0.5 815

CCNCCOCC 6.2 2.1 31802

CCCCCC 9.3 4.8 92788

c1ccccc1 38 19 420299

To benchmark MDF the earlier outlined web applica-tion was used. The database contains 525573 uniquecompounds. The compounds are from Zinc [10] subset13 at reference pH 7 the SD-files 13_p0.0.sdf, 13_p0.1.sdf, 13_p0.10.sdf and 13_p0.11.sdf. The structures arestored in the database as SMILES. Importing each of theSD-files, which contain approximately 131’000 chemicalstructures, took 12 min with the chemical structuresearch index disabled. Rebuilding the index after all SD-files were imported took 22 min on a laptop with 4 GBRAM, Core i5-3220M CPU and a 512GB Samsung 830SSD. That totals to 1 h 10 min to setup a fully indexeddatabase with half a million compounds. As an add-itional reference the same import was done on a desktopPC with 12 GB of RAM, i7-875K @ 3.4 GHz and thedatabase running on a Western Digital Green Drive(5400 RPM). Here the import took 8 min and the con-clusion is that the import is CPU limited rather than be-ing limited by storage-speed. The index generation tookapproximately 22 min on the laptop and 20 min on thedesktop. The conclusion here is that it is limited moreby CPU but drive speed also matters. The import andindex generation performance when storing the struc-tures as molfiles was not benchmarked.The substructure search performance was bench-

marked with different configuration settings. Substruc-ture search is done by Bingo PostgreSQL cartridge [5]and this benchmark therefore reflects its performanceplus any overhead caused by MDF. With the exceptionof c1ccccc1 the author drew chemical structures with nospecific significance and tested the search speed. Thesearch speed was determined by logbacks [11] imple-mentation of org.slf4j.profiler.Profiler.The first benchmark is a reference. This benchmark used

the option to disable all distinct queries and no sorting was

Table 5 Tanimoto similarity search performance: hits withat least 90% similarity

SMILES Firstpage (s)

Secondpage (s)

Hits

CC1(C)Cc2ccccc2/C(=C/C(=O)NC2CCCCC2)/N1 0.40 - 3

COc1ccc(NC([C@H]2CC(O[C@H]2C) =O) = O)cc1 0.40 0.32 11

C[C@H](CCC)C[C@@H](C[NH3+])c1cc[n]cc1 0.41 0.20 17

Kiener Journal of Cheminformatics 2013, 5:48 Page 12 of 13http://www.jcheminf.com/content/5/1/48

done. MDF performs a count of total hits on first occur-rence of a chemical structure search and the count iscached which causes the first page to load slower than sub-sequent pages. Each page contains 4 records. The resultsare shown in Table 2 ordered ascending by number of hits.The benchmark was repeated but this time with

distinct queries enabled. First page load time is dou-bled because the count query is run and then the ac-tual query is run which takes about the same amountof time as the count query due to the distinct clause.The second page then always takes half the time toload compared to the first page for the same reason(Table 3). The number of hits is identical to the onesin Table 2 because all compounds in the databaseconsist of only one component.The results show that Bingo does no optimizations for a

common substructure query like a benzene ring andhence searching for c1ccccc1 in a database in which al-most all molecules have this feature is very slow. To im-prove search speed in such a scenario filtering byadditional properties is recommended. Hence the bench-mark was repeated with an additional filter of compoundname starting with “ZINC34”.Table 4 shows the benefit of the MDF optimization as

a workaround of the cost estimation issue in Bingo Post-greSQL cartridge. Without this optimization the bench-mark would have the same performance as shown inTable 3.MDF also uses Bingo Cartridges similarity searching

functionality. Its performance was tested by searchingfor compounds with a similarity score of 0.9 using theTanimoto similarity measure also known as Jaccard Index[12]. The results are shown in Table 5.

OutlookFor generating the chemical structure depictions theIndigo toolkit is used. The toolkit can be configured togenerate the structures in many ways including the col-oring of heteroatoms, bond length and width and manymore. Currently this is hard coded and cannot be ad-justed by the user. A next step would be to expose theseconfigurations options so they can be set through a Javaproperties file. Also the handling of salts and solvatesmust be implemented to make MDF usable in areaswere such compounds are important.To make use of MDF you must be able to program in

Java and you will need basic knowledge about Springframework and how to configure it. This limits the tar-get audience. When using MDF you need to write someboiler-plate code and hence the next step would be tocreate additional tools to facilitate the usage of MDF likethe automatic generation of entity classes and their re-positories and services. These tools would need to beconfigurable so that a user can define the desired

properties for each of the entities and the desired searchmethods. An option would be a maven plugin. Mavenplugins can generate code like the metamodel creationdone by the QueryDSL-maven plugin. Another optionwould be annotations that generate code on compile likeProject Lombok does [13].The final step would be to create a web application

platform that lets administrators create new web appli-cations with chemical structure search capability by sim-ply entering the desired properties for the entities on aweb form and clicking on a button. It is obvious that thiswould require a significant development effort.

ConclusionsMDF successfully abstracts chemical structure data main-tenance and chemical structure searching and lets softwaredevelopers focus their efforts on other aspects of the appli-cation like the graphical user interface and usability. Thiswas achieved by integrating well maintained and widelyused open-source software components which offer fea-tures like dependency injection, declarative transactions, se-curity or advanced chemical structure search features liketautomer or resonance structure search. MDF therefore of-fers a robust basis for creating database applications withchemical structure search capability.

Availability and requirementsMDF source code is available as mercurial source con-trol repository on MDF’s project page on bitbucket [14]or as Additional file 2. Besides the source code the pro-ject page also contains a Wiki [15] with much more de-tailed information and usage instructions for MDF.MDF is also available as a maven artefact on mavencentral [16] or for a jar file without dependencies seeAdditional file 3. Note that this is a framework andnot a runnable application!MDF is licensed under AGPL 3 [17].MDF is written in Java 7 and was successfully tested in

32 and 64-Bit Windows 7. However all components arecross-platform and it should run on other operating sys-tems too.The source code for the example web application is also

available as mercurial source control repository [18] or asAdditional file 4. In the download section there is also a 7-zip archive of a standalone version [19] of this simple webapplication for Windows 64-bit. No installation is neededas the standalone version already contains a configuredPostgreSQL database and the tomcat servlet container.

Additional files

Additional file 1: Highlighted Source Code ofAbstractStructureSearchRepositoryImpl andChemicalCompoundRepositoryImpl.

Kiener Journal of Cheminformatics 2013, 5:48 Page 13 of 13http://www.jcheminf.com/content/5/1/48

Additional file 2: Molecule Database Framework source code.

Additional file 3: Molecule Database Framework jar file withoutdependencies.

Additional file 4: MDF simple web application source code of themercurial changeset 16f39f4e447b.

AbbreviationsAGPL: Affero General Public License; API: Application programming interface;CRUD: Create read update delete; ELN: Electronic lab notebook; JPA: Javapersistence API; JPQL: Java persistence query language; MDF: Moleculedatabase framework; ORM: Object relational mapping; RDBMS: Relationaldatabase management system; SQL: Structured query language;UML: Unified modelling language; XML: Extensible markup language.

Competing interestsThe author declares that he has no competing interests.

Authors’ informationJK has a M.Sc. in Biology from ETHZ and a MAS in Computer Science fromFFHS. He is employed by Givaudan Schweiz AG as Research Associate.

AcknowledgementsThis work would not have been possible without all the helpful users onstackoverflow. I also thank Dr. Andreas Muheim, Dr. Markus Gautschi andDr. Robin Clery for their support and thorough review of this manuscript.

Received: 30 September 2013 Accepted: 2 December 2013Published: 11 December 2013

References1. Martin E, Monge A, Duret JA, Gualandi F, Peitsch MC, Pospisil P: Building an

R&D chemical registration system. J Cheminf 2012, 4:11.2. ChemAxon Compound Registration. http://www.chemaxon.com/products/

compound-registration/.3. Haider N: Functionality pattern matching as an efficient complementary

structure/reaction search tool: an open-source approach. Molecules 2010,15:5079–5092.

4. Xia B, Tai ZF, Gu YC, Li BJ, Ding LS, Zhou Y: MyMolDB: a micromoleculardatabase solution with open source and free components. J ComputChem 2011, 32:2942–2948.

5. Pavlov D, Rybalkin M, Karulin B: Bingo from SciTouch LLC: chemistrycartridge for Oracle database. J Cheminf 2010, 2(Suppl 1):F1.

6. Weininger D: SMILES, a chemical language and information system. 1.Introduction to methodology and encoding rules. J Chem Inf Model 1988,28:31–36.

7. Indigo ELN: The Open-Source Chemistry Electronic Lab Notebook.http://www.ggasoftware.com/opensource/indigo/eln.

8. Bienfait B, Ertl P: JSME: a free molecule editor in JavaScript. J Cheminf 2013, 5:24.9. DataTables (table plug-in for jQuery). https://datatables.net/.10. Irwin JJ, Shoichet BK: ZINC − A free database of commercially available

compounds for virtual screening. J Chem Inf Model 2005, 45:177–182.11. Logback. http://logback.qos.ch12. Jaccard P: Distribution de la flore alpine dans le bassin des Dranses et

dans quelques régions voisines. Bull Soc Vaud Sci Nat 1901, 37:241–272.13. Custom AST transformations with Project Lombok. http://www.ibm.com/

developerworks/library/j-lombok/.14. MDF – Source Code. https://bitbucket.org/kienerj/moleculedatabaseframework/src.15. MDF –Wiki. https://bitbucket.org/kienerj/moleculedatabaseframework/wiki/Home.16. Maven Central Repository. http://search.maven.org/.17. GNU Affero General Public License. http://www.gnu.org/licenses/agpl-3.0.html.18. MDFSimpleWebApp. https://bitbucket.org/kienerj/mdfsimplewebapp.19. MDFSimpleWebApp Standalone Download. https://bitbucket.org/kienerj/

mdfsimplewebapp/downloads.

doi:10.1186/1758-2946-5-48Cite this article as: Kiener: Molecule database framework: a frameworkfor creating database applications with chemical structure searchcapability. Journal of Cheminformatics 2013 5:48.

Open access provides opportunities to our colleagues in other parts of the globe, by allowing

anyone to view the content free of charge.

Publish with ChemistryCentral and everyscientist can read your work free of charge

W. Jeffery Hurst, The Hershey Company.

available free of charge to the entire scientific communitypeer reviewed and published immediately upon acceptancecited in PubMed and archived on PubMed Centralyours you keep the copyright

Submit your manuscript here:http://www.chemistrycentral.com/manuscript/