IBSurgeon Recovery Guide: chapter from "The InterBase World"

8/14/2019 IBSurgeon Recovery Guide: chapter from "The InterBase World"

1/15

InterBase and Firebird recovery guide

by Alexey Kovyazin, Serg Vostrikov

NOTICE: This document is the chapter from the book "The InterBase World" which was written byAlexey Kovyazin and Serg Vostrikov.

The chapter from book "The InterBase World" devoted to the database repairing.

1. The history of this guide

The Russian book "The InterBase World" was published in September, 2002.

Its pressrun was 3000 copies. After 3 months it was sold out and second, improvededition was published in April, 2003 with pressrun 5000 copies.

Now it is on top in the largest Russian online-bookstores and we intend that it will

sold out very soon.

The authors of the book are Alexey Kovyazin, developer of IBSurgeon andwell-known Russian InterBase specialist, and Serg Vostrikov, CEO of Devracecompany www.devrace.com

Its a funny thing, not a single book devoted to InterBase was published in English!Thousands and thousands of developers use InterBase and Firebird, discuss thetopic in various conferences (take a look here: Links).

The community of Interbase developers averages out to tens of thousands ofpeople. The strong demand on the InterBase books in various countries proves theInterBase community is really big.

We can stake the case of beer on the fact that the edition of 10000 copies will beswept away from Amazon.com in a month. But people in publising companies"knows everything" and they are sure that nobody buy book about InterBase. Its a

2002 AIexey Kovyazin, Serge Vostrikov.

Page 1Copyright 2004 IBSurgeon Team. All rights reserved.
http://www.devrace.com/http://../links.htmlhttp://../links.htmlhttp://www.devrace.com/


2/15

real pity.

Here we'd like to offer you the draft of one chapter of this book devoted to recoveryof InterBase/Firebird databases.

2. How to recover InterBase/Firebird database

2.1. Review of main causes of database corruption

Unfortunately there is always a probability that any information store will becorrupted and some information from it will be lost. Database is not an exception tothis rule. In this chapter we will consider the principal causes that lead to InterBasedatabase corruption, some methods of repairing databases and extracting

information from them. Also we will get to know the recommendations andprecautions that will minimize a probability of information loss from database.

First of all, if we speak about database repairing we should clarify a notion ofdatabase corruption. Database is usually called damaged if trying to extract ormodify some information errors appear and /or the extracting information turns out tobe lost, incomplete or not right at all. There are cases when database corruptionsare hidden and are found only by testing with special facilities, but there are also realdatabase corruptions when it is impossible to be connected to the database, whenadjusted programs-clients show strange errors (when no manipulations wereexecuted with database), or when it is impossible to restore the database frombackup copy.

2.2. Principal causes of database corruption are:

1. Abnormal termination of server computer, especially electric power interruption.For IT- industry it is a real lash and that is why we hope there is no need toremind you once again about the necessity of having a source of uninterruptedpower supply on server.

2. Defects and faults of server computer, especially HDD (hard disk drive), diskcontrollers, main memory of the computer and cache memory of Raid controllers.

3. Not a proper connection string with a multi-client database of one or more users(in versions prior to 6.x ) When connecting via TCP/IP, the path to database must

be pointed server name: drive:/path/databasename /for servers on UNIX platformservername: /path/databasename /, according to NETBEUI protocol\\servername\drive:\path\databasename. Even when connecting database fromthe computer, on which database is located and server is running one should use





3/15

the same line renaming servername for localhost. One cannot use mapped drivein the connection line. If you break one of these rules, server considers that itworks with different databases and database corruption is guaranteed.

4. File copy or other file access to database when server is running. The executionof the command shut-down or disconnecting the users in usual way is not aguarantee that server is doing nothing with the database, if sweep interval is notset to 0, garbage collection can be executed. Generally the garbage collectionis executed immediatelly after the last user disconnects from database. Usually ittakes several seconds, but if before it many DELETE or UPDATE operationswere committed, a process may be longer.

5. Using unstable InterBase server versions 5.1-.5.5.Borland Company officiallyadmitted that there were several errors in these servers and a steady upgrate 5.6removed only after an output of certified InterBase 6 was in free-running mode

for all clients of servers 5.1-5.5 on its site.6. Exceeding size restriction of database file. For most existing, at the moment of

writing of these lines, servers of UNIX platform it is 2 Gb, for windows NT/2000-4Gb, but it is recommended to take 2 Gb. When approaching database size to alimit value, an additional file must be created.

7. Exhaustion of free disk space when working with database.8. For Borland InterBase servers versions under 6.0.1.6 - exceeding of restriction to

a number of generators according to Borland InterBase R & D defined in thefollowing way (look table 1).

Version Page size=1024 Page size=2048 Page size=4096 Page size=8192

Pre 6 248 504 1016 2040

6.0.x 124 257 508 1020

Table 1: Critical number of generators in early InterBase versions For all Borland InterBase servers exceeding of permissible number of

transactions without executing backup/restore. One can know the number oftransactions happened in database from the time of last creation by invoking theutility gstat with a key h- parameter NEXT TRANSACTION ID will be desiredquantity of transactions. According to Ann W.Harrison the critical number oftransactions depends on page size, and has the following values (look table 2):

Database page size Critical number of transactions

1024 byte 131 596 2872048 byte 265 814 016

4096 byte 534 249 472





4/15

8192 byte 1 071 120 384

Table 2: Critical number of transactions in Borland InterBase serversConstraints of Borland InterBase servers enumerated above are not applied toFirebird servers except for the earliest versions 0.x., existence of which has alreadybecome a history. If you use the final version Firebird 1.0 or InterBase 6.5-7.x, youshould not be worried about points 5, 6, 8 and 9 and you should concentrate yourefforts on other causes. Now we will consider the most frequent of them in detail.

2.3. Power supply failure

When shutting-off the power on server, all the activities of data processing areinterrupted in the most unexpected and (according to Murphys law) dangerous

places. As a result of it the information in database may be distorted or lost. Thesimplest case is when all uncommitted data from clients applications were lost as aresult of emergency server shutdown. After power-fail restart server analyzes thedata, notices incomplete transactions related to none of the clients and cancel all themodifications made within the bounds of these dead transactions. Actually suchbehavior is normal and supposing from the beginning by InterBase developers.

However power supply interruption in not always followed by such insignificantlosses only. If server was executing database extension at the moment of powersupply interruption, there is a big probability of having orphan pages in database file(pages that are physically allocated and registered on page inventory page (PIP) ,data writing on which is impossible). If you want to know more about orphan pages

look chapter The structure of InterBase database.

Only the tool of repairing and modification gfix (we will consider it below) is able tofight with orphan pages in database file. Actually orphan pages lead to unnecessaryexpense of disk space and as such are not the cause of data loss or corruption.Power loss leads to more serious damages. For example, after shutting off thepower and restarting a great amount of data, including committed, may be lost (afteradding or modification of which a command commit transaction was executed). Ithappens because confirmed data are not written right to database file on disk. Andfile cache of operating system (OS) is used for this aim. Server process gave datawriting command to OS. Then OS assured server that all the data were saved ondisk and in reality data were stored in file cache. OS doesnt hurry to flush thesedata to disk, because it considers that there is much main memory left and puts offslow operations of writing to disk until main memory is filled.





5/15

2.4. Forced writes cuts both ways

In order to affect the situation tuning of data write mode is provided in InterBase 6.This parameter is called forced writes (FW) and has 2 modes ON (synchronous)and OFF (asynchronous). FW modes define how InterBase communicates with disk.If FW is turned on, the setting of synchronous writes to disk is switched on, whenconfirmed data are being written to disk just after command commit, server is waitingfor writing completion and only then continues processing. If FW is turned offInterBase doesnt hurry to write data to disk after the command of transactioncommit and delegates this task to parallel thread while main thread continues dataprocessing not waiting until writes are done to disk. Synchronous writes mode is oneof the most careful and it minimizes any possible data loss, however it may cause

some loss of performance. Asynchronous writes mode increases a probability of lossof great number of data. In order to achieve maximum performance FW Off mode isusually set. But as a result of power interruption much more number of data is lostduring the asynchronous writes than synchronous. When setting the write mode youshould decide whether a few percents of performance are more significant than afew hours of work if power interruption happens unexpectedly.

Very often users are careless to InterBase. Small organizations save on any trifle,often on computer-server where DBMS server and different server programs (andnot only server) are set as well. If they hang-up people thinking for not a long timepress RESET (it happens several times a day). Although InterBase is very steady tosuch activities comparing with other DBMS and allows to start working with database

just after emergency reboot, but such use isnt desired. The number of orphan pagesincreases and data lose connections among themselves as a result of fault reboots.It may continue for a long time, but sooner or later it will come to an end. Whendamaged pages appear among PIP or generators pages or if database header pageis corrupted, database may never open again and become a big piece of separatedata from which one cant extract a single byte of useful information.

2.5. Corruption of hard disk

Hard disk corruptions lead to missing of database important system pages and/orcorruption of links among the remained pages. Such corruptions are one of the most

difficult cases, because almost always they require low-level interference to restorethe database.

2.6. Mistakes of database design





6/15

Its necessary that you know about some mistakes made by database developers

that can lead to impossibility of database recovery from a backup copy (*.gbk filescreated by gbak program). First of all this is a careless use of constraints ondatabase level. Typical example is constraints NOT NULL. Lets suppose that wehave a table filled with the number of records. Now well add to this table usingALTER TABLE command one more column and point that it mustnt containnon-defined values NULL. Something like this:ALTER TABLE sometable Field/INTEGER NOT NULL

And in this case there will be no servers error as it could be expected. Thismetadata modification will be committed and we wont receive any error or warningmessage that creates an illusion of normality of this situation.

However, if we backup database and try to restore it from a backup copy, wellreceive an error message at the phase of restoring (because Nulls are inserted intothe column that has NOT NULL constraint, and the process of restoring will beinterrupted. (The important note provided by Craig Stuntz - with version InterBase7.1 constraints are ignored by default during restore (this can be controlled by acommand-line switch) and nearly any non-corrupt backup can be restored. It'salways a good idea to do a test restore after making a backup, but this problemshould pretty much go away in version 7.1. ) This backup copy cant be restored. Ifrestoring was directed to file having the same name as the existing database (duringrestoring existing database working file was being rewritten) well lose the wholeinformation.

It is connected with the fact that constraints NOT NULL are implemented by systemtriggers that check only arriving data. During restoring the data from backup copyare inserted into the empty just created tables - here we can find inadmissibleNULLs in the column with constraint NOT NULL.

Some developers consider that such InterBase behavior to be incorrect, but otherone will be unable to add a field with NOT NULL restriction to the database table

A question about required value by default and filling with it at the moment ofcreation was widely discussed by Firebird architects, but wasnt accepted becauseof the fact that programmer is obviously going to fill it according to the algorithm,rather complicated and maybe iterative. But there is no guarantee, whether hell be

able to distinguish the records ignored by previous iteration from unfilled records ornot.

The similar problem can be caused by garbage collection fault because of settingnot a correct path to database (the cause of corruption 3) at the time of connection



http://delphi.weblogs.com/http://delphi.weblogs.com/


7/15

and file access to database files when server is working with it (the cause ofcorruption 4) and records whole filled with Null can appear in some tables. Its verydifficult to detect these records, because they dont correspond to integrity controlrestrictions, and operator Select just doesnt see them, although they get into backupcopy. If it is impossible to restore for this reason, one should run gfix program (lookbelow), find and delete these records using non-indexed fields as search conditions,after it retry to make a backup copy and restore database from it. In conclusion wecan say that there is great number of causes of database corruption and you shouldalways be ready for worst - that your database will be damaged for that or otherreason. You also must be ready to restore and save valuable information. And nowwell consider precautions that guarantee InterBase database security, as well asmethods of repairing damaged databases.

2.7. Precautions of InterBase database corruption

In order to prevent database corruption, one should always create backup copies (ifyou want to know more about backup then look the chapter Backup and restore).Its the most trusted way against database corruption. Only backup gives 100%guarantee of database security. As its described above, as the result of backup wecan get a useless copy (a copy that cant be restored), thats why restoring a basefrom the copy mustnt be performed by writing over the script and backup must bedone according to definite rules. Firstly, backup must be executed as more often aspossible, secondly it must be serial and thirdly, backup copies must be checked forrestoring capability.

Often backup means that its necessary to make a backup copy rather often, forexample once in twenty-four hours. The less data period is between the databasebackup, the fewer data will be lost as a result of fault. Sequence of backup meansthat the number of backups must increase and must be stored at least for a week. Ifthere is a possibility, its necessary to write backups to special devices like streamer,but if there is not just copy them to the other computer. The history of backupcopies will help to discover hidden corruptions and cope with the error that aroselong ago and showed up unexpectedly. One has to check, whether it is possible torestore the received backup without errors or not. It can be checked only in one way- through the test restore process. It should be said that restore process takes 3times more time than backup, and its difficult to execute restore validation every day

for large databases, because it may interrupt the users work for a few hours (nightbreak may not be enough).

It would be better if big organizations didnt save on matches and left one computer





8/15

for these aims.

In this case, if server must work with serious load 24 hours 7 days a week, we canuse SHADOW mechanism for taking snapshots from database and further backupoperations from the immediate copy. Backup process and database restoring isdescribed in detail in chapter Backup and restore. When creating a backup andthen restoring database from it, recreation of all data in database is happening. Thisprocess (backup/restore or b/r) contributes to the correction of most non-fatal errorsin database, connected with hard disk corruptions, detecting problems with integrityin database, cleaning database from garbage (old versions and fragments ofrecords, incomplete transactions), decreasing database size considerably.

Regular b/r is a guarantee of InterBase database security. If database is working,then it is recommended to execute b/r every week. To tell the truth, there are someillustrations about InterBase databases that are intensively used for same yearswithout backup/restore.

Nevertheless, to be on a safe side its desirable to perform this procedure, especiallyas it can be easily automated (look chapter Backup)

If its impossible to perform backup/restore often for some reasons, then one canuse the tool gfix for checking and restoring database. gfix allows to check andremove many errors without b/r.

2.8. Command line tool gfix

Command line tool gfix is used for checking and restoring database. Besides, gfixcan also execute various activities of database control: changing database dialect,setting and canceling the mode read-only, setting cache size for a concretedatabase and also some important functions (you can know about them in InterBase6 Operations Guide [4])

gfix is committed in a command line mode and has the following syntax:Gfix [ options] db name

Options is a set of options for executing gfix , db name is a name of database overwhich operations will be performed, defined by set of options. Table 3 representsoptions gfix related to database repairing:

Option Description

f[ull] This option is used in combination with v andmeans its time to check all fragments of records





9/15

i[gnore] Option makes gfix ignore checksums errors atthe time of validation or database cleaning

m[end] Marks damaged records as not available, as aresult of what they will be deleted during thefollowing backup/restore. Option is used at thetime of preparing corrupted database to b/r.

n[o_update] Option is used in combination with v forread-only database validation without correctingcorruptions

pas[sword] Option allows to set the password whenconnecting database. (Note that it is the error inInterBase documentation -pa[ssword], but theshortcut "-pa" will not work - use "-pas" )

user Option allows to set users name connectingdatabase

v[alidate] Option presetting database validation in the wayof which errors are discovered

-m[ode] Option setting the write mode for database forread only or read/write. This parameter canaccept 2 values read write or read only.

w[rite] {sync | async} Option that turns on and off the modesynchronous/asynchronous forced writes todatabase. sync to turn synchronous writes on

(FW ON); async to turn asynchronous writeson (FW OFF);

Table 1: gfix tool options for database restoring

There are some typical examples of using gfix:gfix w sync user SYSDBA pass masterkey firstbase.gdb

In this example we set for our test database firstbase.gdb synchronous writes mode(FW ON). (Of course, it is useful before the corruption occurs). And below is the firstcommand that you should use to check database after the corruption occurs:

gfix v full user SYSDBA pass masterkey firstbase.gdb

In this example we start checking our test database (option v) and indicate thatfragments of records must be checked as well (option -full). Of course, it is moreconvenient to set various options for checking and restoring process by any GUI, butwell consider the functions of database recovery using command line tools. These





10/15

tools are included to InterBase and you can be sure that their behavior will be thesame on all OS running InterBase. It is very important that they always be near.Besides, the existing tools, allowing to execute database administrating from aclients computer use Services API for it, that isnt supported by InterBase serverClassic architecture. That means you may use third party products with serversarchitecture SuperServer.

2.9. The repairing of corrupted database

Lets suppose there are some errors in our database. Firstly, we have to check theexistence of these errors; secondly, we have to try to correct these errors. Youshould abide the following instructions.

You should stop the InterBase server if its still working and make a copy of file ordatabase files. All the restore activities should be performed only with databasecopy, because the chosen way may lead to unfortunate result, and youll have torestart a restore procedure (from a starting point). After creating a copy well performthe whole database validation (checking fragments of records).

We should execute the following command for it:gfix v full corruptbase gdb user SYSDBA - password

In this case corruptbase.gdb is a copy of damaged database. A command willcheck database for any structure corruption and give the list of unsolved problems. Ifsuch errors are detected, well have to delete the damaged data and get ready for

backup/restore using the following command:gfix mend user SYSDBA password your_masterkey corruptbasegdb

After committing a command you should check if there are some errors in databaseleft. You must run gfix with options v full for it, and when the process is over,perform database backup:gbak b v -ig user SYSDBA password corruptbase.gdb

corruptbase.gbk

This command will perform database backup (option - b says about it) and well getdetailed information about backup process executing (option v). Error regarding tochecksums will be ignored (option - ig)

If you want to know more information about the options of command line tool gbak,you can find it in the chapter Backup and restore





11/15

If there are some errors with backup, you should start it in another configuration:

gbak b v ig -g user SYSDBA password corruptbase.gdbcorruptbase.gbk

Where option g will switch off garbage collection during backup. If often helps tosolve a problem with backup.

Also it may be possible to make a backup of database, if before it we set database inread-only mode. This mode prevents from writing any modifications to database andsometimes helps to perform backup of damaged database. For setting database toread-only mode, you should use the following command: gfix m read _onlyuser SYSDBA password masterkey Disk:\Path\file.gdb

After it you should try again to perform database backup using the parameters given

above.

If backup was done successfully, you should restore the database from backupcopy. You should use the following command:gbak c user SYSDBA password masterkey Disk:\Path\backup.gbk

Disk:\Path\newbase,gdb

When you are restoring the database, you may have some problems, especiallywhen creating the indexes.

In this case options inactive and -one_at_a_time should be added to restorecommand. These options deactivate indexes in creating from database backup and

commit data confirmation for every table.

2.10. How you can try to extract the data from a corrupted database

It is possible that the operations given above will not lead to database recovery.

It means that database is seriously damaged or it cannot be restored as a singlewhole, or a great number of efforts must be made for it is recovery. For example,one can execute a modification of system metadata, use non-documented functionsand so on. It is a very hard, long-lasting and ungrateful work with doubtful chancesfor success. And if it is possible, try to evade it and use other methods. If a damageddatabase opens and allows to perform reading and modification operations with

some data, you should use this possibility and save the data by copying them to anew base, and say god-bye to the old one for good.

So, before transferring the data from the old database, its necessary to create a





12/15

abase destination. If database hasnt been changed for a long time, then you canuse the old backup, from which metadata can be extracted for creating a databasedestination. On a basis of these metadata one has to create a data destination andstart copying the data. The main task is to extract the data from a damageddatabase. Then well have to allocate the data in a new base, but its not verydifficult, even if well have to restore database structure from memory.

When extracting data from tables, you should use the following algorithm ofoperations:

1. At first you should try to execute SELECT* from table N. If it went normally youcould save the data youve got in the external source. Its better to store data inscript (almost all GUI give this function), if only the table doesnt containBLOB-fields. If there are BLOB-fields in the table, then data from them should be

saved to another database by client program that will play a role of a mediator.Maybe youll have to write this trivial program especially for data recovery aims.

2. If you failed to retrieve all data, you should delete all the indexes and try again.Virtually, indexes can be deleted from all the tables from the beginning ofrestoring, because they wont be needed any more. Of course, if you dont havea structure of metadata, same to the corrupted, its necessary to input a protocolof all operations that you are doing with a damaged database-source.

3. If you dont manage to read all the data from the table after deleting the indexes,one can try to do range query by primary key. It means to choose definite rangeof data. For example:

SELECT * FROM table N WHERE field_PK >=0 and field_PK


13/15

a great number of damaged pages). Thus, data from a damaged database must bemoved to a new database or into external sources like SQL-scripts. When you copythe data, pay attention to generators values in damaged database (they must besaved for restarting a proper work in new database. If you dont have a completecopy of metadata, you should extract the texts of stored procedures, triggers,constraints and definition of indexes.

2.11. Restoring of hopeless database

In general, restoring of database can be very troublesome and difficult and thatswhy its better to make a backup copy of database than restore the damaged dataand whatever happened, you shouldnt get despaired because a solution can befound in the most difficult situations. And now well consider 2 cases.

The first case (a classic problem). A backup that cant be restored because of havingNULL values in the column with constraints NOT NULL (restore process was runover the working file). The working file was erased and restore process wasinterrupted because of error. And as a result of thoughtless actions we got a greatnumber of useless data (that cant be restored) instead of backup copy. But thesolution was found. The programmer managed to recollect what table and whatcolumn had constraints NOT NULL. Backup file was loaded to hexadecimal editor.And a combination of bytes, corresponded to definition of this column, was foundthere by searching. After innumerous experiments it turned out that constraint NOTNULL adds 1 somewhere near the column name. In HEX-editor this 1 wascorrected to 0 and backup copy was restored. After that case programmermemorized once and for all how to execute backup process and restore.

The second case.

The situation was catastrophic. Database corrupted on the phase of extensionbecause of lack of disk space.

When increasing the database size, server creates series of critically importantpages (for example, transaction inventory page and page inventory page, additionalpages for RDB$Pages relation) and writes them down to the end of database

As a result, database didnt open either by administration facilities or by utility GBAK.And when we tried to connect database, error message (Unexpected end of file)appeared. When we run utility gfix strange things were happening: The program wasworking in an endless cycle. When gfix was working, server was writing errors to log(file InterBase log) with high speed (around 100 Kb per second). As a result, log filefilled all the free disk space very quickly. We even had to write a program that





14/15

erased this log by timer. This process lasted for a long time gfix was working formore than 16 hours without any results.

Log was filled up with errors of the following view: Page XXX doubly allocated. Instarting InterBase sourses (in file val.#) there is a short description of this error. Itsays that this error appears when the same data page is used twice.

Its obviously that this error is a result of corruption of critically important pages.

As a result, after several days of unfortunate experiments attempts to restore thedata in standard ways were left. And thats why we had to use low-level analysis ofdata stored in damaged database.

Alexander Kozelskiy, a chief of Information technologies department East ViewPublications Inc, is the author of the idea how to extract information from similarunrecoverable databases.

The method of restoring that we got as a result of researches was based on the factthat database has page organization and data from every table are collected by datapages. Every data page contains identifier of the table for which it stores data. It wasespecially important to restore data from several critical tables. There were datafrom the similar tables, received from an old backup copy that worked perfectly andcould be a pattern. Database-pattern was loaded to editor of hexadecimal sourcesand then we searched for the patterns of those data that interested us. These datawere copied to buffer in hexadecimal format and then remains of damaged databasewere loaded to the editor. A sequence of bytes corresponded to the pattern was

found in damaged database, and page was analyzed (on which this sequence wasfound).

At first we defined the beginning page, but it wasnt difficult because the size ofdatabase file is divisible by data page size. A number of current byte divided by pagesize 8192 bytes, approximates the result to integer (and got the number of currentpage). Then multiplied the number of current page-by-page size and got the numberof byte corresponded to the beginning of current page. Having analyzed the header,we defined the type of page (for pages with data the type is 5 look file ods.h fromset of starting InterBase sources and also the chapter The structure of InterBaseDatabase) as well as identifier of necessary table.

Then a program was written, that analyzed the whole database, collected all thepages for necessary table into one single piece and move it to file.

Thus, when we got the data we needed in first term, we started analyzing contentsof selected pages. InterBase is widely using data compression for saving place. For





15/15

example, a string like VARCHAR containing ABC string, it stores sequence offollowing values: string length (2 bytes), in our case it is 0003, and then symbolsthemselves and then checksum. We had to write analyzer of string as well as otherdatabase types that converted data from hexadecimal format into ordinary view. Wemanaged to extract up to 80% of information from several critical tables using amanual method of analyzing database contents. Later on the basis of experienceOleg Kulkov and Alexey Kovyazin, one of the authors of this book, developed theutility InterBase Surgeon that performs direct access to database, bypassing theInterBase engine and allows to read directly and interpret the data within InterBasedatabase in a proper way.

Using InterBase Surgeon, we manage to detect causes of corruption and restore upto 90% of absolutely unrecoverable databases that cant be open by InterBase and

restored by standard methods.You can download this program from official program site www.ib-aid.com.

3. Thanks

I'd like to tender thanks to all who help me to create this guide:Craig Stuntz, Alexander Nevsky, Konstantin Sipachev, Tatjana Sipacheva and allother kind and knowledgeable people of InterBase and Firebird comminity.

If you have any suggestions or questions about this chapter, please feel freeto email.




mailto:[email protected]://delphi.weblogs.com/http://delphi.weblogs.com/

Documents

IBSurgeon Recovery Guide: chapter from "The InterBase World"