Mocca International GmbH _Q500 analysis and Recommendations_Final

Q500 Database Analysis and Recommendations

1 Executive SummaryThe Q500 database server is working too hard and inefficiently, which leads to unsatisfactory performance of the application and web interface. This is due partly to file fragmentation on disk and internal fragmentation of data within the database files and also to an unsuitable database layout. Fragmentation means that the hardware must do a huge amount of work to access a small amount of data and this takes an excessive amount of time. The database layout forces the load onto one or 2 disks, which can’t handle the demand, so there is a large queue of requests, which causes delay in execution. Furthermore, the active database files are almost full, which means that it is a challenge for Sqlserver to find space to store new data. The files do extend automatically, which imposes an extra load on the system at peak times.

Mocca International needs to decide how long the Q500 application can be unavailable for user access, before it poses a risk to the business. This relates to unscheduled downtime. If unscheduled downtime must be minimal, a disaster recovery system must be continuously available, preferably in another location, so that in the event of failure of the main system, service continuity can be maintained with minimal outage. A DR System could provide read access to application users. This could be useful for intense operations like searches.

There are significant concerns about the current hosting arrangements with Basefarm, particularly how long it would take to get back online in the event of a failure requiring the database to be recovered from backups, which are on tape. Customer service responses to questions/requests are slow and inadequate.

1.1 The role of Robert

Robert Golias, the principal application architect and developer, has been the main contact for gathering information for this analysis. He is very familiar with the application and database architecture history. It is apparent, that he is attempting to develop and improve the application, whilst, at the same time managing production. This arrangement may well have worked to the satisfaction of Mocca International management to date but in the long term, it is unsustainable. Robert will need to choose between continuing to develop the application or devoting himself to production server admin and DBA. He cannot do both successfully and it is unfair of the business and risky to expect this of him. His creativity would be best employed in development, whilst not having the distraction of managing the production environment, which is expected to explode in data terms for new markets.

1.2 Cost effectiveness

Considerable emphasis has been made by the business on keeping the cost of fixing problems and weaknesses in the system as low as possible. Cleaning up the system, reorganising the database and then improving the application is the most cost effective option. However, it will take time to set up, test and implement, as implementation should be as fast as possible to keep downtime to a minimum.

Changing hosting and moving the current configuration to, for example Amazon, may be the cheapest option in money terms but the Amazon environment still requires a learning curve and any gain from environment change will be one off. Performance will start to degrade again almost immediately. The current proposed database reorganisation will only be postponed and when it is undertaken, the amount of data involved will be that much greater and potentially require more downtime.

Howard Perry Page 1 of 16 02.12.2010


Besides, the current contract with Basefarm runs into 2011 and presumably it is not possible to exit early.

Costs of a DR system need to be balanced against risk of system outage due to high load or hardware/software/database failure. As the Q500 system is the core and income generator for the entire business, the risk of any kind of failure poses a major threat to the business. As the Q500 database grows, the amount of data and numbers of users will also grow, plus the risk of failure will also increase. This needs to be counterbalanced by an expected increase in revenue.

The database will also need ongoing tuning, capacity expansion and load balancing. This will most likely require future database reorganisations.

The top 3 tables make up 80% of database..

1.3 Principal Recommendations in priority/cost order

Tune current system, database, application – top priority Improve database recovery time Implement Disaster Recovery system Fix current problems, before formal change of hosting partner Amazon EC preferred hosting partner but need thorough testing to confirm. Review IT staff to spread responsibilities and workload

Only consider conversion to another platform e.g. Linux/Mysql, when tuning options on Windows and Sqlserver exhausted (unlikely).

Conversion to another database platform now is bad idea because:a) Expensive (estimate 6 man years+ over 2 years)b) High risk - no guarantee of better performance c) Delay expansion into bigger markets.d) Requires staff training to high level on new platform

Possible long term option to redesign entire system i.e. application and

database for different platform but only after comprehensive cost/benefit analysis.



2 Technical Detail

2.1 Introduction

Mocca International GmbH requested Howard Perry to conduct a database analysis of their web based application, Q500, in order to identify reasons for performance weakness and make recommendations for action in respect of:

2.1.1 Continue with current database platform or switch to an alternative in order to cope with future capacity, particularly introduction to larger markets (Germany, UK, France) than those currently served (Norway, Denmark, Sweden)

2.1.2 Continue current hosting arrangements with Basefarm, Sweden or switch to an alternative

2.1.3 Recommendations to be geared to cost effectiveness

2.2 Q500 Architecture.

The Q500 system consists of the following major components

a) Web server – the interface, to which any web user can gain access using a browser such as Internet Explorer, Firefox etc.

b) Application Server, which processes user requests entered on the Web serverc) Database Server, which returns data requested by the Applicationd) Mail server - which processes emails sent by Q500 system administrators and

individual users.

Q500 operates under the Windows Operating system, currently Windows Server 2003 with the database running under Microsoft Sqlserver 2005. The application software is written in Microsoft C# .NET and calls stored procedures embedded within the database.

2.3 Scope

As a general rule, targets for tuning in descending order of greatest potential impact are:

1. Hardware and Operating System2. Database3. Application

Hardware and operating environment changes have the greatest impact, both positive and negative. Operating system tuning, however, is an area to be approached with extreme caution, as changing system parameters can drastically degrade as well as improve overall performance.

Hardware changes will also have an impact, although for a high growth application the benefit is usually only short term, so over time it’s not the most cost effective option. However, combined with database changes, benefits of hardware changes last longer. It is important to note, that high growth applications need constant performance monitoring to identify areas for tuning. This includes increasing capacity. The growth potential for



Q500 in terms of database size and numbers of users is enormous, given the plans to launch it in markets with populations orders of magnitude larger than the current market served (Norway, Denmark, and Sweden). Assuming the same level of market penetration as current countries served (2.5%); introduction to Germany, UK, and France could increase the database from the current 600 Gb to 10 Terabytes within 2 years.

Investigations have been largely confined to the database server and the database itself, as hardware, operating system, database management system and the database physical design are generally areas where the greatest amount of impact can be achieved for the least amount of change.

Application design does offer scope for tuning, but it will be localized and requires more time for investigation than was available. Consequently there has been no major review of the application at this stage. It has, however, been observed that

1. procedures within the database make extensive use of temporary tables

2. the email tables occupy a disproportionate amount of space compared to the rest of the data. It has been established that there is significant scope for purging certain types of obsolete emails and setting up a purge mechanism for the future to do this regularly.

3. Investigation into performance of queries and stored procedures running inside the database has revealed potential tuning targets. However, this area needs to be more thoroughly investigated after changes to the server environment and the physical database design, as it is possible that the tuning targets will then be different. Indeed, changing too much at once can have adverse effects.

2.4 Database Server Problems and Recommendations

This section is mainly concerned with the database itself, but some references are made to operating environment problems, which impact the database.The database is divided into 2 Filegroups, Primary and History, and occupies about 600Gb of disk space spread over 10 files including 3 transaction log files totaling 60Gb. The History Filegroup consists of 3 files of varying sizes totaling 158 Gb and contains tables which are largely empty and unused. This Filegroup could be reduced to a single file of a much smaller size e.g. 5Gb and other unused tables from the Primary Filegroup moved to it.

2.4.1 In addition, it has been estimated that a further 150Gb data could be removed by removing inactive profiles and email messages.

2.4.2 Complete removal of the unused tables is not advisable, as it would involve changing the application to ensure that there was no possibility that these tables could be referenced. Such application changes would require a complete system test, which would be time consuming and therefore costly.

2.4.3 The Primary Filegroup consists of 4 files of sizes varying from 1 Gb to 257Gb. The 257Gb file is 72% of the entire Primary group and resides on a single disk. Where a Filegroup consists of multiple files, it is not transparent, in which file a specific table is held. Indeed the contents of a table may be spread across several files.

The following table shows the database file components.



DB File LocationSize (Gb) Free(Gb) %Free %PrimaryFG

Q500_DataE:\Microsoft SQL Server\Data\Q500_Data.mdf 53 1 2.25% 14.86%

Q500_Data2E:\Microsoft SQL Server\Data\Q500_Data2.ndf 44 3 6.43% 12.38%

Q500_Data3 F:\db\data\Q500_Data3.ndf 1 0 2.34% 0.28%

Q500_Data4H:\Microsoft SQL Server\Data\Q500_Data4.ndf 257 7 2.64% 72.48%Total Primary Data 355 11 3.05%

Q500_Hist3H:\Microsoft SQL Server\Data\Q500_Hist3.ndf 98 98 100.00%

Q500_HistF:\Microsoft SQL Server\Data\Q500_Hist.mdf 30 30 100.00%

Q500_Hist2F:\Microsoft SQL Server\Data\Q500_Hist2.ndf 30 30 100.00%

Q500_log3H:\Microsoft SQL Server\Log\Q500_log3.ldf 20 19 96.28%

Q500_log2G:\Microsoft SQL Server\LOG\Q500_log2.ldf 20 20 96.28%

Q500_logG:\Microsoft SQL Server\LOG\Q500_log.ldf 20 20 96.28%Total 573 238 41.46%



2.5 Fragmentation

2.5.1 Disk Fragmentation

The table below shows that the disks are heavily fragmented. Fragmented disks require more work to be done in order to read and write data to them. This negatively affects performance. Database system and data files in particular should not be fragmented, nor should they be allowed to become so. Similarly Drive C, where the operating system files reside, is also fragmented. Discussions with Basefarm have indicated that they consider it to be the customer’s responsibility to monitor disks for fragmentation and, presumably, to do any work required to eliminate it

Volume VolSizeGb UsedGb FreeGb Used% Free% VolFrag% Filefrag%OS (C:) 137 26.21 111 19% 81% 17 34DATA (E:) 271 210 61 77% 23% 49 99TEMPDB (F:) 136 72.97 63 54% 46% 27 54 LOG (G:) 68 40.89 27 60% 40% 49 99 DB Disk (H:) 547 429 118 78% 22% 38 77Total 1159 779 380 67% 33%Total(excl OS) 1022 753 269 74% 26%

2.5.2 Database File Fragmentation on each disk

Volume Fragments File Size filenameDATA (E:) 11 113 GB \Log\q500_log.trnDATA (E:) 9 52.75 GB \Microsoft SQL Server\Data\Q500_Data.mdfDATA (E:) 9 43.96 GB \Microsoft SQL Server\Data\Q500_Data2.ndf

TEMPDB (F:) 18 1.95 GB \Microsoft SQL Server\Data\tempdb3.ndfTEMPDB (F:) 18 1.95 GB \Microsoft SQL Server\Data\tempdb.mdfTEMPDB (F:) 18 1.95 GB \Microsoft SQL Server\Data\tempdb2.ndfTEMPDB (F:) 16 1.87 GB \Microsoft SQL Server\Data\tempdev5.ndfTEMPDB (F:) 3 1.95 GB \Microsoft SQL Server\Data\tempdb4.ndfTEMPDB (F:) 2 30.00 GB \Microsoft SQL Server\Data\Q500_Hist2.ndf

LOG (G:) 5 20.31 GB \Microsoft SQL Server\LOG\Q500_log.ldfLOG (G:) 4 20.31 GB \Microsoft SQL Server\LOG\Q500_log2.ldfLOG (G:) 2 200 MB \Microsoft SQL Server\LOG\templog.ldf New DB Disk (H:) 8 257 GB \Microsoft SQL Server\Data\Q500_Data4.ndfNew DB Disk (H:) 4 53.73 GB \q500_trn.bakNew DB Disk (H:) 2 20.02 GB \Microsoft SQL Server\Log\Q500_log3.ldf



2.5.3 Internal Database Fragmentation

Analysis of the Q500 database has confirmed that data within the database is heavily fragmented.

Below is a query extracted from monitoring of the database:

SQL> delete from tblMessageBrowse where ProfileID=@profileid

This is a frequently executed query and was executed 2424 times during the monitoring period of 20 minutes

Average reads were 14,347 with min of 3 and maximum of 612,587

Average writes were 2 with minimum of 0 and max of 60.Average duration was 19,684 ms with min of 117 ms and max 503,904ms

tblMessageBrowse contains 3.75m records occupying total space in the database of ca 500 Mb. Analysis of this table shows fragmentation of 96%, there being 27,007 fragments out of a total of 27,765 pages. The query above will require Sqlserver to do a huge amount of work to find the limited number of records required and then much more work in addition to remove them from the table.

When the recommendations have been implemented, this query should execute with a substantial reduction in reads and duration.

2.6 Disk Cleanup

Based on the existence of certain large files of uncertain origin, it is possible that a cleanup of the disks would recover an additional considerable amount of disk space. For example: E:\Log\q500_log.trn 113 Gb. This could be either an obsolete transaction log or transaction log backup. In either case, it could potentially be removed. 113 Gb is 10% of entire current disk capacity of the database server.

2.7 New database sizing

Plenty of space for expansion needs to be allocated to the new Filegroups. Filegroup files should allocate at least 50% more space than is required for data. When space free within the file drops below 30%, the next reorganisation for expansion should be planned.

2.8 Hosting

Basefarm hosting does not appear to be entirely satisfactory. Basefarm provide basic management and some basic monitoring of system performance. It also takes care of database and transaction log backups. However, recoverability of these backups is not tested, which needs to be done on a regular basis, in case it needs to be done live, due to hardware, software failure or database corruption. Database backup is to tape every 2 weeks and takes 6 hours. Transaction log backup is to disk and then the log backup is presumably archived to tape. As things stand, database restore and recovery to point of failure, would take a long time. How long can Mocca International afford to be without the Q500 system?



Basefarm was asked about the backups and other matters and they took 2 weeks to reply with limited information. Apart from the hosting costs, ca USD 5.000 per month for all systems including database, web, application and mail servers, Mocca International owns the hardware.

Based on document entitled Amazon AWS, the Amazon EC and other offerings were investigated. The Amazon service is what is known as Cloud Computing. This is a new concept in hosting and, as, is common in IT, the definition varies between providers. Some providers offer web hosting, online backups and FTP services only and call this “Cloud Computing”. However Amazon appears to be the only provider, which can offer comprehensive hosting on a suitable scale for Q500’s expected growth, which can be compared with the Basefarm service. The monthly cost of using Amazon appears to be significantly less than Basefarm, whilst at the same time having more resources available in form of CPU power, memory, and disk space.

However, the main difference to Basefarm is that the Amazon systems are rented by the hour and it is necessary to specify the period, for which hosting is required. When the rental period expires, the data is lost. Therefore separate permanent storage is required, which can be rented. The entire concept is far too complex to explain here – see separate document Amazon EC2 FAQ. It should be noted that the hourly charge for Sqlserver instances is 60c higher than the standard windows only instance. Charge for instances shown in table below:

Standard Instances Windows Usage Windows and SQL Server UsageSmall (Default) $0.12 per hour ---Large $0.48 per hour $1.08 per hourExtra Large $0.96 per hour $1.56 per hourHigh-Memory Instances Windows Usage Windows and SQL Server UsageExtra Large $0.62 per hour $1.22 per hourDouble Extra Large $1.24 per hour $1.84 per hour

The recommended configuration is:

a) Small standard instance for web serverb) Extra Large High memory instance with Sqlserver for application, mail server and DR

for database serverc) Extra Large High Memory Instance with Sqlserver for main database serverd) Separate permanent storage space

This is a more powerful configuration than currently in use at Basefarm but it is also more secure, as it also allows for quick recoverability.

The database servers and permanent storage spaces could potentially be in separate locations for extra security. The monthly charge for items (a-c) would be USD 1843.20.

Optionally, there could be separate servers for application and mail server as at present, but this would cost extra and possibility of underutilising the DR database server. If the load justified it, these servers could be added later

The reason for a High memory instance is that database servers, especially sqlserver, perform better when they have large amounts of memory available to hold or cache, as it is known, data. This produces better performance.



Based on AMAZON AWS document, storage requirements for current system would be as shown below

Storage, backup and bandwidth equivalents:

Hardware / Backup S3 GB per month Price per monthMSA 30SB 73GB*14 1022GB $153.3Backup 1600GB $240.0Transferred data total for a month

1600GB $240.0

Storage and bandwidth charges would be USD 633.30 making total for items (a-d) of USD 2476.50 which represents less than 50% of current Basefarm monthly fee. This will probably vary with growth in database and numbers of users, but is a useful starting point. It is assumed that Amazon customers will perform their own system management. Amazon does provide instance management for an extra fee, but is not clear, what this includes.

It is recommended that the Amazon service be tested with view to eventual Q500 hosting if testing is satisfactory. This requires setting up of an account and going through the learning curve of setting up the application and database, plus managing the environment etc. It is possible that Amazon could be used to develop and test the database reorganisation procedures as well as functional and load tests for Q500. This assumes that it would be easy to transfer large volumes of data from Basefarm to Amazon and possibly vice versa.

2.9 Database Platform

The Amazon AWS document proposed a switch of the database platform from Microsoft Windows SQL Server to Linux with MySql. A conversion now would be quite costly, and delay any expansion to bigger markets. Robert is not familiar with Linux/MySql, so he would have a significant learning curve in order to convert the application interfaces and the database stored procedures, together with comprehensive testing of the new system. The database would still require a restructure and tuning.

This option may be worth considering for the long term, although, it would also be better to redesign the application to run on Linux as well as the database, as mixed platforms rarely work as effectively as expected.

Migrating all or part of Q500 to a different platform needs to be approached with extreme caution. Not all migrations are successful. The keys to success are:

a) thorough knowledge of the application and its designb) how it works on the source platform and will work on the targetc) in-depth knowledge of the target platform

It is definitely much more cost effective to fix performance problems on the present platform.



2.10 The next steps

2.10.1 Project Plan to implement recommendations

2.10.2 Develop new database layout and sizing

2.10.3 Develop and test database reorganization

2.10.4 Schedule and Implement Project

2.11 Future Actions

2.11.1 After reorganization of the database and implementation of comprehensive server and database monitoring, data should be collected and regularly reviewed to identify targets for query and stored procedure tuning. This kind of tuning involves reviewing execution plans and determining whether additional indices or changing existing ones will make query faster.

2.11.2 Query and Stored Procedure code may need to be reviewed.

2.11.3 Data Growth and fragmentation within the database must be monitored and the database regularly reorganized to maintain and enhance performance. Expansion into new markets should cause explosive database growth.

2.11.4 The application should be reviewed to look for efficiency improvements.


Appendix A

Most fragmented files on each disk

Volume Fragments File Size Fragmented fileOS (C:) 5,249 183 MB \var\log\tsm\cmdsqlsched.logOS (C:) 1,346 125 MB \Program Files\Microsoft SQL Server 2005\MSSQL.1\MSSQL\LOG\ERRORLOG.5OS (C:) 1,153 24 MB \Program Files\Microsoft SQL Server\90\Setup Bootstrap\LOG\Files\SQLSetup0004_Q5-DB01_Tools.logOS (C:) 1,147 22 MB \WINDOWS\Hotfix\SQLTools9\Logs\SQLTools9_Hotfix_KB913090_sqlrun_tools.msp_1.logOS (C:) 933 21 MB \Program Files\Microsoft SQL Server\90\Setup Bootstrap\LOG\Hotfix\SQLTools9_Hotfix_KB921896_sqlrun_tools.msp.logOS (C:) 900 10 MB \Program Files\Microsoft SQL Server\90\Setup Bootstrap\LOG\Hotfix\SQL9_Hotfix_KB921896_sqlrun_sql.msp.logOS (C:) 779 75 MB \Documents and Settings\Administrator\Local Settings\Temporary Internet Files\Content.IE5\G4RFS1I4\smartstart-7.60-0[1].zipOS (C:) 762 10 MB \Program Files\Microsoft SQL Server\90\Setup Bootstrap\LOG\Hotfix\RS9_Hotfix_KB921896_sqlrun_rs.msp.logOS (C:) 754 8 MB \Program Files\Microsoft SQL Server\90\Setup Bootstrap\LOG\Files\sqlsetup0003_q5-db01_.net framework 2.0.logOS (C:) 696 9 MB \Program Files\Microsoft SQL Server\90\Setup Bootstrap\LOG\Files\SQLSetup0007_Q5-DB01_RS.logOS (C:) 592 9 MB \Program Files\Microsoft SQL Server\90\Setup Bootstrap\LOG\Files\SQLSetup0004_Q5-DB01_PPESku_1.logOS (C:) 538 8 MB \Program Files\Microsoft SQL Server\90\Setup Bootstrap\LOG\Files\SQLSetup0003_Q5-DB01_SQL.logOS (C:) 475 10 MB \Program Files\Microsoft SQL Server\90\Setup Bootstrap\LOG\Hotfix\DTS9_Hotfix_KB921896_sqlrun_dts.msp.logOS (C:) 434 4 MB \WINDOWS\Microsoft.NET\Framework64\v2.0.50727\ngen_service.old.logOS (C:) 401 20 MB \WINDOWS\Hotfix\SQLTools9\Logs\SQLTools9_Hotfix_KB913090_sqlrun_tools.msp.logOS (C:) 390 9 MB \Program Files\Microsoft SQL Server\90\Setup Bootstrap\LOG\Files\SQLSetup0003_Q5-DB01_DTS.logOS (C:) 379 8 MB \WINDOWS\Hotfix\SQL9\Logs\SQL9_Hotfix_KB913090_sqlrun_sql.msp.logOS (C:) 364 9 MB \WINDOWS\Hotfix\DTS9\Logs\DTS9_Hotfix_KB913090_sqlrun_dts.msp.logOS (C:) 363 160 MB \Program Files\Tivoli\TSM\baclient\dsmcrash.dmpOS (C:) 354 4 MB \WINDOWS\Microsoft.NET\Framework\v2.0.50727\ngen_service.old.logOS (C:) 306 10 MB \WINDOWS\Hotfix\DTS9\Logs\DTS9_Hotfix_KB913090_sqlrun_dts.msp_1.logOS (C:) 297 75 MB \Documents and Settings\Administrator\Local Settings\Temporary Internet Files\Content.IE5\LXQXHDV8\5.5.0.4-TIV-TSMBAC-WinX64[1].exeOS (C:) 292 2 MB \WINDOWS\iis6.logOS (C:) 287 1 MB \WINDOWS\FaxSetup.logOS (C:) 276 1 MB \Program Files\Microsoft SQL Server\90\Setup Bootstrap\LOG\Hotfix\Redist9_Hotfix_KB921896_SqlSupport.msi.logOS (C:) 266 20 MB \Program Files\Microsoft SQL Server 2005\MSSQL.1\MSSQL\LOG\log_7147.trc

Volume Fragments File Size Fragmented fileDATA (E:) 11 113 GB \Log\q500_log.trn - Not clear what this file is – could be log backup. If so, 113 Gb is recoverableDATA (E:) 9 52.75 GB \Microsoft SQL Server\Data\Q500_Data.mdfDATA (E:) 9 43.96 GB \Microsoft SQL Server\Data\Q500_Data2.ndfDATA (E:) 4 38 MB \db\disk1\MSDBData001.ndf

TEMPDB (F:) 18 1.95 GB \Microsoft SQL Server\Data\tempdb3.ndfTEMPDB (F:) 18 1.95 GB \Microsoft SQL Server\Data\tempdb.mdfTEMPDB (F:) 18 1.95 GB \Microsoft SQL Server\Data\tempdb2.ndfTEMPDB (F:) 16 1.87 GB \Microsoft SQL Server\Data\tempdev5.ndfTEMPDB (F:) 3 34 KB \RECYCLER\S-1-5-21-3206593322-3623838108-3005304415-1004\INFO2TEMPDB (F:) 3 1.95 GB \Microsoft SQL Server\Data\tempdb4.ndfTEMPDB (F:) 2 30.00 GB \Microsoft SQL Server\Data\Q500_Hist2.ndfTEMPDB (F:) 2 8 KB \60a822b11cefec6fceTEMPDB (F:) 2 1,006 KB \Program Files\Common Files\Microsoft Shared\DW\DW20.EXETEMPDB (F:) 2 485 KB \Program Files\Common Files\Microsoft Shared\DW\DWDCW20.DLL

LOG (G:) 5 20.31 GB \Microsoft SQL Server\LOG\Q500_log.ldfLOG (G:) 4 20.31 GB \Microsoft SQL Server\LOG\Q500_log2.ldfLOG (G:) 2 200 MB \Microsoft SQL Server\LOG\templog.ldf

New DB Disk (H:) 8 257 GB \Microsoft SQL Server\Data\Q500_Data4.ndfNew DB Disk (H:) 4 53.73 GB \q500_trn.bak – could be log backup. If so, what is file on Drive E:? New DB Disk (H:) 2 20.02 GB \Microsoft SQL Server\Log\Q500_log3.ldf

Appendix B

1. Preparation for upgrade to Windows Sever 2008 and Sqlserver 2008

1. Windows Server 2008• Check no incompatibilities in software between Windows 2003 and

Windows 2008• Make any changes required

2. Sqlserver 2008 R2• Download and run Upgrade advisor for SS2008 R2 on database server• Check report and resolve any issues highlighted

2. MAXDOP explanation

MAXDOP is abbreviation for Maximum Degree of ParallelismBy default Sqlserver breaks down queries into components which can be executed in parallel, where multiple CPUs are available. When all subqueries have completed the results have to be reconnected, which often results in waiting time, as shown by the value of CXPacket waits. On Q500 this is the top type of wait time. OLTP systems where response time is important, MAXDOP is usually set to 1, which makes all parts of a query execute serially ie one after another. This means no CXPacket time and thus reduces execution time.

Appendix C

Method for Disk Cleanup

1. Add disks – may need temporary boot disk – refer basefarm.2. Do Image Backup of Drive C3. Backup Q500 database and system databases (master, msdb, model) and Q500 log4. Backup any other files to be kept on drives E,F,G and H5. Change configuration of Tempdb6. Delete any unwanted files from drive C 7. Shutdown Sqlserver8. Restart Sqlserver and check New Tempdb Configuration has been completed

successfully9. Restore Q500 backup to new disks - change database and filenames – this could also

be separate exercise to test restore and roll forward. If this step has already been completed, old database could be dropped and database restored to original disks

10. Drop old Q500 database11. Shutdown Sqlserver12. Reformat original disks - could be useful to use maximum cluster size for database

disks including LOG and Tempdb13. Restore Drive C14. create folders required on Non-OS reformatted disks15. Restore files from other disks backed up at step 416. Reboot server (Basefarm)17. Start Sqlserver

Appendix D

Database reorganization

Method 1

1. Create new Filegroup for unused tables2. Move unused tables to new Filegroup3. Create rest of new Filegroups4. Remove unwanted data

• If entire table contents to be removed, drop table and recreate it in new Filegroup

• If part of table contents to be removed,a) Drop non-clustered indexes not required for selecting deletionsb) Delete unwanted rows (might be slow if many rows to be deleted)

Or

a) Create Temporary table with same columnsb) Drop non-clustered indexes not required for selecting rows c) Insert data to be kept into temporary tabled) Drop and recreate table in new Filegroup without clustered indicese) Insert data into table from temporary tablef) Create non-clustered indices

5. Move tables to new Filegroups6. Backup database7. Remove or shrink any old Filegroups and files8. Backup database9. If any last disk changes required either move individual files or drop database and

restore to final layout.

Method 2

1. Restore database to temporary disks2. Create Q500 empty database with new layout on permanent disks3. Drop Non-Clustered indices4. Set recovery to bulk load5. Copy tables/data to be retained from Temporary database to new Q500 db6. Recreate non-clustered indices7. Set recovery to Full8. Backup new Q500 database and transaction log9. Drop temporary database

Appendix E

1. Current Database Problems

1. Disk fragmentation on database server disks2. High table and index fragmentation within the database 3. Tempdb – too many, small and fragmented files4. 72% of database in use resides on a single disk.5. 27% of entire allocated space is 99% empty, whereas the rest is 97% full6. Database and transaction log backups too infrequent7. Database restore and roll forward never tested8. Concern about current hosting arrangement, particularly in event of a major failure.9. No Disaster Recovery Plan in place10. Server and Database Management Software not latest versions11. Inadequate monitoring of server and database environment12. Database operations make extensive use of local temporary tables13. The top 3 tables, which relate to emails, make up 80% of database.14. Top waits on database are CXPacket indicating Max degree of Parallelism set to 0

2. Recommendations

1. Investigate and Upgrade to Sqlserver 2008 R2 2. Investigate and Upgrade to Windows Server 20083. Add more disks to server. Disks should be as small as possible.4. Defragment all disks5. Increase size of Tempdb to 32 Gb6. Reduce number of Tempdb files to 4 @8Gb, spread over multiple disks7. Restructure database into larger number of Filegroups8. Create separate Filegroups for non-clustered indices 9. Spread Database Filegroups over more disks (not OS, Log nor Tempdb)10. Increase size of transaction logs and put on separate disks from Database11. Set MAXDOP to 1 to eliminate CXpacket waits12. Retain database and log backups on disk between database backups13. Decide on and implement DR Plan e.g. database mirror14. Setup comprehensive server and database monitoring facilities15. Review use of temporary tables – consider multi-user permanent work tables16. After restructure tune queries and stored procedures17. Review email component of database with view to reduction of proportion of database

taken up by this component.18. Investigate and test Amazon Elastic Cloud Computing with view to making it new

hosting platform.