What does it take to get ready for the launch of your new website?

55 killer questions to ask before you launch your new website

Website Performance & Software

Testing

Contents1. Introduction - Failure to Launch

2. Does it work - Functional Testing

3. Will the customers like it? - Usability and User Experience

4. Can it handle the load? - Capacity and Scalability

5. Is it fast enough? - Web Performance Optimisation

6. How do I measure online activity? - Web Analytics

7. How will I know when things go wrong? - Web Monitoring

8. What will I do when things go wrong? - Incident Management

9. Who’s minding the store? - Web Operations

10. Managing changes - Release and Change Management

11. Keeping customers happy - Customer Service

12. Staying safe - Privacy, Security and Compliance

13. How will I get customers to my site? - SEO, SEM, PPC etc

14. Back in the real world - Fulfilment and Returns

15. The people factor - Dealing with Burnout

1. Failure to LaunchWhat are the key things that you need to do to ensure that you don’t join the ranks of Nokia, Apple, Nectar, the French and UK governments, and even Carla Bruni in having very public and high-profile disasters when you “go live”?

Launching a modern transactional website is a complex project, one that is increasing “not just an IT problem” as you have to co-ordinate activity across multiple functional areas of the business e.g. sales, marketing, customer services etc.

Furthermore, delivery of many of these elements might be outsourced to third parties - designers, software houses, hosting providers, digital marketing specialists etc - that might be located in different countries, with different languages, cultures and time zones.

Wouldn’t it be great if someone gave you a list of things to check before “launch day” plus some “killer questions” to ask each team to test if they are “all systems go?”

The goal of this document is exactly that - an easy to understand checklist of 55 “killer questions” about the things you need to do to ensure that your next website launch is a roaring success.

2. Does it work? - Functional TestingIt’s a simple question - “does your new website work or not?” - yet it’s surprising the number of websites that launch with critical bugs that drive customers away.

Regardless of whether you follow a classic “V-Model” methodology or a more agile approach (that might incorporate concepts like “Test-Driven Development” aka TDD) you should have a clear testing strategy to ensure that the new site has been tested and works correctly.

In the run-up to launch your goal is to ensure that the testing strategy has been correctly executed.

Killer questions:1 Is the test plan on schedule?

2 Have we met the test objectives and the entry/exit criteria?

3 How many critical or high-priority bugs are still open? (Watch out for the “re-prioritising bug severity downwards” trick in order to meet the quality goals!)

4 What’s the level of test/code coverage (i.e. how much of my site has been tested)?

3. Will the customers like it?- Usability and User ExperienceYour site is working as designed, you’ve ironed out the bugs, you launch it… and the customers hate it.

They complain it’s too hard to use, not intuitive, the layout is wrong, not as good as the “old one” (or worse, your competitors), it gets slated in the industry press (for example this negative review of the H&M Website (bad user experience) versus etc this positive review of the revised (non-Flash) River Island website? (good user experience).

Your site may have passed the functional testing criteria to meet the specification (“doing it right”) but are you “doing the right thing(s)” to give a great user experience that drives conversion and creates loyal customers?

Killer questions:5 Do we have a clear understanding of our customer wants and needs and/or have we

developed “personas” to visualise the customer segments?

6 Have we set clear usability and user experience goals for each customer segment?

7 Have we done usability and user experience testing across each segment, and were the results aligned with our site and brand goals?

8 What plans or mechanisms do we have in place for continuous usability and user experience testing [e.g. A/B or multivariate testing?]

9 What feedback mechanisms do we have in place to capture the subjective customer experience?

Your site is working as designed, you have ironed out the bugs,

you launch it… and the customers hate it!

4. Can it handle the load?- Capacity and ScalabilityAs shown in the examples in the introduction above, or in the recent rush for Take That concert tickets that caused many ticket websites to crash, sometimes websites can be the “victim of their own success” in that they are unable to cope with the levels of traffic they experience.

Capacity (the level of traffic your site can sustain) and scalability (how easily extra capacity can be added) are complex issues that cut across a wide range of application development and infrastructure engineering disciplines.

Before you launch you need to know the answers to both questions.

Killer questions:10 Do we have a clear plan to manage site capacity?

11 What performance, load and stress testing has been performed so far?

12 How many visitors, visits, page impressions, searches, customerregistrations and online orders can our site currently handle?

13 How will we know when we need more capacity?

14 How quickly can we add more capacity?

5. Is it fast enough?- Web Performance OptimisationThree second rule: 57% of online shoppers will wait three seconds or less before abandoning the site - PhoCusWright

Younger visitors are less patient: 65% of 18-24 year olds expect a site to load in two seconds or less - Akamai

How fast does your website need to be to convert your visitors into loyal shoppers?

There is now a solid body of research from companies like Shopzilla, Amazon, Microsoft and Google, as well as research groups like Forrester and the Aberdeen Group that your site’s performance is a key factor in online success.

Your page download speed directly affects your site conversion rate, and quite literally “every second counts”.

Killer questions:15 What percentage of our site’s pages load in less than three seconds when measured in IE7

on Windows XP over a 2mbps connection?

16 Do our key user journeys (e.g. an end to end e-commerce journey) have less than three second load times for each page in the journey?

17 Have we implemented the key web performance optimisation best practices such as browser caching, Gzip compression and image optimisation?

18 How will we measure and track the performance of our website over time?

A one second delay in page load time equals 11% fewer

page views, a 16% decrease in customer

satisfaction, and 7% loss in conversions - Aberdeen Group

6. How do I measure online activity?- Web AnalyticsYou should not launch your website if you can’t measure the activity of visitors to your website (web analytics) and the performance of your web application and server farm (web monitoring, which we will discuss in the next section).

Web analytics, such as Google Analytics, WebTrends, Omniture or Unica, can give you a wealth of information about who is visiting your site, where they came from, what they did while they were there, and where they went when they left - all of which is very useful information as you seek to optimise your online activity.

Prior to launch it’s vital to ensure that you have the ability to collect the right data and that data is accessible to the right people.

Killer questions:19 What key metrics are we using to measure online activity?

20 What web metrics tools are we using to collect the relevant data?

21 Which teams can access what data?

22 Do they have “dashboards” customised for their needs?

23 If using a tag-based approach, can we easily manage changes to the tags (without breaking the website)?

7. How will I know when things go wrong?- Web MonitoringWeb monitoring is a highly complex topic and many good books have been written about it e.g. Powers & Croll but in the simplest form it breaks down into three main areas:

1. Monitoring the health of your (internal) infrastructure (servers, databases, networks etc)

2. Monitoring the health of your application (custom instrumentation, components, transactions, application error logs, etc)

3. Monitoring from the end user (external) perspective (page and user journey monitoring)

Your website monitoring strategy (at a minimum) needs to cover these three areas to ensure that you have the right level of monitoring for your site.

Killer questions:24 What monitoring do we have in place to detect problems with our infrastructure and

application from both an internal and external perspective?

25 Who is responsible for responding to alerts (RACI Matrix) and how are they notified? What happens if that person can’t solve it (escalation matrix)?

26 Is there a centralised dashboard or monitoring console that shows the health of the site?

27 What error and performance logging (instrumentation) is built into our application?

It is critical to monitor the status

of all the various systems that make up

your website so that you know when things go wrong (and ideally

before it impacts your customers)

8. What will I do when things go wrong?- Incident ManagementServers fail, networks go down, data gets corrupted, systems get overloaded, or there is a bug in the new release.

In ITIL terminology this is called an “incident” and how you deal with it is called “incident management”.

The secret is, of course, the “5 P’s” (or 6 or 7) i.e. “proper planning prevents poor performance” – you need to have rehearsed your incident management plan across a range of different scenarios to know that you can respond efficiently and effectively to any situation that might occur.

An effective incident management plan, along with the accompanying communication strategy, can dramatically reduce the commercial and reputational damage that can ensue from of a major outage.

Killer questions:28 What is our incident management plan for dealing with site problems?

29 How will we communicate about the situation internally?

30 How will we communicate the situation externally with our customers, investors and partners?

31 Do we have a plan to manage social media channels in the case of a major outage?

9. Who’s minding the store - Web OperationsBeyond just responding to “incidents”, modern websites need proactive management to ensure that they continue to meet the customer and business needs. This management task is normally the responsibility of the “web operations team” within the IT department.

The “WebOps” team is responsible for proactive and reactive management of the website including managing the health of the end-to-end application which is deployed across your web, application and database servers.

One surprisingly common area of confusion is “why do I need a web operations team if I have ‘managed hosting’?”.

As a general rule, unless you have paid for extra services, “managed hosting” is ONLY the provision of the core hosting platform (e.g. Windows/ IIS or Linux/ Apache). It does not include the management of the application which is deployed on top of that core platform!

Killer questions:32 Who is responsible for the day-to-day management of our website?

33 Have we outsourced any of the services to a hosting or managed services provider, and do we have a clear RACI matrix of “who does what” if responsibility is split across teams?

34 What level of support do we offer in terms of response time goals (1 hour? 4 hours?) and coverage (24x7x365, 8x5, “on call” etc)?

Inevitably things will go wrong on your website

10. Managing change- Release and Change ManagementYou are going to have to change your site, that’s a given… but HOW you change your site and the impact it will have on current operations needs careful thought.

What you should be aiming for is this:

1. Zero downtime when you roll out changes (Just say “no” to site maintenance pages!)

2. Only change what you need to change (aka “incremental deployment” as unnecessarily changing the “last modified” dates on files can have a significant performance impact on the end-user experience)

3. “One click” deployment that automates as much of the change as possible, which reduces human error and will scale as your site grows to use more and more servers.

4. An easy to use system to keep track of what got changed and by whom, who authorised it and why (e.g. an ITIL-based change management system like www.service-now.com).

Killer questions:35 How quickly can we deploy changes to our site?

36 How much of it is automated versus manual change

37 Do we need to take the site offline when we deploy changes?

38 How do we track what was deployed, when and why?

11. Keeping customers happy- Customer ServiceThere are many surveys that show that online customer service and communication is poorly implemented by many of the major online retailers (for example this one by the DMA or the annual US survey by E-tailing Group) and the key message is clear - online retailers are losing money and turning away customers by not meeting their customer service expectations. Conversely, companies like Zappos in the US are reaping the benefits of great customer service.

There are six common channels of communication in use: Email, Phone, Online forms, Online Chat (e.g. using tools like LivePerson or WhosOn), Online forums (including peer-to-peer support) and Social Media (Facebook, Twitter etc).

Which mix of these is right for your site depends on your target audience, type of product, available resources etc but it is worth noting that whilst these channels can be “reactive” (i.e. responding to customer-initiated enquiries) they can also be “proactive” channels used to help your sales and marketing goals.

Killer questions:39 How can our customers contact customer service?

40 How will we track our customer service delivery across channels?

41 What are our response time and customer satisfaction goals?

42 What resources do we have in place to meet these goals?

Good websites are not static - they

change, evolve and adapt to changes in the marketplace, the

customers wants and needs, technological progress and many

other factors

12. Staying safe- Privacy, Security and CompliancePrivacy, security and compliance are crucial for any website for three reasons: your customers are worried about it, your payment vendors demand it (e.g. PCI Compliance), and it’s the law (e.g. the Data Protection Act).

With security concerns behind up to 63% of shoppers abandoning a purchase, and the potential costs of security breaches running into millions it is worth ensuring that you follow security best practice for both your website and your internal data processing before you launch.

Killer questions:43 Have we followed industry best practice on our website to re-assure our customers

regarding privacy and security e.g. Trustmark schemes, privacy policy etc

44 Do we follow industry best practice regarding cyber-security of our application and infrastructure e.g. SANS guidance?

45 Has our site passed external penetration testing to verify it’s secure?

46 Is our site PCI compliant (if relevant)?

13. How will I get customers to my site?- SEO, SEM, PPC etcIn order to drive traffic to your website there are well known marketing channels such as search engines (organic search and paid-for search engine marketing), pay-per-click (PPC) banner advertising, affiliate marketing and “word of mouth” referrals via social media, each of which may have a different cost of acquisition.

Before you launch your site you need to have a clear understanding of your key marketing channels, since in many cases you will need to implement features on your site to support those channels.

For example, search engine optimisation (SEO) requires specific “on-page” features like unique titles, meta-data etc that need to be built into your site.

Similarly your marketing efforts might benefit from custom landing pages that are designed to convert specific target audiences.

Killer questions:47 Do we have a clear marketing strategy covering areas like SEO, SEM, PPC, affiliates,

social media etc?

48 Have we “built-in” the necessary features into the website to support these channels?

49 Can we track our marketing ROI effectively via our web analytics system?

There is little point in having a great website if no-one knows about it or

can’t find it!

About NCC GroupNCC Group is a leading global information assurance firm, providing freedom from doubt that all critical material is available, protected, and operating as it should be at all times. Information assurance is delivered through escrow and verification, security testing, audit and compliance, website performance and software testing services.

About Website Performance & Software TestingNCC Group’s website performance and software testing services assure the quality of your software, the integrity of your infrastructure and the performance of your websites.

We provide independent business analysis, software testing, web performance and load testing services to over 550 clients worldwide. Our services are completely independent, so we don’t rely upon, or recommend, the use of any one testing tool or process.

NCC Group Manchester Technology CentreOxford Road Manchester M1 7EF

For more information on Website Performance services from NCC Group, please contact:

+44 (0) 161 209 5200response@nccgroup.comwww.nccgroup.com

14. Back in the real world- Fulfilment and ReturnsSo, you’ve sorted out the online needs and you have customers visiting your highly-available, high-performance, user-friendly, secure website backed up with great customer service… and then you fail to deliver “offline” and undermine the whole experience.

It may be that your site offers a purely online service with no physical component but for many online vendors the fulfilment takes place offline by the delivery of a physical good or service and it’s this gap between the online and offline where things can “fall through the cracks” because of poor co-ordination between the online and offline channels.

For example, some sites let themselves down with slow delivery, poor packaging or a returns process that is a laborious because it wasn’t integrated into the website.

Prior to launch you need to test your end-to-end fulfilment process (including returns) to ensure a seamless high-quality customer experience regardless of whether it is online or offline.

Killer questions:50 Have we tested our offline fulfilment processes to ensure a seamless customer experience?

51 Can our fulfilment process scale to meet the expected online demand?

52 Is there sufficient integration between online and offline systems e.g. stock availability or payment systems to meet our goals

13. The people factor - Dealing with BurnoutYou and your team have been working long and hard to prepare for launch, everyone is tired, everyone has postponed their annual leave, and you’re all hanging in there waiting for the launch so you can all take time off to recover. Sound familiar?

Employee burnout is a very common issue with high-profile, high-pressure website launches and a common fallacy is to believe that “the pressure will come off once we launch”.

Unfortunately, website launches are rarely perfect and if things go seriously wrong it’s common for the pressure to increase after launch, not decrease. Therefore, careful management of the “human resource” is just as crucial as any of the more technical issues discussed earlier.

Killer questions:53 Are we effectively monitoring and managing the stress levels of the team members?

54 How large is the backlog of unused annual leave and how will this impact our delivery capability for the rest of the year?

55 Have we planned a great post-launch party for EVERYONE involved in making the site launch an awesome success?

You’ve sorted out the online needs...

and then fail to deliver “offline”