Diy user engagement, white paper

1

October 2011 Product Doctor Insights from Over The Air 2011

DIY USER ENGAGEMENT

A guide on how to engage users in product development

This White Paper is based on a Product Doctor Drop in Surgery that I ran with Katrina Damianou during Over The Air 2011, 36 hours of Mobile Development. We offered 25

minute free Product Health Checks to the developers in attendance. Some of the examples of products brought to us are referenced in this paper.

___________________________________________________________________________

Contents

Agree this premise first!

1. It is never too early (or too late) to engage end users 2. What do you show users? 3. How to find your end users 4. Can you have the conversation with end users? 5. How to begin the conversation 6. Write a test script 7. “I can’t explain what my product does” 8. Showing a prototype 9. Testing for usability 10. Keep checking back with users as you develop and improve each new feature 11. Build and test your product before developing your brand 12. Be honest with yourself as to why you are developing the App

Conclusion

Agree this premise first!

Before reading any further, we have to first accept that engaging users throughout the development cycle and from the concept phase (i.e. before you build anything) is good practice. I have proved both of these points over the past years, time and time again.

A. Engaging users WILL give your product a far greater chance at success. We can all think of great examples of products that may work beautifully, but fail as they do not meet a user need or desire.

B. Furthermore, products that are not tested properly with users often are too feature heavy -there may be a handful of features that are used while the effort spent to build the rest has been wasted.

When I asked patients whether they had engaged users at any stage of their development, those with live Apps did say proudly that they certainly studied their App Store feedback.

2


App Store feedback is not enough! Not only is it once the App is out there, but throughout user tests I have carried out, I have found a bias on males giving written feedback on the App Stores. Even then, it is only a certain type of personality that gives written feedback on the App stores. They may give you suggestions on new features or how to improve their user experience, but this will often be on one or two actions / features they are carrying out. You are unlikely to get confirmation that the needs / desires you are trying to meet are valid – again, this can throw the whole validity of the App in to question.

Remember that these people who give you feedback are likely to be early adopters. We have to remember that Apps are still in the “introduction phase” of the product lifecycle. This is the point where all the product and experience glitches/bugs will be highlighted and ironed out. So by all means, you may well get good feedback on what is and isn’t working but not on the bigger picture.

So here is my guide to DIY User Engagement

1. It is never too early (or too late) to engage end users

I am a bit of a broken record on this one, but seriously, it is something that recurs time and time again! Before you put your time in to developing, at any stage, please test out what you are intending that the product is going to do. It is crucial to check that the user needs or desires exist and that the product and features you are going to build are a good solution to those needs or desires.

2. What do you show users?

One of our patients was very surprised to hear that he did not need a prototype to start consulting with users and he could in fact use a statement of the Product Concept at a very early stage. They were part way through development of a home energy monitoring App. They could see how this would save time and energy and focus them on spending their time developing the right features.

So, you do not have to show people a prototype, you can just talk about your concept - try to define it in one paragraph. It may help to write your concept up as a statement / user story

"...As a user I need something that does "x" so that I can "y"..."

For example, "...As a user of the Boris Bikes scheme, I want to be able to easily find a free bay to dock back my bike so that I don't have to waste my time trying to find one..."

You can also show functionality that you haven’t yet beautified on the front end (like the energy App mentioned above). In fact, people are more comfortable to give you constructive feedback if they see the product isn’t finished yet.

3


3. How to find your end users

On questioning, every patient could identify what sort of people they thought would use their product, but very few have checked their ideas out with end users that are not relations or friends. Note that those close to you will want to be encouraging and supportive and so their feedback may be biased.

Here are examples of how to find your users:

...for the Boris Bikes scheme App - go stand at the bike docking bays and chat to users there;

...for an App to learn to Chinese - go to International House and talk to students and teachers

...for navigating around a festival - go to a festival!

Once you talk to users you are likely to understand more about where your product sweet spot is. You will find people who you thought would be interested in using your product that are not; and you are highly likely to find scenarios and other target groups that you would not have thought of. In my 10 years of talking to users on a large range of digital products I have always found out something I had not imagined!

With many products, it makes sense to test in pairs of friends. Often the conversation between the two respondents will be very open and you will gauge more open and honest feedback from the discussion between them.

In some cases, it may be appropriate to ask your friends / family if they know anyone that fits your criteria. Importantly, unless your product is specifically aimed at the tech community, you really want someone outside of our industry. Sometimes, even tweeting or facebooking who you are looking for can help.

You may also find groups around particular interests that may be happy to test out your product – you can find them in Meet Up; Facebook groups; Ning and so on. They may be happy to do this also without you have to pay them. Perhaps showing up with a tray of donuts will do the job! If you have a product that is targeted to the youth, talk to me about how to work with schools / colleges / universities as these sorts of exercises can double up as learning experiences for young people. Many teachers really appreciate real life products and people to use as case studies not only for enterprise skills development but also to compliment particular courses, such as Product Design, Technology, Marketing, Business Studies etc.

You may well need to reward the respondents. You can offer cash or a voucher; the amount obviously depends on how long the interview is going to be. For the last set of user groups I ran, I offered £40 for a 90 minute interview, so around £15 for 30 minutes. Sometimes a free pint in the pub works – again – if that is where you can find your target group.

4


4. Can you have the conversation with end users?

Some of our patients expressed concerns about their comfort levels with facilitating the conversation with end users. If you are not comfortable, you could ask someone else - a friend or family member who you think could help you. Of course, you can always ask Product Doctor as we may be able to do it for you on an affordable hourly rate!).

Getting feedback from user is a practised skill; something I have been trained to do and have practised over many years, but if you do not have any budget to hire someone in, please try to do this yourself. H0nestly, it is better than not doing it all. Take a recording of the interview – on your phone or on a dictaphone – that you can play back after the event. You will pick up on comments that you missed at the time and also be able to improve on your interview techniques, for example you will hear where you might have “lead the witness”. If you want to be good at this, you can be and I am always happy to give you any pointers you may want. You may even be able to provide a real life learning experience for young people who are still studying – they could carry out the surveys for you – again, get in touch if you would like to talk about this in more detail.

5. How to begin the conversation

Start by setting the scene; say (yes, lie if need be!!) that you did not develop this yourself, you are just helping out a friend – people will be less likely to be honest if you seem too invested in the product. Do not be passionate about the product. Do not appear to be selling. Try to be as neutral as you can.

6. Write a test script

In addition to obviously being able to explain what the product is going to do, list out all the assumptions you have made. This will include items like:

a. The purpose of the product / product concept b. The user needs / desires that you think your product is meeting c. Assumptions for financial modelling – e.g. how often the user is going to use your

product d. The full features that you have / are intending to build These are examples. Of course, you should also test out pricing, naming and so on.

So your questions will follow the above – example

a. Can you put in your own words, what this product does? b. Which of the following needs can you relate to? Perhaps there are things we have

not thought of / how could the product help you? If it is not for you, do you know anyone else that might use it? You can keep coming back to the user needs throughout the interview as you will find they may well change throughout the interview

c. How often could you see yourself using this product (based on scenarios you have thought of above)

5


d. Rate these features in order of importance to you – please do not rate any features that you would not use at all. Are there any features you would like to see that are not listed here? How would you prioritise those in this list? Feature over-load is one of the most common mistakes I see being made. Sometimes the killer benefit of the product is a feature that you are thinking of as a side-product. This exercise will help you to find what is really going to make your product stand out. One of our patients had an App relating to finding a taxi, which had a number of different features. He had one feature that seemed particularly unique and instinctively, it felt as if this could form the basis of the entire App (patient confidentiality stops me from giving any detail).

7. “I can’t explain what my product does”

This was a wonderful quote from one of our patients. He had an idea that he felt would help with person to person transactions. Since the rise of E-Bay we have certainly seen an increase in person to person transactions. More traditionally, they have always existed – direct purchases with no middle man - buying and selling cars and landlord/tenant transactions. He talked us through his idea and then reflected, saying “I can’t really explain what the product is” but he knew he was on to something. The prescription in this case, was to go talk to people that have had experience of these person to person transactions. Find out whether the pain he felt he could address really existed. From there, he could develop a one pager for each of these user scenarios that he could pull out when presenting what his product did. Stories are a great way to explain what your product does and how it meets a user need. Products can be easily explained by exposing the solution to a user need in a real life scenario.

8. Showing a prototype

As above, it is not always necessary to show the user something – testing out the concept before build is crucial in product development. However, if you do have a prototype, remember not to sell the product! Do not say things like “this bit is really cool”. Don’t look excited, you are just trying to demo the functionality.

Once you have shared the prototype and worked through your questions, it is a good idea to ask users how they would recommend you showed the product in the App Store. How they would describe it and what screens they would show. You can also ask users how they would expect to hear about the App. This will help you with your marketing plan.

9. Testing for usability

Once you have a prototype / something to show, you will also need to check usability and design. The guide for usability testing is to list out a number of user tasks. Let’s take an example: London Journey Planner App – so sample tasks might be

i) Find the route from here to Oxford Circus ii) Find the current status of the Circle Line

6


iii) Find the W7 Bus Route

These tasks should reflect all the different actions that users can take. Give them the task. Then stop talking. Watch and listen. Try to not jump in and show a user how to do it. After 5 interviews (as per Jacob Neilson, the Godfather of Usability for his guidelines), you are likely to see the same usability difficulties repeated. You will find the phrase “did that behave as you expected?” to be a very useful question to ask. Try to replay for clarity “So when you pressed that you expected….but instead it took you to….” Confirming what you heard the user say is very important, you may find that you have made some interpretations that are not correct.

Testing out design is part usability and part look and feel. Let your users know that they should feel free to make suggestions and improvements as they go through. Again, try to not feedback to their suggestions as to whether you like them or not or whether they are technically possible. In some cases, user suggestions need further working through. So hypothesise with them and walk through the change they suggest to make sure that it has a logical conclusion.

In general, be very sparing with your words. State…Ask them what they think of that…Be quiet and listen! If you find yourself doing any interpreting in your head, replay that back – so “when I said that blah blah you said blee blee, does that mean bleugh bleugh?” Seek clarity and try not to get side tracked in to a conversation off topic!

If users have difficulties with navigating through a particular feature, do ask them how they would make it easier. Same goes for any icons that you are using. Just tips, and again, one of my mantras, do not reinvent the wheel. If there is a good and well known user interface out there that you can adopt for your product then use it. As much as users should be able to navigate through an intuitive user interface, they love it when the interface is familiar and will often suggest you should build it just like they do on x-another App.

10. Keep checking back with users as you develop and improve each new feature Gathering user input is a necessary on-going exercise. If you are working with Scrum, you can build it in to the sign off of the user stories for each iteration. 11. Build and test your product before developing your brand I am a strong believer that you live and die by the strength of your product – product is king. Brand and design are super important, but is nothing if your product is not meeting user needs and desires and providing a great user experience. One of our patients was a designer, rather than a developer. He had developed a brand before his product was up to scratch. To me, this is the cart before the horse. I can empathise with developers that are driving new product development as they have to fund developers to build their vision. So many of our patients were developers hacking in their spare time – while they may need to engage designers down the line, at least they are not having to pay out at this stage. Recently I was asked to help a company come up with a name for their product, but they could not explain in one paragraph what their product did. This illustrates the point well. I do not see how you can get a product name or develop your branding without defining what

7


your product does. Being product-centric, this view should not surprise you. Naming a product and developing a brand is also an exercise where users need to be engaged.

12. Be honest with yourself as to why you are developing the App

I often meet young developers who are working on their pet App in their own time. They may be doing this as their day jobs are not giving them the chance to work on particular code or areas that they want to so they are supplementing their education and ability to show their skill in their own time. Great. But in some cases, this is not about developing profitable Apps. This is just about enhancing their CV – great to be able to say, “I developed an App that has had x downloads and is rated as 4/5”. Leave it at that! Don’t drive yourself crazy to find the revenues as that is not your core motivation. When I ask patients “what do you see yourself doing in 5 years’ time?” very few can see themselves as running a company that takes them away from their core love, which is developing! Many do not want to build up the skills to market and PR their Apps for example. “Just leave me with the fun bit – the coding”.

On the other hand, some developers absolutely are looking to make the killer App and want to make their million. In these cases, it is really important to think through

a) The revenue model, examples

This can include partnership conversations with revenue share

Exit strategies where they may be able to sell their App

App stands on its own two feet and revenue models work independently

b) The go to market strategy In my opinion, you cannot rely on just putting your App in the App stores and hoping for the best. You need to “DIY PR” to coin Lisa Devaney’s phrase – for example - how to build up an online profile for the product and generate online noise so that for example, industry bloggers write about your App. Where are your target customers going to find out about your App – we had some great examples of brainstorming with some of our patients of societies, online groups centred around particular interests, magazines and known bloggers and so on. There is no excuse; you can find groups on Meet Up, Facebook, Ning and so on

Additional Notes: (i) Proving in the niches first If you have a product, like Audio Books, that is going to have pretty much mass market appeal, rather than try to tackle the market en mass, you can prove in the niches first and work your way out from there. There may be some obvious places to start; or perhaps places where the social media machine will work for you best; maybe you already have contacts that can help you in those niches. (ii) Nothing wrong with backing a few different horses A few of our patients had products that could be own brand or white label. I am sure we can all name many products that work in this way and we know the benefits of working

8


with each tranche. Ultimately, the core functionality will be the same, with a different wrap. If you have the capacity to manage both, then best to run with the two horses and see what happens.

c) Your product positioning & understanding your competition

How do you sit in relation to your competition – around half of the patients we saw did not have an up to date grip on what was out there and how their product was better / different.

What are the 3 key product benefits to using your App

Again, no surprise here, but these entire elements can also be tested out with users. For example for “go to market”, you can ask the users that say they would use your product “How would we best reach you”. For Product Positioning “What do you think are the 3 key benefits of using this product”?

I hope that you find this Guide useful, if you want to contact me for any clarification or advice please do contact me:

T. 07956 376472 E. [email protected]

W. productdoctor.co.uk

mailto:[email protected]