Testing API behind Mobile Aap · 3. Integration testing Internal API with complete backend No regression impact of other projects 4. Acceptance testing External integration API with

2013/7/11

1

ChinaTest, Our TEST!

Testing the API behind a mobile app

Tutorial

Marc van ‘t Veer


Content

• What is an API

• Why use an API

• How to use an API

• How-to test an API

• Exercises

– Group exercises (learning concepts – 7)

– Individual exercises (apply concepts – 4)

• Lessons learned

http://www.molens.nl/site/dbase/zoeken.php

2013/7/11

2


What is an API - Definition

• API, stands for application program interface:

– Set of routines, protocols, tools and blocks for

building applications (systems of systems) and so

reducing complexity

• Definition within the context of this training:

– An internet web service that opens systems and

giving standard way of access to external users

4 Marc van ‘t Veer – Polteq

2013/7/11

3


What is an API - API model

• Lego bricks are standard building blocks

• Blocks are a library to build

• Figure is showing which resources there are

• Sales box is an example of the possibilities


What is an API - API model

2013/7/11

4


What is an API – Typical structure

• API is built upon existing internal architecture

Backend (Enterprise Service Bus)

Adapter

Desktop web

Browser

API

Webservices Adapter

app

Mobile web

SoapUI

Adapter

Test app’s app

Facebook


Why use an API - example

Telecom companies provide usage details on their

websites. App developers used screen scraping to

get this data into their apps.

Resulting in:

• Misinterpretation of data

• Malformed data

• Telecom company gets

blamed when the app

provides wrong information

2013/7/11

5


Why use an API - example

API provides the telecom companies:

• More control over the data

• Control over the content used in an app

– Key needed to use the API

– App developer needs to prove that it works

correct before getting access


Why use an API

• Scaling development

– Enhance a program without editing itself

• Converting competitors into partners

– Allow competitors to build on top of your product

• Scaling market reach

– Open up new markets

• Empowering users

– Allow people to use the product for things it

wasn’t designed for

2013/7/11

6


How to use an API – Mashup

• A mashup is a web page or web application

that uses and combines data, presentation or

functionality from two or more sources to

create new services

– Easy, fast integration

– Frequently using API

– Produce enriched results from the raw source

data


How to use an API – Demo of API’s

• Local examples

– Command line on Windows or Linux

– Local VB script with Windows API

• Web API examples

– Overview of API’s: Programmableweb.com

– Demo of WordPress blog API

C:/Windows/System32/cmd.exe

../../70 Special Development Program/60 Testing API behind mobile app/20 Tutorial/Use windows API to get username and computername.xlsm

http://www.programmableweb.com/

http://192.168.8.50/

2013/7/11

7


How to use an API – Concepts

• Concepts of the API

– Division of data and presentation

– Root (Start URL, http or https)

– Navigation

– View of a command-line

– Hierarchy

– Resources

– Account (private API)

– Structure


How to use an API – Who uses it?

• Apps: Android, Iphone, Windows, Blackberry

• Internal organizations

• User via browser version of the API

• Social media: Facebook, twitter, etc.

• (internal/external) developers:

– Documentation

– Base line code for new App (with guidelines)

• Operations (monitoring)

2013/7/11

8


Summary – Introduction to an API

• Public available via Internet

• No-frontend

• Interfaces

• API is part of an end-to-end solution

• Architecture / protocol focus

• External users / devices

• Test tools are needed

• Main goal is: a generic API


How-to test an API – start from scratch

2013/7/11

9


How to test an API – start from scratch

• Architecture

– REST vs. SOAP

– Stateless vs. Session

– Media types vs. WSDL

• text/xml or application/vendor specific

– Verbs (GET) and resources

– Headers and caching results

• Tools

– SoapUI, Fiddler, new add-ons for Firefox

• Architecture

– REST vs. SOAP

– Stateless vs. Session

– Media types vs. WSDL

• text/xml or application/vendor specific

– Verbs (GET) and resources

– Headers and caching results

• Tools

– SoapUI, Fiddler, new add-ons for Firefox


How to test an API – RESTful verbs

• Verbs are actions, actions like CRUD on

database

• POST: Create a new data element

• GET: Read a data element

• PUT: Update a data element

• DELETE: Delete a data element

2013/7/11

10


How to test an API – Headers

• HTTP header fields are components of the

message header of requests and responses in

the Hypertext Transfer Protocol (HTTP).

• They define the operating parameters of an

HTTP transaction

• Request example:

• Accept: text/plain

• Cache-Control: no-cache


How to test an API – Cache

• A web browser stores web content for reuse

• If the back button is pressed, the local cached

version of a page is displayed instead of a new

request being sent to the web server.

• Risks

– Old data

– Cached errors

– Layered caching (app – API – server)

2013/7/11

11


How to test an API - Typical risks

• Unknown integration

• Big variation of customer data

• No control of the E2E chain

• Load is unknown

• Wrong use of API

• Dynamic scope


How to test an API - Strategy

• Early integration test with complete

infrastructure

– Integration test and dog fooding during

Development and System Testing

– Multiple integration phases

– Prototype app (on Acceptance environment)

2013/7/11

12


How to test an API - Dogfooding

• Dogfooding: Don’t try to build the sort of API you think people will want, but use the public

API’s your self everywhere

• Examples

– All Google tools use their own public API’s

– Windows 7 is build with windows 7

– Twitter.com uses it’s own public API


How to test an API - Test approach

1. Development (with dogfooding)

2. System testing (API with stubs and SIT)

3. Integration testing (API with backend)

4. Acceptance testing (API with prototype app)

5. Production integration (in phases)

6. Regression testing (automated)

2013/7/11

13


How to test an API - Test approach

Backend (Enterprise Service Bus)

Adapter

Desktop web

Browser

API

Webservices Adapter

app

Mobile web

SoapUI

Adapter

Test app’s app

Facebook


How to test an API - Mocks and stubs - 1

• API’s are part of a big E2E chain

• Testing is only possible if a stub framework is

available

• Stub, driver, mock

• Example of a stub framework

2013/7/11

14



• Replacement of a reply from backend system



• If there is no connection to a backend system

• Re-routing an interface to a local tool like

SoapUI

2013/7/11

15


Group exercises

• Minimum of three implementations is needed

for a quality API

• Learning

– One implementation: it’s a trick

– Two implementations: it’s a method

– Three implementations: it’s your own way of working


Group exercises – learning concepts

• Exercise 1 – Lorum ipsum text

• Exercise 2 – Colour lovers and Color picker

• Exercise 3 – Authorization

• Exercise 4 – Status and error codes

• Exercise 5 – Header injections

• Exercise 6 – Business rules

• Exercise 7 – Current and future version

2013/7/11

16


Exercise 1 – Lorum ipsum text

• Description

– Generation of Lorum ipsum text

• Assignment

– Get a number of lines of bacon ipsum text

– Tool: Firefox


Exercise 1 details – Lorum ipsum text

• API

– http://baconipsum.com/api/

• API documentation

– Open two tabs

• One with documentation: http://baconipsum.com/api/

• One to enter commands like

– ?type=meat-and-filler

http://baconipsum.com/api/?type




2013/7/11

17


Exercise 2 – Colour lovers and Color picker

• Description

– API provide content in many different types, like

generation of colors

• Assignment

– Find a favorite color with colour lover API and

validate the Hex-code with Color picker

– Tool: Firefox


Exercise 2 details – Color lovers and Color picker

• API

– http://www.colourlovers.com/api/colors?

– http://www.colorpicker.com/


– http://www.colourlovers.com/api/

http://www.colourlovers.com/api

http://www.colourlovers.com/api

http://www.colorpicker.com/

http://www.colourlovers.com/api/

2013/7/11

18


Exercise 3 – Authorization

• Description

– Difference between public and private API

– Testing of functional security

• Assignment

– Get access to API by using access tokens

– Tools: Firefox


Exercise 3 details – Authorization

• API

– http://openexchangerates.org/api/latest.json

– Access key:

• 66e62d8337b545ec9f0508c1215764b5


• https://openexchangerates.org/documentation

• https://openexchangerates.org/documentation#app-

ids

http://isbndb.com/docs/api/index.html

http://openexchangerates.org/api/latest.json?app_id=66e62d8337b545ec9f0508c1215764b5

https://openexchangerates.org/documentation




2013/7/11

19


Exercise 4 – Status and error codes

• Description

– Explore what is a good and what is a bad request

• Assignment

– Find status and error codes and rules to trigger

them

– Which error code is shown when (priority)

– When do you expect an error code

– Tools: Firefox + Firebug


Exercise 4 details – Status and error codes

• API

– http://openexchangerates.org/api/latest.json?app

_id=66e62d8337b545ec9f0508c1215764b5


– https://openexchangerates.org/documentation

– https://openexchangerates.org/documentation#e

rrors

– http://en.wikipedia.org/wiki/List_of_HTTP_status

_codes






http://en.wikipedia.org/wiki/List_of_HTTP_status_codes

http://en.wikipedia.org/wiki/List_of_HTTP_status_codes

2013/7/11

20


Exercise 5 – Header injections

• Description

– Explore what header elements are

• Assignment

– Compare results with and without a header

elements

• If-Modified-Since and If-None-Match

– Tools: Firefox + Rest Client


Exercise 5 details - Header injections

• API

– http://openexchangerates.org/api/latest.json?app

_id=66e62d8337b545ec9f0508c1215764b5

• Documentation

– https://openexchangerates.org/documentation#e

tags

– http://en.wikipedia.org/wiki/List_of_HTTP_heade

r_fields





http://en.wikipedia.org/wiki/List_of_HTTP_header_fields


2013/7/11

21


Exercise 6 – Business rules

• Description

– Compare output between different customer

types. Which resource has which output?

• Assignment

– Find output of a prepaid and a business customer

– Find a invoice for a prepaid customer

– Tool: Firefox


Exercise 6 details – Business rules

• API

– https://capi.t-mobile.nl/

– Access key

• {Bearer} with:

– {StubbedResidentialPostPaid}

– {StubbedBusinessPostPaid}

– {StubbedResidentialPrePaid}

• Documentation

– https://capi.t-mobile.nl/documentation 42 Marc van ‘t Veer – Polteq

https://capi.t-mobile.nl/



https://capi.t-mobile.nl/documentation



2013/7/11

22


Exercise 7 – Versions of resources

• Description

– Resources have a life cycle which is organized in

versions for now and the future

• Assignment

– Find a resource that is active, have multiple

versions and is end of life

– Tool: Firefox


Exercise 7 details - Versions of resources

• API

– https://capi.t-mobile.nl/

– Access key

• {Bearer} with:

– {StubbedResidentialPostPaid}

– {StubbedBusinessPostPaid}

– {StubbedResidentialPrePaid}

• Documentation

– https://capi.t-mobile.nl/documentation








2013/7/11

23


Individual exercises – Apply concepts

• Exercise 8 - Design review

• Exercise 9 – Testing design techniques

• Exercise 10 – Explore test application

• Exercise 11 - Search of the defects


Test levels for testing an API Phase Structure Element Test case

-2. Create designs Review -Perform review on API design

-1. Test case design Design -Create test cases from designs

0. Setup Test application Setup - Stub frame work

- Tooling

- Interfaces and test data

1. Development Framework Dog fooding Check easy of use framework and how good it functions

2. System testing Functionality with and

without stubs

• With and without stubs

• Test applications

URL Structure, parameters

• Header request : character set, media types, Verbs (GET/POST/UPDATE), HTTP(S), status/error codes

• Logging: Support defect analysis (server and database)

• HTTP-status: Error and related status

Common validation/Authentication/Authorization

• Access codes (tokes)

• Profiles (subscription active, SIM active, token status)

• Customer/contract types

• Caching: Duration, authorization, clearing, content, multiple users

• Navigation: between resources

Resources

• Profiles: Business rules for each resource with variation customer data

Output

• Presentation forms and versions (supported not supported)

Monitoring

• Reporting and performance (load filter, backend availability

Documentation

SIT No stubs

Integration with Middleware

Variation in customers, rate plans, prices, contract types, history, options with no stubs

3. Integration testing Internal API with complete backend No regression impact of other projects

4. Acceptance testing External integration API with prototype First integration of app with API

5. Production testing Real customers API with live app Real customers with app on production API

6. Regression Regression Reruns - Manual and Automated regression tests

2013/7/11

24


Test application

• JSON API as plugin for WordPress

• CMS system

• Customizable and Open source

• Based on PHP language and MySQL database

• Also has mobile application for all platforms

• See also:

http://en.wikipedia.org/wiki/WordPress


Test environment setup

Test tool

API

Verification

Request

Reply

Request

Reply

WordPress

server

Data


2013/7/11

25


IP address WordPress


Exercise 8 – Design review

• Description

– An API design should give all information about

how it operates

• Assignment

– Use a checklist to review design of JSON API plugin

2013/7/11

26


Exercise 8 details – Design review

• API

– http://wordpress.org/extend/plugins/json-

api/other_notes/

– http://<local IP address>/documentation/json-

api/

• Documentation

– https://github.com/WhiteHouse/api-standards

– http://<local IP address>/documentation/design-

review-checklist/

51 Marc van ‘t Veer – Polteq


Exercise 8 - Design review result

• Result of design review on JSON API plugin

–

–

–

–

–

–

–

–

http://wordpress.org/extend/plugins/json-api/other_notes/



https://github.com/WhiteHouse/api-standards



2013/7/11

27


Exercise 9 – Test design techniques

• Description

– Normal test techniques are useful

• Assignment

– Create test cases for the JSON API plugin of

Wordpress

– Use techniques like

• State transition

• Data combination / classification tree

• Equivalence Class partitioning

• Boundary Value analysis


Exercise 9 – Testing design techniques

• Use scrap paper to write test cases

• See example workout

• Documentation

– http://wordpress.org/extend/plugins/json-

api/other_notes/




2013/7/11

28


Exercise 10 – Explore Test application

• Description

– Before you can test the API you should first

explore it

• Assignment

– Open the JSON API and try it out


Exercise 10 – Explore Test application details

• API

– http://<local IP address>/?json=

• Documentation

• http://wordpress.org/extend/plugins/json-

api/other_notes/

• http://<local IP address>/json-api/

• To comment or post with RestClient use: – “name” = “Content-Type” and “value” = “application/x-www-

form-urlencoded” 56 Marc van ‘t Veer – Polteq




2013/7/11

29


Exercise 11 – Search the defects

• Description

– Search defects on the JSON API in WordPress

• Assignment

– Use the design test cases

– Use the learned tooling and techniques

– Analyze the results

– Tool: Firefox + Rest Client


Exercise 11 – API details

• API

– http://<local IP address>/?json=

• Documentation

• http://wordpress.org/extend/plugins/json-

api/other_notes/

• http://<local IP address>/json-api/

• To comment or post with RestClient use: – “name” = “Content-Type” and “value” = “application/x-www-

form-urlencoded”




2013/7/11

30


Exercise 11 - Search the defects

• Which defects are found

–

–

–

–

–

–

–

–

–

–

––


Evaluation of exercises

• What went well?

–

–

–

• What went wrong?

–

–

–

• Conclusion

–

––

2013/7/11

31


Lessons learned

• API

• API/app communication

• Testing


API

• New test type: production tests

• Command line

• Scope

• Different skills

– Security

– Performance

– Automation

• No backup tricks

2013/7/11

32


API/app communication

• Provider of the API is seen as responsible for

the presentation of the data in the app

• Presentation errors of data will always occur

• Very tempting to fix defects in app instead of

API

• More explanation needed with secured API


Testing

• Normal test techniques

• Instrument for defect analysis

• Automated regression tests on production

• New responsibilities

• SOA/Interfaces/HTTP protocol/Tools

• Experience with building interfaces

2013/7/11

33


More information

Marc van 't Veer

Test consultant

Polteq Amersfoort (Netherlands)

+31 (0) 6 46 63 61 48 (mob)

http://www.polteq.com

[email protected]


References

Source Link

[Apple, 2009]

iPhone 3g Commercial "There's An App For That"

http://www.youtube.com/watch?v=szrsfeyLzyg

[T-Mobile, 2011] Customer API T-Mobile: https://capi.t-mobile.nl/

[API design, 2013] http://apidesign.com

[SmartBear Software, 2011] SoapUI with REST:

http://www.soapui.org/REST-Testing/getting-started.html

API testing:

http://blog.smartbear.com/software-

quality/bid/273613/practicing-safe-sex-with-third-party-apis

[Molen animation,2013] http://www.animatieplaatjes.nl/molens.html

Test design technique http://www.pragmaticmarketing.com/resources/why-api-as-a-

strategy

http://www.polteq.com/

mailto:[email protected]



http://www.pragmaticmarketing.com/resources/why-api-as-a-strategy









2013/7/11

34


References, continued

Source Link

[Tweakers, 2012] More secure public API from NS:

http://tweakers.net/nieuws/81389/ns-wil-apps-die-prijs-

treinreis-tonen-vooraf-keuren.html

WordPress http://www.wordpress.org


http://developer.wordpress.com/docs/api/


Practical API Design Jaroslav Tulach, chapter 9: Keep Testability in Mind

Firefox add-ons http://www.garethhunt.com/modifyheaders/help/

http://jsonview.com/

http://restclient.net/

https://addons.mozilla.org/en-us/firefox/addon/make-address-

bar-font-size-big/



Source Link

[Wiki,2012] http://en.wikipedia.org/wiki/Application_programming_interfac

e#Use_of_APIs_to_share_content

REST architecture:

http://en.wikipedia.org/wiki/Representational_state_transfer

Internet Media Types:

http://en.wikipedia.org/wiki/Internet_media_type#Type_applica

tion

http://en.wikipedia.org/wiki/Mashup_%28web_application_hyb

rid%29

http://www.mediawiki.org/wiki/API:Main_page

http://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol#Requ

est_methods


http://en.wikipedia.org/wiki/Web_browser

http://developer.wordpress.com/docs/api/

https://addons.mozilla.org/en-us/firefox/addon/make-address-bar-font-size-big/













http://en.wikipedia.org/wiki/Web_browser

2013/7/11

35



Source Link

[Wiki,2012], continued http://en.wikipedia.org/wiki/Service-oriented_architecture

Sequence diagrams:

http://nl.wikipedia.org/wiki/Unified_Modeling_Language

Dogfooding:

http://en.wikipedia.org/wiki/Eating_your_own_dog_food

Learning REST API

http://www.restapitutorial.com/

http://toc.oreilly.com/2013/02/a-publishers-job-is-to-provide-a-

good-api-for-books.html

http://net.tutsplus.com/tutorials/other/a-beginners-

introduction-to-http-and-rest/


HTTP design http://www.ietf.org/rfc/rfc2616.txt

App testing http://www.kohl.ca

HTTP Header elements http://themayesfamily.com/blogs/b/2011/05/rest-client-for-

firefox-sample-post-request/

http://www.ietf.org/rfc/rfc2616.txt

http://themayesfamily.com/blogs/b/2011/05/rest-client-for-firefox-sample-post-request/













Documents

Testing API behind Mobile Aap · 3. Integration testing Internal API with complete backend No regression impact of other projects 4. Acceptance testing External integration API with