Distributed QoS Evaluation for Real-World Web Services

Zibin Zheng, Yilei Zhang, and Michael R. Lyulyu@cse.cuhk.edu.hk

July 07, 2010

Department of Computer Science & EngineeringThe Chinese University of Hong Kong

Hong Kong, China

ICWS 2010, Miami, Florida, USAJuly 05 - 10, 2010

2

Outlines

• Introduction

• Distributed Evaluation of Web Services

• How to Use the Datasets

• Conclusion and Future Work

1. Introduction

44

IntroductionWeb applications are becoming more and more important!

55

Introduction• The age of Web 2.0

– Web pages and Web services

• Web services are Web APIs that can be accessed over a network and executed on remote systems.– Open standards– Interoperability

6

Quality-of-Service

• Quality-of-Service (QoS): Non-functional performance. – User-independent QoS properties.

• price, popularity.• No need for evaluation

– User-dependent QoS properties. • failure probability, response time, throughput.• Different users receive different performance.

7

QoS-Driven Approaches

• Web service selection

• Web service composition

• Fault tolerant Web services

• Web service ranking

• Web service recommendation

Limited real-world Web service QoS datasets for experimental studies

8

Real-world Web Service Evaluation

• Obtain 21,358 publicly available Web services from the Internet.

• WS invocation and evaluation:– 235,262,555 lines of Java codes.

• Two large-scale distributed evaluations are conducted and first hand experiences are provided.– Dataset 1: 150 users * 100 Web services– Dataset 2: 339 users * 5825 Web services

2. Distributed Evaluation of Web services

10

Location Information

• 21,358 Web services from 89 countries. • The top 3 countries provide 55.5% of the obtained Web s

ervices.– United States: 8867 Web services, – United Kingdom: 1657 Web services, – Germany: 1246 Web services

11

Obtaining Web Service Address

• Web service portals or directories – xmethods.net, – webservicex.net,

– webservicelist.com,

• Web service searching engines– seekda.com, – esynaps.com,

• Obtain 21,358 addresses of WSDL files

12

WSDL File Infomation

13

Java Code Generation

• Axis 2 to generate Java codes for 13,108 Web services. • Totally 235,262,555 lines of Java codes are produced.

14

Dataset 1: Failure Probability

15

Overall Performance

• Mean of failure probability

• Standard deviation of failure probability

16

Overall Performance

• Average failure probabilities of all of the 100 Web

services and all the 150 service users are larger th

an 0.

• The standard deviation first becomes larger with th

e increase of mean and begins to decrease after a

certain threshold.

• The standard deviations are large, indicating that p

erformance of the same Web service observed by

different service users can vary widely.

17

Failure Types(1) Web service invocations can fail easily.

(2) Providing reliable Web services is not enough for building reliable service-oriented system.

(3) The Web service invocation failures are unavoidable in the unpredictable Internet environment; therefore, service fault tolerance approaches are becoming important.

(4) Service fault tolerance mechanisms should be developed at the client-side.

18

Dataset 2: Response-time & Throughput

• Each service user makes one invocation on each Web services.

19

Overall Response-Time• Web services with poor average response time performance tend to have la

rge performance variance to different users.

• Large response time of a Web service can be caused by the long data transferring time or the long request processing time at the server-side.

• Influenced by the client-side network conditions, different service users observe quite different average response time performance on the same Web services.

20

Overall Throughput

• Influenced by the poor server-side network conditions, there is a small part of Web services providing very poor average throughput performance (<1 kbps).

• Service users with large average throughput values are more likely to observe large throughput performance variance on different Web services.

3. How to Use the Datasets?

22

Web Service Recommendation

Zibin Zheng, Hao Ma, Michael R. Lyu, Irwin King, “WSRec: A Collaborative Filtering based Web Service Recommender System”, ICWS2009.

23

Fault Tolerant Web Services

Zibin Zheng and Michael Lyu, “A QoS-Aware Fault Tolerant Middleware for Dependable Service Composition ”, DSN2009.

Global constraint: Success-rate of the whole service plan > 99%.

Stateless Web services

Stateful Web services

24

More Research on the Datasets

• Web service selection.

• Web service search.

• Web service ranking.

• Other QoS-driven approaches of Web services.

4. Conclusion and Future Work

26

Conclusion and Future Work

ConclusionDistributed evaluation of Web servicesDataset 1: 150 * 100, failure probabilityDataset 2: 339 * 5825, response-time and throughput

Future work Investigating more QoS properties Incentive mechanisms for collecting user dataEvaluating more Web services from more locations

Thank you!

Web service QoS datasets: http://www.wsdream.net