Deployed Semantic Services for the Common User of the Web:

A Reality Check

Matthias Klusch and Xiguo ZhingGerman Research Center for Artificial Intelligence

Stuhlsatzenhausweg 3, 66121 Saarbruecken, Germanyklusch@dfki.de

Abstract

Where are all the semantic Web services today? Inthis paper, we report on quantitative results of search-ing the surface Web and the prominent citeseer archiveas one element of the deep Web for publicly accessiblesemantic Web service descriptions written in OWL-S,WSML, WSDL-S or SAWSDL by means of the special-ized meta-search engine Sousuo 1.6.

1 Introduction

Regarding the impressive efforts, advances, and in-vestments in semantic Web technology during the pastdecade, the common user of the Web might expect areasonably fair number of publicly accessible semanticservices at her fingertips when searching for them to-day. However, according to a focussed search for suchservice descriptions we conducted throughout the year2007 with a specialized meta-search engine Sousuo thisdoes not seem to be the case. We found not morethan about one hundred semantic service descriptionsin prominent formats like OWL-S, WSML, WSDL-Sand SAWSDL in the surface Web as well as in the scien-tific archive citeseer. This quantity appears tiny com-pared to, for example, the more than half million RDFsources indexed by the semantic Web search engineSwoogle, and several hundreds of validated Web ser-vice descriptions in the standard format WSDL foundby Sousuo 1.6 in the Web.

Of course, one could argue that this low number ofpublic semantic Web services comes at no surprise fortwo reasons. First, semantic Web service technologywith a standard description language, SAWSDL, an-nounced just recently (in August 2007), is still consid-ered immature by both academia and industry whichprovides insufficient common ground supporting its ex-

ploitation by potential end users. Independent fromthis (WSDL became official recommendation of theWeb Consortium only few months before SAWSDL,in June 2007), one could have expected the massiveresearch and development activities of the communityaround the globe in the past ten years to have produceda considerable amount of publicly visible semantic Webservice descriptions that are not merely stored in inter-nal project repositories.

Second, one might argue that it is not clear whetherthe surface Web and academic publications are theright place to look for semantic Web services, asmany of them would be intended for internal or inter-enterprise use but not visible for the public. Thoughthis is one possible reason of the low quantities reportedabove, there is no experimental evidence in favor of, oragainst this claim yet. In any case, this merely quan-titative result might motivate the community to investmore into raising the public awareness and taking upof developed semantic business service applications bythe common Web user.

In our initial search experiment we were particularlyinterested in the following: How many semantic Webservice descriptions are actually accessible to everyonesearching the Web for them? How many of these de-scriptions are written in the standard format SAWSDL,and the non-standard but prominent ones like OWL-S and WSML? What is the distribution of their geo-graphic locations and application domains? How manyof these semantic service descriptions are syntacticallyvalid? Finally, how many links to semantic Web ser-vice descriptions can only be found in academic publi-cations such as those in the prominent scientific archiveciteseer as one element of the deep Web?

In the remainder of this paper, we summarize ourquantitative experimental results. 1 In section 2, we

1Please note that in this first experiment, we searched for se-mantic Web service descriptions independent from whether they

Figure 1. General architecture of Sousuo.

briefly describe the meta-search engine Sousuo 1.6 withwhich the initial search has been performed togetherwith its testing environment. The performance of boththe search engine and its topic crawler followed by thesearch results are provided in section 4, while we con-clude our report in section 5.

2 Meta-search engine Sousuo

The purpose of the specialized meta-search engineSousuo for semantic Web services is to search the sur-face Web and the scientific archive citeseer for semanticWeb service descriptions in OWL-S, WSML, WSDL-Sand SAWSDL.

2.1 Architecture

The general architecture of Sousuo is shown in figure1.

Overview. Users may select any combination of thefollowing search methods of Sousuo to search for se-mantic Web services.

• Meta-search (MS) through most prominent searchengines Google with A9,

• Sousuo’s own focussed topic crawler (TC) basedon the WebSphinx crawler,

• Inverse ontology based search (IOS) via Swoogle,and

• Full text scientific archive search (FTAS).

Sousuo considers returned links to syntacticallyvalid service descriptions relevant. For validation,Sousuo uses the validators of publicly available OWL-SAPI, WSMO4J API, and SAWSDL4J API. Dependingon the validation result, it determines the relevance

are grounded in deployed Web services in WSDL, or not. Infact, a comparative analysis of world widely deployed Web ser-vices and semantic Web services is out of the scope of this paper.We are currently working on Sousuo 2.0 to support such furtherexperiments.

ranking score of each link, that is a score of 1.0 if theservice description is syntactically valid, 0.5 if valida-tion failed due to minor syntax errors, and 0 else. Val-idation is complemented by checking whether the linkhas been already stored in the local database (OpenBerkeley XML database[4]).

This database can be queried and evaluated by theuser according to the distribution of geographic loca-tions, description formats, domains and categories ofsemantic Web services, as well as the coverage of thetotal result returned for any combination of the dif-ferent search methods. Sousuo 1.4 also informs aboutthe actual performance of both its topic crawler andthe whole meta-search engine. Sousuo has been imple-mented in Java and is publicly available through thesoftware portal semwebcentral.org dedicated to seman-tic Web software.

Meta-search and topic crawler. The meta-searchof Sousuo is restricted to querying Google, Swoogle,and A9 through their API with predefined and usergiven search keys. Predefined search keys focus on linksto files of type, for example, filetype:{wsdl sawsdl, wsmlwsml, owls service, owls profile, owl jp, owl kr, owl tw,owl cn, owl service Profile, owl profile, wsdl annotation}. This is complemented by a simple focussed topiccrawler which performs a recursive depth-first searchtaking the continuously growing set of (initially given)links in the local database as root and base set, andterminates with a time-out or given maximum of searchdepth reached. Validation, ranking, and redundancychecking of found links are as described above.

Inverse ontology based search. This searchmethod is looking for services that reuse ontologies im-ported by services that have been already located bySousuo. For this purpose, Sousuo first checks for eachlink to a service stored in the database the assigned setof individual ontologies required to understand its se-mantics. It then searches for service descriptions thatcontain one or multiple of these selected ontology linksthrough respective queries to Swoogle, A9 and Google.The intuition behind this inverse ontology based searchis that semantic Web services share ontologies but maynot be fully indexed.

Full text archive search. Sousuo queries the scien-tific archive citeseer via its API for references to pa-pers on semantic Web services, retrieves the respec-tive documents in the answer set formatted in pdf, andscans each of them for embedded relevant links to SWSdescriptions. These links then get validated, ranked,

2

checked for redundancy, and stored in the local data-base by Sousuo as mentioned above.

Querying the local database. Finally, Sousuo al-lows the user to search for relevant semantic Webservices in its local database, open Berkely XML ofOracle, by means of full text and structured XMLbased search. SOusuo is available as open source athttp://projects.semwebcentral.org/projects/sousuo/

2.2 Testing environment

Search period and hardware. We ran our searchexperiment from January 1, 2007, to July 25, 2007, No-vember 2007, and January to February 2008, executingSousuo once every two weeks over 24 hours on a note-book with CPU Intel Core 2 Duo 2.0GHZ, 2GB RAM,and LAN 10 Mbit/s access to the Internet.

Search space and index. Sousuo does not searchany surface Web directory such as yahoo. In fact, ourrespective pre-experiments showed that the resultinganswer sets returned by the directories were alreadycovered by the one produced by Sousuo 1.6. However,we are currently working to enlarge the search space ofSousuo version 2.0 by incorporating specialized searchengines that claim to provide access to parts of thedeep Web such as clusty, intute, and infomine with anopen, non-commercial API for inquiries.

The search space of Sousuo 1.6 is the union of theindices of queried search engines, the discovered realmof the focussed topic crawler plus the index of the sci-entific archive citeseer. This size is, in general, im-possible to determine accurately due to the privacy ofinformation on size, redundancy, and coverage. How-ever, the size of the index of Google is estimated with11.3 billion links2, while the size of the Swoogle indexand citeseer archive is estimated with 1.7 million 3, and767,558 links, respectively. Besides, our focussed topiccrawler explored 11.2 million links in total during theexperiment. With an admittedly speculative 90% of anoverlap of the latter indices with the Google index, thesearch space of Sousuo might be estimated with 13 bil-lion pages. The current size of Sousuo’s index equalsthat of its total answer set for the whole search experi-ment which amounts to 1439 non-redundant, validatedlinks stored in the local database of Sousuo.

2See http://www.linksandlaw.de/news234-indexgroesse.htm3See http://swoogle.umbc.edu/index.php?option=com-

swoogle-stats

3 Performance

As the real relevance set of semantic Web servicesin the Web is unknown, and impossible to deduce fromneither the set of crawled pages nor the answer setsof particular sources queried, we approximate the clas-sical performance measures of precision and recall forSousuo’s topic crawler by means of the so called tar-get recall and target precision as defined in figure 2according to [5].

Figure 2. Target precision and recall.

3.1 Performance of focussed topic crawler

Figure 3 shows the average target precision and re-call of the focussed topic crawler of Sousuo. It exploredaround 11.2 million links in total with fairly reasonablethroughput of 46 links per minute during the experi-ment. Regarding its focussed search the target preci-sion is comparably fair enough as well [5].

3.2 Performance of Sousuo

For measuring the precision of the search engingeSousuo, we determine the classical ratio between thesize of the intersection of its answer set AS with therelevance set RS and the size of AS. The answer setof Sousuo equals the set of valid links taken from thosereturned by its topic crawler, the search using Googleand A9 API which answer set is limited to 1k, respec-tively, 10k links per day, the inverse ontology basedsearch via Swoogle, and the full text search throughciteseer. The relevance set, however, is restricted to thesubset of (manually determined) relevant links of thetotal union of answer sets produced during the searchperiod.

3

Figure 3. Target precision and recall ofSousuo’s focussed topic crawler.

Figure 4 displays the fairly high average precisionand recall of Sousuo while its target precision and recallis shown in figure 5.

4 Experimental results

The total number of semantic Web services in OWL-S, WSML, WSDL-S and SWASDL including the testcollections amounts to 1439 of which only about fourpercent (65 services) are available outside these collec-tions in the surface Web and through citeseer. Figure6 shows the number of relevant links found by each ofthe individual search methods of Sousuo without re-dundancy checking and test collections OWLS-TC2 [2]and SWS-TC [3]. These quantities did not change as aresult of our final search run in December (67 semanticWeb services, and 600 non-redundant WSDL servicesfound in December).

Meta-Search Citeseer TC IOSMeta-Search 35 4 6 10

Citeseer 4 11 4 2TC 6 4 29 8IOS 10 2 8 20

Table 1: Answer sets with links to semanticWeb service descriptions (in OWL-S, WSML,

Figure 4. Local precision and recall ofSousuo.

Figure 5. Target precision and recall ofSousuo.

WSDL-S and SAWSDL) returned by Sousuofor individual search methods.

The overlapping of answer sets from individualsearch methods of Sousuo for those services not in-cluded in the above mentioned test collections is sum-marized in table 1. It shows, for example, that thefull text archive search returned valid links to semanticWeb services in the archive citeseer that were not re-turned by Google and A9, and a few that have not beenreturned by any other method. Despite its functionalsimplicity, the same result holds with our focussed topiccrawler (TC). This is in contrast to the inverse ontol-ogy based search (IOS) which answer set is completelycovered by those of the other search methods.

4.1 Support of semantic Web service for-mats

Figure 7 shows the quantitative distribution of se-mantic Web service descriptions in prominent formats,

4

Figure 6. Number of relevant links found byindividual search methods of Sousuo.

that are OWL-S, WSML, WSDL-S and SAWSDL,without counting those in the test collections, whilefigure 8 shows its relation to the number of found Webservice descriptions in WSDL (.wsdl) that are syntac-tically valid.

Given the historic evolution of semantic Web servicetechnology, and its reasonably fair software support to-day, it comes at no surprise that the quantities of se-mantic service descriptions in OWL-S still outnumberthe considered alternatives. Remarkably, there are lesspublic WSML services than for WSDL-S and SAWSDLtogether. Though SAWSDL became a proposed rec-ommendation by the Web Consortium just recently,its software support world wide still appears compara-tively negligible - which might rapidly change in nearterm in response to its standardisation.

Apart from two rather medium sized semanticWeb service retrieval test collections for OWL-S,that are the OWLS-TC2 [2] and the SWS-TC[3],we did not find any other retrieval test collectionin the Web. Please note that a mere collection ofservices such as the one available for SAWSDL athttp://www.w3.org/2002/ws/sawsdl/ is not sufficientfor this purpose, since it would have to be extendedwith set of queries and respective relevance sets.

Figure 7. Distribution of semantic Web ser-vice formats OWL-S, WSML, WSDL-S and thestandard SAWSDL.

4.2 Geographic distribution

Figures 9 and 10 provide an overview of the geo-graphic distribution and locations of the publicly ac-cessible semantic Web services. Regarding the historyof the semantic Web vision, in particular the early jointstart between researchers in the US and the EU onDAML+OIL, OWL and OWL-S, as well as the massivefunding of WSMO related projects in this field by theEuropean Commission, it might not come at a surprisethat, according to the quantities reported, the domainappears clearly dominated by the US and Europe whilebeing remarkably close to each other.

However, what surprised us most is that, thoughmajor projects in the area exist, we could not find anyvalid semantic Web service description published in therest of the world publicly, in particular the Asia andPacific rim. Additional personal communication withfew selected research groups at universities in these re-gions revealed that, if semantic Web service descrip-tions do exist at their site, the public retrieval fromspecific project related repositories is prohibited, henceinvisible to any search engine. In general, we appreci-ate any reference by the interested reader to publiclyaccessible semantic Web services in one of the consid-ered formats, in particular if published in the abovementioned geographic regions.

4.3 Internet domains and business cate-gories

Figures 11 and 12 show the distribution of the do-main and business category of found services outside

5

Figure 8. Support of Web services in WSDLVs. semantic Web services.

Figure 9. Geographic distribution of semanticWeb services.

the test collections. Business and travel are the mostcommon categories, followed by finance and education.

In compliance with the prevalent geographic distrib-ution and location of semantic Web service links in theUS and the EU, the majority of links from Internet do-mains devoted to commerce (.com, 8 %), organisational(.org, 28%) and educational institutions (.edu, 15%) ishosted in these world regions. Remarkably, most ofthese semantic Web services publicly accessible in theEU are located in the UK.

The most common business domain for semanticWeb services according to their naming or statementin the profiles are business (16%) and travel (17%), fol-lowed by education (11%), finance (11%), and govern-ment (6%). One third of found semantic Web servicelinks, however, belongs to a variety of other domains

Figure 10. Geographic locations of semanticWeb service providers.

Figure 11. Internet domains of semantic Webservices.

of smaller size such as sports and health.

5 Conclusions

The preliminary results of our experimental search-ing for semantic Web services in the surface Web byuse of a specialized meta-search engine is rather de-sillusioning. We found not more than around 1500 in-dexed semantic service descriptions in OWL-S, WSML,WSDL-S or SAWSDL in the Web, of which only aboutfour percent are located outside special test collectionslike the OWLS-TC2. This quantity appears tiny com-pared with the sheer volumes of estimated thirty billionand one million indexed resources in the Web, respec-tively, semantic Web encoded in RDF and OWL.

As mentioned above, the reported preliminary ex-perimental result does not reflect the strong researchefforts carried out in the SWS domain world wide in

6

Figure 12. Business categories of semanticWeb services.

the past few years, independent from the status of ma-turity of SWS technology and implied low adoption byend users yet. Nevertheless, the result might encouragethe community as a whole to increase its visibility andawareness to the common Web user outside the com-munity and savvy project teams also by publishing asignificant number of SWS show cases in the surfaceWeb.

Although one could have expected these results, inparticular the majority of semantic Web services beingpublished in protected internal project repositories andother sites of the deep Web[6], there was no experimen-tal evidence available in favor of this claim or against.On the other hand, there still is plenty of space left tosearch both the surface and the deep Web: The searchspace of Sousuo in its current version is limited to onlyfew selected indices of freely accessible and prominentsearch engines with open API, that are Google, A9,Swoogle, and the scientific archive citeseer.

However, raising awareness by a significant numberof show cases seems senseless if it is not complementedby community efforts to equip end users with easy touse software support for building, sharing and reusingsemantic Web services. Unfortunately, this is supportis missing either, despite the variety of SWS relatedsoftware available at relevant open source softwareportals such as semwebcentral.org and sourceforge.net.Though, it remains to be seen whether and to whatextent these tools or the experience of developing themcan be exploited to support the standard SAWSDLfor which, apart from the project internal MWSDIinfrastructure and Lumina search engine, no publicsoftware support exists yet. Sousuo is available athttp://projects.semwebcentral.org/projects/sousuo/

Ongoing work is on improving the search methods ofSousuo for an updated version 2.0.

References

[1] L. Ding, T. Finin, A. Joshi, R.S. Cost, J. Sachs:Swoogle: A Semantic Web search engine and meta-data engine. Proceedings of the Thirteenth ACMConference on Information and Knowledge Man-agement, 2004.

[2] OWLS-TC2:http://projects.semwebcentral.org/projects/owls-tc/.

[3] SWS-TC:http://projects.semwebcentral.org/projects/sws-tc

[4] Oracle’s Open Berkeley XML Database:http://www.oracle.com/database/berkeley-db/xml/index.html

[5] G. Pant, P. Srinivasan, F. Menczer: Crawling theWeb. In M. Levene and A. Poulovassilis, eds.: WebDynamics, Springer, 2004.

[6] C. Sherman, G. Price: The Invisible Web: Un-covering Information Sources Search Engines Can’tSee. Cyberage Books, 2001. The Deep Web:http://en.wikipedia.org/wiki/Deep-web.

[7] SouSuo 1.3:http//:projects.semwebcentral.org/projects/sousuo/.

[8] Swoogle:http://swoogle.umbc.edu/

7