18
Open Market or Ghost Town? The Curious Case of OpenBazaar James E. Arps and Nicolas Christin CyLab, Carnegie Mellon University, USA {jarps,nicolasc}@cmu.edu Abstract. OpenBazaar, a decentralized electronic commerce market- place, has received significant attention since its development was first announced in early 2014. Using multiple daily crawls of the OpenBazaar network over approximately 14 months (June 25, 2018–September 3, 2019), we measure its evolution over time. We observed 6,651 unique participants overall, including 980 who used Tor at one point or another. More than half of all users (3,521) were only observed on a single day or less, and, on average, only approximately 80 users are simultaneously active on a given day. As a result, economic activity is, unsurprisingly, much smaller than on centralized anonymous marketplaces. Furthermore, while a majority of the 24,379 distinct items listed seem to be legal of- ferings, a majority of the measurable economic activity appears to be related to illicit products. We also discover that vendors are not always using prudent security practices, which makes a strong case for imposing secure defaults. We conclude that OpenBazaar, so far, has not gained much traction to usher in the new era of decentralized, private, and le- gitimate electronic commerce it was promising. This could be due to a lack of user demand for decentralized marketplaces, lack of integration of private features, or other factors, such as a higher learning curve for users compared to centralized alternatives. Keywords: Measurement; peer-to-peer systems; electronic commerce. 1 Introduction OpenBazaar, originally called DarkMarket, is a peer-to-peer electronic commerce platform that has attracted significant media attention [11, 14, 15]. It was first en- visioned in 2014 as a response to the government takedown of the Silk Road [6] online anonymous (“darknet”) marketplace [11]. However, OpenBazaar devel- opers quickly pivoted away from the darknet marketplace space and toward a decentralized e-commerce platform. The project raised several million dollars in seed funding through a startup called OB1 [14, 15]. While the default OpenBazaar search engine is developed by OB1 and filters undesirable items such as narcotics [16], the OpenBazaar project itself is open- source and the developers cannot prevent vendors from listing such items using the platform.

Open Market or Ghost Town? The Curious Case of ...Open Market or Ghost Town? The Curious Case of OpenBazaar 5 [peer ID]), and add them to a list, recursively continuing until we no

  • Upload
    others

  • View
    3

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Open Market or Ghost Town? The Curious Case of ...Open Market or Ghost Town? The Curious Case of OpenBazaar 5 [peer ID]), and add them to a list, recursively continuing until we no

Open Market or Ghost Town?The Curious Case of OpenBazaar

James E. Arps and Nicolas Christin

CyLab, Carnegie Mellon University, USA{jarps,nicolasc}@cmu.edu

Abstract. OpenBazaar, a decentralized electronic commerce market-place, has received significant attention since its development was firstannounced in early 2014. Using multiple daily crawls of the OpenBazaarnetwork over approximately 14 months (June 25, 2018–September 3,2019), we measure its evolution over time. We observed 6,651 uniqueparticipants overall, including 980 who used Tor at one point or another.More than half of all users (3,521) were only observed on a single dayor less, and, on average, only approximately 80 users are simultaneouslyactive on a given day. As a result, economic activity is, unsurprisingly,much smaller than on centralized anonymous marketplaces. Furthermore,while a majority of the 24,379 distinct items listed seem to be legal of-ferings, a majority of the measurable economic activity appears to berelated to illicit products. We also discover that vendors are not alwaysusing prudent security practices, which makes a strong case for imposingsecure defaults. We conclude that OpenBazaar, so far, has not gainedmuch traction to usher in the new era of decentralized, private, and le-gitimate electronic commerce it was promising. This could be due to alack of user demand for decentralized marketplaces, lack of integrationof private features, or other factors, such as a higher learning curve forusers compared to centralized alternatives.

Keywords: Measurement; peer-to-peer systems; electronic commerce.

1 Introduction

OpenBazaar, originally called DarkMarket, is a peer-to-peer electronic commerceplatform that has attracted significant media attention [11,14,15]. It was first en-visioned in 2014 as a response to the government takedown of the Silk Road [6]online anonymous (“darknet”) marketplace [11]. However, OpenBazaar devel-opers quickly pivoted away from the darknet marketplace space and toward adecentralized e-commerce platform. The project raised several million dollars inseed funding through a startup called OB1 [14,15].

While the default OpenBazaar search engine is developed by OB1 and filtersundesirable items such as narcotics [16], the OpenBazaar project itself is open-source and the developers cannot prevent vendors from listing such items usingthe platform.

Page 2: Open Market or Ghost Town? The Curious Case of ...Open Market or Ghost Town? The Curious Case of OpenBazaar 5 [peer ID]), and add them to a list, recursively continuing until we no

2 James E. Arps and Nicolas Christin

Interestingly, the takedown of the Silk Road marketplace failed to curb illicitactivity on centralized anonymous marketplaces. On the contrary, Silk Road’ssuccessors have been thriving [10,19,22,31]: As of 2017, the leading marketplaceswere grossing hundreds of millions of dollars per year [10], with no decline insight, despite adversarial events such as law enforcement operations and “exitscams,” in which some marketplace operators abruptly shut down their servers,taking with them any money left in escrow on the platform.

Given that OpenBazaar, thanks to its decentralized architecture, could pre-vent or mitigate most of these adversarial events, and given the clear economicdemand for such services, we would expect thriving economic activity on a peer-to-peer platform such as OpenBazaar. This is what we measure in this paper.

For approximately 14 months (June 25, 2018–September 3, 2019), we havebeen crawling the OpenBazaar network on a near-daily basis to get a sense of thenumber of participants and the size of the overall network. By scraping listingsoffered by each vendor and the associated feedback left by buyers (similar totechniques used in related work, e.g., [6, 19, 31]), we also estimate the economicactivity taking place on OpenBazaar. Finally, we can examine the extent of users’mindfulness about operational security when trading in illicit or illegal products.

Our findings are rather sobering. While there is undeniable network activity,and a reasonably large number of items available on the OpenBazaar network (atleast 24,379 distinct listings observed during our measurement period), economicactivity remains modest, and appears to be mostly generated by illicit productsales. Furthermore, many vendors appear to misunderstand the security guar-antees offered by OpenBazaar or fall prey to some configuration defaults, andpublicly reveal some potentially compromising information about themselves:Nearly a fifth of the vendors that use OpenBazaar over the Tor network [9] haveaccidentally revealed their IP address at some point in time.

The remainder of this paper is organized as follows. We discuss relevantbackground on OpenBazaar in Section 2, describe our collection methodology inSection 3, and present our results in Section 4. We explain how our work differsfrom related efforts in Section 5, and conclude in Section 6.

2 The OpenBazaar System

Imagine Alice wishes to join the OpenBazaar network and purchase an item.First, she downloads the OpenBazaar client from https://openbazaar.org, whereshe is presented with a “Get Started” page which allows her to edit her profile.Since all users on the platform can act as both buyers and vendors, she may alsobegin creating item listings if she were so inclined. Crucially, if Alice proceedswith the default setup process even once, her public IP address will be leaked tothe network. Indeed, using Tor with OpenBazaar requires an extra configurationstep, which is not immediately evident to first-time users. While the OpenBazaarclient prompts the user if it detects that Tor is already running on the user’smachine, we do not expect this to be common for most users.

Page 3: Open Market or Ghost Town? The Curious Case of ...Open Market or Ghost Town? The Curious Case of OpenBazaar 5 [peer ID]), and add them to a list, recursively continuing until we no

Open Market or Ghost Town? The Curious Case of OpenBazaar 3

Once Alice creates her account, she can explore the listings on the platform.OpenBazaar’s peer-to-peer backbone is the InterPlanetary File System (IPFS),an open-source protocol which allows content storage and addressing across anetwork of distributed nodes [3]. IPFS can be used as an alternative to a tra-ditional client/server architecture – instead of clients requesting data from aserver, here, a node provides a hash of the content it is requesting to its peers,who are either able to provide the requested content or query their own peersfor its location.

Since IPFS does not support a centralized repository of item listings, searchis implemented through a third-party search engine. At the time of writing,the only search engine enabled by default in the OpenBazaar GUI is createdby the OB1 developers – other known search engines, such as SearchBizarreand a service operated by BlockStamp, require a manual addition to the clientwhich may not be intuitive for new users. These search engines operate theirown crawlers which travel the network to index users and listings, where theirresults can then be queried by users who have added their search engine to theOpenBazaar client. Existing third-party search engines exist mainly to providean uncensored view of listings on the platform, as OB1’s search engine does notinclude listings for illicit products.

Listings are stored over IPFS using a Distributed Hash Table (DHT), wherenodes store the hash of a given user, item, or feedback along with their locationon the network. If a node visits a vendor page or views one of its listings, it alsostores the information it receives locally for a certain period of time, allowingfor an added level of redundancy across the network. This enables vendors notto be perpetually logged in. The system has been designed with the intentionthat individual listings will be re-seeded for about a week after their owner waslast seen online [23].

When Alice clicks on a listing she is interested in, her client attempts tofetch the relevant information from her peers by querying the DHT for the itemhash. If Bob, the store owner, is online, Alice will often be able to receive thelisting directly from Bob – otherwise, Bob’s data is typically seeded by otherIPFS peers for a set period of time. If Alice can retrieve the item hash, she isbrought to a listing page which contains information about the item such as itsprice, description, shipping details, preferred cryptocurrency payment method,and pictures of the item. Currently, OpenBazaar supports payment in Bitcoin,Bitcoin Cash, Litecoin, and Zcash [2]. Zcash required a full installation of itsbinary for more than half of the duration of our study, which raises usabilityconcerns; Litecoin was added only in the second half of our measurement interval.

When purchasing an item through OpenBazaar, Alice may opt to directlysend cryptocurrency to Bob to pay him, or to use a moderator service to holdthe funds in escrow until the item is received. Moderators are OpenBazaar userswho volunteer to mediate disputes between other users and decide the eventualdistribution of funds using multisignature transactions, and usually do so in ex-change for a small fixed-percentage fee. Moderators receive community feedback,but individual moderators are chosen by the vendor at the time of listing cre-

Page 4: Open Market or Ghost Town? The Curious Case of ...Open Market or Ghost Town? The Curious Case of OpenBazaar 5 [peer ID]), and add them to a list, recursively continuing until we no

4 James E. Arps and Nicolas Christin

ation. Regardless, Alice sends her payment details and shipping information toBob through the platform and the sale proceeds as it would on existing servicessuch as eBay.

3 Data Collection Methodology

We turn now to our data collection methodology, outlining our objectives first,before discussing the mechanisms we use and their limitations.

3.1 Objectives

Contrary to traditional dark-web marketplaces [31], data collection on Open-Bazaar is encouraged rather than discouraged, which eschews most of the con-straints one faces when attempting to scrape a network stealthily. In fact, sinceOpenBazaar is not regulated by any central authority, our node cannot be easilybanned from the network. While it could be blocked by individual nodes, we re-ceived no indication that this happened at any point during the study. Therefore,our primary focuses were on data completeness and collection speed.

We elected to scrape data ourselves, despite the existence of several searchproviders for OpenBazaar. Some OpenBazaar search providers service the Open-Bazaar GUI (i.e., they return JSON parseable by the OpenBazaar client GUI),while others simply serve their results on public-facing webpages. Regardlessof the method used, we noticed some search engines – in particular, the mostpopular search engine, run by OB1 at http://openbazaar.com – do not displayresults for illicit products. As a result, we needed to build our own OpenBazaarcrawling infrastructure to obtain uncensored data.

Ethics of data collection. The data we collect are volunteered by OpenBazaarparticipants. In particular, listings, descriptions, and user feedback are all madepublicly available for everyone to see. We previously referred to our InstitutionalReview Board (IRB) to determine whether data collected in related work [6,31]could be publicly reshared. Our IRB had opined this line of work was not human-subject research. The only difference with previous research is that here, we docollect IP addresses of other clients. But, in doing so, our crawler does not collectany information a regular OpenBazaar node would not collect for operationalpurposes. In short, like most peer-to-peer network measurement research (seee.g., [7]), our measurements do not put users at additional risk compared toparticipating in the peer-to-peer network in the first place.

3.2 Crawler design

Our crawler runs over Tor and leverages the OpenBazaar API [25], which is amodified version of the IPFS API. First, our crawler queries a list of its connectedpeers (GET http://localhost:4002/ob/peers/). For each of those peers, we re-trieve their closest connected peers (GET http://localhost:4002/ob/closestpeers/

Page 5: Open Market or Ghost Town? The Curious Case of ...Open Market or Ghost Town? The Curious Case of OpenBazaar 5 [peer ID]), and add them to a list, recursively continuing until we no

Open Market or Ghost Town? The Curious Case of OpenBazaar 5

[peer ID]), and add them to a list, recursively continuing until we no longer findany new peers (in our testing, this usually took between five and seven rounds).Using each peer’s unique user ID, we can then make separate API calls to retrieveall of their current listings (GET http://localhost:4002/ob/listings/[peer ID]) aswell as all reviews left for those listings. Users, items, and reviews are all uniquelyidentified within OpenBazaar by a 46-character alphanumeric hash, which allowsus to reliably track them across different scrapes. We log the approximate geo-graphic location of peers with public IP addresses with FreeGeoIP (now calledipstack, [1]).

The crawler makes use of Python’s Requests library [27], and the relativelysmall size of the network means that it is very fast (often completing in an houror less). However, to avoid too much redundancy and to reduce strain on theTor network, we chose to scrape the network once every two to four hours. Thisprovided sufficient coverage on a daily basis, and by leaving our OpenBazaarnode running while the scraper was not in use, we also contributed to the overallhealth of the network.

3.3 Potential limitations

0.6

0.8

1.0

0 25 50 75 100

Number of scrapes

Cov

erag

e

Fig. 1. Estimated coverage as a function of the number of scrapes.

Network coverage One risk of measuring a decentralized marketplace is thatour node could lack a complete view of the network, and it was not uncommonfor our scraper’s requests to the DHT to time out for certain items or users.Figure 1 depicts the relative coverage percentage of items seen on the networkduring April 2019. We observed 6,292 items on the network during this time,and saw 85% of them during the first two days (i.e., the first 24 scrapes), encoun-tering the rest over the following seven. The large number of scrapes required toobserve this proportion of the network is indicative of the high levels of churn we

Page 6: Open Market or Ghost Town? The Curious Case of ...Open Market or Ghost Town? The Curious Case of OpenBazaar 5 [peer ID]), and add them to a list, recursively continuing until we no

6 James E. Arps and Nicolas Christin

discuss in Section 4.1. Non-seller nodes (which make up the majority of networkparticipants) also have little incentive to leave their OpenBazaar clients openfor long periods of time, as they do not need their listings to be seeded by othernetwork participants. The diminishing returns shown by the curve suggest thatthere are few items remaining to be found by the crawler.

While the number of items we observe is less than the number reported bythe OB1 developers [5], we often found through manual inspection that manyitems returned in the later pages of a given search result performed through theirservice failed to load in our client. This is likely because the OpenBazaar protocolattempts to cache data from inactive nodes for approximately a week [23] beforethey are “forgotten” by the network. For example, if Alice owns a store anddo not log on for a few days, other nodes should still be able to access Alice’sstore for approximately a week because it will be cached collectively by the othernodes in the network. Since OB1’s search engine returns results indexed by itscrawlers and then stored on its own servers, it is likely that its results containstale listings that are no longer reachable on the network.

We therefore believe that our view of the network is typical of that seen byan average node; this motivates the need for our crawler to frequently scrape theentire network to increase coverage.

Economic coverage Unlike many other marketplaces, leaving a review is notmandatory after a purchase on OpenBazaar. We were unable to purchase anyitems ourselves as a part of this study due to our legal counsel’s concerns aboutinadvertently participating in money laundering, but we confirmed this with oneof the OpenBazaar developers. As a result, our sales numbers in Section 4.2 area lower bound, even if we assume complete coverage of the network.

Despite this caveat, we are confident that our results are useful for two rea-sons. First, social norms on anonymous marketplaces have proven quite strongover the years: even when leaving feedback is not mandatory, buyers often do so,especially if they plan on buying from the same seller again in the future [31].Soska and Christin’s analysis, based on feedback ratings [31], showed numbersvery close to those obtained externally through criminal complaints, when ven-dors were arrested or through marketplace takedowns, even when leaving feed-back was not actively enforced by all marketplaces. The importance of feedbackon those marketplaces was also evident when the original Silk Road changedthe way feedback was tallied (shifting from per-item feedback, to aggregate,per-vendor feedback) and quickly reversed course in the face of customer com-plaints [6].

Furthermore, our reported distribution of sales by category is likely valid.One could think that users purchasing illicit items on OpenBazaar would be lesslikely to leave a public review than users purchasing legal products. As we willsee, our results do not support this hypothesis.

Page 7: Open Market or Ghost Town? The Curious Case of ...Open Market or Ghost Town? The Curious Case of OpenBazaar 5 [peer ID]), and add them to a list, recursively continuing until we no

Open Market or Ghost Town? The Curious Case of OpenBazaar 7

4 Results

We next turn to our measurement results. We first describe network-level metricssuch as the size of the peer-to-peer network, or the underlying churn in IPaddresses we see participating in the network, before turning to discussing theeconomic activity that appears to take place on OpenBazaar. We then examinethe security and privacy precautions vendors take.

4.1 Network-level metrics

The OpenBazaar peer-to-peer network Over our entire measurement interval,we have observed 6,651 distinct network participants, using 6,116 distinct IPaddresses.

Table 1. OpenBazaar demographics. Users do not have any product listings; sell-ers have at least one active listing; active sellers have realized at least one documentedsale.

With Tor Without Tor Total

Users 743 4,487 5,230Sellers 197 1,057 1,254Active sellers 40 127 167

Total 980 5,670 6,651

Table 1 breaks down these participants into finer-grained demographics, dis-tinguishing between users, who do not list any product, and are therefore as-sumed to be solely browsing or buying items; sellers, who list at least one prod-uct, but do not have any documented sale – that is no one left any feedback forthem; and active sellers, who have received at least one piece of feedback. Wealso break down these participants between those who use Tor and those whodo not.

Figure 2 shows, for each day, the number of OpenBazaar hosts we have en-countered during our multiple scrapes on that day. The lighter curve denotes thefraction of hosts that are using the Tor network. We observe that the populationhas been relatively steady, at approximately 80 users simultaneously online onany given day throughout our measurement interval. The few downward spikesdenote measurement issues rather than network instability.

The gaps in the plot denote times during which our measurement infrastruc-ture was disabled, or otherwise unable to properly collect data.

The seemingly decreasing number of Tor users toward the end of the mea-surement interval might be a slight undercount. Through experimentation withour own test nodes, we discovered that OpenBazaar nodes in version 2.3.1 andhigher running over Tor, sometimes failed to appear in our crawls, despite being

Page 8: Open Market or Ghost Town? The Curious Case of ...Open Market or Ghost Town? The Curious Case of OpenBazaar 5 [peer ID]), and add them to a list, recursively continuing until we no

8 James E. Arps and Nicolas Christin

0

30

60

90

120

Jul 2018 Oct 2018 Jan 2019 Apr 2019 Jul 2019

Date

Num

ber

of h

osts

All Users

Tor Users

Fig. 2. Number of hosts on the OpenBazaar peer-to-peer network. Eachpoint is the cumulative number of different hosts seen over all measurements taken ona specific day.

online and reachable (that is, if one knew their node ID). This coincides, andmay be due to the backwards-incompatible [24] software upgrade on March 19,2019 (OpenBazaar 2.3.1), which may have affected some long-term participantsover Tor. Overall, compared to Table 1, this plot seems to indicate that the vastmajority of OpenBazaar users are actually rarely online. To better understandthe dynamics of the OpenBazaar network, we next turn to a survivability analy-sis. Similar to Christin [6], we estimate vendor “lifespan” by recording the timewe first saw a vendor, and the time we saw them last. They may have left andrejoined in the meantime—here we are looking at the vendor lifetime, regardlessof their transient activity. To account for measurement effects (e.g., vendors stillbeing present on the last day of measurement), Figure 3 depicts a Kaplan-Meierestimator [17] that shows the probability a given user seen on day 0 will be seenagain after x days, broken down by the categories defined in Table 1.

Churn is high among regular users: more than two thirds of them stay lessthan a day. Vendors, on the other hand tend to stay longer, especially vendorsthat have documented sales (i.e., that have received feedback). Roughly threequarters of all participants do not stay more than a week; a few users remainon the network throughout our measurement interval. Log-rank tests confirmthat visually striking differences between the survival curves for all of these usercategories are statistically significant (p < 0.0001). In short: vendors tend to belong-lived, while regular users are not, and usage of Tor is positively correlatedwith longer presence on the network.

Geographical considerations Figure 4 shows the geolocation of the IP addressesof participants that are not using Tor. OpenBazaar seems to be fairly interna-tional, with the usually observed concentration of users in North America and

Page 9: Open Market or Ghost Town? The Curious Case of ...Open Market or Ghost Town? The Curious Case of OpenBazaar 5 [peer ID]), and add them to a list, recursively continuing until we no

Open Market or Ghost Town? The Curious Case of OpenBazaar 9

++

++

+

+

+

+++

+++++ +

++ + + ++

+

+

+

+++++++++++ + + + + + + +0.00

0.25

0.50

0.75

1.00

0 100 200 300 400

Time

Sur

viva

l pro

babi

lity

++

++

++

Active seller (w/ Tor)

Active seller (w/o Tor)

Seller (w/ Tor)

Seller (w/o Tor)

User (w/ Tor)

User (w/o Tor)

Fig. 3. Survivability analysis of OpenBazaar users. Kaplan-Meier estimatorthat shows the probability a given user seen on day 0 will be seen again after x days,broken down for different types of users. Shaded areas indicate 95% C.I.

Europe. A couple of points with strange locations (e.g., Easter Island) suggestcertain participants use VPNs, some of which are known to advertise implausiblelocations that fool geolocation databases [33]. We generally do not observe mean-ingful differences between different types of users, even though Western Africa,India and Thailand/Malaysia feature a larger proportion than we expected ofactive sellers – again, it is hard to tell whether this could be due to VPN usage.

4.2 Economic activity

We next turn to a study of economic activity on OpenBazaar. We observed24,379 distinct item listings during our measurement intervals. The apparenthigh average ratio of listings per seller (≈40) is due to the ease of creatinglistings (including test listings) and to a few “power users” who have thousandsof listed items on the platform.

Item listing survivability We start with a survival plot in Figure 5 shows theprobability an item seen at time zero will still be available on the network x dayslater. The median item stays online for approximately three weeks. One quarterof all items disappears (i.e., are delisted) within a day, which further motivatesthe need to repeatedly crawl the network for completeness. A handful of itemswere present throughout our measurement interval. The “jumps” observed inthe graph correspond to a large number of items belonging to a given vendor

Page 10: Open Market or Ghost Town? The Curious Case of ...Open Market or Ghost Town? The Curious Case of OpenBazaar 5 [peer ID]), and add them to a list, recursively continuing until we no

10 James E. Arps and Nicolas Christin

● ●

●●

●●

●●

●●

● ●

● ●

●●

●●

●●

●●

●●

●●

●●●

● ●

●●

●●●

●●

●●●

●●●

● ●●

● Active Seller Seller User

Fig. 4. Geolocation of OpenBazaar users. Users are network participants; Sellersare users with at least one listing; Active sellers are sellers with at least one feedback.

being all delisted on the same day, presumably because the vendor node hadbeen offline for long enough to have its listings cleared from the IPFS cache.1

Overall, the survivability analysis paints a picture eerily similar to that ofthe early days of online anonymous marketplaces [6]: most items are very short-lived. As discussed in prior work [6, 31], short-lived items are usually indicativeof vendors holding low stocks, which in turn suggests that vendors operatesprimarily in the retail space, with small product quantities, and (usually) lowsales volumes.

Preliminary economic analysis We next examine economic activity on Open-Bazaar, across item categories. We initially attempted to feed each of the 24,379item listings we observed into the 16-category item classifier proposed by Soskaand Christin [31], who trained their item classification with listings from theAgora and Evolution marketplaces and showed very high (>98%) accuracy whenevaluating a priori unknown listings.

1 Item listings do not automatically disappear when an item is sold out. The sellerneeds to either delete the listing, or be offline for a long enough period of time forthe listing to stop being seeded by the network. We differentiate between the twocases by only counting visible reviews for an item as a sale and not consideringdisappearing listings as possible sales.

Page 11: Open Market or Ghost Town? The Curious Case of ...Open Market or Ghost Town? The Curious Case of OpenBazaar 5 [peer ID]), and add them to a list, recursively continuing until we no

Open Market or Ghost Town? The Curious Case of OpenBazaar 11

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

+ ++ ++ ++0.00

0.25

0.50

0.75

1.00

0 100 200 300 400

Time

Sur

viva

l pro

babi

lity

Fig. 5. Survivability analysis of OpenBazaar items. The plot is a Kaplan-Meierestimator that shows the probability a given item seen on day 0 will be seen againafter x days. The median lifetime of an item is approximately 22 days. The curve fullyoverlaps the very narrow 95% confidence interval.

We realized, however, that items classified in the “Miscellaneous” categoryrepresented a disproportionately high fraction of all listings, which led us tofurther break this category down into additional categories: Adult Toys, Art,Clothing, Jewelry, Print Media, and Souvenirs. We took 200 new items fromour OpenBazaar corpus, hand-labeled them,2 and added them five times (i.e.,an extra 1,000 items) to the original training set consisting of 62,989 labeleditems from Evolution and Agora. Our resulting classifier operates on 22 cate-gories. Table 2 shows very good performance metrics, with precision and recall,overall, being over 0.96. This is unsurprising, given that the modified classifieris very close to the original classifier from Soska and Christin – we merely added1,000/63,989≈1.6% of new training data to the original classifier. The support,here, is much larger than the number of items we observed in OpenBazaar, sincewe are evaluating with 10-fold cross-validation on the original dataset providedby Soska and Christin3 [31] and the OpenBazaar data. Certain categories (e.g.,Adult Toys) do appear very rarely, though. The overall support for the new cat-egories is fairly small (240 items); this again is unsurprising, as the imbalance intraining sets means that an item needs to closely resemble a training example tobe classified as one of the new sub-categories. In other words, our modified clas-sifier closely mimics what Soska and Christin used; in a few “obvious” cases, itwill manage to further identify a subcategory, but will do so very conservatively.

Next, we present the category breakdown for these 24,379 listings in Table 3.Simply looking at listing counts (columns 2 and 3), close to half of the listingsare in the “Misc.,” “Print Media,” and “Souvenirs” categories, which generally

2 A single researcher was tasked with this labeling, hence we do not report agreementnumbers.

3 The dataset is available from IMPACT, https://www.impactcybertrust.org.

Page 12: Open Market or Ghost Town? The Curious Case of ...Open Market or Ghost Town? The Curious Case of OpenBazaar 5 [peer ID]), and add them to a list, recursively continuing until we no

12 James E. Arps and Nicolas Christin

Table 2. Classifier performance.

Category Precision Recall F1-score Support

Adult Toys 1.00 1.00 1.00 3Art 1.00 1.00 1.00 24Benzos 0.97 0.98 0.97 21,132Cannabis 1.00 1.00 1.00 113,516Clothing 1.00 1.00 1.00 38Digital Goods 0.94 0.96 0.95 158,162Dissociatives 1.00 0.99 0.99 7,172Drug Paraphernalia 0.97 0.98 0.97 15,740Ecstasy 1.00 0.99 0.99 49,184Electronics 0.96 0.94 0.95 5,379Jewelry 0.76 1.00 0.87 13Misc 0.87 0.80 0.83 47,651Opioids 0.98 0.98 0.98 25,511Prescription 0.95 0.93 0.94 23,023Print Media 1.00 0.96 0.98 113Psychedelics 1.00 1.00 1.00 31,023Sildenafil 0.98 0.97 0.97 31,22Souvenirs 1.00 0.90 0.95 49Steroids 0.99 1.00 0.99 17,044Stimulants 0.98 0.99 0.99 55,555Tobacco 1.00 1.00 1.00 3,966Weapons 0.99 0.97 0.98 5,341

Total 0.96 0.96 0.96 582,761

denote legitimate items. The third largest category, “Digital goods,” containsa mix of legitimate (e.g., e-books and other guides) and illegitimate (porno-graphic website account passwords) items. In short, a majority of items listedon OpenBazaar shops appear to be for legal products.

However, looking at actual sales (columns 4 through 6) paints a very differentpicture. To compute sales for a given item listing, using the same technique asin related efforts [6, 31], we add up the price of the item to its total sales ateach time a feedback for a sale of that item is recorded. Feedback that wererecorded prior to our monitoring the OpenBazaar network are counted if thecorresponding item was still listed when we scraped the network, and feedbackwas still accessible. As noted before, reviews are not mandatory in OpenBazaar,so that the sales numbers presented are a lower bound. Even with this caveat,sales volumes appear to be very modest. We only count around $217,000 in totalsales over our measurement interval. By comparison, sales on Silk Road [6, 31]and AlphaBay [22], two of the largest online anonymous marketplaces, reachedhundreds of millions of dollars in revenue.

Looking a bit more carefully at the data revealed an interesting outlier: onevendor, P...A, appeared to be single-handedly responsible for 60% of all saleson OpenBazaar. In particular, that vendor had three pieces of feedback for a$36,000+ listing for a kilogram of cocaine, which should have accounted formore than $100,000 in sales by itself. When we manually inspected this specificvendor, we discovered that for most of their items, the feedback appeared to be

Page 13: Open Market or Ghost Town? The Curious Case of ...Open Market or Ghost Town? The Curious Case of OpenBazaar 5 [peer ID]), and add them to a list, recursively continuing until we no

Open Market or Ghost Town? The Curious Case of OpenBazaar 13

Table 3. Sales observed during our measurement interval. The values incolumns 4–6 exclude vendor P...A, whose feedback seems highly questionable, andlikely fake.

Category Listings Listings Sales Sales Sales(count) (%) (count, (USD, (%,

corrected) corrected) corrected)

Adult Toys 182 0.747 0 0 0.0Art 331 1.358 10 975 1.13Benzos 155 0.636 8 1,650 1.92Cannabis 1,910 7.835 318 22,450 26.1Clothing 1,881 7.72 30 375 0.437Digital Goods 2,701 11.079 139 1,312 1.53Dissociatives 21 0.086 2 1,050 1.22Drug Paraphernalia 829 3.4 13 224 0.261Ecstasy 243 0.997 10 4,895 5.69Electronics 1,906 7.818 23 2,389 2.78Jewelry 354 1.452 9 208 0.243Misc 6,796 27.876 333 4,028 4.69Opioids 223 0.915 17 2,207 2.57Prescription 289 1.185 23 808 0.941Print Media 4,902 20.107 6 123 0.144Psychedelics 242 0.993 125 17,653 20.5Sildenafil 45 0.185 41 792 0.921Souvenirs 791 3.245 60 3,587 4.17Steroids 69 0.283 2 11 0.0128Stimulants 242 0.993 24 20,835 24.2Tobacco 34 0.139 1 5 0.005Weapons 233 0.956 8 374 0.436

Total 24,379 100 1,202 85,954 100

fake: very closely clustered timestamps, all with highly positive ratings and unin-formative messages for highly priced items. This strongly suggested an attemptat manipulation by the vendor.4 While in centralized marketplaces, paddingfeedback with misleading information is prohibited, and frequently results inbanning the vendors engaging in such deceptive practices, the decentralized na-ture of OpenBazaar makes this kind of enforcement difficult. We do note, though,that OpenBazaar supports moderators that can assist in ensuring transactionsare conducted satisfactorily (see Section 2); unsurprisingly, all of P...A’s list-ings were unmoderated. We removed this vendor, and the 33 associated sales,from further consideration in columns 4–6, to obtain an hopefully more accuratepicture of sales on OpenBazaar – excluding P...A, the total amount of sales isactually around $86,000.

4 Interestingly, one of their items seemed to have legitimate feedback, which pre-datedall of the seemingly deceptive feedback discussed here.

Page 14: Open Market or Ghost Town? The Curious Case of ...Open Market or Ghost Town? The Curious Case of OpenBazaar 5 [peer ID]), and add them to a list, recursively continuing until we no

14 James E. Arps and Nicolas Christin

As Table 3 shows, the category distribution of items that do sell is verydifferent from the category distribution of items that are merely listed. Over25% of all recorded sales are for cannabis-related products (including seeds),and more than three quarters of all recorded sales are for drugs – prescriptiondrugs or narcotics.5 This higher economic activity occurs despite the fact thatthe OpenBazaar developers have taken active measures to try to make illicititems harder to find, by excluding them from their built-in search engine searchresults, which suggests that the demand for illicit offerings far outpaces that forlegitimate goods available on OpenBazaar.

Table 4. Distribution of feedback ratings. 5 is best, 1 is worst.

Score Count Percent

5 1,302 85.32%4 28 1.83%3 34 2.22%2 29 1.90%1 133 8.71%

Feedback ratings An alternative hypothesis would be that buyers of legitimateproducts are somehow less likely to leave feedback than buyers of illicit goods.We found no evidence to support that hypothesis. Specifically, we present thefeedback ratings we observed in Table 4. OpenBazaar uses a 5-point rating scale,where higher scores are better, i.e., vendor strive to obtain 5-star feedback. Theratings we see heavily skew positive, as has been observed in general (legitimate)e-commerce platforms [28], and feedback distribution presents striking similari-ties with with that obtained for feedback left by Silk Road patrons [6, Table 3]:5’s dominate, followed by 1’s, and other ratings are less frequently used.

4.3 Operational security

While much of the core OpenBazaar vendor base tends to connect over Tor, notall of these users are truly anonymous. Indeed, user IDs are persistent acrosssessions. By comparing the unique user IDs of the 980 nodes seen connectingover Tor with those seen connecting over public IP addresses, we found that173 users (17.7%) had revealed an IP address at some point in time. Not all ofthese users may have wished to remain anonymous during the entire collectioncycle, but we did observe some obvious lapses in operational security, such asUS vendors selling marijuana offering global shipping. It appears a version of

5 The disproportionate volume of Psychedelics sales seems to be mostly influencedby one specific vendor, who has been highly successful on various online anonymousmarkets, and also operates their own vendor shop, and has presence on OpenBazaar.

Page 15: Open Market or Ghost Town? The Curious Case of ...Open Market or Ghost Town? The Curious Case of OpenBazaar 5 [peer ID]), and add them to a list, recursively continuing until we no

Open Market or Ghost Town? The Curious Case of OpenBazaar 15

the OpenBazaar client which is pre-bundled with Tor is in development, whichshould alleviate this problem over time [26].

At the beginning of our measurement timeframe, OpenBazaar nodes weretied to a single cryptocurrency – if a vendor wished to, for example, sell thesame items both with Bitcoin and Bitcoin Cash, they were required to maintaintwo distinct nodes with identical listings. Furthermore, for some time using Zcashon the platform also required running a full Zcash node. As a result, activityon the platform was conducted almost entirely in Bitcoin until the release ofa multiwallet feature on January 17, 2019, which added native support for thethree currencies mentioned plus Litecoin. Previous studies (e.g., [21]) have shownthat Bitcoin is highly traceable, meaning that early purchases on the platformmay be able to de-anonymize certain vendors. Since the introduction of themultiwallet, however, there has been large growth in Zcash usage, with at least6

12,441 observed listings accepting payment in the currency.

5 Related work

This work follows a long lineage of peer-to-peer network measurements, whichcan be traced back to studies of file-sharing networks such as Napster [30], orGnutella [29]; or, beyond file-sharing, of Skype [12]. Later papers focused onspecific metrics to better understand user behavior. For instance, Gummadi etal studied peer churn on Kazaa [13], while others investigated overall peer avail-ability [4,32], or resilience to poisoning attacks [7,20]. A number of papers lookedinto the economics of online anonymous marketplaces and have documented theirrise in popularity [6, 10, 31]. In comparison, our analysis of OpenBazaar showsfairly modest revenues. Finally, also related to the operational security aspectswe outline in this paper are attempts to quantify cryptocurrency traceability –notably efforts to trace Bitcoin [21], Monero [22], and Zcash [18].

6 Conclusion

We conducted multiple daily crawls of the OpenBazaar distributed marketplaceover approximately 14 months (June 25, 2018–September 3, 2019). More thanhalf of the 6,651 participants we observed were present only for less than a day,but users relying on Tor tended to be much longer lived, particularly if theywere actively selling items. Economic activity is orders of magnitude smallerthan on centralized anonymous marketplaces, and while most listed items arefor legitimate products, the majority of items that do result in sales are fornarcotics. Finally, vendors are not always using prudent security practices, leak-ing for instance their IP address despite generally connecting over Tor, whichmakes a strong case for imposing secure defaults—fortunately, the OpenBazaardevelopers are already reportedly working on this [26].

6 A bug in our parser code for items with multiple currencies prevented us from pre-cisely computing the number of such listings, but we could recover this lower bound.

Page 16: Open Market or Ghost Town? The Curious Case of ...Open Market or Ghost Town? The Curious Case of OpenBazaar 5 [peer ID]), and add them to a list, recursively continuing until we no

16 James E. Arps and Nicolas Christin

We conclude that OpenBazaar, so far, has not gained much traction to usherin the new era of decentralized, private, and legitimate electronic commerce itwas promising.7 This could be due to a lack of user demand for decentralizedmarketplaces, lack of integration of private features, or other factors, such as ahigher learning curve for users compared to centralized alternatives.

7 Acknowledgments

We thank our shepherd, Ben Edwards, and the anonymous reviewers for nu-merous comments that greatly improved this paper. We are also grateful toKyle Soska for extensive discussions about this work. This research was partiallysupported by DHS Office of Science and Technology under agreement numberFA8750-17-2-0188.

References

1. Apilayer: ipstack (2019), https://ipstack.com/. Last accessed: January 8, 2020.2. Ben-Sasson, E., Chiesa, A., Garman, C., Green, M., Miers, I., Tromer, E., Virza,

M.: Zerocash: Decentralized anonymous payments from bitcoin. In: IEEE Sympo-sium on Security and Privacy. pp. 459–474. IEEE Computer Society (May 2014)

3. Benet, J.: IPFS - content addressed, versioned, P2P file system. Tech. rep. (Jun2014), draft 3. https://arxiv.org/abs/1407.3561

4. Bhagwan, R., Savage, S., Voelker, G.: Understanding availability. In: Proceedingsof the 2nd International Workshop on Peer-to-Peer Systems (IPTPS’03). pp. 256–267. Berkeley, CA (Feb 2003)

5. CC EF JTF: Did OpenBazaar ever take off? (Apr 2019), OpenBazaar devel-oper comments on total number of listings. https://www.reddit.com/r/Bitcoin/comments/bad0nh/did openbazaar ever take off/ekbd2k6/

6. Christin, N.: Traveling the Silk Road: A measurement analysis of a large anony-mous online marketplace. In: Proceedings of the 22nd World Wide Web Conference(WWW’13). pp. 213–224. Rio de Janeiro, Brazil (May 2013)

7. Christin, N., Weigend, A., Chuang, J.: Content availability, pollution and poisoningin peer-to-peer file sharing networks. In: Proceedings of ACM EC’05. pp. 68–77.Vancouver, BC, Canada (Jun 2005)

8. DHS S&T – CSD: Information marketplace for policy and analysis of cyber-risk & trust (IMPACT), retrieved January 8, 2020, from https://www.dhs.gov/csd-impact

9. Dingledine, R., Mathewson, N., Syverson, P.: Tor: The second-generation onionrouter. In: Proceedings of the 13th USENIX Security Symposium (Aug 2004)

10. European Monitoring Centre for Drugs and Drug Addition and Europol: Drugs andthe darknet: perspectives for enforcement, research and policy (Nov 2017), http://www.emcdda.europa.eu/system/files/publications/6585/TD0417834ENN.pdf

11. Greenberg, A.: Inside the ‘darkmarket’ prototype, a Silk Road the FBI can neverseize (Apr 2014), https://www.wired.com/2014/04/darkmarket/

7 We sent an earlier preprint of this paper to the OpenBazaar developers, but did notreceive any feedback.

Page 17: Open Market or Ghost Town? The Curious Case of ...Open Market or Ghost Town? The Curious Case of OpenBazaar 5 [peer ID]), and add them to a list, recursively continuing until we no

Open Market or Ghost Town? The Curious Case of OpenBazaar 17

12. Guha, S., Daswani, N.: An experimental study of the skype peer-to-peer voipsystem. Tech. rep., Cornell University (2005)

13. Gummadi, K., Dunn, R., Saroiu, S., Gribble, S., Levy, H., Zahorjan, J.: Measure-ment, modeling, and analysis of a peer-to-peer file-sharing workload. In: Proceed-ings of ACM SOSP’03. pp. 314–329. Bolton Landing, NY (Oct 2003)

14. Higgins, S.: OpenBazaar raises $1 million for decentralised marketplace (Jun 2015),https://www.coindesk.com/openbazaar-raises-1-million-from-silicon-valley-giants

15. Higgins, S.: Bitcoin-powered marketplace OpenBazaarraises $3 million (Dec 2016), https://www.coindesk.com/bitcoin-powered-marketplace-openbazaar-raises-3-million

16. Jeffryes, J.: Remove BlockBooth from search providers (Apr 2019), OpenBazaardeveloper comments on OB1 censorship policy. https://github.com/OpenBazaar/openbazaar-desktop/issues/1569

17. Kaplan, E., Meier, P.: Nonparametric estimation from incomplete observations.Journal of the American Statistical Association 53, 457–481 (1958)

18. Kappos, G., Yousaf, H., Maller, M., Meiklejohn, S.: An empirical analysis ofanonymity in Zcash. In: Proc. USENIX Security (Aug 2018)

19. Kruithof, K., Aldridge, J., Decary Hetu, D., Sim, M., Dujso, E., Hoorens, S.:Internet-facilitated drugs trade: An analysis of the size, scope and the role of theNetherlands. Tech. Rep. RR1607 (2016), https://www.rand.org/pubs/researchreports/RR1607.html

20. Liang, J., Kumar, R., Xi, Y., Ross, K.: Pollution in P2P file sharing systems. In:Proceedings of IEEE INFOCOM’05. Miami, FL (Mar 2005)

21. Meiklejohn, S., Pomarole, M., Jordan, G., Levchenko, K., McCoy, D., Voelker,G.M., Savage, S.: A fistful of bitcoins: characterizing payments among men withno names. In: Proceedings of the ACM/USENIX Internet measurement conference.pp. 127–140. Barcelona, Spain (Oct 2013)

22. Moser, M., Soska, K., Heilman, E., Lee, K., Heffan, H., Srivastava, S., Hogan, K.,Hennessey, J., Miller, A., Narayanan, A., Christin, N.: An empirical analysis oftraceability in the Monero blockchain. In: Proceedings of PETS. vol. 3. Barcelona,Spain (Jul 2018)

23. OB1 Team: Openbazaar seller guide – what to expect in this de-centralized marketplace (Oct 2017), https://openbazaar.org/blog/openbazaar-seller-guide-what-to-expect-in-this-decentralized-marketplace/

24. OpenBazaar: OpenBazaar 2.3.1 is released with IPFS rebase (Mar 2019), https://www.openbazaar.org/blog/openbazaar-2-3-1-release-ipfs-rebase/

25. OpenBazaar Team: Openbazaar API, https://api.docs.openbazaar.org/

26. Patt, S.: Create “openbazaar-private” installer (Mar 2019), https://github.com/OpenBazaar/openbazaar-desktop/issues/1735

27. Reitz, K.: Requests: Http for humans (May 2019), https://requests.readthedocs.io

28. Resnick, P., Zeckhauser, R.: Trust among strangers in internet transactions: Em-pirical analysis of eBay’s reputation system. Advances in Applied Microeconomics11, 127–157 (2002)

29. Ripeanu, M., Foster, I.: Mapping the gnutella network: Macroscopic properties oflarge-scale peer-to-peer systems. In: international workshop on peer-to-peer sys-tems. pp. 85–93. Springer (2002)

30. Saroiu, S., Gummadi, K., Gribble, S.: A measurement study of peer-to-peer filesharing systems. In: Proceedings of SPIE/ACM MMCN’02. pp. 156–170. San Jose,CA (Jan 2002)

Page 18: Open Market or Ghost Town? The Curious Case of ...Open Market or Ghost Town? The Curious Case of OpenBazaar 5 [peer ID]), and add them to a list, recursively continuing until we no

18 James E. Arps and Nicolas Christin

31. Soska, K., Christin, N.: Measuring the longitudinal evolution of the online anony-mous marketplace ecosystem. In: Proceedings of the 24th USENIX Security Sym-posium (USENIX Security’15). pp. 33–48. Washington, DC (Aug 2015)

32. Stutzbach, D., Rejaie, R.: Understanding churn in peer-to-peer networks. In: Pro-ceedings of the 6th ACM SIGCOMM conference on Internet measurement. pp.189–202. ACM (2006)

33. Weinberg, Z., Cho, S., Christin, N., Sekar, V., Gill, P.: How to catch when proxieslie: Verifying the physical locations of network proxies with active geolocation. In:Proceedings of the 17th ACM Internet Measurement Conference (IMC’17). Boston,MA (Oct 2018)