How Tracking Companies Circumvented Ad Blockers Using ... · WebSockets which are used by A&A...

How Tracking Companies Circumvented Ad Blockers Using WebSockets

Muhammad Ahmad Bashir, Sajjad Arshad, Engin Kirda, William Robertson, Christo Wilson

Northeastern University

Online Tracking

Online Tracking• Boom in online advertising.

• Ad networks pour in billions of dollars.

• Value for their investment?

• Extensive tracking to serve targeted ads.

• User concern over tracking

• This has led to the proliferation of ad blockers

• User concern over tracking

• This has led to the proliferation of ad blockers

• Ad networks fight back

• E.g Using anti-ad blocking scripts

Google & Safari

• Google evaded Safari’s third-party cookie blocking policy (Jonathan Mayer)

• … by submitting a form in an invisible iFrame

• Google was fined $22.5M by FTC

This Talk

How Ad Networks leveraged a bug in Chrome API to bypass Ad Blockers

using WebSockets

This Talk

How Ad Networks leveraged a bug in Chrome API to bypass Ad Blockers

using WebSockets

• What caused this?

• How this bug was leveraged by ad networks?

Web Sockets

HTTP/S

Web Sockets

HTTP/S

Web Sockets

HTTP/S request

response

Web Sockets

HTTP/S request

response

Chatting App

Web Sockets

HTTP/S request

response

Chatting Appanything new?

Web Sockets

HTTP/S request

response

Web Socket

Web Sockets

HTTP/S request

response

Web Socketbidirectional

ws:// or wss://

• Both client and server can send/receive data • This is a persistent connection

Ad Blockers

• Chrome extension chrome.webRequest API• Extension can inspect / modify / drop outgoing requests

Ad Blockers

webRequest API

Ad Blockers

http://cnn.com/logo.jpegwebRequest API

Ad Blockers

Rule List Usually borrowed from EasyList

Ad Blockers

Rule List

Usually borrowed from EasyList

Ad Blockers

Rule List

Ad Blockers

Rule List

Ad Blockers

Rule List

webRequest API

Ad Blockers

Rule List

http://doubleclick.com/s1.js

webRequest API

Ad Blockers

Rule List

webRequest API

Ad Blockers

Rule List

webRequest API

Ad Blockers

Rule List

webRequest API

AdBlock Evasion

• Due to a bug in chrome.webRequest API

• All ws/wss requests bypassed this extension

AdBlock Evasion

2012 2013 2014 2015 2016 2017 2018

AdBlock Evasion

2012 2013 2014 2015 2016 2017 2018

Original bugreported

AdBlock Evasion

2012 2013 2014 2015 2016 2017 2018

Users report unblocked ads

AdBlock Evasion

2012 2013 2014 2015 2016 2017 2018

Users report unblocked ads Patch

Landed

AdBlock Evasion

2012 2013 2014 2015 2016 2017 2018

Landed

Chrome 58 released

AdBlock Evasion

2012 2013 2014 2015 2016 2017 2018

Landed

Chrome 58 released* Represents when our crawls were done

AdBlock Evasion

2012 2013 2014 2015 2016 2017 2018

* * * *

Landed

Chrome 58 released* Represents when our crawls were done

Data Crawling

100K websites sampled from Alexa

Data Crawling

100K websites sampled from Alexa Visit 15

links / website

Collected chains for all inclusion resources

Data Crawling

links / website

This means we know which resource included

which other resource

Data Crawling

links / website

Filter all resources which end in web sockets

Filter WebSockets

Data Crawling

links / website

Filter all resources which end in web sockets

Filter WebSockets

Detect A&A WebSocketsMark web sockets

which are used by A&A domains

Companies involved in Advertising and Analytics are collectively referred as A&A

High-Level Numbers

Crawl Dates %Websites with sockets

% Socketswith A&AInitiators

% Socketswith A&AReceivers

#Unique A&A

Initiators

#Unique A&A

ReceiversApr 02-05, 2017 2.1 60.2 72.0 72 14

Apr 11-16, 2017 2.4 61.0 73.0 61 16

May 07-12, 2017 1.6 60.2 68.3 17 13

Oct 12-16, 2017 2.5 54.9 55.1 19 14

Before Chrome 58

High-Level Numbers

#Unique A&A

Initiators

#Unique A&A

ReceiversApr 02-05, 2017 2.1 60.2 72.0 72 14

Apr 11-16, 2017 2.4 61.0 73.0 61 16

May 07-12, 2017 1.6 60.2 68.3 17 13

Oct 12-16, 2017 2.5 54.9 55.1 19 14

Before Chrome 58

After Chrome 58

High-Level Numbers

#Unique A&A

Initiators

#Unique A&A

ReceiversApr 02-05, 2017 2.1 60.2 72.0 72 14

Apr 11-16, 2017 2.4 61.0 73.0 61 16

May 07-12, 2017 1.6 60.2 68.3 17 13

Oct 12-16, 2017 2.5 54.9 55.1 19 14

• ~2% websites use web sockets.

Before Chrome 58

After Chrome 58

High-Level Numbers

#Unique A&A

Initiators

#Unique A&A

ReceiversApr 02-05, 2017 2.1 60.2 72.0 72 14

Apr 11-16, 2017 2.4 61.0 73.0 61 16

May 07-12, 2017 1.6 60.2 68.3 17 13

Oct 12-16, 2017 2.5 54.9 55.1 19 14

• 55-61 % sockets are initiated by A&A domains

Before Chrome 58

After Chrome 58

High-Level Numbers

#Unique A&A

Initiators

#Unique A&A

ReceiversApr 02-05, 2017 2.1 60.2 72.0 72 14

Apr 11-16, 2017 2.4 61.0 73.0 61 16

May 07-12, 2017 1.6 60.2 68.3 17 13

Oct 12-16, 2017 2.5 54.9 55.1 19 14

• 55-73 % sockets contact an A&A domain

Before Chrome 58

After Chrome 58

High-Level Numbers

#Unique A&A

Initiators

#Unique A&A

ReceiversApr 02-05, 2017 2.1 60.2 72.0 72 14

Apr 11-16, 2017 2.4 61.0 73.0 61 16

May 07-12, 2017 1.6 60.2 68.3 17 13

Oct 12-16, 2017 2.5 54.9 55.1 19 14

• # Initiators drops after Chrome 58 release.

Before Chrome 58

After Chrome 58

High-Level Numbers

#Unique A&A

Initiators

#Unique A&A

ReceiversApr 02-05, 2017 2.1 60.2 72.0 72 14

Apr 11-16, 2017 2.4 61.0 73.0 61 16

May 07-12, 2017 1.6 60.2 68.3 17 13

Oct 12-16, 2017 2.5 54.9 55.1 19 14

• # Initiators drops after Chrome 58 release.

• Small but persistent A&A receivers.

Before Chrome 58

After Chrome 58

Initiators and Receivers

Initiator ReceiverJavaScript

Initiator Receiverws/s

JavaScript

A&A Initiator #A&AReceivers

facebook 11google 11

doubleclick 9youtube 8addthis 8

hotjar 6googlesyndication 6

cloudfront 4sharethis 4

adnxs 3

Top A&A Initiators

JavaScript

adnxs 3

Top A&A Initiators

JavaScript

adnxs 3

A&A Receiver #A&AInitiators

realtime 2733across 19intercom 15

disqus 13zopim 12hotjar 11feedjit 10

lockerdome 8inspectlet 6

smartsupp 4

Top A&A Initiators

Top A&A Receivers

JavaScript

adnxs 3

smartsupp 4

Top A&A Initiators

Top A&A Receivers

• Disqus provides comment board services.

JavaScript

adnxs 3

smartsupp 4

Top A&A Initiators

Top A&A Receivers

• Zopim, Intercom, Smartsupp provide live chat services.

JavaScript

adnxs 3

smartsupp 4

Top A&A Initiators

Top A&A Receivers

• 33across & Lockerdome are advertising platforms.

JavaScript

adnxs 3

smartsupp 4

Top A&A Initiators

Top A&A Receivers

• 33across & Lockerdome are advertising platforms.

• Inspectlet & Hotjar are session replay services.

JavaScript

Sent Items Over Web Sockets

Cookie

User IDs

Fingerprinting Variables

% Requests

0 20 40 60 80

WebSocketsHTTP/S

•Stateful Identifiers like Cookie and User IDs

Cookie

User IDs

% Requests

0 20 40 60 80

WebSocketsHTTP/S

• Fingerprinting data in ~3.4% WebSockets. 97% is 33across

Cookie

User IDs

% Requests

0 20 40 60 80

WebSocketsHTTP/S

• Fingerprinting data in ~3.4% WebSockets. 97% is 33across

• ~1.5% WebSockets sends the entire DOM to Hotjar

Cookie

User IDs

% Requests

0 20 40 60 80

WebSocketsHTTP/S

Received Items Over Web Sockets

Received Items Over Web SocketsHTML

JavaScript

Images

% Responses

0 10 20 30 40 50

WebSocketsHTTP/S

JavaScript

Images

% Responses

0 10 20 30 40 50

WebSocketsHTTP/S

JavaScript

Images

% Responses

0 10 20 30 40 50

WebSocketsHTTP/S

Received Items Over Web Sockets

Ads served from Lockerdome

JavaScript

Images

% Responses

0 10 20 30 40 50

WebSocketsHTTP/S

Summary• ~67% of socket connections are initiated or received by A&A domains.

• Major companies like Google, Facebook, Addthis adopted WebSockets. Abandoned after Chrome 58 was released.

• The culprits:

• 33across was harvesting fingerprinting data.

• HotJar was exfiltrating the entire DOM

• Lockerdome downloaded URLs to serve ads.

• We need to keep with the current practices of A&A companies.

Summary• ~67% of socket connections are initiated or received by A&A domains.

• Major companies like Google, Facebook, Addthis adopted WebSockets. Abandoned after Chrome 58 was released.

• The culprits:

• 33across was harvesting fingerprinting data.

• HotJar was exfiltrating the entire DOM

• Lockerdome downloaded URLs to serve ads.

• We need to keep with the current practices of A&A companies.

Questions?ahmad@ccs.neu.edu

Discussion Points

• What’s Next?

• Can these findings be used to fine advertisers or shape new policies?

• Major Ad Exchanges abandoned WebSockets — Why?

• New web standards.

• Can be problematic? Where should we intervene?

• Surprising that it took few years to patch this bug

• WebRTC aspect of it.

Backup Slides

Inclusion Chain

</body> </html> </iframe>

</body> </html>

pub/ index.html

tracker/ script.js

tracker/ img.jpg

ads/ script.js

ads/ frame.html

ads/ script_12.js

ads/ img_a.jpg

adnet/ data.ws

Source code for ads/script_12.js let ws = new WebSocket(“ws://adnet/data.ws”, …); ws.onopen = function (e) {ws.send(“…”);}

DOM Tree Inclusion Tree

How Tracking Companies Circumvented Ad Blockers Using ... · WebSockets which are used by A&A...

Documents

Transforming WebSockets

Realtime with-websockets-2015

HTML5 WebSockets {week 13 }

Asynchronous WebSockets using Django - VilniusPyvilniuspy.lt/slides/DjangoWebSockets.pdfWEBSOCKETS DJANGO CHANNELS EXAMPLE WEBSOCKETS (A.K.A.ASYNCHRONOUS WEBSOCKETS) I WebSocket is

vlavrynovych - WebSockets Presentation

Hacking websockets

WebSockets för applikationstestare

Programming WebSockets - O'Reilly Mediaassets.en.oreilly.com/1/event/45/Programming WebSockets... · Programming WebSockets Sean Sullivan OSCON ... • JavaScript API is easy to learn

Using WebSockets with ColdFusion

How Tracking Companies Circumvented Ad Blockers Using ...using WebSockets 4. This Talk How Ad Networks leveraged a bug in Chrome API to bypass Ad Blockers using WebSockets 4 1. What

Building interactivity with websockets

WebSockets On Fire

HTML5 WebSockets {week 14 }

Comet, WebSockets, HTML5, SSE

How Tracking Companies Circumvented Ad …...WebSockets Mark web sockets ! which are used by A&A domains A&A = Advertising and Analytics e.g. DoubleClick, Criteo, Adnxs This means

Programming WebSockets - OSCON 2010

Node.js and websockets intro

Cowboy rabbit-websockets

Ember and WebSockets

How Tracking Companies Circumvented Ad Blockers Using ... · web sockets Filter WebSockets Detect A&A WebSockets Mark web sockets ! which are used by A&A domains A&A = Advertising