Importance of data

Importance of dataJay Daley, ICANN Helsinki 2016

• Small registry - Big data pioneer22 server Hadoop cluster installed in 2012Full packet capture of DNS servers

• Data science oriented research team18% of total staff

• World class registrar data portalMain business priority for new development

• Actively building data productsRecent success with broadbandmap.nz

• Open data portal for .nz data

Why is .nz presenting?

June 2016ICANN 56 Helsinki 2

Case study – https://idp.nz


• Evidence based policy

• Organisational/community development

• Cleaner and safer DNS

• Business - more, new, better

• Public trust

Why data matters


Evidence based policy


• Already some excellent examplesRSTEPIANA SLE approximations by Marc Blanchet

• But data is not public and so …No reproducibilityAgenda set by those who pay the analystsVery low throughput of research

• Some people have their own dataGives them real power in the debatee.g. Root server operators and WPAD debate“data is the new oil”

• Open Data means Open Debate

Current use of data in policy


Case study – hoarding?


1

10

100

1000

10000

100000

10000001 2 3 4 5 6 7 8 9 10

11-2

021

-30

31-4

04

1-50

51-6

06

1-70

71-8

081

-90

91-

100

101-

200

201-

300

301-

40

04

01-

500

501-

60

06

01-

700

801-

90

09

01-

100

010

01-

200

020

01-

300

0

Num

ber

of r

egis

tran

ts

Portfolio size

Number of registrants by portfolio size

Organisational development


Openness and transparency

• Applied toDiversity, remuneration, expenses, etc

• Significant organisational benefitsReinforces best behaviourCreate culture of community/customer audit

Modern business principle


• ICANN funds travel for some attendees• Until recently this data was hard to use

Only published in PDFsSome meetings missingNames spelled inconsistentlyNo summaries or multi-meeting analysis

• At Dublin meeting spent several hoursExtracting data from PDFsTidying up namesCreating reportsPublishing data

Case study – Travel funding


Example PDF


Example output


$-

$2,000

$4,000

$6,000

$8,000

$10,000

$12,000

$14,000

0 2 4 6 8 10 12 14 16 18 20

Average travel funds received by Number of meetings attended

Example publication


• Report from AFNIC on ICANN Diversity• Based on public data – but not easy!

Case study – Diversity


Example output


Cleaner and safer DNS


• Highly developed use of dataMultiple research teams, cooperative forums, NFP services and commercial providersMultiple data resources, extensive data sharingTools: Entrada, Turing, hadoop-pcap, zonemaster

• Data collection – source evidencePassive monitoring (e.g. DNSDB, PassiveTotal)Hand produced by threat researchers

• Strong sharing culture – via data feeds40+ feeds available (both NFP and commercial)Track domains, IPs, credentials, URLs, etc

Very active area


• Data shared with cooperative forum – then shared with registrars

Case study – threat sharing


Business – more, new, better


• Market intelligence• Targeting marketing• New products, same customers• New products, new customers

Data driven business


• ”Registrants prefer shorter names” – right?

Market intelligence - basic


0

5000

10000

15000

20000

25000

30000

35000

40000

45000

50000

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

Number of domain names by number of characters

• Domain name categorisation by industry

Market intelligence - advanced


0% 5% 10% 15% 20% 25% 30% 35%

A — Agriculture, forestry, fishing and hunting

B — Mining

C — Manufacturing

D — Electricity, gas and water supply

E — Construction

F — Wholesale trade

G — Retail trade

H — Accommodation, Food Services

I — Transport and storage

J — Information Media and Telecommunications

K — Finance and insurance

L — Rental, Hiring and Real Estate Services

M — Professional, Scientific and Technical Services

N — Administrative and Support Services

O — Public Administration and Safety

P — Education and Training

Q — Health Care and Social Assistance

R — Arts and Recreation Services

S — Other Services

Registry Registrar

Case study - registrar portal


• Domains most likely to renew for 10 yearsThose in top 20% by observed traffic

• Domains in danger of cancellingNo MX record

• TLD cross-sell opportunitiesSimple data matching

• Industry verticalsMachine learning classifier using web site text

• Expiring domainsValued by algorithm

Targeted marketing


Case study – expiring domains


• Domain name popularity

New products, same customers


• SaaS product market sizingSpecific DNS record indicators for each productCounted by regular zone scansData sold for competitor analysis

New products, new customers


Public trust

June 26, 2016Add Presentation Name 28

“Sunlight is the best disinfectant”

• Open data can prove:Effectiveness of markets and regulatorsSocial benefitWhere problems remain

Open data and public trust


Case study – Web Index


• Competition, Consumer Trust and Consumer Choice (CCT) Metrics Reporting

• So near but yet so far …

Case Study - CCT


• 204 datasets only one from ICANN (NRO)!

Case study - ISOC


Taking this to the next level


• ICANN F17 Strategic Plan includes:

“Deploy automated systems to collect data and compute ratio of registered domain names to active IP addresses.”“Deploy automated systems to collect data and compute ratio of registered domain names to Internet users regionally and globally.”“Publish analyses of data collected, implications of changes in data over time, and other topics relevant to the use of unique identifiers and evolution of identifier technologies”“Document growth in ratios in developing regions”

Toes in the water


Data enabled community & industry

• Data openly shared by all market participants• Data is easy to find and use and can be

trusted • Strong community use of data• Authoritative publications

ICANN Industry ReportDomain Names Contribution to Society Index

A goal to consider


• Commit – employ a Chief Data Officer• Begin cultural change

Broaden the principle of openness to include dataSet the vision of the benefits

• Engage community – “social license”Community expectations of openness vs privacy

• Put legal framework in placeAdjust contracts, policies and processes to support open dataDetermine privacy protection rules

Steps to make it happen


Contact:www.nzrs.net.nz

[email protected]

ThanksAny questions?