Vulnerability Disclosures: Law and Ethics in Security Research

Vulnerability Disclosures: Law and Ethics in Security Research

Robert Carolina, Senior FellowInformation Security [email protected]; +44 7712 007 095

Robert Carolina

§ Royal HollowayUniversity of London§ Law & Regulation module leader,

Information Security Group, (1999- )

§ Lawyer (England & US)§ Law & regulation of ICT;

law & ethics in cyber security

§ General Counsel, ISC (BIND DNS, Kea DHCP, internet F-Root) (2020- )

§ BA (Dayton, 1988)Juris Doctor (Georgetown, 1991)LL.M (London School of Economics, 1993)

2

N.B. The content of this presentation does not constitute the provision of legal advice. All comments are personal and not for attribution to any client, employer, institution, agency, collective, or enterprise.

What seems to be the problem?

3

4

The CyBOK challenge Dear Rob,

Write a “knowledge area” for security practitioners and educators globally about law and regulation relevant to cybersecurity practice

… and cover “Ethics”

Sincerely,CyBOK project team

• Professional advisory board• Academic advisory board• International group of authors, editors

and reviewers• Feb 2017 – Oct 2019 v1.0• 19 Knowledge Areas• 854 Pages (20 MB pdf)• Sources cited: 1839

Plan A: publish the Merman Borgnine Conjecture

5

𝐸𝑡ℎ𝑖𝑐𝑠 𝐺𝑢𝑖𝑑𝑎𝑛𝑐𝑒 : [𝐶𝑦𝑏𝑒𝑟𝑠𝑒𝑐𝑢𝑟𝑖𝑡𝑦]

: :

𝑀𝑎𝑟𝑟𝑖𝑎𝑔𝑒 𝑡𝑜 𝐸𝑟𝑛𝑒𝑠𝑡 𝐵𝑜𝑟𝑔𝑛𝑖𝑛𝑒 : [𝑀𝑒𝑟𝑚𝑎𝑛– 𝑎𝑛 𝑎𝑢𝑡𝑜𝑏𝑖𝑜𝑔𝑟𝑎𝑝ℎ𝑦]


6

𝑐𝑦𝑏𝑒𝑟𝑠𝑒𝑐𝑢𝑟𝑖𝑡𝑦 𝑒𝑡ℎ𝑖𝑐𝑠 𝑔𝑢𝑖𝑑𝑎𝑛𝑐𝑒 ∈(I.e., nothing worth writing about)

𝑠𝑢𝑏𝑠𝑡𝑎𝑛𝑐𝑒 ∈ ∅


§ Support for the M B Conjecture

§ Early writing on ethics stresses compliance with intellectual property laws: anti-piracy patter

§ Heavy focus on “legal” compliance without further enlightenment

§ “Comply with the law”

§ “Comply with all applicable laws”

§ “Be aware of all legal obligations”

§ Heavy reliance on bromides

§ “First do no wrong”

§ “Don’t be evil”

§ “You’ve always had the power my dear, you just had to learn it for yourself”

7

§ Bottom line

§ Although security practitioners

§ Operate using special skills

§ Work outside the glare of public supervision

§ Are placed in unique positions of trust

§ … providing asymmetric power which is easily abused

§ I could find no clear statement of principles and guidance that provide a cybersecurity practitioner with a framework to understand

§ Responsibility to clients

§ Responsibility to society

§ … and how to balance these

Plan B: write something about ethics

8

• Pages: 112 (0.7 MB pdf)• End Notes: 204• Sources cited: 269

• Introduction

• Introductory principles of law and legal research

• Jurisdiction

• Privacy laws in general and electronic interception

• Data protection• Computer crime• Contract (voluntary obligations)

• Tort (involuntary obligations, e.g. negligence)

• Intellectual property• Internet intermediaries – shields

from liability and take-down procedures

• Dematerialisation of documents and electronic trust services

• Other regulatory matters• Public international law

• Ethics• Conclusion: legal risk

management

• Cross-reference table

• End notes (204: 15 pages)

• References (269: 12 pages)

• Acronyms

• Glossary


§ Codes of conduct

§ Association for Computing Machinery (ACM) Code of Ethics and Professional Conduct§ ACM is a professional society of computer scientists

§ Code extensively revised in 2018 to address cyber related issues. Good support & training materials with case studies.

§ Focus on social responsibility; nothing specific about client relationship

§ The CREST Code of Conduct forCREST Qualified Individuals§ Initially founded to professionalise penetration

testing; membership increasingly required to secure high profile client work in UK and other countries

9

§ Professional-client relationship

§ Mostly abstract observations attempting to draw out nature of client relationship

§ “It remains to be seen whether cyber security practitioner-client relationships will become the subject of formal state regulation or licensure in due course.”


§ Vulnerability discovery

§ The process of testing

§ “Unauthorised” access?

§ DIY vulnerability inserted into open source?

§ DRM hack – violation of copyright law

§ Disclosure…

§ Facilitating and acting on disclosure§ Obligations on system vendors

§ ISO 29147 (receiving and handling disclosures)

§ ISO 30111 (verifying and remediating vulnerabilities)

10

“A system being publicly accessible is not sufficient grounds on its own to imply authorization”– ACM Code § 2.8

Vulnerability disclosure basics

§ No disclosure

§ Retain for Offensive Cyber Operations (“balancing equities”)

§ “More trouble than it’s worth”

§ Public disclosure

§ Publish everything immediately

§ “Responsible disclosure”

§ Initial private disclosure followed at some later time by public disclosure

11

“A computing professional has an additional obligation to report any signs of system risks that might result in harm.”– ACM Code § 1.2 (Avoid harm)

“… capricious or misguided reporting of risks can itself be harmful.”– ACM Code § 1.2 (Avoid harm)


§ Privately disclose – to whom?

§ Supply chain

§ Upstream to component vendor?

§ Downstream to all vendors who adopt the vulnerable component?

§ What about vulnerability in a widely adopted industry standard?

12

§ Rewarding disclosure

§ Vendor bug bounty

§ Third party information broker who sells vulnerability find to vendors

§ Market trades in anticipation of disclosure (e.g., buying put options; short selling)

§ Non-monetary rewards. E.g., consider the relationship between publication of research and academic promotion.


§ Delay between private and public disclosure

13

𝑡0

Private disclosure

Public disclosure

𝑡! 𝑡"

∆𝑡 = 𝑡" − 𝑡!lim∆$→&

ℎ𝑜𝑝𝑒 𝑜𝑓 𝑎𝑐𝑎𝑑𝑒𝑚𝑖𝑐 𝑝𝑟𝑜𝑚𝑜𝑡𝑖𝑜𝑛 = 0%

Case study: Megamos

14

Background of this case study

§ The Paper§ Carolina & Paterson, “Megamos Crypto,

Responsible Disclosure, and the Chilling Effect of Volkswagen Aktiengesellschaft vs Garcia, et al” (2013) (15 pages A4)

§ Written for an audience of both technologists and lawyers.

§ Discussed the legal case. The core legal issue was the status of the target crypto algorithm as “trade secret”. The nature of “responsible disclosure” was a secondary issue.

§ Written while the case was pending in the High Court of England.

15

§ Independent investigation

§ Online sources concerning the trade secret

§ Ongoing discussions over the nature of “responsible disclosure”

§ Opinions on responsible disclosure

§ Opinions expressed about responsible disclosure are outside the scope of the 2013 paper and are attributable solely to Carolina – who will be exploring this further with Paterson J

The security research project

The paper in controversy

§ “Dismantling Megamos Crypto: Wirelessly Lockpicking a Vehicle Immobilizer”(accepted 2013; published 2015)

17

§Authors:§ Roel Verdult

Radboud University Nijmegen§ Flavio D. Garcia

University of Birmingham(formerly of RU Nijmegen)

§ Baris EgeRadboud University Nijmegen

18

Slide from R Verdult presentation at Usenix Security 2013

Supply chain for Megamos immobiliser

§ Megamos algorithm (apparently) originated from Thales in 1990’s

§ Thales licensed the algorithm to crypto chip manufacturer (EM)

§ EM sells the chips to immobiliser hardware manufacturer/integrator (including Delphi)

§ Immobiliser hardware is then sold to automobile manufacturers (including Volkswagen)

§ Resulting product is installed in “millions” of automobiles

19

Image from R Verdult presentation slide at Usenix Security 2013

How do researchers carry out cryptanalysis?

§ Problem: the algorithm is not generally published

§ Solution: researchers decide to obtain algorithm through reverse engineering

§ Why? A secret found by reverse engineering a lawfully manufactured and lawfully obtained product is generally not a violation of trade secret rights

20

§ Problem: grabbing the algorithm from the chip is a relatively expensive reverse-engineering project (cost of chip slicing estimated at €50,000)

§ Solution: Find something else that embodies the algorithm that is cheaper and easier to reverse engineer:

Enter… TANGO PROGRAMMER

Tango Programmer

§ Hardware & software product used to code immobiliser transponders

§ Manufactured by company in Bulgaria

§ Sold openly online and via resellers in multiple EU jurisdictions

§ Megamos functionality in product from 2009

§ €1,000 base price plus additional fees for each type of transponder you wish to code, plus recurring fee

§ Software does not include cryptanalysis or secret key grabber

21Top: www.tangoprogrammer.com (Bulgaria)Bottom: www.obdsvs.com (China)

The project proceeds

§ Method§ Tango Programmer software reverse

engineered & Megamos algorithm uncovered

§ Cryptanalysis

§ Three weaknesses found § Weak algorithm

§ Weak secret keys

§ Poor key updating practices

22

§Disclosure§ Researchers disclose results to chip

manufacturer (EM) in November 2012

§ Submitted to peer review

§ Paper is accepted for publication at USENIX, scheduled for August 2013; more than 9 months following vulnerability disclosure to EM

The Lawsuit

Who initiated; where; on what grounds?

§ Claimant is Volkswagen, AG

§ Thales added as an additional Claimant during the preliminary hearing on the theory that the trade secret “belongs” to them

§ High Court of England

24

§ Effort to protect confidential information (i.e., “trade secret”)

§ The trade secret argument:

1 While it’s OK to reverse engineer a trade secret from our product, you took it from Tango Programmer

2 Tango Programmer embodies a misappropriated trade secret

3 You reverse engineered the wrong thing and therefore have no right to use or publish the trade secret

Who was sued?

§ Five defendants:

§ Flavio D. Garcia (UK)

§ University of Birmingham (UK)

§ Roel Verdult (NL)

§ Baris Ege (NL)

§ Radboud University Nijmegen (NL)

§ The 2 UK-resident Defendants and the 3 NL-resident Defendants were represented by separate legal teams

25

§ Who is missing from the case?

§ USENIX conference organiser and publisher who already holds a copy of the paper (US)

§ Manufacturer of Tango Programmer (Bulgaria; web site suggests branch sales office in the UK)

Timeline of the case

§ November 2012: researchers disclose to crypto chip manufacturer (EM)

§ May 2013: Volkswagen files case in High Court of England

§ Emergency ex parte temporary injunction. The researchers and universities were not notified or represented in court; the first notice they get would include the temporary court order saying “though shalt not publish”

§ Mid-June 2013: full preliminary hearing with parties present

26

§ 25 June 2013: Decision

§ Temporary injunction confirmed pending trial

§ Summer 2015: “amicable” settlement

§ Researchers allowed to publish their articlewith one line deleted (the secret algorithm)

§ No other terms disclosed, such as legal costs§ In England loser pays winner’s legal costs. Many

settlements turn on resolving this issue.

Highlights from thepreliminary decision

A few observations by the Court…

§ Tango Programmer§ "I think it is obvious that Tango Programmer does

not derive from a legitimate source in the automotive industry.”

§ “I find that the Tango Programmer’s website is clear, that this is a product sold by someone who knows it is likely to facilitate crime.”

§ Researchers’ knowledge of Tango Programmer

§ “In my judgment, the [academics] have taken a reckless attitude to the probity of the source of the information they wish to publish”

§ “…the [academics] have made no effort to make enquiries about the legitimacy of Tango Programmer. That is not to impose a unreasonable burden on them.”

28

A few observations by the Court…

§ VW’s case§ "The claimants do not have an overwhelming case

on the merits, not even a very strong one.” It’s “much more than … merely arguable … [but] not overwhelming”.

§ Freedom of speech

§ Must be balanced against “the security of millions of Volkswagen cars”

§ Responsible disclosure:“I think the defendants’ mantra of ‘responsible disclosure’ is no such thing. It is a self-justification by defendants for the conduct they have already decided to undertake and it is not the action of responsible academics.”

§ BOTTOM LINE:

§ Court orders Defendants to stop publication of (parts of) the paper

29

After the case is over: where are we with responsible disclosure?

Responsible disclosure

§ Protocols for responsible disclosure?§ Degree of agreement within the academy?

§ Degree of agreement within industry?

§ Degree of agreement in the embedded devices space?

§ Should responsible disclosure rely upon consensus among all stakeholders?

31

§ ISO 29147, Vulnerability disclosure

§ Outlines what a vendor should do when receiving a vulnerability report from a finder

§ Silent about the responsibility of finder

§ 5.4.4 “… For the purposes of this International Standard, it is expected that a finder will attempt to inform a vendor or coordinator about a vulnerability. In practice, a finder may choose not to attempt to inform a vendor or coordinator or the attempt may fail”


§ Liability for publication§ What “duty of care” (if any) do researchers owe to

existing car owners? Or to component manufacturers? Or to main manufacturers?

§ Is it reasonably foreseeable that this publication will result in my Porsche or Volvo being stolen? If publication contains errors, will it harm the business of anyone in the supply chain?

§ Is publication of a vulnerability that cannot be remediated in an economical fashion the act of a “reasonable person”?

§ In other words – is it “negligent” to publish, and could foreseeable victims sue the researchers?

32

§ Vicarious liability for publication

§ If a victim can prove that publication constitutes a negligent act by the researcher, then the liability of the researcher will be attributed to his employer

§ Remember the rule of lawsuits: only sue people and organisations who have lots of money (or lots of insurance)

§ This sort of lawsuit could be launched in the US, and US tort law would most likely be applicable


§ Disclose to the entire supply chain?§ Pro: earlier notice to effected parties

§ Con: the crypto vendor might hate your guts and never talk to you again

§ Con: if you get it (even slightly) wrong, the crypto supplier could attack you for disrupting their business

33


§ Perhaps disclose in stages?For example, notify EM:

§ “We will disclose details to Delphi in 3 weeks.”

§ “We will disclose details to VW (and other manufacturers) in 6 weeks.”

§ “We will publish a non-technical announcement to the market in 8 weeks.”

§ “We will generally publish technical paper with details in 6 months.”

34

§ Practical challenge§ How to identify the upstream supplier?

§ Megamos research team was able to identity EM

§ Did they have any way of knowing the role of Thales in developing and licensing the algorithm?

§ How to identify the downstream product integrators?

𝑡0

Private disclosure A

Full public disclosure

𝑡!' 𝑡"(

Private disclosure B

𝑡!(

Private disclosure C

𝑡!)

Partial public disclosure

𝑡"'

What about the merits of the legal case?

35

What did VW need to prove torestrain publication of this paper pending trial?

Human Rights Act

To stop publication pending trial, VW must convince the Court (based on what little we know now) that a full trial is "likely to establish that publication should not be allowed” -HRA s.12(3)

Does this mean only if court believes thatchances of winning at trial > 50%?

Not necessarily. Available evidence must be "sufficiently favourable to justify such an order being made in the particular circumstances of the case”

Even if chances of winning < 50%, a temporary order may be appropriate "where the potential adverse consequences of disclosure are particularly grave, or where a short-lived injunction is needed to enable the court to hear and give proper consideration to an application for interim relief pending the trial or any relevant appeal” Cream Holdings v. Banerjee, [2004] UKHL 44

36

The Court’s understanding of crypto algorithm

§ "For the process to be secure, both pieces of information need to remain secret - the key and the algorithm.”

§ The Court does not explain or support this assertion.

§ The Claimant’s team have clearly done a better job presenting their picture of secure systems

37

Are potential adverse consequences of disclosure“particularly grave”?

§ Published decision lacks any credible risk analysis

§ Court clearly concerned about the security of “millions” of cars

§ No clear explanation of how this publication would influence the economics of car theft

38

Where did Tango Programmer get the secret algorithm?

§ Judge acknowledges chip-slicing has been done by others, but rejects the idea that this was the source of the information (why?)

§ The judge believes VW will be able to prove that Tango Programmer was built using a misappropriated trade secret (how?)

§ Court heavily influenced by its opinion that the device can be used for evil, that the device manufacturer knows this, and that the researchers know this

39 siliconzoo.org

What are the researchers supposed to knowabout Tango Programmer?

§ Court suggests that researchers have a duty to perform due diligence on Tango Programmer

40

§ But why?

§ Contra UK Supreme Court decision: VestergaardFrandsen A/S et al v Bestnet Europe Ltd et al (May 2013, only one month before Megamos decision)

§ Third party in receipt of misappropriated trade secret liable only if they “knew” or turned a “blind-eye” to obvious misappropriation – equivalent of dishonesty.

§ Megamos court seems to apply lower standard: “should have known” or “with a bit of diligence, would have figured it out”

Unexplained delay in enforcing rights

§ Court believes that Tango Programmer embodies a misappropriated trade secret

41

§ Product has been on the market since 2009; complaining parties believe it is used for criminal purposes

§ So WHY has it taken so long to enforce trade secret rights?

§ And WHY was the manufacturer of Tango Programmer not named as a defendant?

A question Carolina and Paterson did not discuss…

§ Why did the researcher’s state of mind matter at all?

§ To restrain publication or further use of a trade secret, one merely needs to prove the existence of the secret –the potential publisher’s knowledge is irrelevant to granting of restraining order

§ State of mind is relevant if one is attempting to apportion liability (not discussed)

§ Will this lead to a claim for money damages? (“Your guilty use of our stolen trade secret damaged our product”)

OR

§ Will this be the basis for claiming legal costs? (“Your guilty use of our stolen trade secret forced us to spend way too much money on lawyers”)

42

“I think, therefore I am… liable?”

Why did this go so badly wrong?(Educated guess – not in paper)

Time pressure

§ Expedited hearing was forced by the academics

§ Weak evidentiary record

§ The Court was openly annoyed at being forced to confront a complex record this quickly

§ A Court working too quickly is very often a Court that gets things wrong

§ (Be careful what you wish for)

44

Failure to tell a compelling story

§ Defendant as fact witness AND expert witness?

§ Court needed a careful education about crypto and security research

§ Only source of this education seems to have been the defendants themselves (who have a personal interest in the outcome)

45

Over-confidence of the legal team?

§ On the face of it, this looks like a “slam-dunk” for the defence team

§ Algorithm not much a “secret”§ The chip has been sliced.

§ Third party software products are in the market.

§ Application of Human Rights Act§ Free speech rules for “prior restraint” mean heavier

scrutiny.

§ The legal burden of proof for all of the difficult-to-prove elements of the case rest with VW

46

The feet of clay hypothesis

§ Judges are human, too§ This was a newly appointed judge. What would

you do to minimise the risk of going down in history as “the judge who allowed millions of (expensive) cars to be stolen”?§ Would you tell three crypto researchers that they have

to delay publishing their academic research for 6 months? Or 12? Or 24?

§ Would you (hypothetically) get annoyed with a crypto researcher who repeats “responsible disclosure” in your court room whenever he is asked about potential harms?

47

§ And besides…§ This is preliminary - everyone will have their day in

court (at the trial)

§ And furthermore…

§ The judge makes it pretty clear that VW may have serious difficulty proving its case at trial…

§ (Judge-speak for “settle this, guys”)

Vulnerability Disclosures: Law and Ethics in Security Research

Robert Carolina, Senior FellowInformation Security [email protected]; +44 7712 007 095

Documents

Vulnerability Disclosures: Law and Ethics in Security Research