Vendor Analysis: FICO...• Cyber risk management. • Insurance risk. • Regulatory requirements including Basel 2 and 3, Dodd-Frank, MiFID II and Solvency II. Chartis is solely

Vendor Analysis: FICOCyber Risk Quantification Solutions, 2019

© Copyright Infopro Digital Services Limited 2019. All Rights Reserved2 | Vendor Analysis: FICO; Cyber Risk Quantification Solutions, 2019

Chartis Research is the leading provider of research and analysis on the global market for risk technology. It is part of Infopro Digital, which owns market-leading brands such as Risk and WatersTechnology. Chartis’ goal is to support enterprises as they drive business performance through improved risk management, corporate governance and compliance, and to help clients make informed technology and business decisions by providing in-depth analysis and actionable advice on virtually all aspects of risk technology. Areas of expertise include:

• Credit risk.• Operational risk and governance, risk and

compliance (GRC).• Market risk.• Asset and liability management (ALM) and

liquidity risk.• Energy and commodity trading risk.• Financial crime including trader surveillance, anti-

fraud and anti-money laundering.• Cyber risk management.• Insurance risk.• Regulatory requirements including Basel 2 and

3, Dodd-Frank, MiFID II and Solvency II. Chartis is solely focused on risk and compliance technology, which gives it a significant advantage over generic market analysts.

The firm has brought together a leading team of analysts and advisors from the risk management and financial services industries. This team has hands-on experience of implementing and developing risk management systems and programs for Fortune 500 companies and leading consulting houses.

Visit www.chartis-research.com for more information.

Join our global online community at www.risktech-forum.com.

© Copyright Infopro Digital Services Limited 2019. All Rights Reserved.

No part of this publication may be reproduced, adapted, stored in a retrieval system or transmitted in any form by any means, electronic, mechanical, photocopying, recording or otherwise, without the prior permission of Infopro Digital Services Limited trading as Chartis Research (‘Chartis’).

The facts of this document are believed to be correct at the time of publication but cannot be guaranteed. Please note that the findings, conclusions and recommendations that Chartis delivers will be based on information gathered in good faith, whose accuracy we cannot guarantee. Chartis accepts no liability whatever for actions taken based on any information that may subsequently prove to be incorrect or errors in our analysis. See ‘Terms and conditions’.

RiskTech100®, RiskTech Quadrant®, FinTech Quadrant™ and The Risk Enabled Enterprise® are Registered Trade Marks of Infopro Digital Services Limited.

Unauthorized use of Chartis’ name and trademarks is strictly prohibited and subject to legal penalties.

https://www.infopro-digital.com/terms-conditions/research-and-marketing-services/?lang=en


Table of contents

1. Report context 5

2. Quadrant context 8

3. Vendor context 11

4. Methodology 16

5. Further reading 19


List of figures and tables

Figure 1: The increasing value of CRQ 7

Figure 2: The operational scope of quantification 7

Figure 3: RiskTech® Quadrant for cyber risk quantification solutions, 2019 9

Figure 4: The FICO ESS – benchmarking example 12

Figure 5: ESS Portrait – example screen 12

Figure 6: ESS Landscape – example screen 13

Figure 7: FICO Cyber Risk Score – third-party risk management example 14

Table 1: Completeness of offering – FICO (cyber risk quantification solutions, 2019) 10

Table 2: Market potential – FICO (cyber risk quantification solutions, 2019) 10

Table 3: FICO – company information 11

Table 4: Evaluation criteria for Chartis’ cyber risk quantification solutions report 17


1. Report context

This Vendor Analysis is based on the Chartis quadrant report Cyber Risk Quantification Solutions, 2019: Market and Vendor Landscape (published in May 2019). This section summarizes the key theses in that report; subsequent sections take a detailed look at FICO’s quadrant positioning and scoring, and Chartis’ underlying opinion and analysis.

Key thesis

We want quant

As the frequency and severity of cyber breaches continue to grow, cyber crime is now one of the biggest challenges facing financial institutions (FIs). Adding to their problems, FIs must also address the growing risk of technology outages – established FIs’ legacy networks and newer challengers’ untested systems have both fallen victim to cyber incidents.

Clearly this is an issue, and it’s a costly one too. Overall, breaches and outages can cost the average FI millions of dollars annually, and the figure increases significantly for the largest institutions. Facing a rise in threats, institutions of all types are spending big on their cybersecurity systems.

Yet amid the fog of spending and hype surrounding the latest cybersecurity defenses, the task of systematically quantifying firms’ relative cyber risks has until recently gone unaddressed. This lack of functionality has also prevented FIs and vendors from assessing the relative effectiveness of different cybersecurity systems. Most current solutions used against malicious attacks and potential system failures – from passwords and firewalls to AI-powered enterprise systems – often do not rigorously quantify the benefits of the reduced risk they offer.

FIs and vendors have sought to quantify cyber risk before, but increasingly they are spending such large sums on cybersecurity systems that they require defensible risk scores for their cyber domains. And only now is there technology available to automate analysis and leverage the vast datasets required to properly quantify cyber risk.

Demand for cyber risk quantification (CRQ) solutions is coming from insurers – keen to assess the risk in counterparties’ infrastructure – and more general financial services firms, which want to assess the risk in the systems they rely on for their

1 Business impacts may also include regulatory enforcement and reputational damage, although not all systems can or will quantify these.2 Chief risk officers, chief technology officers and chief information security officers.

operations. This is becoming more pressing as FIs’ IT systems and risk-management infrastructures become more complex.

Vendors to the rescue

Increasingly, Chartis believes, vendors of CRQ solutions will develop specific functionality across four key functional and operational areas: the cyber risk score, loss estimation, portfolio optimization, and attribution. Vendors currently approach CRQ from two angles: externally, assessing a firm’s network in relation to that of other firms; and internally, mapping the risk of cyber events occurring on a firm’s own network. By partnering and cooperating, vendors can start to offer comprehensive solutions that will enable them to exploit the ever-growing CRQ market.

Demand-side takeaways

Defining cyber risk quantification

Chartis defines the components of CRQ as:

• Cyber risk. The likelihood and severity of a loss due to the breach or failure of IT systems. These losses are the business impacts1 that might result from the theft of confidential data, the compromise of information integrity, or the loss of systems availability, as well as any costs required to rectify issues.

• Cyber risk quantification. The evaluation of cyber risk using mathematical models to produce a numerical score or ranking. These components can be fashioned into better tools for risk managers to measure the level and location of the cyber risk their FIs carry. In addition, by viewing security systems through the lens of risk, risk professionals can demonstrate the value of those solutions to the broader business.

This approach offers CROs, CTOs, and CISOs2 a valuable joint lexicon to communicate their


institution’s stance on risk to all relevant staff in a language they already share.

The drivers of demand for CRQ solutions

Until recently, assessments of cyber risk were limited to manual analyses of a firm’s own systems or those of a third party (with its consent), whereby examiners inspected networks on a case-by-case basis. FIs relied on manual assessment using qualitative standards such as ISO 27001 or the NIST Cybersecurity Framework, both of which require significant modification to suit specific organizational contexts.

Modern CRQ techniques attempt to circumvent the persistent obstacles to statistically measuring operational risk – a lack of data and a lack of unbiased analytics. As CRQ represents a quantification of operational risk, FIs remain wary of relying too much on cyber risk models. As a result, demand for solutions is currently immature, although it is growing rapidly as FIs realize the importance of adequately understanding their cyber risk exposure.

There are two key sources of demand for CRQ solutions:

• Insurance firms, which use CRQ to assess the cyber risk that exists in a counterparty’s infrastructure.

• More generally, financial services firms – including insurers – will use CRQ to assess the cyber risk in the native and/or third-party systems that form the infrastructure on which their institutions rely.

Quantification: why now?

As FIs’ employees come to rely more on technology, standardized CRQ methods will be more in demand. Quantitative risk assessments will help FIs’ boards grapple with cyber risk by aggregating possible bad outcomes into layers of risk that are appropriate to a given level of the risk management hierarchy. This clear-eyed view of risk can overcome the superiority bias that often manifests in cybersecurity, whereby staff often believe that their institution is better protected from breaches compared to its peers. A board that knows the relative cyber risks it faces will be well placed to set and execute a secure and efficient technology strategy. The penalties for observing risks and choosing to dismiss them are likely to be high.

3 https://www.msci.com/documents/10199/5915b101-4206-4ba0-aee2-3449d5c7e95a

Tangled infrastructure is risky infrastructure

The broad demand for CRQ solutions arises because FIs continue to add new systems to their legacy cores. The complexity of FIs’ infrastructure – and its inherent risk – has grown to cope with demands on systems that cover everything from incorporating multiple market data streams to offering extended functionality to retail customers. These systems are often provided by third parties – such as market data suppliers in the first instance and application developers in the second.

CRQ is applied broadly at all points in FIs’ provision of services to support business processes. Regulators such as the US Office of the Comptroller of the Currency expect banks to ‘practice risk management regardless of whether the bank performs the activity internally or through a third party.’ Firms must thus monitor the risk they carry due to their use of third-party services, an increasingly pressing responsibility as FIs come to rely more on loosely regulated FinTech firms that may lack mature risk management oversight.

Supply-side takeaways

CRQ on the increase, but it must be robust

CRQ is becoming a formalized component of risk management in financial services, and risk management departments will own it if solutions can demonstrate statistical rigor. CRQ’s nascent adoption is somewhat akin to the early years of value at risk (VaR), a metric developed for gauging market risk. While many trading desks implemented VaR, awareness of the technique outside discrete groups of quantitative analysts was low, until J.P.Morgan opened up access to its VaR methodology and hived it off into RiskMetrics3. CRQ is undergoing a similar process – FIs increasingly acknowledge its usefulness (see Figure 1), and an ecosystem of vendors that are exploiting recent technological advances is growing to support this demand.

Yet risk managers will resist incorporating CRQ into their portfolios unless measurement methodologies are shown to be robust and consistent across business contexts. FIs have seen cybersecurity as a set of binary outcomes – attackers get in, or they don’t; a breach occurs, or it doesn’t. This is not the language of risk, which deals in probabilities. The myriad of functionality that helps firms’ security divisions defend their networks obscures the value of a CRQ tool. Vendors must focus on ensuring that their scores are explainable and have predictive value. They must also speak the language of risk, not just cybersecurity, in designing and deploying their

https://www.msci.com/documents/10199/5915b101-4206-4ba0-aee2-3449d5c7e95a

https://www.occ.gov/news-issuances/bulletins/2013/bulletin-2013-29.html




systems. Wide adoption of any given system among FIs will occur only if vendors demonstrate their solutions’ accuracy and defensibility to risk experts.

Developing the right functionality

Cyber risk represents the first type of operational risk for which quantification methodologies properly exploit the vast amounts of data now available to develop a more statistically defensible, data-intensive methodology. Vendors have overcome the historical obstacles to quantifying operational risk: a lack of data – much of it unstandardized – and human bias.

The internet’s constituent networks contain multitudes of information, most of it in standard formats. Increases in data transfer speeds mean that relevant information, like server security configurations and routing tables (akin to maps of connected computers), can be collected and updated frequently. Vastly higher data storage densities give vendors commercially viable warehouses in which to trap this stream. Machine learning techniques offer tools capable of processing these variable, complex and large data sets.

Chartis predicts that, as CRQ matures, vendors will be pushed to develop certain functionality, across four main operational areas (see Figure 2).

Vendor landscape

Vendors currently approach quantification from one of two angles:

• Externally, assessing the risk in a firm’s network in relation to other networks; or

• Mapping and analyzing networks internally, calculating the risk of different cyber events occurring on that particular network.

Both external and internal approaches quantify cyber risk. Vendors using the external approach, however, quantify the cyber risk of a firm’s network relative to that of other firms. In contrast, those offering an internal method quantify the risk of a particular attack succeeding on a firm’s network relative to other attacks on the same infrastructure. Both produce quantitative risk scores, but one presents network risk, while the other presents attack risk.

Solutions that employ external analysis excel at giving risk scores for third parties. These solutions are designed primarily to support comparisons across different firms and of different characteristics of those firms. Due to their extensive time series

data, they also prove strong in tracking changes in a firm’s risk score over time. This maintained history also allows retrospective analysis of attacks that exploited vulnerabilities not publicly known at the time. This enables analysts to devise more accurate risk scores for networks identified as harboring those vulnerabilities, once their details become known.

Advanced CRQ deployments will allow FIs to weight the relative benefits of different controls by simulating their protective benefits to the network. CRQ, used mindfully, will also help FIs’ systems development, by allowing cyber risk to be assessed from the design stage through to implementation.

Vendors of all types stand to gain from cooperation. By combining complementary datasets and analytical approaches, vendors will be better placed to develop and exploit the growing field of CRQ.

Figure 1: The increasing value of CRQ

Security systems and policy strategy

Network optimization

Quantifying risk allows organizations to attack problems at various levels of strategic complexity…

Insurance(and other risk-mitigation strategies)

Business strategy

Operational optimization and planning

Source: Chartis Research

Figure 2: The operational scope of quantification

Cyber risk score

Loss estimation

Portfolio optimization

Attribution



2. Quadrant context

Introducing the Chartis RiskTech® Quadrant

This section of the report contains:

• The Chartis RiskTech® Quadrant for CRQ solutions for 2019.

• An examination of FICO’s positioning and its scores as part of Chartis’ analysis.

• A consideration of how the quadrant reflects the broader vendor landscape.

Summary information

What does the Chartis quadrant show?

The RiskTech® Quadrant uses a comprehensive methodology that involves in-depth independent research and a clear scoring system to explain which technology solutions meet an organization’s needs. The RiskTech® Quadrant does not simply describe one technology option as the best CRQ solution; rather it has a sophisticated ranking methodology to explain which solutions are best for specific buyers, depending on their implementation strategies.

The RiskTech® Quadrant is a proprietary methodology developed specifically for the risk technology marketplace. It takes into account vendors’ product, technology and organizational capabilities. Section 4 of this report sets out the generic methodology and criteria used for the RiskTech® Quadrant.

How are quadrants used by technology buyers?

Chartis’ RiskTech and FinTech quadrants provide a view of the vendor landscape in a specific area of risk, financial and/or regulatory technology. We monitor the market to identify the strengths and weaknesses of different solutions, and track the post-sales performance of companies selling and implementing these systems. Users and buyers can consult the quadrants as part of their wider research when considering the most appropriate solution for their needs.

Note, however, that Chartis Research does not endorse any vendor, product or service depicted in its research publications, and does not advise technology users to select only those vendors with

the highest ratings or other designation. Chartis Research’s publications consist of the opinions of its research analysts and should not be construed as statements of fact.

How are quadrants used by technology vendors?

Technology vendors can use Chartis’ quadrants to achieve several goals:

• Gain an independent analysis and view of the provider landscape in a specific area of risk, financial and/or regulatory technology.

• Assess their capabilities and market positioning against their competitors and other players in the space.

• Enhance their positioning with actual and potential clients, and develop their go-to-market strategies.

In addition, Chartis’ Vendor Analysis reports, like this one, offer detailed insight into specific vendors and their capabilities, with further analysis of their quadrant positioning and scoring.

Chartis Research RiskTech® Quadrant for cyber risk quantification solutions, 2019

Figure 3 illustrates Chartis’ view of the CRQ vendor landscape, highlighting FICO’s position.


Figure 3: RiskTech® Quadrant for cyber risk quantification solutions, 2019

MA

RK

ET P

OTE

NTI

AL

COMPLETENESS OF OFFERING

Best of breed Category leaders

Point solutions Enterprise solutions

BitSight

Corax

eFortresses

FICO

foreseetiIBMRiskLens

RiskSenseAon

Marsh

RiskReconSecurityScorecard

UpGuard

CyberPoint

Willis Towers Watson


Quadrant Dynamics

General quadrant takeaways

The CRQ quadrant comprises two broad groups:

• Those vendors that assess cyber risk from an external perspective, and whose solutions quantify the cyber risk in a firm’s network relative to that of other firms. An external approach is well suited to quantifying third-party risk. This group splits further into:

o Vendors that employ CRQ to support their own insurance operations.

o Vendors that provide cyber risk scores for use by their clients, whether they are insurers or other financial services firms looking to assess third-party cyber risk.

• Vendors that assess cyber risk from an internal perspective. These vendors’ solutions quantify the relative cyber risk of different slices of a firm’s internal network, or a firm’s current cyber risk relative to its historical cyber risk. Such an approach is optimal for assessing first-party cyber risk.


Vendor positioning in context – completeness of offering

FICO scored well on the CRQ quadrant’s completeness of offering axis, due to the rigor of its cyber risk scoring methodology and its expansive underlying data set, which incorporates 100 billion internet-wide data points, including 40 million malicious IP addresses. FICO’s solution uses this network node-level information, arrayed in a five-year historical time series, to examine changes in firms’ security postures over time. This provides the foundation for the quantification of cyber risk based on a firm’s organizational security practices (such as time to patch).

Machine learning (ML) applied to this dataset derives correlations between breaches and given security configurations, as well as a firm’s higher-order behavioral factors. This supports a 24x dynamic range – a very high degree of granularity separating the least risky firms from the most risky. Married with firmographic information this method enables a financial loss-oriented quantification of cyber risk, a key functional area for those users, such as insurers, looking to assess third-party risk.

Table 1 shows Chartis’ rankings for FICO’s coverage against each of the completeness of offering criteria.

Vendor positioning in context – market potential

On the market potential axis, FICO scored well for market penetration. Though today’s CRQ solutions are relatively new, FICO has leveraged the strength of its brand in credit scoring to appeal to financial services firms using a familiar framework.

FICO’s growth strategy is strong, and the company has a robust plan to embed its CRQ solution in financial services firms. This is supported by an educational program aimed at explaining its solution to risk and insurance functions – a crucial component in boosting wider uptake and deeper deployment.

Table 2 shows Chartis’ rankings for FICO’s coverage against each of the market potential criteria.

Table 1: Completeness of offering – FICO (cyber risk quantification solutions, 2019)

Completeness of offering criterion Coverage

Scoring/event risk quantification High

Risk modeling and aggregation Medium

Allocation, attribution, and impact analysis Medium

Visualization and analytics High

Financial quantification Medium


Table 2: Market potential – FICO (cyber risk quantification solutions, 2019)

Market potential criterion Coverage

Business model High

Customer satisfaction Medium

Market penetration High

Growth strategy High

Financials High



3. Vendor context

Overview of relevant solutions/capabilities

Table 3 gives an overview of FICO and its CRQ solution.

The FICO Cyber Risk Score is designed to quantify cyber risk and provide a forward-looking benchmark of an organization’s security posture. It is based on empirical data analysis and derived from a ML model that is formulated to forecast a well-defined objective outcome – the likelihood that an organization will suffer a material breach event in the next 12 months. Like other operational risk metrics engineered by FICO, the output is a three-digit score ranging from 300 to 850, which reflects relative risk and translates directly to

outcome odds which are provided by FICO in a detailed model report.

Users can access the FICO Cyber Risk Score via the FICO Enterprise Security Suite (ESS; see Figure 4). This cloud-based application suite provides access to three separate portals, each serving different users’ needs: ESS Portrait, ESS Portrait Premium and ESS Landscape. ESS Portrait (see Figure 5) provides the Cyber Risk Score of a user’s organization, for free. It is for self-assessment purposes and is used to monitor score results over time. ESS Portrait Premium is an enhanced version of ESS Portrait and provides access to the Cyber Risk Score, as well as a more detailed view of an organization’s specific security risk indicators. Finally, ESS Landscape allows users to view the FICO Cyber Risk Score of third-party organizations. With ESS Landscape

Table 3: FICO – company information

Company FICO

Headquarters San Jose, California, US

Other offices Global offices include:

North America – San Diego, San Rafael, Miami, Bozeman, Austin, Ann Arbor, New York City, Montreal, Toronto and Roseville, MN.

Latin America – Brazil and Chile.

EMEA – UK, Turkey, Germany, Italy, South Africa, Spain and Russia.

Asia-Pacific – Singapore, Australia, China, India, Japan, Korea, Malaysia, Philippines and Thailand.

Description Founded in 1956 and based in Silicon Valley, FICO uses data science to help its clients improve their operational decisions in risk management, fraud control, security, logistics and marketing. FICO solutions are used by businesses in more than 100 countries across a range of applications, including fraud protection, credit scoring and logistics.

Solution The FICO Cyber Risk Score relies on a comprehensive and diverse set of cybersecurity risk signals, collected at internet-scale, to measure organizations’ forward-looking security risk. It is engineered to provide insights into security risk that encompass both technical and policy-related shortcomings.

By analyzing time-series compilations of risk signals and comparing them to past behaviors of organizations that have, and have not, suffered a material data breach, FICO is able to produce an empirically derived score that forecasts the likelihood of a breach event over a subsequent 12-month period.

Source: FICO


users can monitor cyber risk across the supply chain, including partner assessments, procurement assessments, and potential merger and acquisition targets. Within ESS Landscape cyber insurance underwriters are provided with tailored dashboards designed to support the underwriting process and aggregate risk assessment across their portfolios (see Figure 6).

Given FICO’s heritage in risk quantification and ML, the underpinnings of the FICO Cyber Risk Score leverage a rich set of IP in feature engineering, designed to expose and amplify signals used to quantify forward-looking risk outcomes. Notable features of the solution include:

• Empirically derived. The FICO Cyber Risk Score is built using a supervised analytic model. This means that the algorithm that computes the

Figure 4: The FICO ESS – benchmarking example

Source: FICO

Figure 5: ESS Portrait – example screen

Source: FICO


score leverages mathematical relationships between signal data, inferred behaviors, and real-world security outcomes from both breached and non-breached organizations.

• Focused on risk quantification. While vulnerability inventories are important, they can also serve to mask underlying risk. As a result, organizations may confuse security activity (e.g., patching cadence) with effectiveness, and distract security teams from focusing on impactful change.

• Depth and breadth of signals. The key risk signals leveraged by the FICO Cyber Risk Score are based on a deep database of time-series historical information, collected by FICO, which spans the entire internet address space for six years. This allows FICO to correlate conditions and behaviors to cyber incidents, regardless of delays in disclosure, and

enables it to immediately generate scores for companies worldwide.

In aggregate, the FICO Cyber Risk Score assesses both the condition and scale of internet-facing network assets, as well as the behavior and performance of organizations in managing their security posture. Rather than grading the current state of the network, FICO evaluates forward-looking risk by employing a ML model that is trained to a well-defined objective outcome – the likelihood of a material data breach event in the next 12 months. This provides an easy to interpret result that applies across self-assessment, third-party risk management (see Figure 7), and cyber insurance underwriting.

Figure 6: ESS Landscape – example screen

Source: FICO


Figure 7: FICO Cyber Risk Score – third-party risk management example

Source: FICO

Client leading practices

The FICO Cyber Risk Score serves as an impartial assessment of the effectiveness of an organization’s security controls and adherence to network management best practices, at a time when new regulations, such as the EU General Data Protection Regulation (GDPR), are requiring increased scrutiny of third-party data processors. It offers an internet-scale alternative to static questionnaires, allowing for faster vetting of new partners and ongoing monitoring of existing relationships.

Risk indicators used in the FICO Cyber Risk Score

In producing its Cyber Risk Score, FICO derives signals from time-series observations of the internet-facing assets on an organization’s network. These signals help inform an understanding of how both conditional and behavioral indicators are related to breach outcomes. Conditional indicators relate to items such as the presence of misconfigurations and compromised hosts – i.e., the condition of network components. Behavioral indicators relate to the performance of the people tasked with managing a network, such as the

time required to recognize misconfigurations, or the frequency with which compromised hosts are found on an organization’s network. These signals help form an understanding of network hygiene practices, consistency in policy, and the network management track record of an organization. Examples of these signals include:

• Internet surface. The nature and scale of internet-exposed IT assets associated with an organization through the various IP address registries, as corrected and adjusted through the asset curation process.

• Endpoint posture. This may include indicators of compromise such as increased spam or botnet activity, which may indicate users’ willingness to click on untrusted links or email attachments. The frequency and duration of these incidents help account for the strength of network management practices as well as end-user behavior.

• Infrastructure posture. The long-term presence of latent threats that may cause harm to an organization, or other organizations. For example, the potential for infrastructure to


be leveraged in a distributed denial-of-service (DDOS) attack.

• Services posture. Services represent the policies and configurations used to manage a network. Examples of risk indicators include unsecure protocols that are externally exposed, expired security certificates, and databases responding to external probes.

• Firmographic data. While not risk indicators in themselves, the size, sector and location of an organization help inform and weight risk signals in order to reflect an organization’s inherent risk.

Ensuring scoring accuracy

FICO offers real-time, self-service network asset curation. This means that users curate the assets (i.e., validate the IP address blocks that define the footprint of the organization) that are used in the creation of an organization’s score. The score can be focused on an entire organization, a business unit or subsidiary of an organization, or a geographic region of an organization.

How the model works

The data used to create the Cyber Risk Score ML models, as well as the data used to derive an organization’s score, is continuously collected and captured by FICO at internet scale. This data asset contains six years of history on the global IP address space. This gives FICO a perspective of organizational behavior in the time period leading up to the breach event – whenever that might have been. This forms the basis for predicting future outcomes as time-series indicators associated with organizations that have, and have not, suffered a breach. This helps FICO identify the signals most indicative of increased risk.

Model performance

The goal of any risk model is to quantify the likelihood of a targeted outcome. In the case of the FICO Cyber Risk Score the model is focused on determining the odds of a future material data breach event. While several statistical methods can be used to assess model performance, the measures are intended to assess a model’s ability to separate goods and bads with the fewest false positives (or false negatives) while focused on a given operating point in the score distribution.

One of the measures used to assess the predictive performance of a model is dynamic range. The dynamic range measures the relative

outcome odds across a model’s score range. For example, the FICO Cyber Risk Score operates on a scale of 300-850 (higher scores indicate lower risk), with a dynamic range of 24x. This means that an organization scoring 300 is 24 times as likely to suffer a material cyber event in the next 12 months as an organization scoring 850.

It’s important to note that even organizations receiving the best score (i.e., 850) are not impervious to internal and external threats and subsequent data breach. However, their odds of suffering a breach are significantly lower than organizations receiving lower scores. This granularity of risk separation helps risk professionals better categorize the severity of forward-looking risk rather than merely reviewing the latest inventory of misconfigurations and transient network threats.

Operationalizing the Cyber Risk Score

Cyber risk quantification is intended to support better decisions by providing an easy to interpret benchmark of network hygiene and adherence to network management best practices. For most organizations this information can support decisions related to incremental security investments, training needs, and triaging of vendor relationships. In the cyber insurance space this information is being used to inform underwriting processes and manage aggregate portfolio risk.


4. Methodology

Overview

Chartis is a research and advisory firm that provides technology and business advice to the global financial services industry. Chartis provides independent market intelligence regarding market dynamics, regulatory trends, technology trends, best practices, competitive landscapes, market sizes, expenditure priorities, and mergers and acquisitions. Chartis’ RiskTech and FinTech Quadrant™ reports are written by experienced analysts with hands-on experience of selecting, developing and implementing financial technology solutions for a variety of international companies in a range of industries including banking, insurance and capital markets. The findings and analyses in our quadrant reports reflect our analysts’ considered opinions, along with research into market trends, participants, expenditure patterns, and best practices.

Chartis seeks to include RiskTech and FinTech vendors that have a significant presence in a given target market. The significance may be due to market penetration (e.g., a large client base) or innovative solutions. Chartis uses detailed ‘vendor evaluation forms’ and briefing sessions to collect information about each vendor. If a vendor chooses not to respond to a Chartis request for information, Chartis may still include the vendor in the report. Should this happen, Chartis will base its opinion on direct data collated from technology buyers and users, and from publicly available sources.

Chartis’ research clients include leading financial services firms and Fortune 500 companies, leading consulting firms and financial technology vendors. The vendors evaluated in our quadrant reports can be Chartis clients or firms with whom Chartis has no relationship.

Chartis evaluates all vendors using consistent and objective criteria, regardless of whether or not they are Chartis clients. Chartis does not give preference to its own clients and does not request compensation for inclusion in a quadrant report, nor can vendors influence Chartis’ opinion.

Selection criteria

Chartis selected a set of vendors that cover the two main approaches to CRQ – internal and

4 Note that vendors do not always respond to requests for briefings; they may also choose not to participate in the briefings for a particular report.

external. Vendors that sell risk scores formed the bulk of those we assessed, although we also looked at vendors that provide modeling suites, as well as those that use CRQ modeling for insurance underwriting. Note also that some vendors did not respond to our invitation to brief us for this report.

Briefing process

We conducted face-to-face and/or web-based briefings with each vendor4. During these sessions, Chartis experts asked in-depth, challenging questions to establish the real strengths and weaknesses of each vendor. Vendors provided Chartis with:

• A business update – an overview of solution sales and client satisfaction.

• A product update – an overview of relevant solutions and R&D roadmaps.

• A product demonstration – key differentiators of their solutions relative to those of their competitors.

In addition to briefings, Chartis used other third-party sources of data, such as conferences, academic and regulatory studies, and publicly available information.

Evaluation criteria

We develop specific evaluation criteria for each piece of quadrant research from a broad range of overarching criteria, outlined below. By using domain-specific criteria relevant to each individual risk, we can ensure transparency in our methodology, and allow readers to fully appreciate the rationale for our analysis. The specific criteria used for CRQ are shown in Table 4.

Completeness of offering

• Depth of functionality. The level of sophistication and amount of detailed features in the software product (e.g., advanced risk models, detailed and flexible workflow, domain-specific content). Aspects assessed include: innovative functionality, practical relevance of features, user-friendliness, flexibility, and


embedded intellectual property. High scores are given to those firms that achieve an appropriate balance between sophistication and user-friendliness. In addition, functionality linking risk to performance is given a positive score.

• Breadth of functionality. The spectrum of requirements covered as part of an enterprise risk management system. This will vary for each subject area, but special attention will be given to functionality covering regulatory requirements, multiple risk classes, multiple asset classes, multiple business lines, and multiple user types (e.g. risk analyst, business manager, CRO, CFO, Compliance Officer). Functionality within risk management systems and integration between front-office (customer-facing) and middle/back office (compliance, supervisory and governance) risk management systems are also considered.

• Data management and technology infrastructure. The ability of risk management systems to interact with other systems and handle large volumes of data is considered to be very important. Data quality is often cited as a critical success factor and ease of data access, data integration, data storage, and data movement capabilities are all important factors. Particular attention is given to the use of modern data management technologies, architectures and delivery methods relevant to risk management (e.g., in-memory databases, complex event processing, component-based architectures, cloud technology, and Software as a Service). Performance, scalability, security and data governance are also important factors.

• Risk analytics. The computational power of the core system, the ability to analyze large amounts of complex data in a timely manner (where relevant in real time), and the ability to improve analytical performance are all important factors. Particular attention is given to the difference between ‘risk’ analytics and standard ‘business’ analytics. Risk analysis requires such capabilities as non-linear calculations, predictive modeling, simulations, scenario analysis, etc.

• Reporting and presentation layer. The ability to present information in a timely manner, the quality and flexibility of reporting tools, and ease of use, are important for all risk management systems. Particular attention is given to the ability to do ad-hoc ‘on-the-fly’ queries (e.g., ‘what-if’ analysis), as well as the range of ‘out of the box’ risk reports and dashboards.

Market potential

• Business model. Includes implementation and support and innovation (product, business model and organizational). Important factors include size and quality of implementation team, approach to software implementation, and post-sales support and training. Particular attention is given to ‘rapid’ implementation methodologies and ‘packaged’ services offerings. Also evaluated are new ideas, functionality and technologies to solve specific risk management problems. Speed to market, positioning, and translation into incremental revenues are also important success factors in launching new products.

• Market penetration. Volume (i.e. number of customers) and value (i.e. average deal size) are considered important. Rates of growth relative to sector growth rates are also evaluated. Also covers brand awareness, reputation, and the ability to leverage current market position to expand horizontally (with new offerings) or vertically (into new sectors).

• Financials. Revenue growth, profitability, sustainability, and financial backing (e.g. the ratio of license to consulting revenues) are considered key to scalability of the business model for risk technology vendors.

• Customer satisfaction. Feedback from customers is evaluated, regarding after-sales support and service (e.g. training and ease of implementation), value for money (e.g. price to functionality ratio) and product updates (e.g. speed and process for keeping up to date with regulatory changes).

Table 4: Evaluation criteria for Chartis’ cyber risk quantification solutions report

Completeness of offering Market potential

Scoring/event risk quantification Business model

Risk modeling and aggregation Market penetration

Allocation, attribution, and impact analysis

Financials

Visualization and analytics Customer satisfaction

Financial quantification Growth strategy



• Growth strategy. Recent performance is evaluated, including financial performance, new product releases, quantity and quality of contract wins, and market expansion moves. Also considered are the size and quality of the sales force, sales distribution channels, global presence, focus on risk management, messaging, and positioning. Finally, business insight and understanding, new thinking, formulation and execution of best practices, and intellectual rigor are considered important.

Quadrant construction process

Chartis constructs its quadrants after assigning scores to vendors for each component of the completeness of offering and market potential criteria. By aggregating these values, we produce total scores for each vendor on both axes, which are used to place the vendor on the quadrant.

Definition of quadrant boxes

Chartis’ quadrant reports do not simply describe one technology option as the best solution in a particular area. Our ranking methodology is designed to highlight which solutions are best for specific buyers, depending on the technology they need and the implementation strategy they plan to adopt. Vendors that appear in each quadrant have characteristics and strengths that make them especially suited to that particular category, and by extension to particular users’ needs.

Point solutions

• Point solutions providers focus on a small number of component technology capabilities, meeting a critical need in the risk technology market by solving specific risk management problems with domain-specific software applications and technologies.

• They are often strong engines for innovation, as their deep focus on a relatively narrow area generates thought leadership and intellectual capital.

• By growing their enterprise functionality and utilizing integrated data management, analytics and Business Intelligence (BI) capabilities, vendors in the point solutions category can expand their completeness of offering, market potential and market share.

Best-of-breed

• Best-of-breed providers have best-in-class point solutions and the ability to capture significant market share in their chosen markets.

• They are often distinguished by a growing client base, superior sales and marketing execution, and a clear strategy for sustainable, profitable growth. High performers also have a demonstrable track record of R&D investment, together with specific product or ‘go-to-market’ capabilities needed to deliver a competitive advantage.

• Because of their focused functionality, best-of-breed solutions will often be packaged together as part of a comprehensive enterprise risk technology architecture, co-existing with other solutions.

Enterprise solutions

• Enterprise solution providers typically offer risk management technology platforms, combining functionally rich risk applications with comprehensive data management, analytics and BI.

• A key differentiator in this category is the openness and flexibility of the technology architecture and a ‘toolkit’ approach to risk analytics and reporting, which attracts larger clients.

• Enterprise solutions are typically supported with comprehensive infrastructure and service capabilities, and best-in-class technology delivery. They also combine risk management content, data and software to provide an integrated ‘one stop shop’ for buyers.

Category leaders

• Category leaders combine depth and breadth of functionality, technology and content with the required organizational characteristics to capture significant share in their market.

• They demonstrate a clear strategy for sustainable, profitable growth, matched with best-in-class solutions and the range and diversity of offerings, sector coverage and financial strength to absorb demand volatility in specific industry sectors or geographic regions.

• They will typically benefit from strong brand awareness, a global reach, and strong alliance strategies with leading consulting firms and systems integrators.


Cyber Risk Quantification Solutions, 2019; Market and Vendor Landscape

Spotlight: quantifying cyber risk in financial institutions

Financial Crime Risk Management Systems: AML and Watchlist Monitoring; Market Update and Vendor Landscape, 2019

Financial Crime Risk Management Systems: Enterprise Fraud; Market Update 2018

Financial Crime Risk Management Systems: Trade Surveillance – Transaction Monitoring; Overview and Vendor Landscape, 2019

Artificial Intelligence in Financial Services, 2019: Demand-Side Analysis

For all these reports, see www.chartis-research.com

5. Further reading

http://www.chartis-research.com

Documents

Vendor Analysis: FICO...• Cyber risk management. • Insurance risk. • Regulatory requirements including Basel 2 and 3, Dodd-Frank, MiFID II and Solvency II. Chartis is solely