Immunoscore workflow enhanced by Ar�ficial Intelligence Assil Benchaaben, Felipe Machado Guimaraes, Emmanuel Prestat, Alboukadel Kassambara, Mounia Filahi, Caroline Laugé, Thomas Sbarrato, Jacques FieschiHalioDx, Marseille, France

Assil.Benchaaben@haliodx.com & Jacques.Fieschi@haliodx.com Poster # 870

= –Workflow Turnaround Time (TAT) Automated Tissue Classifica�on

Automated Quality Control

Immunoscore Accuracy

Method

Background

ML Workflow (min) CV Workflow (min)Wai�ng

TimeAnalysis

Time TAT Wai�ngTime

AnalysisTime TAT

Case 1 0 10 10 277 20 302

Case 2 0 15 15 277 12 292

Case 3 0 3 3 172 12 187

Case 4 0 11 11 171 16 191

Case 5 0 17 17 155 16 175

Case 6 0 60 60 123 28 157

Case 7 0 7 7 161 16 181

Case 8 0 4 4 101 16 121

Case 9 0 4 4 42 12 57

Case 10 0 10 10 2 12 17

Total 0 141 141 1481 160 1680

Machine Learning Computer VisionROI Tumor Healthy Epithelium Tumor Healthy EpitheliumCase 1 0.87 0.70 1.00 0.92 0.82 1.00Case 2 0.89 0.82 1.00 0.91 0.88 1.00Case 3 0.81 0.77 0.20 0.83 0.85 0.00Case 4 0.77 0.76 1.00 0.78 0.76 0.60Case 5 0.79 0.69 0.36 0.89 0.65 1.00Case 6 0.80 0.64 0.46 0.80 0.61 1.00Case 7 0.87 0.78 0.85 0.51 0.50 0.36Case 8 0.87 0.77 0.86 0.44 0.54 0.38Case 9 0.89 0.58 1.00 0.89 0.80 0.49Case 10 0.89 0.57 1.00 0.91 0.96 0.57Case 11 0.60 0.35 0.69 0.90 0.74 1.00Case 12 0.76 0.44 0.77 0.87 0.59 1.00Case 13 0.51 0.70 0.76 0.32 0.26 0.64Case 14 0.44 0.74 0.68 0.65 0.36 0.90Case 15 0.83 0.59 0.53 0.72 0.26 0.00Case 16 0.83 0.61 0.72 0.71 0.24 0.00Case 17 0.61 0.58 1.00 0.63 0.66 1.00Case 18 0.73 0.68 1.00 0.73 0.60 1.00Case 19 0.69 0.30 0.07 0.53 0.15 0.00Case 20 0.68 0.44 0.05 0.54 0.36 0.00Median 0.79 0.66 0.76 0.76 0.60 0.62

Original imageGround truth

(Manual annota�on by Expert)Automated detec�on

by Machine Learning trained algorithmAutomated detec�on

by Computer Vision trained algorithm

• IuO was calculated for each ROI between the ground truth and both automated methods on 20 sec�ons from FFPE �ssue blocks of colon cancer.

• ML outperforms CV especially for Healthy Tissue and Normal Epithelium detec�on.

• ML was closer to Ground Truth than Computer vision that could not detect normal epithelium (Green) nor holes (white).• Detec�on of normal �ssue was also be�er with ML requiring far less manual correc�on.

• The ML workflow was trained to detect necrosis, folds and DAB deposits, main sources of false posi�ve cells detec�on.

• Automated annota�ons are validated or corrected by the expert user if needed.

• Automated ar�facts removal results in reduced hands-on �me and improved user experience.

• The analysis of 10 cases was launched simultaneously on both workflows:o CV workflow uses a First Come First Serve scheduling algorithmo ML workflow uses the power of the cloud to run analyses in parallel

• Dashed lines are mean percen�les cut-offs.• Miss-classified samples are highlighted in red.

Mean Percen�le (CV method)

Mea

n Pe

rcen

�le

(ML m

etho

d)

• 37 FFPE �ssue samples from colon cancer pa�ents were evaluated in both CV and ML based methods.

• A high degree of correla�on (r²=0.98 is observed and the overall agreement between methods is 94.6%.

Definition: The TAT is the total �me needed to complete an analysis and includes the actual compu�ng �me but also the wai�ng �me induced by unparalleled processes

• Digital Pathology (DP) has extended the capabili�es of Pathology Labs. For instance. when counts of cells of interest in Regions of Interest (ROI) are required to establish a score.

• Un�l the development of Machine Learning (ML). Computer Vision (CV) was the method of choice to iden�fy ROI on IHC stained �ssue sec�ons counterstained with Hematoxylin.

• HalioDx Immunoscore® is an CE-IVD / CLIA test helpful to assess the risk of relapse of colon cancer pa�ents and to predict the response to chemotherapy1,2. To determine the Immunoscore CD3+ and CD8+ cells must be counted in two ROI: the core of the tumor (CT) and the Invasive Margin (IM).

• In the present work we replaced CV by ML to detect ROI and introduced a new ML based feature to automa�cally subtract staining artefacts (e.g. DAB deposits, Necrosis or folds) from ROI.

• The ML based workflow was validated against a trained operator considered as ground truth and also compared to the ini�al CV based workflow.

• We demonstrated that ML applied to the Immunoscore for ROI detec�on results in reduced �me-to result and overall improved robustness of the analysis.

• Each Digital Pathology step is executed by a dedicated applica�ons

• Posi�ve (IHC-stained) cells are detected by an applica�on based on a proprietary computer vision algorithm

• Tissue Classifica�on and Quality Control applica�ons use a Convolu�onal Neural Network to detect Regions of Interest (ROI) and various ar�facts present on the scan of the stained �ssue sec�on

• The dataset is composed of cases from FFPE �ssue samples of colon cancer pa�ents

• We ran the ML and CV workflows on 10 samples to evaluate the turnaround �me and �ssues classifica�on for both workflows

• We evaluated also the impact of the ML workflow on the Immunoscore accuracy by calcula�ng the overall agreement between both workflows on 37 samples.

1. F. Pagès et al. Annals on Oncology (2020); h�ps://doi.org/10.1016/j.annonc.2020.03.3102. F. Sinicrope et al. JNCI Cancer Spectrum (2020); h�ps://doi.org/10.1093/jncics/pkaa023

References

SAMPLE

Ar�facts Detec�on &

Dele�on

Tissues Classifica�on

Cell Detec�on

ROIs edi�on & review

Pathologist Valida�on

ReviewImmunoscore

Calcula�on

168

14

CV WOR K FL OW M L WOR K FL OW

M EAN T AT PER S AM PL E( M I N UT ES )

0

0,1

0,2

0,3

0,4

0,5

0,6

0,7

0,8

0,9

Tumor Tissue Epithelium

Machine Learning Computer VisionMachine LearningComputer Vision

Example of automa�c ar�facts detec�on and dele�on

Valida�on of the Machine Learning based workflow

The TAT is a cri�cal metric in the context of a Pathology Laboratory

Machine Learning Based Workflow

Datasets for algorithms training and valida�on

Example of automated �ssue classifica�on compared to ground truth established by a trained pathology expert.

Intersec�on over Union (IuO) of automated methods vs ground truth annota�ons

Original Image Ar�facts Annota�ons Ar�facts Dele�on

DAB

y=0.9556x+1.8912

Conclusions This study highlights how Machine Learning can be successfully applied to Digital Pathology in the context of in-vitro diagnos�c

Mean TAT of the analysis workflow is reduced to 14 minutes per case, 12 �mes faster than the reference Computer Vision workflow, with superior performance in terms of �ssue classifica�on and automa�c ar�facts detec�on, reducing hands-on �me and possible human errors

This in-house Deep Learning algorithm will minimize the need for manual annota�on of Digital Pathology images, and improve accuracy of complex �ssue classifica�on

Such an innova�ve tool is applicable both for diagnos�c use to ensure robustness and limit human-induced variability, and for clinical research where the spa�al distribu�on of certain cell types should be precisely quan�fied

Perspec�ves

Necrosis

Fold

deposits