Proficiency Tests in genomics – for GMI · 2018. 9. 28. · • GMI-134, GMI-148, GMI-150, GMI-156, GMI-179 • The individual participants reports for 2017 pending • PT report

9/28/2018

Prof. Rene S. Hendriksen, PhDResearch Group of Genomics EpidemiologyNational Food Institute, Technical University of Denmarkwww.compare-europe.euwww.genomicepidemiology.org

Proficiency Tests in genomics –for GMI

28th Sep 2018, DTU, Denmark

Objectives of GMI PT

• To facilitate the production of reliable laboratory results of consistently good quality within the area of whole genome sequencing (WGS) by

– Selecting two strains of three species of public health importance– Selecting species that range in sequencing difficulties– Assessing the sequencing quality based on a set of quality markers

e.g. N50, no of contigs etc. but also the ability to identify epidemiological markers such as MLST and resistance genes

– Identify participants underperforming

• To facilitate harmonization and standardization of whole genome sequencing and data analysis setting tentative arbitrary quality control thresholds

Structure of GMI PT, wet-lab

Component 1aMaterial provided: Bacterial cultures (lyophilized)• DNA extraction, purification• Library-preparation, and whole-genome-sequencing of six bacterial

cultures

Component 1bMaterial provided: Purified DNA (pre-prepared, dried)• Library-preparation, and whole-genome-sequencing of the same six

bacterial cultures

Results• Submission reads (via a portal or ftp site) • Survey response

– Method details – MLST (optional) – Resistance genes (optional)

http://www.google.dk/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0ahUKEwjR-p-YttjTAhWDB5oKHUWeCJ0QjRwIBw&url=http://clipartall.com/clipart/5801-building-blocks-clipart.html&psig=AFQjCNGf0MiWKZckyv96wOvGSxVU6-J-lw&ust=1494062302703621

Development of GMI PT

2014 – pilot PT20152017

Salmonella (2)E. coli (2)S. aureus (2)

2016 K. pneumonia (2)L. monocytogenes (2)C. coli (1) C. jejuni (1)

http://www.google.dk/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&ved=0ahUKEwihmNysttjTAhWGA5oKHWqfCckQjRwIBw&url=http://www.clipartkid.com/germs-bacteria-cliparts/&psig=AFQjCNG0yNKxnmAoIUgjeCYzTxpYWD-lVQ&ust=1494062408405202

DTU Food, Technical University of Denmark

Participation in the 2016/2017 GMI PT

• 46 laboratories in 22 countries had provided data for at least one of the PT components

– Australia (3), Austria, Belgium (2), Canada (2), Denmark (3), Finland, France, Germany (3), Hong Kong, Italy (7), Latvia, Luxembourg, Mexico, the Netherlands (3), Poland, Portugal, Singapore (2), Sweden (2), Switzerland, Taiwan, the United Kingdom (2), the United States (6)

• 66 laboratories in 27 countries had provided data for at least one of the PT components

– Australia (5), Austria, Belgium (3), Canada (2), China, Denmark (3), Finland, France (2), Germany (3), India, Ireland (3), Italy (8), Japan (2), Luxembourg, Mexico (2), the Netherlands (3), Peru, Poland, Portugal, Scotland, Senegal, Singapore (2), Sweden, Taiwan, the United Kingdom (3), the United States (10)


Participation in the 2017 GMI PT

2016 2017


• Number of reads mapped to – reference total DNA sequence– reference chromosome– reference plasmid #1 – reference plasmid #2 – reference plasmid #3– and unmapped reads

• Proportion of reads mapped to the above• Depth of coverage, of the above• Size of assembled genome • Size of assembled genome per total size of DNA sequence (%)• Total number of contigs• Number of contigs > 200 bp• N50• NG50

Measured QC parameters

http://www.google.dk/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0ahUKEwiMxLvTttjTAhWsApoKHaQbDWsQjRwIBw&url=http://alphaislandemirates.com/responsibilities/quality-control/&psig=AFQjCNGkPmI_NWJK4klLz5SOqIsg_sPrbA&ust=1494062473702915http://www.google.dk/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0ahUKEwiMxLvTttjTAhWsApoKHaQbDWsQjRwIBw&url=http://alphaislandemirates.com/responsibilities/quality-control/&psig=AFQjCNGkPmI_NWJK4klLz5SOqIsg_sPrbA&ust=1494062473702915http://www.google.dk/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0ahUKEwiMxLvTttjTAhWsApoKHaQbDWsQjRwIBw&url=http://alphaislandemirates.com/responsibilities/quality-control/&psig=AFQjCNGkPmI_NWJK4klLz5SOqIsg_sPrbA&ust=1494062473702915

Individual participants reports

Pending for the 2016/ 2017 trial


Sequencing method

Qubit Nanodrop DNA concentration notmeasured

Other0,00%

10,00%

20,00%

30,00%

40,00%

50,00%

60,00%

70,00%

80,00%

90,00%

For bacterial cultures, DNA concentration (ng/µl) prior to library preparation was measured on (please select one

answer)


Sequencing method

Bioanalyser Nanodrop DNA quality not measured Other0,00%

10,00%

20,00%

30,00%

40,00%

50,00%

60,00%

For bacterial cultures, DNA quality (e.g. RIN, 260/280 ratio and/or 260/230 ratio) prior to library preparation was

measured on (please select one answer)


Sequencing method


Sequencing method

Single-end Paired-end Not relevant0,00%

20,00%

40,00%

60,00%

80,00%

100,00%

120,00%

Sequencing details #1 (please select one answer)


QC parameters output

• 2 times standard deviation• 3 times standard deviation


• The proportion of reads produced which map directly to the closed genome of the same strain. (=> cannot exceed 100%)

Proportion of reads mapped to reference DNA sequence (%)


• The proportion of reads produced which map directly to the closed genome of the same strain. (=> cannot exceed 100%)

Proportion of reads mapped to reference DNA sequence (%)

% %

Outlier

Outlier

Consistency across samples• Indication of contamination or strain mix up

• #22 and 87 - missing reads• #31 – too many reads

Outlier

Outlier


• The proportion of contigs which map directly to the closed genome of the same strain (=> should not exceed 100%)

Size of assembled genome per total size of DNA sequence (%)

%%


• The proportion of contigs which map directly to the closed genome of the same strain (=> should not exceed 100%)

Size of assembled genome per total size of DNA sequence (%)

Outlier

%%

Outlier

• Assembly exceed the expected size of the reference - contamination

• #13, #32 Assembly lower that the expected size of the reference – poor performance

• #4; #5, #83

Outlier

Outlier


• Definition: The length for which the collection of all contigs of that length or longer contains at least half of the sum of the lengths of all contigs, and for which the collection of all contigs of that length or shorter also contains at least half of the sum of the lengths of all contigs. A N50 more than 15000 normally indicate good quality.

N50

bp bp


• Definition: The length for which the collection of all contigs of that length or longer contains at least half of the sum of the lengths of all contigs, and for which the collection of all contigs of that length or shorter also contains at least half of the sum of the lengths of all contigs. A N50 more than 15000 normally indicate good quality.

N50

15.000 Outlier

bp bp

Poor performance – short contigs• #22 for the Bact sample

15.000 Outlier


• The total number of contigs assembled. A number of contigs less than 200 normally indicate good quality.

Total number of contigs

bp bp


• The total number of contigs assembled. A number of contigs less than 200 normally indicate good quality.

Total number of contigs

Outlier

bp

200

bp

Poor performance – large number of contigs• #22, #23, #41, #37, #83

200 Outlier

Outlier


N50 No ctgs longest total bps species11905 793 60618 5413102 Escherichia coli58951 268 229096 5259803 Escherichia coli

490118 32 1549500 4703228 Salmonella enterica

”What comes together goes together”


Q1-ScorePhred quality score (Q score) between Q30 and Q40 indicate a 99.9 % base call accuracy with the probability of an incorrect base call in 1 out of 1,000 bp


Q-ScorePhred quality score (Q score) between Q30 and Q40 indicate a 99.9 % base call accuracy with the probability of an incorrect base call in 1 out of 1,000 bp

30

Some laboratories obtained a base calling accuracy, Phred quality score lower than 30 indicating some quality issue with the sequencing

Outlier

30 Outlier


SNP analysis

#83

Strain Sample Type Number of

SNPs

GMI16-001 Culture 3 DNA 0

Number of SNPs per strain

3 SNPs difference to the ref. (#83)

#83

Strain

Sample Type

Number of SNPs

GMI16-001

Culture

3

DNA

0


Overall Score – based on CDC scoring(different user codes)


Overall Score 2017, Salmonella 001/002

100 99-90 89-80 79-70 69-60 59-50 49-40 39-30 29-20 19-10 9-0


• GMI17-001 BACT:– GMI-150

• GMI17-001 DNA:– GMI-134, GMI-148, GMI-150, GMI-179

• GMI17-002 BACT:– GMI-150

• GMI17-002 DNA:– GMI-150

• 5 outliers

Summary 2017, Salmonella 001/002(different user codes)


Overall Score 2017, E. coli 003/004

100 99-90 89-80 79-70 69-60 59-50 49-40 39-30 29-20 19-10 9-0


• GMI17-003 BACT:– GMI-150, GMI-156,

• GMI17-003 DNA:– GMI-148, GMI-150, GMI-179

• GMI17-004 BACT:– GMI-150,

• GMI17-004 DNA:– GMI-134, GMI-148, GMI-150, GMI-179

• 5 outliers

Summary 2017, E. coli 003/004(different user codes)


Overall Score, S. aureus 005/006

100 99-90 89-80 79-70 69-60 59-50 49-40 39-30 29-20 19-10 9-0


• GMI17-005 BACT:

• GMI17-005 DNA:

• GMI17-006 BACT:– GMI-179

• GMI17-006 DNA:– GMI-179

• 1 outliers

Summary 2017, S. aureus 005/006(different user codes)


Summary of PT 2017

• Most laboratories performed well but six labs seems to have problems with

– Contaminations– Poor WGS performance – low yield – Need for optimization of WGS

• GMI-134, GMI-148, GMI-150, GMI-156, GMI-179

• The individual participants reports for 2017 pending

• PT report 2016 / 2017 published as a scientific paper in 2017

• No PT in 2018 – Consideration to terminate the PT– If continued in 2019

• focus on K. pneumonia, L. monocytogenes, C. coli / C. jejuni

AcknowledgementSusanne Klarsmose Pedersen (DTU Food)

Pimlapas Leekitcharoenphon (DTU Food)

Rolf Sommer Kaas (DTU Food)

Inge Marianne Hansen (DTU Food)

Jacob Dyring Jensen (DTU Food)

Frank Aarestrup (DTU Food)

Ole Lund (DTU Systems Biology)

Jose Luis Bellod Cisneros (DTU Systems Biology)

James Pettengill (US FDA)

Division of Microbiology (CFSAN/FDA)

Anthony Underwood (PHE)

Brian Beck (Microbiologics)

Isabel Cuesta de la Plaza (ISCIII)

Angel Zaballos (ISCIII)

Jorge De La Barrera Martinez (ISCIII)

…..and the rest of WG 4 (‘advisory group’)

GMI is supported by:


Thank you for your attention

Oksana Lukjancenko, PhD

Research Group Genomic Epidemiology

WHO Collaborating Centre for Antimicrobial Resistance in Food borne Pathogens

and Genomics

European Union Reference Laboratory for Antimicrobial Resistance

National Food Institute, Technical University of Denmark

[email protected]

mailto:[email protected]

Slide Number 1Objectives of GMI PTStructure of GMI PT, wet-labDevelopment of GMI PTParticipation in the 2016/2017 GMI PT�Participation in the 2017 GMI PT�Measured QC parametersIndividual participants reports�Sequencing methodSequencing methodSequencing methodSequencing methodQC parameters outputProportion of reads mapped to reference DNA sequence (%)Proportion of reads mapped to reference DNA sequence (%)Size of assembled genome per total size of DNA sequence (%)Size of assembled genome per total size of DNA sequence (%)N50Slide Number 19N50Total number of contigsTotal number of contigs”What comes together goes together”Slide Number 24Slide Number 25SNP analysisSlide Number 27Slide Number 28Slide Number 29Slide Number 30Slide Number 31Slide Number 32Slide Number 33Summary of PT 2017GMI is supported by:Slide Number 36

Documents

Proficiency Tests in genomics – for GMI · 2018. 9. 28. · • GMI-134, GMI-148, GMI-150, GMI-156, GMI-179 • The individual participants reports for 2017 pending • PT report