Current DICOM Topics: 1. The new surface segmentation ... · • Segmentation algorithm used to create the surface • Whether post processing has been done on the segmented surface

RADIOLOGY

RESEARCH

Current DICOM Topics:

1. The new surface segmentation class

2. Compression: JPEG 2000 & MPEG2

Donald Peck, PhD

Michael Flynn, PhD

AA

PM

20

08

2

Recent DICOM Enhancementsthat are important to Medical Physicists

• CT Radiation Dose Reporting

• X-Ray 3D Storage SOP Class

• Email Transport

• Enhanced PET Image

• Deformable Spatial Registration Storage SOP Class

• Breast Tomosynthesis Image Storage SOP Class (Ballot)

• Segmentation Storage SOP Class

• Surface Segmentation Storage SOP Class (Ballot)

• JPEG 2000 Interactive Protocol

• JPEG 2000 Part 2 Multi-component Transfer Syntaxes

AA

PM

20

08

3

Part I

• Surface Segmentation Storage SOP Class

Part II

• Compression: JPEG 2000 and MPEG2

AA

PM

20

08

4

DICOM Review

• Utilizes an open standards developmentprocess that encourages the involvementand consensus of both vendors and users

• Specification of a conformance mechanismso that a user can decide whether or notdevices are likely to interoperate

AA

PM

20

08

5

DICOM Review

• DICOM Standards Committee (DSC)

• DICOM’s executive body whose membersare imaging equipment manufacturers,physician organizations, and others

• 26 manufacturer members

• 20 other members

• DICOM's activities are coordinatedthrough a general secretariat at MITA

• formerly NEMA

AA

PM

20

08

6

DICOM Standards Committee (DSC)

• Development of DICOM Standard is donethrough committees termed “WorkingGroups” (WG)

• AAPM Members can participate in any WGto develop the Standard

• WG propose work items based onsuggestions from members or at thedirection of the DSC

AA

PM

20

08

7

DICOM “workflow”

• Work items are divided into Supplementsand Change Orders

• Supplements define new objects, content orstructure

• Change Orders modify the existingStandard

AA

PM

20

08

8


• Output of any work item is submitted tothe Base Standards Working Group (WG6)for review throughout the process

• WG6 may request changes or clarification tothe work item to ensure it complies withStandard

• Once the work item is complete WG6petitions the DSC to approve work item andsend out for public comment

AA

PM

20

08

9


• After public comment and all issues listed in thecomments are responded to, the DSC authorizesthe work item for letter ballot by DICOMmembers

• Change proposals do not require the same rigorousreview process and are often “batched” for publiccomment and letter ballot,

• Some Change proposals can be instituted withoutcomment or ballot if the change is required toreduce potential patient care issues or it istypographical or trivial in nature

AA

PM

20

08

10

Changes to Standard

• Currently over 100 approved Supplementsincorporated into Standard

• Over 25 in progress or in comment stage

• Currently over 600 Change Proposalsincorporated into Standard

• Ongoing process with changes beingreviewed at every WG6 meeting

AA

PM

20

08

11

DICOM Parts

• PS 3.1 Introduction and Overview

• PS 3.2 Conformance

• PS 3.3 Information Object Definitions

• PS 3.4 Service Class Specifications

• PS 3.5 Data Structures and Encoding

• PS 3.6 Data Dictionary

• PS 3.7 Message Exchange

• PS 3.8 Network Communication Support for Message Exchange

• PS 3.10 Media Storage and File Format for Data Interchange

• PS 3.11 Media Storage Application Profiles

• PS 3.12 Media Formats and Physical Media for Data Interchange

• PS 3.14 Grayscale Standard Display Function

• PS 3.15 Security Profiles

• PS 3.16 Content Mapping Resource

• PS 3.17 Explanatory Information

• PS 3.18 Web Access to DICOM Persistent Objects (WADO)

Describes how theStandard is organized

AA

PM

20

08

12

Changes to the Standard

On a yearly basis all changes to the Standard arepublished at: http://medical.nema.org/dicom/

AA

PM

20

08

13

Surface Segmentation Storage SOP Class

• Supplement 132, currently under revisionfollowing Letter Ballot

• Developed by a joint effort of WG 17 (3D)and WG 24 (Surgery)

AA

PM

20

08

14

Current DICOM Objects

• All images are stored as 2D frames

• If more than one image is acquired, and there issome information linking the images (e.g. crosssectional images, temporal images, etc.) the imagescan be stored as multi-frame objects

• The images are still single 2D frames

AA

PM

20

08

15

Current DICOM Objects

• 3D reconstructions can be created from the multi-frame objects

• Yet results can only be saved as “snapshots” ofspecific views (i.e. single- or multi-frame images)

AA

PM

20

08

16

Rendering of 3D objects

• Surface renderingsare commonlyderived frompolygonal surfaceprimitives

• Need a method inDICOM to storethese primitivesinstead of images

From Osirix

AA

PM

20

08

17

Surface Segmentation Storage SOP Class

• All surfaces used for 3D objects firstrequire the objects to be segmented

• Supplement introduces a new Surface MeshModule that may be used to encode thesegmentation data

• The Surface Segmentation can specify asurface derived from any DICOM modalityor from non-DICOM measurementtechniques.• Can be used to specify designed surfaces,

e.g. implants.

AA

PM

20

08

18

The supplement supports the following features:

• The segmentation shapes are encoded in apolygonal representation of the surface.

• More than one segmentation object is supportedper Imaged Object Definition (IOD) instance.

• Position and orientation are defined within thepatient coordinate system as specified by theFrame of Reference.

• Segmentation is in the patient space rather than theimage data, i.e. it is not slice based.

AA

PM

20

08

19

Surface Mesh Primitives

• Encoded information was developed to alloweasy utilization of the object data bygraphic hardware and software

• The encoded data has the format of simple“primitive” structures.

AA

PM

20

08

20


• Vertex - a single vertex, referencing a single point

• Edge - an edge, referencing two points

• Line - a series of connected points describing a path

AA

PM

20

08

21


• Triangle - a triangle, referencing three points

• Triangle Strip - a triangle strip with n triangles

AA

PM

20

08

22


• Triangle Fan- a set of triangles where one point iswithin each triangle, i.e. the center of the fan.

• Facet - a planar polygon, referencing n points

A Facet can be easily madefrom Triangles, but inclusionof this primitive allows less

points to be saved

AA

PM

20

08

23

Information Encoded

• Segmentation algorithm used to create the surface

• Whether post processing has been done on thesegmented surface

• Recommended display/presentation properties

• If the surface is a finite volume or is a manifold

• Normal to surface

AA

PM

20

08

24

Segmentation Algorithms

1. Neighborhood Analysis2. Adaptive Filtering3. Edge Detection4. Morphological Operations5. Histogram Analysis6. Multi-Scale/Resolution Filtering7. Cluster Analysis8. Multispectral Processing9. Manual Processing10. Artificial Intelligence11. Deformable Models

This is a required attribute, i.e. Type 1

AA

PM

20

08

25

Surface processing

• Surface modification such as smoothingoperations, which remove redundantvertices, or decimation which will modifythe resolution of the surface.

• Can reduce the amount of data needed todefine the surface

Algorithm description and data reduction ratioare required if this processing is done,

i.e. this attribute is a Type 2

AA

PM

20

08

26

Recommended Presentation

• Recommended Presentation Opacity

• Percentage of transmission of light through thesurface.

• Recommended Presentation Type• Defined terms are:

• SURFACE; Render the surface as a solid, applying theopacity specified.

• WIREFRAME; Represent the surface as a series oflines connecting the vertices to form the definedprimitive faces.

• POINTS; Represent the surface as a cloud of points.

These are all required attributes

AA

PM

20

08

27

Finite Volume

• A Finite Volume is defined when thesurface mesh generated by the primitivesis topologically closed and has an insideand an outside.• A surface mesh is closed if it has no rim

(every facet has a neighboring facet alongeach edge).

• Not all closed surfaces contain a finitevolume or are required to considered afinite volume

These points can be defined to contain

one or two closed/finite volumes

This is a required attribute

AA

PM

20

08

28

Manifold objects

• A surface embedded into an n-dimensional vectorspace is called an n-1 manifold if it resembles ann-1 dimensional Euclidian space in a neighborhoodof every point lying on the surface.

• A sphere in 3D-space is a 2-dimensional manifold:Every point has a neighborhood that looks like a plane.

Shared points along central surface

makes these two tetrahedrons

Non-Manifold

Object is a Manifold

This is a required attribute

AA

PM

20

08

29

Surface normal determinations

• If the “normal” for every surfacepoint is provided, the normal for aprimitive may be computed bycombining the normal's for eachvertex making up the primitive.

• If the Normal for all surface pointsare not provided, the normal for aprimitive shall be determined bycomputing the cross product of theprimitive segments.

• The primitive vertices are taken inthe order specified within thePrimitive Point Index List attribute.

AA

PM

20

08

30

Current Issues

• Back from Letter Ballot

• Formatting of the Supplements grammaticalstructure was a concern, so edits are inprogress

• Should be approved October 2008

AA

PM

20

08

31

Part I

• Surface Segmentation Storage SOP Class

Part II

• Compression: JPEG 2000 and MPEG2

• Previous JPEG 2000 compression standardsare now extended to support progressiveweb communication using JPIP.

• Previous MPEG2 standards are beingextended to support high resolution objects.

AA

PM

20

08

32

Topics

A. Basic Concepts

B. JPEG 2000

C. JPEG 2000 p2 – JPIP

D. JPEG 2000 p2 – 3D

E. MPEG2 MPML & MPHL

F. Image Quality

AA

PM

20

08

33

Encode - Decode

• Encode

Transform the digital image pixel data intoa binary code stream of reduced size.

• Decode

Interpret a compressed code stream toretrieve the original image pixel data.

AA

PM

20

08

34

Image management

A basic image management system will store andtransmit image data in the acquired image format

ACQUISITION STORAGE TRANSMISSION PRESENTATION

8 MB 8 MB 8 MB 8 MB

AA

PM

20

08

35

Image management

Reversible (lossless, bit-preserving) compression

• Storage and transmission size reduced by a factor of 2-3

• Decoded images values are identical to the original


8 MB 3 MB 3 MB 8 MB

AA

PM

20

08

36

Image management

Irreversible (lossy) compression

reduces bit storage and transmission by a factor of 10 ormore, however the decoded image values are slightly altered.


8 MB 0.5 MB 0.5 MB 8 MB

AA

PM

20

08

37

Compression ratio

• Compression ratio,

The size of the original image data is related tothe size of the encoded data

2:1 , 8:1 , 16:1 , 80:1

• Bit-rate,

The average bits per pixel computed from thesize of the encoded data.

8 , 2 , 1.0 , 0.2

The bit-rate is generally a better indicator ofimage distortion for irreversible compression.

AA

PM

20

08

38

Topics

A. Basic Concepts

B. JPEG 2000


D. JPEG 2000 p2 – 3D


F. Image Quality

AA

PM

20

08

39

DICOM SUP 61 (2002), JPEG 2000

• Retired 14 unused JPEG transfer syntaxes

• Retained;• 1.2.840.10008.1.2.4.50 JPEG baseline 8 bit

• 1.2.840.10008.1.2.4.51 JPEG baseline 12 bit

• 1.2.840.10008.1.2.4.70 JPEG lossless

• 1.2.840.10008.1.2.4.80 JPEG LS lossless

• 1.2.840.10008.1.2.4.81 JPEG LS lossy

• 1.2.840.10008.1.2.5 RLE lossless

• Added;• 1.2.840.10008.1.2.4.90 JPEG 2000 lossless

• 1.2.840.10008.1.2.4.91 JPEG 2000

AA

PM

20

08

40

DICOM part 3.5 – Data Structures and Encoding

From DICOM part 3.5 we learn,

• Pixel data conveyed in the Pixel Data Element(7FE0,0010) may be sent either in a Native(uncompressed) Format or in an EncapsulatedFormat (e.g. compressed) defined outside theDICOM standard.

• If sent in an Encapsulated Format (i.e. other thanthe Native Format) the Value Representation OB isused. The Pixel Cells are encoded according to theencoding process defined by one of the negotiatedTransfer Syntaxes (see Annex A).

AA

PM

20

08

41


DICOM part 3.5 further explains,

• Inclusion of a JPEG 2000 coded image in a DICOMmessage is indicated by the use of specificTransfer Syntaxes that are defined in Annex A.

• Only the features defined in JPEG 2000 Part 1(ISO/IEC 15444-1) are permitted for [the twoJPEG 2000] Transfer Syntaxes.

• Wavelet based multiple component transformationsusing the same two wavelet filters as used in Part 1of JPEG 2000 (5-3 reversible wavelet and 9-7irreversible wavelet).

• A single JPEG 2000 process is used for bit depthsup to 16 bits.

AA

PM

20

08

42


DICOM part 3.5 further explains,

• Note in particular that the JP2 file header isnot sent in the JPEG 2000 bitstream that isencapsulated in DICOM.

*.jpg – image compressed using the jpeg standard.

*.jp2 – image compressed using jpeg 2000.

‘Similar to JPEG-1, JPEG2000 defines both afile format and a code stream. Whereas thelatter entirely describes the image samples, theformer includes additional meta-information asthe resolution of the image ... ‘

en.wikipedia.org/wiki/JPEG2000

AA

PM

20

08

43

J.P.E.G.

• The Joint Photographic Experts Group (JPEG) is a jointcommittee between ISO and ITU-T which created theJPEG and JPEG 2000 standards.

• Its proper name is ISO/IEC Joint Technical Committee 1,Subcommittee 29, Working Group 1.

• Resources;

• www.jpeg.org

• the jpeg committee home page

• www.ece.uvic.ca/~mdadams/jasper/

• the Jasper project,

• open source JPEG 2000 reference implementation.

• www.kakadusoftware.com

• highly conformant, very fast,

• nearly free software for JPEG 2000 and JPIP .

AA

PM

20

08

44

DICOM SUP 61 (2002), JPEG 2000

The introduction of Supplement 61 notes;

• The use of [JPEG 2000] does not necessarily result inimproved compression performance for any particularapplication in terms of

• quantitative or qualitative measures of image fidelity,

• preservation of diagnostically significant information,

• consumption of resources (memory, [encode/decode] speed).

• However, JPEG 2000 offers additional features that maybe important for some medical applications. Thesefeatures include;

• progressive and embedded spatial and contrast resolution,

• progression to lossless reconstruction,

• regions of interest and ...

AA

PM

20

08

45

JPEG 2000 - features

Multiple resolution representation

• JPEG2000decomposes the imageinto a multipleresolutionrepresentation in thecourse of itscompression process.

• This representationcan be put to use forother imagepresentation purposesbeyond compressionas such.

AA

PM

20

08

46

JPEG 2000 - features

Progressive transmission by pixel and resolution accuracy

•JPEG2000 provides efficientcode-stream organizations whichare progressive by pixel accuracyand by image resolution.

•After a small part of the wholefile has been received, the viewercan see a low quality version ofthe final picture.

•The quality then improvesprogressively by downloadingmore data bits from the source.

AA

PM

20

08

47

Image management

Progressive transmission

• Some clients can view the original image

• Other clients can view with slightly altered image values


8 MB 3 MB 3 MB 8 MB

AA

PM

20

08

48

Topics

A. Basic Concepts

B. JPEG 2000


D. JPEG 2000 p2 – 3D


F. Image Quality

AA

PM

20

08

49

DICOM SUP 106 (2006), JPIP

DICOM JPIP REFERENCED TRANSFER SYNTAX

1.2.840.10008.1.2.4. 94

“This Supplement extends the Pixel Data Module toallow reference to a JPIP URL to access pixel data,rather than encoding it in the image instance.”

“Data Element (7FE0,0010) Pixel Data shall not bepresent, but rather pixel data shall be referenced viaData Element (0028,7FE0) Pixel Data Provider URL”

AA

PM

20

08

50

DICOM with JPIP: Diagram

Weisfeiler, Dicom WG4, Feb 2006

AA

PM

20

08

51

JPIP Application Block Diagram

Weisfeiler, Dicom WG4, Feb 2006

AA

PM

20

08

52

Image Streaming GUIs

JPIP facilitates advanced image viewing GUIs

AA

PM

20

08

53

DICOM PS 3.18 - 2008

Web Access to DICOM Persistent Objects (WADO)

Annex B - Examples (Informative)B.3 RETRIEVING A REGION OF A DICOM IMAGE

Retrieving a region of a DICOM image, converted if possible in JPEG2000,with annotations burned into the image containing the patient name and technicalinformation, and mapped into a defined image size:

https://aspradio/imageaccess.js?requestType=WADO&studyUID=1.2.250.1.59.40211.12345678.678910&seriesUID=1.2.250.1.59.40211.789001276.14556172.67789&objectUID=1.2.250.1.59.40211.2678810.87991027.899772.2&contentType=image%2Fjp2;level=1,image%2Fjpeg;q=0.5&annotation=patient,technique&columns=400&rows=300&region=0.3,0.4,0.5,0.5&windowCenter=-1000&windowWidth=2500

AA

PM

20

08

54

Topics

A. Basic Concepts

B. JPEG 2000


D. JPEG 2000 p2 – 3D


F. Image Quality

AA

PM

20

08

55

DICOM SUP 105 (2005), JPEG 2000 p2 - Multi-component

This supplements adds support for themulti-component extensions in part 2 ofthe JPEG 2000 standard.

• 1.2.840.10008.1.2.4.92

JPEG 2000 Part 2 MulticomponentImage Compression (Lossless Only)

• 1.2.840.10008.1.2.4.93

JPEG 2000 Part 2 MulticomponentImage Compression

AA

PM

20

08

56

DICOM SUP 105 (2005), JPEG 2000 p2 - Multi-component

Multicomponent transforms for 3D compression

• The multi-component transformations in Part 2 of JPEG2000 can be effective in compressing volumetricdatasets because the correlation between adjacentimages can be exploited to achieve better compressionthan if each image were compressed independently.

• The use of component collections is advantageous for3D imagery that is highly correlated over a smallnumber of components (slices), but less correlated overthe entire sequence. Image data can then be retrievedfrom within the 3D stack (unlike video codecs)

AA

PM

20

08

57

3D Compression performance

512 x 512 CT, 620 slices

Tzannes, RSNA, 2003

15:1 40:1

AA

PM

20

08

58

Topics

A. Basic Concepts

B. JPEG 2000


D. JPEG 2000 p2 – 3D


F. Image Quality

AA

PM

20

08

59

DICOM SUP 42 (2004), MPEG2 MP@ML

Supplement 42

• Working Group 13, Visible Light, 2004

• A DICOM Transfer Syntax for MPEG2Image Compression shall be identifiedby a UID value of

1.2.840.10008.1.2.4.100

corresponding to MPEG2 MP@ML optionof the ISO/IEC MPEG2 Video standard.

• Annex X, DVD MPEG2 InterchangeProfiles (includes DICOMDIR).

AA

PM

20

08

60

MPEG2

• MPEG2 is a widely used format forbroadcast and cable television and for DVDvideo programs.

• MPEG2 Profiles and Levels

• The Main profile (MP) applies to square pixelsin 4:3 or 16:9 format (not scalable).

• Main Level (MP) – 720 x 576 (15 Mb/S) max

• High Level (HL) – 1920 x 1152 (80 Mb/S) max

AA

PM

20

08

61

DICOM SUP 137 (2008 public comment), MPEG2 MP@HL

• This Supplement describes a newtransfer syntax to embed an additionalhigher level of MPEG2 encoding inDICOM.

• The syntax uses MPEG2 lossycompression permitting higher spatialresolution quality, called Main Profile /High Level (MP@HL), that does notexceed 80 Mbit/s. This goes beyond thedata rates typically defined by the "Blu-ray” consortium.

AA

PM

20

08

62

DICOM SUP 137 (2008 public comment), MPEG2 MP@HL

• The syntax would typically be used forencoding HD video sequences, though itis not constrained from encoding otherforms of multi-frame pixel data.

• Targeted modalities include microscopicand endoscopic applications such asGastro-Enterology, Laparoscopy,Orthopedics, Ophthalmology, ENT,Gynaecology, Bronchoscopy, Pathology,and (Microscopic) Surgery.

AA

PM

20

08

63

MPEG2 vs MPEG4

MPEG4 is widely used in computer videoapplications. It was not used in thissupplement because;

• MPEG2 is supported by well-establishedequipment and applications. In particular,tools for editing MPEG2 video are moreavailable in the short term than for MPEG4.

• By comparison with MPEG2, softwaredecoding of MPEG4 is more computationallyintensive and may be possible in the shortterm only with high end computer terminalsor with hardware assistance.

AA

PM

20

08

64

Topics

A. Basic Concepts

B. JPEG 2000


D. JPEG 2000 p2 – 3D


F. Image Quality

AA

PM

20

08

65

Foos, SPIE, 2000

No Difference

Just Noticeable

Conspicuous

Significant

Unacceptable 1

2

3

4

5

6

7

8

9

0.00 0.50 1.00 1.50 2.00 2.50

CR Portable ChestCR ShoulderThoravision

Digitized FilmFlat Panel Digital

f97

f97_csf

f97_csf_msk

wtcq_lra_csf

I53_csf

jpeg_csf

Statistical Groupings

Average Ratings7 obs , 5 imgs

Bits-Per-Pixel

Compressed Radiographic Image Quality

AA

PM

20

08

66

IQ

Canon

50

g,16

0m

icro

npix

el

SideToSide

1 b/pix

16:1

AA

PM

20

08

67

IQ

Canon

50

g,16

0m

icro

npix

el

Replace

1 b/pix

16:1

AA

PM

20

08

68

IQ

Canon

50

g,16

0m

icro

npix

el

Flicker

1 b/pix

16:1

AA

PM

20

08

69

IQ

Canon

50

g,16

0m

icro

npix

el

Subtract

1 b/pix

16:1

AA

PM

20

08

70

Canadian Compression StandardCanadian Compression Standard

Canadian Association of RadiologistsStandards for Irreversible

Compression in Digital DiagnosticImaging within Radiology

Approved June 2008

www.car.ca

• Two independent literature reviews, sponsored by Canada HealthInfoway, were completed as a part of the study

• Two independent and comprehensive legal reviews, sponsored byCanada Health Infoway, were completed as a part of thebroader study into the use of irreversible compression.

• Reference ACR and FDA positions on irreversible compression.

• CAR sponsored a pan Canadian clinical evaluation on JPEG-2000irreversible compression in medical imaging. The evaluationproject was directed by Dr. David Koff, Sunnybrook Hospital .

AA

PM

20

08

71

Questions?

?

AA

PM

20

08

72

Comment

• AAPM has recently applied tobecome a member organization ofthe DICOM Standards Committee.

• Members are being solicited forparticipation in specific workinggroups that are closely aligned withMedical Physics.

Documents

Current DICOM Topics: 1. The new surface segmentation ... · • Segmentation algorithm used to create the surface • Whether post processing has been done on the segmented surface