Transcript Standards for the Uninitiated - Digital Pathology Association

Supplement 145 Whole Slide Imaging –
background and design decisions
Harry Solomon
GE Healthcare
DICOM BASICS
2
Sup 145 WSI
Harry Solomon
DICOM Image Information Object Definition
DICOM Composite
Information Model
Hierarchy
Patient
Information
Patient
Module
Study
Information
General
Study
Module
Series
Information
Image
(Instance)
Information
General
Series
Module
General
Image
Module
Modality
Image
Module
3
Sup 145 WSI
Harry Solomon
Patient Name
Patient ID
Patient Sex
Patient Birthdate
Patient
Study
Module
General
Frame of
Reference Equipment
Module
Module
Image
Plane
Module
Multiframe
Module
Study Unique ID
Accession Number
Study Date/Time
Study Description
Referring MD
Contrast/
Bolus
Module
VOI
LUT
Module
Rows/Columns
Bits per Pixel
Photometric
Image
Pixel
Module
…
SOP
Common
Module
Dwight Simon
Data Element Encoding
Data Set
order of transmission
Data Elem. Data Elem.
Data Elem.
Data Element
Value
Tag
Representation
Value
Length
Data Elem.
Value Field
Attributes are the logical
concepts associated
with an information
entity
Data elements are how
attributes are encoded in
an information object
Similar to TIFF
optional field - dependent on
negotiated Transfer Syntax
0020000Dhex UI
Study Instance
Unique Identifier
(0020,000D)
4
Sup 145 WSI
Harry Solomon
26hex 1.2.840.1.113709.9.0.0.5743.14575602.1
Instance UID encoded as “dotted decimal”
Part of a DICOM object
(0008,0005)
CS
30
ISO 2022 IR 13\ISO 2022 IR 87
(0008,0008)
CS
22
ORIGINAL\PRIMARY\AXIAL
(0008,0016)
UI
26
1.2.840.10008.5.1.4.1.1.2
(0008,0018)
UI
58
1.2.392.200036.9116.2.6.1.48.1211393615.1211874194.564494
(0008,0020)
DA
8
20080527
(0008,0021)
DA
8
20080527
(0008,0022)
DA
8
20080527
(0008,0023)
DA
8
20080527
(0008,0030)
TM
10
163836.000
(0008,0031)
TM
10
164306.390
(0008,0032)
TM
10
164039.850
(0008,0033)
TM
10
164040.397
(0008,0050)
SH
8
5162581
(0008,0060)
CS
2
CT
(0008,0070)
LO
8
TOSHIBA
5
OW
Sup 145 WSI
Harry Solomon
524288
Series
Information
Image
(Instance)
Information
Tags in increasing numeric order
Value length always an even number
Attributes related to modules and
information model levels all jumbled up
…
(7FE0,0010)
Patient
Information
Study
Information
00 00 00 00 ff ff ff ff 00 0f 4c 4a 49 46 00 01...
Attributes
Logical concepts in the description of an Information
Entity
May have 0, 1 or many Values
• 0 (empty) means the creating application doesn’t know the value
of the attribute, e.g. Accession Number (0008,0050)
• Multi-value example: Specific Character Set (0008,0005)
• Value Multiplicity (VM) specified in Part 6 (possibly further
constrained in Part 3)
Attribute value will be a complex data structure for a
Sequence attribute
6
Sup 145 WSI
Harry Solomon
Sequence Attributes and Items
Sequence attribute has a “value” of a structure of subsidiary
attributes
• Sequence Attribute name typically includes word “Sequence”
• Subsidiary attributes specified in Part 3 with > character
Each instantiated set of attributes is a Sequence Item
Number of allowed Items specified in Part 3
For editorial convenience the attributes of a Sequence are often
documented in a separate Table as a Macro
• Include ‘x Macro’ Table m-n
• Facilitates reuse of structure in other sequence attributes
7
Sup 145 WSI
Harry Solomon
Example: Scheduled Protocol Code Sequence attribute
8
Sup 145 WSI
Harry Solomon
Scheduled Protocol Code Sequence attribute
Scheduled Protocol Code Sequence
(0040,0008)
>Code Value
(0008,0100)
>Coding Scheme Version
(0008,0103)
>Coding Scheme Designator
(0008,0102)
>Code Meaning
(0008,0104)
>Protocol Context Sequence
(0040,0440)
>>Value Type
(0040,A040)
>>Concept Name Code Sequence
(0040,A043)
>>>Code Value
(0008,0100)
>>>Coding Scheme Designator
(0008,0102)
>>>Coding Scheme Version
(0008,0103)
>>>Code Meaning
(0008,0104)
>>DateTime
(0040,A120)
>>Person Name
(0040,A123)
>>Text Value
(0040,A160)
>>Concept Code Sequence
(0040,A168)
>>>Code Value
(0008,0100)
…
9
Sup 145 WSI
Harry Solomon
>>Content Item Modifier Sequence
(0040,0441)
>>>Value Type
(0040,A040)
>>>Concept Name Code Sequence
(0040,A043)
>>>>Code Value
(0008,0100)
>>>>Coding Scheme Designator
(0008,0102)
…
Sequence attribute encoding
Sequence Items are the “values” of Sequence attributes
• Structure placed in the Data Element Value Field
Item structure is a “nested data set” of attributes
• Attributes in each Item in tag order
• Item “wrapped” using special data elements specified in Part 5
Sequence attributes and wrappers may have an “undefined length” flag
• Length of Sequence or Item terminated by explicit Delimiter data elements
May be “undefined length”
Sequence Data Element
Value
Tag
Represen-
tation SQ
Value
Length
Item
Introducer
Value Field
Attribute
1
Attribute
2
Specifies length of Item, or
may say “undefined length”
Item
Delimiter
Item
Introducer
Attribute
1
Required if “undefined
length” Item
Attribute
2
Item
Delimiter
Sequence
Delimiter
Required if “undefined length”
Sequence Attribute
Image Compression
Pixel data can be monochrome, color (RGB or YCbCr), or
palette color (monochrome colorized through LUT)
• No definitions yet for hyperspectral, but it has been discussed
Pixel data can be ‘native DICOM’ (with color by-plane or bypixel)
Pixel data can be compressed using standard compression
schemes, and compressed stream put in pixel data element
• JPEG, JPEG-LS, JPEG2000 (each lossy or lossless)
• MPEG2
• Run-Length Encoding (Packbits)
Private compression schemes can also be used
11
Sup 145 WSI
Harry Solomon
Compressed Image Encoding
Uses structure similar to Sequence attribute
• Allows “undefined length” attribute – eliminates 232 byte limitation
• 1st Item is ‘Basic Offset Table’ - pointers to individual frames of a multi-frame
image (optional)
JPEG and JPEG2000Part1 encode each frame of a multi-frame image in a
separate Encapsulated Stream Fragment
JPEG2000Part2 (multicomponent) allows arbitrary mapping of frames to
stream fragments to allow component collections (inter-frame compression)
May be “undefined length
Pixel Data Element
Value
Tag
Represen(7FE0,0010) tation OB
Value
Length
Item
Introducer
Basic Offset
Table
Specifies length of
Basic Offset Table
Value Field
Item
Introducer
Encapsulated Stream
Fragment 1
Specifies length of
Stream Fragment
Item
Introducer
Encapsulated Stream
Fragment 2
Sequence
Delimiter
Required if “undefined length”
Pixel Data Attribute
MULTIFRAME IMAGES
13
Sup 145 WSI
Harry Solomon
Enhanced Multi-frame paradigm
Basic concept used for all new multiframe IODs
• MR (Image and Spectroscopy), CT, XA, US, PET
Multi-frame object to support 1000+ image studies
• Dynamic image header supports functional or acquisition
attributes changing during scan
• Dimensions allow multiple views of data
File size flexibility through concatenations
14
Sup 145 WSI
Harry Solomon
Single-frame to MultiFrame
N Objects, N Headers
N Frames, One Header
Fixed Header
15
Sup 145 WSI
Harry Solomon
Per-frame header
Dimension data
Pixel data
Functional Groups and the Per-Frame Header
Other attributes
Shared Functional Groups Sequence
> Functional Group A
…..
Functional Groups with
attributes constant for all
frames
> Functional Group K
Per-frame Functional Groups Sequence
Item 1 (Frame 1)
> Functional Group B
> Functional Group C
Sequence of
Functional Groups
for each individual frame
…..
> Functional Group M
Item 2 (Frame 2)
…..
> Functional Group B
> Functional Group C
…..
> Functional Group M
…..
Pixel Data
Frame 1
Item n (Frame n)
> Functional Group B
> Functional Group C
…..
> Functional Group M
Other attributes
16
Sup 145 WSI
Harry Solomon
Frame 2
…..
Frame n
Functional Groups
Collection of set of closely related attributes
• A “mini Module”
Structured as a sequence of (usually 1) item under a main
Sequence attribute
Invoked as a ‘Macro’ in either Shared Functional Groups
Sequence or Per-Frame
Functional Groups
Sequence
Table C.7.6.16-8
CARDIAC SYNCHRONIZATION MACRO ATTRIBUTES
Attribute Name
Keeps items together in
encoding under the main
Sequence attribute
17
Sup 145 WSI
Harry Solomon
Tag
Type
Cardiac Synchronization Sequence
(0018,9118)
1
>Nominal Percentage of Cardiac Phase
(0020,9241)
1C
>Nominal Cardiac Trigger Delay Time
(0020,9153)
1
>Actual Cardiac Trigger Delay Time
(0020,9252)
1C
>Intervals Acquired
(0018,1083)
3
>Intervals Rejected
(0018,1084)
3
>Heart Rate
(0018,1088)
3
>R – R Interval Time Nominal
(0020,9251)
1C
>Low R-R Value
(0018,1081)
3
>High R-R Value
(0018,1082)
3
Attribute Description
Dimensions – properties that may change
echo
cardiac phase
b-value
orientation
time
position
volume
time
18
Sup 145 WSI
Harry Solomon
Multi-phase / Multi-slice
Phase (Time) Position Index
1
Slice Order for
phase 1
2
3
Phase order for
slice 2
Physical
Location
(Stack)
Index
6
5
6
4
3
2
1
Frame number 1-6
5
6
4
3
2
5
1
Frame number 13-18
4
3
2
1
Frame number 7-12
Image frames can be sorted/displayed independent of encoded frame order
19
Sup 145 WSI
Harry Solomon
Concatenations
What is a concatenation?
•
•
•
•
•
set of image objects
in the same series
with the same dimension indexes
uniquely identified with a Concatenation UID (0020,9161)
“contained” image objects must have the same Instance Number
Why?
• file system limits – e.g., 600 MB CD-R
• pseudo real-time transfer of a stream of images
– workstation needs to post process images in near real time to figure
out when the scan is to be terminated
20
Sup 145 WSI
Harry Solomon
Concatenations
Legend:
Fixed Header
Per-frame header
Dimension data (not on scale)
Pixel data (not on scale)
An object may be split up into two or more SOP Instances, using the same
concatenation UID
21
Sup 145 WSI
Harry Solomon
IMAGE RETRIEVAL
22
Sup 145 WSI
Harry Solomon
DICOM Query/Retrieve
Allows a system to query another system for a list of available
images (query)
Also allows a system to request another system to send
images (retrieve)
Workstation
Query Request
PACS
Query Match(es)
Retrieve Request
Image(s) Send
Query/Retrieve
SCU
Store Response(s)
Retrieve Response
23
Sup 145 WSI
Harry Solomon
Query/Retrieve
SCP
Hierarchical Query
DICOM query is not a full SQL-type feature
• Limited attributes, no Join capability
• Directed toward production imaging department
requirements
Hierarchical data structure
• (Patient), Study, Series, Image levels
– Patient attributes typically subsumed in Study level
• Query at any level requires specification of unique
entity at each higher level
24
Sup 145 WSI
Harry Solomon
Typical Hierarchical Query
Workstation
Level: STUDY
Patient ID: D73001
Date: 20090521-20090524
Study ID: 09-35541
Study UID: 1.2.789.45.63
Patient ID: D73001
Date: 20090521
Query/Retrieve
SCU
PACS
Study ID: 09-35602
Study UID: 1.2.789.87.11
Patient ID: D73001
Date: 20090522
Study ID: 09-35819
Study UID: 1.2.789.154.3
Patient ID: D73001
Date: 20090524
Level: SERIES
Study UID: 1.2.789.87.11
Study UID: 1.2.789.87.11
Series Num: 1
Series UID: 1.2.405.31.1
Modality: CT
Study UID: 1.2.789.87.11
Series Num: 2
Series UID: 1.2.405.31.2
Modality: CT
Query/Retrieve
SCP
Level: IMAGE
Study UID: 1.2.789.87.11
Series UID: 1.2.405.31.1
Study UID: 1.2.789.87.11
Series UID: 1.2.405.31.1
Image UID: 1.2.405.31.1.99.1
25
Sup 145 WSI
Harry Solomon
Study UID: 1.2.789.87.11
Series UID: 1.2.405.31.1
Image UID: 1.2.405.31.1.99.2
Study UID: 1.2.789.87.11
Series UID: 1.2.405.31.1
Image UID: 1.2.405.31.1.99.3
Classical Hierarchical Retrieve
Retrieve can be at any hierarchical level
• (Patient), Study, Series, Image
• Retrieve at any level requires unique ID of entity at each higher level
Object transfer can be on separate Association (C-MOVE) or
on same Association (C-GET)
• C-MOVE object transfer can be directed to third party
Examples:
• Retrieve all objects under Study UID 1.2.789.87.11
• Retrieve all objects under Study UID 1.2.789.87.11 / Series UID
1.2.405.31.1
• Retrieve single object Study UID 1.2.789.87.11 / Series UID
1.2.405.31.1 / Instance UID 1.2.405.31.1.99.1
Retrieved objects sent and confirmed as wholes
26
Sup 145 WSI
Harry Solomon
Interactive JPIP Retrieve
Image Store SCU and SCP can negotiate a JPEG 2000
Interactive Protocol (JPIP) Transfer Syntax
• Image header (i.e., entire object minus pixel data) transferred and
confirmed as usual
• Pixel data replaced by URL to JPIP service for this image
Limitations
• Pixel data must be in JPEG 2000 format
• Storage Commitment not allowed
• Duration of availability of JPIP not specified or guaranteed
Capabilities
• Retrieve subset of image (ROI)
• Retrieve at a lower resolution (e.g., for quick navigation)
27
Sup 145 WSI
Harry Solomon
Frame-based retrieve
Retrieve subset of frames from a multi-frame image
•
•
•
•
Selected frames of a volumetric stack (ROI)
Decimated volume (e.g., every 10th slice)
Single dimension of a multi-dimensional image
Time snippet of motion image (video)
SCU & SCP negotiate “Instance Root Retrieve” SOP Class
SCU specifies selected frames or time interval
SCP creates new multi-frame image with derivation attributes
• Frame Derivation Module and Contributing Equipment Sequence
• Correct subset of Functional Group Sequence Items
28
Sup 145 WSI
Harry Solomon
VOCABULARY AND
STRUCTURED REPORTING
29
Sup 145 WSI
Harry Solomon
Vocabulary-intensive messaging
There’s a lot of things we want to say about
imaging that cannot be pre-defined in fixed
DICOM attributes
•E.g., specimen processing
How do we define message attributes to handle
what we need to say?
30
Sup 145 WSI
Harry Solomon
Name-value pairs
00180015
ABDOMENPELVIS
00180015 = Body Part Examined
< BodyPartExamined “ABDOMENPELVIS” />
<el>
<name “BodyPartExamined” />
<value “ABDOMENPELVIS” />
</el>
<el>
<name code=00180015 system=DICOM meaning=“Body Part
Examined” />
<value code=R-FAB57 system=SNOMED meaning=“Abdomen and
pelvis” />
Why would we</el>
want to do this?
31
Sup 145 WSI
Harry Solomon
External coded/concept terminologies
Flexibility and extensibility
Leverage externally defined/maintained concepts
Semantic rigor through referenced dictionary/
ontology
General structure – higher layer of abstraction
Allows generalized messaging applications
Shared vocabulary across disparate systems
32
Sup 145 WSI
Harry Solomon
SNOMED
Systematized Nomenclature of Medicine
Most comprehensive clinical healthcare terminology
• 375,000 concepts; 900,000 relationships between concepts
• Multi-hierarchically organized
Primary external vocabulary system for DICOM
• Anatomy
• Procedures (including radiographic views and methods)
• Clinical findings
Originally developed by the College of American
Pathologists, now managed by an international
consortium of governmental agencies (IHTSDO)
33
Sup 145 WSI
Harry Solomon
LOINC
Logical Observation Identifier Names and Codes
Standard coding system for laboratory and clinical
observations
• Hosted by Regenstrief Institute
• Supported by National Library of Medicine
Particularly focused on names of laboratory and
clinical tests
• 50,000 codes; over 275,000 relationships
Major external code system for DICOM and HL7
34
Sup 145 WSI
Harry Solomon
Code Sequences
Table 8.8-1 Common Attribute Set for Code Sequence Attributes
(Invoked as “Code Sequence Macro”)
Attribute Name
Tag
Type
Attribute Description
Code Value
(0008,0100)
1
See Section 8.1.
Coding Scheme
Designator
(0008,0102)
1
Coding Scheme
Version
(0008,0103)
1C
Code Meaning
(0008,0104)
1
DICOM
Part 3
See Section 8.2.
See Section 8.2. Required if the value of
Coding Scheme Designator (0008,0102)
is not sufficient to identify the Code Value
(0008,0100) unambiguously.
See Section 8.3.
“Triplet coding” :
code value,
scheme,
meaning
(version seldom used)
Context Groups (Value Sets)
DICOM
Part 3
DICOM
Part 16
Content Items
Generic Name:Value pair using external coding for Name concept
Encoded as Item in Sequence attributes:
•
•
•
•
Acquisition Context Sequence (in image IODs)
Protocol Context Sequence (in Modality Worklist)
Content Sequence (in Structured Reporting IODs)
Specimen Preparation Step Sequence (in Specimen Module)
Content Item
Concept Name
Sequence
(0040,A043)
Code
(0008,0100)
DateTime Value
(0040,A120)
Person Name Value
(0040,A123)
Text Value
(0040,A160)
UID Value
(0040,A124)
Referenced SOP
Sequence
(0008,1199)
SOP Class UID
(0008,0050)
SOP Instance UID
(0008,0055)
Value Type
(0040,A040)
Scheme
(0008,0102)
Concept Value
Sequence
(0040,A168)
Numeric Value
(0040,A30A)
Measurement Units
Sequence
(0040,08EA)
Meaning
(0008,0104)
Code
(0008,0100)
Scheme
(0008,0102)
Meaning
(0008,0104)
Code
(0008,0100)
Scheme
(0008,0102)
Meaning
(0008,0104)
Templates
Structure for Content Items - like Modules are a structure for
Attributes
Specified in DICOM Part 16
Coding
Scheme
Code Value
Code Meaning
HL7 v3 ActClass
equivalent
SRT
P3-02000
Specimen collection
SPECCOLLECT
SRT
P3-05013
Specimen receiving
CONTREG
SRT
P3-4000A
Sampling of tissue specimen
PROC
SRT
P3-00003
Staining
SPCTRT
ANNOTATION AND
SEGMENTATION
39
Sup 145 WSI
Harry Solomon
DICOM annotation principles
Annotations are conveyed in information objects
separate from the original image
Annotations may be created at a time much later
than the image acquisition, and in a completely
different environment
Multiple annotation objects can reference the
same image
Selection of an annotation object for display
implicitly invokes display of the referenced image
40
Sup 145 WSI
Harry Solomon
Annotation types
Presentation States
Structured Reporting
Segmentation
41
Sup 145 WSI
Harry Solomon
Presentation State
Softcopy Presentation States define how referenced image(s)
will be displayed
•
•
•
•
Transforms to device independent grayscale/color space (LUTs)
Selection of display area (ROI) of the image
Image rotate or flip
Graphical and textual annotations, overlays, shutters
Grayscale, color, and pseudo-color SPSs
Blending SPS overlays a pseudo-color image on a grayscale
image
• E.g., for PET/CT
• Blending on grayscale originals (currently no standard for blending of
color originals)
42
Sup 145 WSI
Harry Solomon
Structured Reporting
Presentation State annotations are for human
reading, not interoperable for automated applications
• No controlled and coded vocabulary, no structural
semantics (relationships between annotations)
SR important for (semi-)automated imaging analysis
and review processes
43
Sup 145 WSI
Harry Solomon
Key Image Note
SR-type object that provides a classification and a
textual comment for a referenced object
• Formally known as “Key Object Selection”, but commonly
denoted “Key Image Note” after IHE use case and profile
Classifications typically identify intended subsequent
use of referenced objects
• “For Referring Provider”, “For Research”, “For Report
Attachment”
• “Rejected for Quality Reasons”, “Signed Complete Study
Content”
44
Sup 145 WSI
Harry Solomon
Segmentation
Derived image object
• Uses enhanced multi-frame mechanism
Multiple segments per object
• Each segment linked to a categorization
• Pixels show presence of category at pixel location
• Binary (1-bit/pixel) or fractional (probability or occupancy)
Segmentation object is typically in same Frame of
Reference as source image
Segments can be displayed as overlays on source
image
45
Sup 145 WSI
Harry Solomon
Segmentation Example
Binary Segmentation Results
White Matter
Segment 1
Gray Matter, CSF
and Partial Volume
Segment 2
Lesion – Part 2
Segment 4
2
Other attributes
Segment Sequence
Item 1 (Segment 1)
Segment Number = 1
Segment Name = White Matter
Segment Category = Brain
Segment Type = White Matter
Item 2 (Segment 2)
Segment Number = 2
Segment Name = Grey Matter
Segment Category = Brain
Segment Type = Grey Matter
Item 3 (Segment 3)
Segment Number = 3
Segment Name = Lesion Part 1
Segment Category = Brain
Segment Type = Lesion
Item 4 (Segment 4)
Segment Number = 4
Segment Name = Lesion Part 2
Segment Category = Brain
Segment Type = Lesion
Shared Functional Groups Sequence
Item 1 (Shared – all frames)
Derivation Image Sequence
Item 1
Reference to source
image (external object)
Lesion – Part 1
Segment 3 3
Per-frame Functional Groups Sequence
Item 1 (Frame 1)
Segment Identification Sequence
Item 1
Referenced Segment
Number = 1
Item 2 (Frame 2)
Segment Identification Sequence
Item 1
Referenced Segment
Number = 2
Item 3 (Frame 3)
Segment Identification Sequence
Pixel Data
Frame 1
Item 1
Referenced Segment
Number = 3
Frame 2
Frame 3
Item 4 (Frame 4)
Segment Identification Sequence
Item 1
Referenced Segment
Number = 4
46
Sup 145 WSI
Harry Solomon
Frame 4
4
PATHOLOGY IN DICOM –
SPECIMEN AND WORKFLOW
47
Sup 145 WSI
Harry Solomon
What’s NOT in Sup145
All the modules already standardized
• Patient, Study, Series, Equipment, General Image
• Multi-Frame Functional Groups and Dimensions
• Sup122 Specimen Module
Explicit description of workflow
• Use of Modality Worklist, Modality Performed
Procedure Step, Image Availability Notification, etc.
48
Sup 145 WSI
Harry Solomon
Sup 122 Specimen Identification
Support for pathology lab workflow, specimen-based imaging
• Gross specimens, blocks, vials, slides
• Image-guided biopsy samples
Specimen Module at image level of hierarchy
• Identification, processing history
• May be used with current Visible Light image object definitions
Update to Modality Worklist to convey Specimen Module
• Enables automated slide scanning devices to fully populate header
Update to Modality Performed Procedure Step to identify
imaged specimen
• Allows LIS/APLIS to track images for specimens
49
Sup 145 WSI
Harry Solomon
Specimen Imaging Information Model
Patient
1
1
Basic DICOM
Information
Model
Disambiguates specimen and
container
Is
source
of
Has
n
Study
Container is target of image
n
Container may have more
than one specimen
1
Contains
Specimens have a physical
derivation (preparation)
from parent specimens
n
Equipment
1
Modality
n
Creates
Series
1
When more than one
specimen in an imaged
container, each specimen is
distinguished (e.g., by
position or color-coding)
Contains
n
Image
1
Is
acquired
on
1
Component
Base, Coverslip
1
n
Has
Container
Box, Block, Slide, etc.
1
n
Specimen
Contains
Physical object
n
Is child of
50
Sup 145 WSI
Harry Solomon
1
1
n
Has
Preparation
Step
Collect, Sample,
Stain, Process
Preparation Step
0-n Preparation Steps per Specimen
Each Preparation Step described by 1-n structured Content Items
(name:value pairs)
• Acquisition Context plus structuring into steps
DICOM Template 8001 Specimen Preparation
Coding
Scheme
Code Value
Code Meaning
HL7 v3 ActClass
equivalent
SRT
P3-02000
Specimen collection
SPECCOLLECT
SRT
P3-05013
Specimen receiving
CONTREG
SRT
P3-4000A
Sampling of tissue specimen
PROC
SRT
P3-00003
Staining
SPCTRT
SRT
P3-05000
Specimen processing
SPCTRT
Preparation steps example
Managed Workflow Concepts (IHE)
ORDER :
A request for departmental service
REQUESTED PROCEDURE :
Report
Unit of work resulting in one
with associated codified, billable acts
PROCEDURE STEP :
The smallest unit of managed work
in the workflow
Scheduled Procedure Step:
Performed Procedure Step:
53
Sup 145 WSI
Harry Solomon
‘A unit of work to do’
‘A unit of work done’
Simple Workflow
One Order – One Procedure – One Study – One Report
DICOM
Modality Worklist
ORDER
A request for
Departmental
Service
Scheduled
Procedure
Step
Requested
Procedure
Imaging Department
Performed
Procedure
Step
One or
more series
of images
Report
Set of
Codifiable,
Billable, Acts
Acquisition
Modality
54
Sup 145 WSI
Harry Solomon
Charles Parisot - IHE
Multiple Modality Steps
ORDER
A request for
Departmental
Service
DICOM
Modality WorklistImaging Department
Scheduled
Procedure
Step A
Requested
Procedure
Scheduled
Procedure
Step B
DICOM
Modality Worklist
Performed
Procedure
Step P1
One or
more series
of images
Performed
Acquisition
Procedure
Step P2 Modality
One or
more series
of images
Acquisition
Modality
Report
Set of
Codifiable,
Billable, Acts
Anatomic Pathology Imaging Workflow
Interpretation Worklist
by accession
Pathology order
Slide preparation
Slide preparation
history data
LIS /
APLIS
Specimen
accessioning data
Modality Worklist Query
by slide barcode
Gross
specimen
accessioning
Images
Images w/
slide prep history
Images – X-ray, U/S,
optical, etc.
56
Imaging task
completion w/
list of images
and specimen IDs
Whole Slide
Scanner
Surgical or
biopsy
procedure
Workstation
Imaging task w/
slide preparation
history data
Sup 145 WSI
Harry Solomon
PACS
Images
SUP145 WHOLE SLIDE
IMAGING PROPOSAL
57
Sup 145 WSI
Harry Solomon
Sup145 multi-frame tiling concept
Use multi-frame image objects (not object per tile)
Thumbnail Image
Intermediate Image Tiles
Baseline
Image Tiles
Single frame image
Multi-frame image
(single object)
Multi-frame image
(single object)
may include multiple
Z-planes, color planes
In 1 or more DICOM Series
58
Sup 145 WSI
Harry Solomon
Alternate approach (not in draft!)
Remove 64k2 image matrix restriction
Can leverage JPEG2000 Part2 multi-component compression
Use JPEG Interactive Protocol capabilities
JPIP low-res view of
baseline image
JPIP medium-res view
of baseline image
Baseline
Image
59
Sup 145 WSI
Harry Solomon
Multi-frame (Z-planes,
colors) image
(single object)
Total Pixel Matrix
Total
Pixel
Matrix
Origin
Columns →
Rows
Total pixel matrix origin at top left
hand corner of imaged volume
Frame (tile) rows and columns
align with total pixel matrix rows
and columns
Frames limited to 216 columns
and rows
Total pixel matrix limited to 232
columns and rows
Frame
Pixel
Matrix
Origin
60
Sup 145 WSI
Harry Solomon
Z-planes
Z-planes are identified as
nominal physical height of image
focal plane above reference
surface (μm)
Z-plane information is used for
relative spatial positioning of
image planes, and nominal interplane distance
↑
Z
↑
Z
61
Sup 145 WSI
Harry Solomon
Cover slip
Specimen
Slide substrate
(glass)
An image plane may track
variable specimen thickness /
surface contour, but only one Zvalue used
Z planes track curved surface
Z plane 1, Z plane 2, Z plane 3, Z plane 4
62
Sup 145 WSI
Harry Solomon
Viktor Sebestyén Varga – 3DHISTECH Ltd.
Organization of tiles into objects
All valid:
Single Multi-frame image
Multi-frame image per Z-plane
Multi-frame image per spatial region
63
Sup 145 WSI
Harry Solomon
Sparse tiling
Multi-frame med-res image
Multi-frame hi-res image
Only selected tiles encoded
Full image matrix might be encoded at lower resolution
64
Sup 145 WSI
Harry Solomon
Localizer Image Flavor
Thumbnail image (single frame) plus
multi-resolution navigation links
Each tile of other resolution images
has its corresponding area identified
in thumbnail
Full description of target tiles
• Object UID and frame #
• Resolution
• Z-plane
Multiple target frames can overlap
• Different resolution, Z-plane, color, etc.
Presentation and any interactive
behavior is not defined in standard
65
Sup 145 WSI
Harry Solomon
Label Image Flavor
Purpose is to capture slide label
Label
• Any specimen captured is irrelevant
Image IOD includes Slide Label
Module
• Barcode (if deciphered)
• Label Text (if deciphered)
Burned In Annotation (0028,0301)
might be “NO” if the label includes
only a specimen identifier and not
patient identifying data
66
Sup 145 WSI
Harry Solomon
C.8.12.2 Slide Coordinates
Used in VL Slide-Coordinates
Microscopic Image IOD
Label
• Single frame image, typically from
microscope-mounted camera
↑
Specimen
Y
X→
Slide
Coordinates
Origin
↑
Z
67
Sup 145 WSI
Harry Solomon
VL SCM
Image area
Used to localize center of VL
SCM Image
DICOM Frame of Reference
associated with slide corner origin
Cover slip
Specimen
Slide substrate
(glass)
Reproducibility not guaranteed
across different mountings of
slide, even on same equipment
C.7.4.1 Frame Of Reference Module
When a Frame of Reference is identified, it is not important
how the [imaging target] is positioned relative to the imaging
equipment or where the origin of the Frame Of Reference is
located. It is important that the position of the [imaging target]
and the origin are constant in relationship to a specific Frame
Of Reference
The Position Reference Indicator may or may not coincide with
the origin of the fixed frame of reference related to the Frame
of Reference UID. The Position Reference Indicator shall be
used only for annotation purposes and is not intended to be
used as a mathematical spatial reference.
68
Sup 145 WSI
Harry Solomon
WSI Image Pixel Matrix
Columns →
Rows
↓
Image
Matrix
Origin
SlideCoordinates
Origin
↑
Z
69
Sup 145 WSI
Harry Solomon
Image Matrix not necessarily aligned to
slide edge, nor to Slide-Coordinates
Image Matrix origin (top left hand corner)
located relative to Slide-Coordinates
Frame of Reference origin (X,Y in mm)
Direction of rows and columns given as
cosines in Slide-Coordinates Frame of
Reference
Cover slip
Specimen
Slide substrate
(glass)
Each tile (frame) TLHC located relative to
Image Matrix origin (column, row)
Each tile center located relative to SlideCoordinates origin (X,Y in mm)
Optical paths
Each combination of light source, lenses, illumination method,
detected wavelengths, etc. used in an acquisition is an optical
path
• Each path described in an Item of the Optical Path Sequence
Examples:
• Full spectrum light, transmission, RGB color sensors
• uV light, excitation, blue monochrome sensor
Each frame may specify a different optical path
• Allows different colors in a single object, including hyperspectral (n
monochrome planes)
• Identified in Optical Path Functional Group by reference to Optical Path
Sequence Item
70
Sup 145 WSI
Harry Solomon
Optical paths
What parameters are needed?
• To be added in Part 16 Context Groups
Is a “macro image” simply a selected optical path?
Illumination:
Color(s)
Intensity
Type (laser)
71
Sup 145 WSI
Harry Solomon
Filters:
Color(s)
Polarization
Lens:
Illumination
Method:
Transmission
Reflection
Scatter
Excitation
Lens:
Filters:
Color(s)
Polarization
Sensor:
Color(s)
WSI Functional Groups
Standard
• Pixel Measures (pixel spacing, layer thickness) – shared
• Frame Content (datetime, dimensional location) – per-frame
• Referenced Image, Derivation (if required for individual
frames)
WSI Specific
• Plane Position (relative to total matrix and to SCM FoR)
• Optical Path
• Specimen Reference (if multiple specimens on slide are
automatically distinguishable)
72
Sup 145 WSI
Harry Solomon
Pixel Measures functional group
Attribute Name
Pixel Measures Sequence
Tag
(0028,9110)
>Pixel Spacing
(0028,0030)
Type Attribute Description
1
Identifies the physical characteristics
of the pixels of this frame. Only a
single Item shall be permitted in this
sequence.
1C Physical distance in the imaging
target (patient, specimen, or
phantom) between the centers of
each pixel, specified by a numeric pair
- adjacent row spacing (delimiter)
adjacent column spacing in mm. See
10.7.1.3 for further explanation of the
value order.
Note: In the case of CT images ...
Required if Volumetric Properties
(0008,9206) is other than
DISTORTED or SAMPLED. May be
present otherwise.
>Slice Thickness
(0018,0050)
1C
Nominal reconstructed slice thickness
(for tomographic imaging) or depth
of field (for optical imaging) in mm.
See C.7.6.2.1.1 and C.7.6.16.2.3.1
for further explanation.
Required if Volumetric Properties
(0008,9206) is VOLUME or
SAMPLED. May be present otherwise.
73
Sup 145 WSI
Harry Solomon
Redefinition
Plane Position (Slide) functional group
Attribute Name
Plane Position
(Slide) Sequence
Tag
(gggg,nn1A)
Type
1
>Position In Image
Pixel Matrix
(gggg,nn1F)
1
>Image Center Point (0040,071A)
Coordinates
Sequence
1
>>X Offset in Slide
Coordinate System
>>Y Offset in Slide
Coordinate System
>Z Offset in Slide
Coordinate System
74
Sup 145 WSI
Harry Solomon
(0040,072A)
1
(0040,073A)
1
(0040,074A)
1
Attribute Description
Describes position of frame in the Total Pixel Matrix and
in the Slide Coordinate System Frame of Reference.
Only a single Item may be present in this Sequence.
The coordinate of the top left pixel of the frame in the
Total Pixel Matrix (see C.8.12.X3.1.1), given as
column\row. Column is the horizontal position and row is
the vertical position. The coordinate of the top left pixel
of the Total Pixel Matrix is 1\1.
Identifies the coordinates of the center point of this
frame in the Slide Coordinate System Frame of
Reference. Only a single Item shall be permitted in this
sequence. See Section C.8.12.2.1.1 for further
explanation.
Note: This attribute allows simplified transformation
of a single frame of a multi-frame VL WSI SOP
Instance into an instance of the VL Slide
Coordinates Microscopy SOP Class.
The X offset in millimeters from the Origin of the Slide
Coordinate System. See Figure C.8-16.
The Y offset in millimeters from the Origin of the Slide
Coordinate System. See Figure C.8-16.
The Z offset in microns from the Origin of the Slide
Coordinate System, nominally the surface of the glass
slide substrate. See Figure C.8-17
Note: Required even if only a single focal plane was
acquired.
Do we need to
separate to two
attributes to support
independent
dimensions?
Do we really need
this sequence
introducer
(consistency w/
C.8.12.2)? Could
just specify X and
Y as center point
coordinates
Dimensions
Based on attributes in functional groups (i.e., values that change on a perframe basis)
Typical dimensions for WSI:
•
•
•
•
Total Matrix Column Origin
Total Matrix Row Origin
Z-Plane
Optical Path (color/polarization)
Attributes used for Dimensions specified in Multi-frame Dimension Module
Each frame specifies its dimensional indexes in Frame Content functional
group
• Index values (ordinals) mapped to dimensional attribute values
• E.g., with (Column, Row) dimensions, and 40962 pixel frames, frame with index
value (2,3) would have origin column\row values of 8193\12289
75
Sup 145 WSI
Harry Solomon
Annotations of WSI - Segmentations
Segmentations can be created frame-by-frame / pixel-by-pixel
against selected frames of original image
• Reference through Derivation Image Functional Group
• 1-bit/source-pixel, or 8-bits/source-pixel
Segmentations can be created against arbitrary areas within a
specified Frame of Reference
• Requires Plane Position and Plane Orientation Functional Groups – may
not be usable with slide coordinates Frame of Reference
Display of segmentation can implicitly invoke a nonstandardized overlay or blending with source image
• Segmentation can specify its preferred color
76
Sup 145 WSI
Harry Solomon
Annotations of WSI - Presentation States
Color Presentation State supports annotation of a source image
Displayed Area Selection allows up to 231-1 rows/columns, currently
relative to frame-based rows/columns
• Proposed enhancement (with new attribute and new enumerated value)
to allow Displayed Area Selection and annotation location relative to
WSI total matrix, rather than to frame
• Implicitly applies to all dimensions (Z-planes, colors), only constrained
by explicit frame numbers; should there be a general mechanism to limit
by dimension (as is done for segments)?
Placement of annotations limited to 24-bit precision (IEEE 754 32-bit
float)
• May be image relative or selected display area relative
• Allows sub-pixel resolution up to 8M rows/columns for image relative
77
Sup 145 WSI
Harry Solomon
Presentation State - Displayed Area
Selection
Attribute Name
Displayed Area
Selection Sequence
Tag
(0070,005A)
Type
1
Attribute Description
A sequence of Items each of which describes the displayed area selection
for a group of images or frames. Sufficient Items shall be present to
describe every image and frame listed in the Presentation State
Relationship Module.
One or more Items shall be present.
>Referenced Image
(0008,1140)
1C
Sequence
>>Include ‘Image SOP Instance Reference Macro’ Table 10-3
>Pixel Origin
(gggg,bb01)
1C
For a referenced multi-frame image, specifies whether the Displayed
Interpretation
Area Top Left Hand Corner (0070,0052)
and Displayed Area Bottom RIght Hand Corner (0070,0053) are to be
interpreted relative to the individual frame pixel origins, or relative to
the Total Pixel Matrix origin (see C.8.12.X3.1.1).
Required if the Referenced Image Sequence (0008,1140) >Referenced
SOP Class UID (0008,1150) value is 1.2.840.10008.5.1.4.1.1.xxx (VL
Whole Slide Microscopy Image). May be present otherwise.
Enumerated Values:
FRAME
VOLUME
If not present, TLHC and BRHC are defined relative to the frame pixel
origins.
78
>Displayed Area Top
Left Hand Corner
>Displayed Area Bottom
Right Hand Corner
…
Sup 145 WSI
Harry Solomon
(0070,0052)
1
(0070,0053)
1
Presentation State - Graphic Annotation
Units
Attribute Name
Graphic Annotation
Sequence
Tag
(0070,0001)
Type
1
Attribute Description
A sequence of Items each of which represents a group of annotations composed of
graphics or text or both.
One or more Items shall be present.
…
>>Bounding Box
Annotation Units
(0070,0003)
1C
Units of measure for the axes of the text bounding box.
Defines whether or not the annotation is Image or Displayed Area relative. Both
dimensions shall have the same units.
Enumerated Values:
PIXEL = Image relative position specified with sub-pixel resolution such that the
origin at the Top Left Hand Corner (TLHC) of the TLHC pixel is 0.0\0.0, the Bottom
Right Hand Corner (BRHC) of the TLHC pixel is 1.0\1.0, and the BRHC of the
BRHC pixel is Columns\Rows (see figure C.10.5-1). The values must be within the
range 0\0 to Columns\Rows.
DISPLAY = Fraction of Specified Displayed Area where 0.0\0.0 is the TLHC and
1.0\1.0 is the BRHC. The values must be within the range 0.0 to 1.0.
MATRIX = Image relative position specified with sub-pixel resolution such that
the origin at the Top Left Hand Corner (TLHC) of the TLHC pixel of the Total
Pixel Matrix is 0.0\0.0, the Bottom Right Hand Corner (BRHC) of the TLHC
pixel is 1.0\1.0, and the BRHC of the BRHC pixel of the Total Pixel Matrix is
Total Pixel Matrix Columns\Total Pixel Matrix Rows (see C.8.12.X3.1.3). The
values must be within the range 0\0 to Total Pixel Matrix Columns\Total Pixel
Matrix Rows. This value is valid only if the Referenced Image Sequence
(0008,1140) >Referenced SOP Class UID (0008,1150) value is
1.2.840.10008.5.1.4.1.1.xxx (VL Whole Slide Microscopy Image).
Required if Bounding Box Top Left Hand Corner (0070,0010) or Bounding Box Bottom
Right Hand Corner (0070,0011) is present.
79
Sup 145 WSI
Harry Solomon
Modality Worklist
Scheduled Specimen Sequence added to MWL in Sup122
• Allows query by Container ID (slide barcode)
• Allows return from SCP of complete Specimen Module (slide processing
history to be used for imaging set up and/or inclusion in WSI header
Other parameters can be passed in Protocol Context
Sequence
• Template specification for Content Items
Proposed Protocol Optical Paths Sequence
• Parallel to Protocol Context Sequence
• General VL attributes
80
Sup 145 WSI
Harry Solomon