1. Tools for Manipulating JPEG-2000 Based Whole Slide Image Formats
Toby Cornish, M.D., Ph.D.
Department of Pathology
Johns Hopkins Medical Institutions
Baltimore, MD

2. Overview
JPEG-2000 is a highly-efficient compression standard well-suited to storage of whole slide images
JP2 and Aperio SVS are two WSI file formats that employ JPEG2k compression
Although the file formats are different, the data is essentially stored in the same format
Few tools exist for manipulating these formats
It should be possible to open, manipulate, and interconvert these formats using readily available, free and open libraries

3. JPEG-2000 File Formats Design Results Compression Codestreams JP2
Aperio SVS
Design
Access SVS Data
Convert SVS to JP2
Results

4. JPEG-2000 File Formats Design Results Compression Codestreams JP2
Aperio SVS
Design
Access SVS Data
Convert SVS to JP2
Results

5. JPEG-2000 A standard for wavelet-based image compression
Joint Photographic Expert Group as a replacement for JPEG
ISO/IEC and (parts 1 and 2)
Part 1 defines a self-contained codestream (JPC) as well as a file format (JP2) that can wrap the codestream

6. JPEG-2000, cont.
JPIP (ISO/IEC ) is a related standard for streaming of large JPEG2k codestreams
Several encoders/decoders both free and proprietary for JPEG2k
Vary widely in the implementation of the standard, but most support JPEG2k part 1
Mostly used for high capacity image storage in videogames, satellite imagery, and, more recently, medical imagery

7. JPEG-2000, cont. Features: Highly efficient compression
Native tiling support for large images
Native multiresolution decompression from a single codestream
Support for an arbitrary number of image components (i.e. channels)
JPEG2k codestreams are self-contained and can be embedded in other file formats

8. JPEG-2000 File Formats Design Results Compression Codestreams JP2
Aperio SVS
Design
Access SVS Data
Convert SVS to JP2
Results

9. JPEG-2k codestreams
The codestream are capable of indepently being decoded into an image
All codestreams are tiled, but the simplest and most common form is the single tile codestream
Single tile images are good for small images
Multiple tile images better for handling large image data such as WSI

10. JPEG2k codestream, single tile1
One codestream – One tile

11. JPEG2k codestream, single tile
start of codestream
SOC
image and tile size
SIZ
coding style defaults
COD
quantization defaults
QCD
comments
JPEG2k
codestream
COM
Main header
Tile header 1
start of tile
SOT
Tile data 1
start of data
SOD
Packet 1
EOC
Packet n
end of codestream
EOC

12. JPEG2k codestream, multiple tiles11 12 13 14 15 16 1 2 3 4 5 6 7 8 9 10
One codestream – Multiple tiles

13. JPEG2k codestream, multiple tiles
start of codestream
SOC
SIZ
image and tile size
coding style defaults
COD
quantization defaults
QCD
Main header
comments
COM
Tile header 1
JPEG2k
codestream
start of tile
SOT
Tile data 1
start of data
SOD
Packet 1
Tile header n
Packet n
Tile data n
start of tile
SOT
start of data
SOD
Packet 1
EOC
Packet n
end of codestream
EOC

14. JPEG-2000 File Formats Design Results Compression Codestreams JP2
Aperio SVS
Design
Access SVS Data
Convert SVS to JP2
Results

15. JP2 Wrapped JPEG2k codestream
Composed of mandatory and optional header boxes
Extensible via UUIDs (Universal Unique Identifiers) and/or XML

16. Contiguous codestream box
JP2 file format, cont.
Main header
Tile header 1
JPEG2k signature box
Tile data 1
File type box
JP2 header box
Contiguous codestream box
JPEG2k codestream
Tile header n
XML boxes
Tile data n
UIUD boxes
EOC

17. JPEG-2000 File Formats Design Results Compression Codestreams JP2
Aperio SVS
Design
Access SVS Data
Convert SVS to JP2
Results

18. Aperio SVS file format Developed by Aperio, Inc. (San Diego, CA)
Essentially, a TIFF file with a tiled layout
Tile data are JPEG2k codestreams
Can consist of multiple images (pages) derived from slide scanning with one Image File Directory (IFD) per image
Image metadata is stored in Tags
The highest resolution data is always tiled

19. SVS/TIFF structure
Image
Pyramid
Thumbnail
Label
Macro

20. SVS/TIFF structure, cont.
Byte order
Version number
IFD 1
IFD offset
Tag entry count
Tag 1
IFD 2
Tag n
IFD 3
Image data
IFD 4
Tag entry count
IFD 5
Tag 1
Tag n
IFD 6
Image data

21. SVS/TIFF Tiling Multiple tiles – one codestream per tile 11 12 13 1415 16 1 2 3 4 5 6 7 8 9 10
Multiple tiles – one codestream per tile

22. SVS/TIFF tiling, cont. JPEG2k codestream Tiff Tiles IFD 1
Tag entry count
Tag 1
Tile 1
Main header
Tile 2
Tile header 1
Tile 3
Tile data 1
IFD 1
Tag n
Image data
Tile n
EOC

23. SVS/TIFF v JP2 Differences: Similarities:
JP2 contains one JPEG2k codestream, which contains multiple tiles
SVS/TIFF contains multiple tiles, each of which contains one codestream with exactly one tile
Similarities:
The tile data is essentially the exact same, but packaged
Should permit direct copying of tile data between formats

24. JPEG-2000 File Formats Design Results Compression Codestreams JP2
Aperio SVS
Design
Access SVS Data
Convert SVS to JP2
Results

25. Design Open source libraries (LibTiff, ImageMagick/Jasper)
C programming language for portability
First two goals:
Read SVS/TIFF image data using custom software
Convert SVS/TIFF files to JP2 without decompressing

26. Read SVS/TIFF Tiff Tiles TIFF Header IFD SVS Image data Tile 1 Tile 2
Main header
Tile header
Decode
Tile data 1
EOC
TIFF Header
Tiff Tiles
IFD
Main header
SVS
Image data
Tile 1
Tile header
Tile data 2
Tile 2
Decode
EOC
Tile n
Main header
Tile header
Tile data n
Decode
EOC

27. SVS/TIFF to JP2 conversion
Merged
codestream
Main header
Tile header
Tile data 1
Main header
Tile header
EOC
Tile data 1
TIFF Header
Tiff Tiles
JP2 Header
IFD
Main header
Tile 1
Tile header
Tile header
JPEG2k
Codestream
SVS
Image data
Tile data 2
Tile data 2
Tile 2
EOC
Tile header
Tile n
Tile data 2
Main header
Tile header
EOC
Tile data n
EOC

28. Issues SVS format “quirks” Somewhat unusual component subsampling
Limits the number of decoders that will read the data without complaining
SVS tiles are transformed into YCrCb color space
Requires a transform to RGB color space before use
A compliant JP2 reader should do this, but not all do

29. JPEG-2000 File Formats Design Results Compression Codestreams JP2
Aperio SVS
Design
Access SVS Data
Convert SVS to JP2
Results

30. Results Several tools were created, allowing:
Direct access to SVS tile data
Manipulate, extract, and montage tiles
See LOCI Bioformats project
Direct conversion of SVS files to JP2
Requires the SVS tile dimensions to be multiples of 64
Embedded ICC profiles not propogated

31. Future work Implement software in Java
Create a JP2  SVS/TIFF converter
Investigate limitations:
Why must tile dimensions be multiples of 64?
Why do most decoders (notably the JAI decoders) fail to decode JPEG2k codestreams from SVS?
Include ICC profiles
Develop a generic LibTIFF plugin for encoding/decoding JPEG2k image data

32. Acknowledgements Johns Hopkins Kristen Lecksell
University of Wisconsin-Madison, LOCI
Melissa Linkert
Curtis Rueden
References
Taubman and Marcellin, JPEG2000 Image Compression Fundamentals, Standards and Practice
Aperio Technical Docs (
Libraries
LibTIFF (
ImageMagick (
Jasper (