Micro-Manager can save files in three formats: separate image files, Image file stack (OME-Tiffs) and NDTiff. These formats differ in flexibility, performance, and ability to be used with downstream applications. Since all formats are Tiffs, image data can be read by any appliation that can read TIFFs. However, not every application can read all of the format’s metadata and can load image data/metadata at high speed. Below is a comparison of the different formats.
TL;DR
Image file stacks will likely have the most widespread downstream compatibility due to implementation of the OME-Tiff standard which can be opened by many different programs. However, for high speed or large (> 10GB ish) datasets, NDTiff is superior. It is also likely the better choice if you are writing your own analysis code, especially if it is in Python. If you are performing a customized or non-standard experiment NDTiff is the most useful due to its flexible model for organizing data. For this reason, NDTiff is the default format of Pycro-Manager. Only use seperate image files if you absolutely must save one image per file.
Details
Seperate image file datsets are a directory full of many TIFF files, each of which contains a single image.
Image file stack datasets embed multiple images into one or more multipage TIFF, each of which can be up to 4GB in size. They also implement the OME-Tiff specification, which enables them to be imported into many downstream applications.
NDTiff datasets also embed multiple images into a single file. This format contains many performance optimizations compared to Image stack datasets to enable high speed reading and writing and the ability, unlike the other formats, to organize images along arbitrary axes (not just channel/slice/frame/position). NDTiff is the default format of Pycro-Manager
The appropriate choice of format varies depending on the downstream use case. Important parameters are outlined in the table below.
| Format | Read/Write speed | Save images along axes | Metadata | Reccomended python reader | Customize which images go in which files |
|---|---|---|---|---|---|
| Seperate TIFFs | Slow | channel/slice/frame/position only | Seperate text file | Tifffile | ❌ |
| Image stack file | Medium | channel/slice/frame/position only | embedded in TIFF (with optinal seperate text file) | Tifffile | ✅ |
| NDTiff | 🔥Fast🔥 | arbitrary | embedded in TIFF | Pycro-Manager | ❌ |
More information about the features and internal structure of Seperate image files and Image file stacks is written below. A similar description for NDTiffs can be found here.
Separate image files
Acquired images are saved to disk as separate TIFF files, each containing a single grayscale image. The file naming convention is “img” prefix followed by frame number, channel name and slice number (img_00000000t_channel_00z.tif). In addition, the folder will contain a file named “metadata.txt” that contains the metadata in JSON format.
Image file stack
A TIFF file or group of TIFF files that contain multiple acquired images per a single file. These files conform to the OME-TIFF specification, allowing them to be easily imported into a variety of analysis applications including anything that utilizes the Bio-Formats library. They also store an index map of the byte offsets of images within a file to improve reading performance.
Image file stacks are designed to be easily imported into ImageJ without the need for a special reader plugin. A stack file can be dragged onto the ImageJ toolbar and will automatically open as a hyperstack with the same contrast settings used in Micro-Manager. Any acquisition comments typed into the Multi-Dimensional Acquisition window or the comments tab of the main Micro-Manager GUI can be viewed by pressing “i” with one of these files open in ImageJ.
By default, one file is created for each XY stage position (up to a maximum of 4 GB per file). In the tools-options menu, this can be changed to save all XY positions in a single file. This is especially useful for acquisitions using a large number XY positions. Since OME-TIFFs require that an identical String of XML metadata be embedded in each file in an acquisition, acquisitions that have a large number of XY positions with a small amount of data at each one waste space on disk and time by writing the same String of metadata in each file at the acquisition’s conclusion.
Writing to these files results in faster performance than writing to Seperate Image Files, in part because it minizes the number of system calls to create new files. This can be advantageous in situations where disk write speed is a limiting factor (i.e. writing to a server or collecting data at a high rate). However, these files cannot be written as fast as NDTIFF files, which contain several additonal optimizations.
Internal structure
Header
| Bytes | Content |
|---|---|
| 0-7 (0x0-0x7) | Standard TIFF Header |
| 8-11 (0x8-0xb) | Index map offset header (54773648 = 0x0343C790) |
| 12-15 (0xc-0xf) | Index map offset |
| 16-19 (0x10-0x13) | Display settings offset header (483765892 = 0x1CD5AE84) |
| 20-23 (0x14-0x17) | Display settings offset |
| 24-27 (0x18-0x1b) | Comments offset header (99384722 = 0x05EC7D92) |
| 28-31 (0x1c-0x1f) | Comments offset |
| 32-35 (0x20-0x23) | Summary metadata header (2355492 = 0x0023F124) |
| 36-39 (0x24-0x27) | Summary metadata length |
| 40- (0x28-) | summary metadata (UTF-8 JSON) |
Image File Directories
The first IFD starts immediately after the summary metadata. Each IFD will contain the same set of TIFF tags, except for the first one in each file, which contains two ImageJ metadata tags, and two copies of the ImageDescription tag. One of these contains a string needed by ImageJ to recognize these files, and the other contains OME metadata. Although these tags appear in the first IFD, their values will not be written until the end of the file, when it is closed. The tags are written in the following order (non-standard TIFF tags have the values listed after them), following the TIFF specification requirement that they be sorted numerically:
ImageWidth (256 = 0x0100)
ImageHeight (257 = 0x0101)
BitsPerSample (258 = 0x0102)
Compression (259 = 0x0103)
PhotometricInterpretation (262 = 0x0106)
ImageDescription (270 = 0x010e) (first IFD only)–contains OME XML metadata
a 2nd ImageDescription (270 = 0x010e) (first IFD only)-–contains ImageJ file opening information
StripOffsets (273 = 0x0111)
SamplesPerPixel (277 = 0x0115)
RowsPerStrip (278 = 0x0116)
StripByteCounts (279 = 0x0117)
XResolution (282 = 0x011a)
YResolution (283 = 0x011b)
ResolutionUnit (296 = 0x0128)
IJMetadataByteCounts (first IFD only) (50838 = 0xc696)
IJMetadata (first IFD only) (50839 = 0xc697)
MicroManagerMetadata (51123 = 0xc7b3)
Immediately after these tags are written:
- 4 bytes containg the offset of the next IFD (per the TIFF specification)
- The pixel data
- In RGB files only, 6 bytes containing the values of the BitsPerSample tag Pixel values
- 16 bytes containing the values of the XResolution and YResolution tags
- The value of the MicroManagerMetadata tag: image metadata (UTF-8 JSON)
Index map
As the files are being written, an “Index map” describing where each image is located in the file is written as well. This allows a specific image to be quickly accessed without having to parse the entire file and read in image metadata. It consists of the following:
- A 4 byte header (3453623 = 0x0034b2b7)
- 4 bytes containing the number of entries in the index map
- 20 bytes for each entry, with 4 bytes each allocated to the image’s channel index, slice index, frame index, position index, and byte offset of the image’s IFD within the file
End of file
After the last IFD, the following constructs are written:
ImageJ Metadata
A subset of the metadata used by the ImageJ TIFF writer (ij.io.TiffEncoder.java), which allows contrast settings and acquisition comments to propagate into ImageJ. The position and size of this metadata is specified by the IJMetadataCounts and IJMetadata tags in the first IFD.
OME XML Metadata
A string containing the OME XML metadata for this data set. This String is referenced by the first of the two ImageDescription tags in the first IFD of the file, in accordance with the OME-TIFF specification. Since this String must be identical for all files in a data set, it is not written for any file until the entire data set is closed at the conclusion of an acquisition.
ImageJ Image Description String
The ImageJ image description String that allows these files to opened correctly as hyperstacks in ImageJ. This String is referenced by the second of the two ImageDescription tags in the first IFD of the file.
–Henry Pinkard 7:35, 20th March 2013 (PDT)
– Updated Henry Pinkard 11:05, 23rd August 2022 (PDT)