top bar

Multi-Dimensional Acquisition Programming

Revision as of 15:13, 29 June 2012 by Arthuredelstein (Talk | contribs)

Multi-dimensional acquisition is an important functionality of Micro-Manager. MDA capabilities have been dramatically revised and extended between 2010-2012.

Architecture

This image shows the flow of TaggedImages through Micro-Manager during multi-dimensional acquisition. The most important interfaces and classes are described below.

TaggedImageFlow.png

JSONObject: All metadata is stored in JSONObjects. Summary metadata refers to the metadata for a particular multi-dimensional acquisition, in contrast to image metadata which is attached to every image plane.

TaggedImage: A pure data object with two fields: pixels (a byte or short array) and tags (a JSONObject for the image's metadata). TaggedImages flow sequentially through a "pipeline" in Micro-Manager. A TaggedImage is born in the swig (C++ to java) layer when the java layer calls core.getTaggedImage or similar methods. During an MDA, after this function is called, AcquisitionEngine2010 adds appropriate tags to the TaggedImage and then adds it to its output queue. DataProcessors can then modify these TaggedImages, and then the images are finally sent for display and/or storage (by default, VirtualAcquisitionDisplay and MMImageCache, respectively).

There are three important tag names that are necessary for Micro-Manager to handle images correctly: Width, Height, PixelType. Width and Height should have integer values, and PixelType should be a string: one of GRAY8, GRAY16, RGB32, RGB64. RGB images are stored as byte or short arrays.

Several optional tags have important meanings: ChannelIndex, FrameIndex, PositionIndex, SliceIndex are integer tags that determine where in a multi-dimensional image set an image is placed. Omitted indices will default to zero.

SequenceSettings: A pure data object set up by the Multi-Dimensional Acquisition setup dialog (AcqControlDialog). The settings within this object include the size of and parameters for each dimension (frames, positions, slices, and channels), the way dimensions are to be nested, as well as settings for saving and autofocus. The SequenceSettings object fully specifies a desired acquisition sequence as carried out by an IAcquisitionEngine2010.

IAcquisitionEngine2010: This interface (created in 2010) is concerned solely with sequencing of hardware events and acquisition of images. Its most important method, run(...), accepts a single argument of type SequenceSettings and returns a BlockingQueue of TaggedImages.

AcquisitionEngine2010: This class (written in clojure, but usable by Java code) implements IAcquisitionEngine2010 and carries out all hardware steps necessary to produce a Multi-Dimensional data set. It runs through channels, slices, xy positions and time points according to the SequenceSettings argument of run(...) and acquires images in the correct order at the appropriate times (frames). The engine runs on its own thread so that display and saving can be carried out independently. The engine code includes an algorithm to use sequence (burst) acquisition whenever possible, either when a zero time interval is used, or when hardware triggering is set up to switch between channels or z slices. The engine takes care of hardware errors (attempting to reload devices), and may be paused or interrupted at any time. It allows extra hardware events (Runnables) to be attached to particular points in the MDA sequence (for example, allowing photobleaching events at a particular time point).

DataProcessor<TaggedImage> (abstract class): Plugins and scripts can implement their own DataProcessors. Each DataProcessor runs on its own thread and has an input and output queue: one or more TaggedImages can be received on the input queue (by the poll() call) and modified, combined, or multiplexed, and then sent out on the output queue. Dimensional indices (such as ChannelIndex) should be adjusted by the DataProcessor to ensure that each TaggedImage is display in the right place in the MDA data set. The ImageFlipper plugin shows a simple example.

ProcessorStack: This object connects up a list of active DataProcessors. It is designed to received images from the queue returned by IAcquisitionEngine2010.run(...) and produces an output queue that yields processed TaggedImages.

LiveAcq: This object has a loop running on its own thread that receives TaggedImages from the ProcessorStack output queue, and pumps them into the ImageCache.

MMImageCache (interface ImageCache): An object that provides a way to store images either in RAM or on disk. The MMImageCache has a pluggable saving mechanism that allows different file formats to be saved (currently two are available: single TIFF and multi-page TIFF). Incoming TaggedImages are automatically indexed according to their "Index" metadata tags (ChannelIndex, FrameIndex, SliceIndex, PositionIndex) and these can be retrieved by calling getImage with the appropriate indices. The ImageCache is wired up to the display components of Micro-Manager so that display settings and comments are also stored in RAM or saved on disk whenever they are adjusted by the user. When the user opens an existing data set, it is loaded into a new MMImageCache. When a disk-based ImageCache is attached to a VirtualAcquisitionDisplay, it essentially operates in "virtual" mode.

TaggedImageStorage: An interface for code to store images and metadata in a multi-dimensional data set, in RAM or on disk (or potentially in a database). Currently there are three implementations: TaggedImageStorageRam holds images in memory, TaggedImageStorageDiskDefault saves individual TIFF files to each image plane (and produces a collection of files compatible with earlier versions of Micro-Manager) and TaggedImageStorageMultipageTiff stores images in one or a very few multipage TIFF files, and reads and writes data substantially faster than the default saving method. The MMImageCache internally uses TaggedImageStorage instances.

VirtualAcquisitionDisplay: The default display class for Micro-Manager. This display module uses ImageJ windows to display multi-dimensional data sets stored in a given ImageCache. The VirtualAcquisitionDisplay object is also connected to the MetadataPanel, which shows the current metadata, comments, and channel settings. The display is designed to adapt to different numbers of channels, slices, positions and frames and show appropriate sliders and colors. In most cases, if color images or (because of DataProcessors) extra channels, slices, positions or frames are detected, the VirtualAcquisitionDisplay will adapt to show the full range of these dimensions. It expects a single height, width, and pixel type, however.

DefaultTaggedImagePipeline: A small class that provides Micro-Manager's default behavior for multi-dimensional acquisition. It instantiates and connects the various components in the TaggedImage pipeline: namely the AcquisitionEngine2010, the ProcessorStack, the LiveAcq and (through the MMAcquisition interface) the VirtualAcquisitionDisplay and the MMImageCache.

AcquisitionWrapperEngine (interface AcquisitionEngine): the traditional "acquisition engine," available in the script panel as acq or by calling gui.getAcquisitionEngine(). This wrapper uses a DefaultTaggedImagePipeline to set up the data flow of TaggedImages.

© Micro-Manager : Vale Lab, UCSF 2006-2011 | All Rights Reserved | Contact