Multi-dimensional acquisition is an important functionality of Micro-Manager. The newest and most powerful tool for this type of programming is Pycro-manager.

The info below describes capabilities developed in 2010-2012.

Architecture

This image (from Grant Harris) shows the flow of TaggedImages through Micro-Manager during multi-dimensional acquisition. The most important interfaces and classes are described below.

JSONObject: All metadata is stored in JSONObjects. Summary metadata refers to the metadata for a particular multi-dimensional acquisition, in contrast to image metadata which is attached to every image plane.

TaggedImage: A pure data object with two fields: pixels (a byte or short array) and tags (a JSONObject for the image’s metadata). TaggedImages flow sequentially through a “pipeline” in Micro-Manager. A TaggedImage is born in the swig (C++ to java) layer when the java layer calls core.getTaggedImage or similar methods. During an MDA, after this function is called, AcquisitionEngine2010 adds appropriate tags to the TaggedImage and then adds it to its output queue. DataProcessors can then modify these TaggedImages, and then the images are finally sent for display and/or storage (by default, VirtualAcquisitionDisplay and MMImageCache, respectively).

There are three required image metadata tag names that are necessary for Micro-Manager to handle images correctly: Width, Height, PixelType. Width and Height should have integer values, and PixelType should be a string: one of GRAY8, GRAY16, RGB32, RGB64. RGB images are stored as byte or short arrays. These tags are automatically produced by the core java wrapper (mmc in the Beanshell scripting panel) whenever getTaggedImage is called.

In addition, four essential tags permit multi-dimensional image data:

• ChannelIndex
• FrameIndex
• PositionIndex
• SliceIndex

These are integer tags, used by the ImageCache and the VirtualAcquisitionDisplay to determine where in a multi-dimensional image set an image is placed. If any of these tags are omitted from a TaggedImage, they will be added set by Micro-Manager to zero. In other words, a TaggedImage with no ChannelIndex, FrameIndex and PositionIndex, but a SliceIndex of 3, will be placed at frame zero, position zero, slice 3, and channel zero.

A variety of other useful optional tags are produced by the multi-dimensional acquisition engine and are interpreted by various parts of Micro-Manager in various useful ways, particularly by the image display window. Of course these tags can also be added by plugins or scripts before they are added to a Multi-Dimensional image window:

• Binning (an integer)
• BitDepth (an integer with values such as 8, 12, 14, 16, 24, 32; used by the contrast panel)
• CameraChannelIndex (an integer specifying which “camera channel” from a multi-channel camera has produced this image)
• Channel (a string with the channel’s name)
• ElapsedTime-ms (the time the image was acquired since acquisition started)
• Exposure-ms (the exposure setting of the camera when the image was acquired)
• Frame (identical to FrameIndex)
• NextFrame (the time on the computer clock when the next image is expected to be acquired)
• PixelSizeUm (the size of the pixel)
• PositionName (a string)
• Slice (identical to SliceIndex)
• SlicePosition (a float)
• Source (the camera that produce the image)
• Summary (a map of summary metadata)
• Time (the time and date that the image was acquired)
• a random, unique UUID (a unique universal ID number)
• WaitInterval (the requested delay between successive time points)

SequenceSettings: A pure data object set up by the Multi-Dimensional Acquisition setup dialog (AcqControlDialog). The settings within this object include the size of and parameters for each dimension (frames, positions, slices, and channels), the way dimensions are to be nested, as well as settings for saving and autofocus. The SequenceSettings object fully specifies a desired acquisition sequence as carried out by an IAcquisitionEngine2010.

IAcquisitionEngine2010: This interface (created in 2010) is concerned solely with sequencing of hardware events and acquisition of images. Its most important method, run(…), accepts a single argument of type SequenceSettings and returns a BlockingQueue of TaggedImages.

AcquisitionEngine2010: This class (written in clojure, but usable by Java code) implements IAcquisitionEngine2010 and carries out all hardware steps necessary to produce a Multi-Dimensional data set. It runs through channels, slices, xy positions and time points according to the SequenceSettings argument of run(…) and acquires images in the correct order at the appropriate times (frames). The engine runs on its own thread so that display and saving can be carried out independently. The engine code includes an algorithm to use sequence (burst) acquisition whenever possible, either when a zero time interval is used, or when hardware triggering is set up to switch between channels or z slices. The engine takes care of hardware errors (attempting to reload devices), and may be paused or interrupted at any time. It allows extra hardware events (Runnables) to be attached to particular points in the MDA sequence (for example, allowing photobleaching events at a particular time point).

**[DataProcessor](https://github.com/micro-manager/micro-manager/tree/svn-mirror/mmstudio/src/org/micromanager/api/DataProcessor.java)** (abstract class): Plugins and scripts can implement their own DataProcessors. Each DataProcessor runs on its own thread and has an input and output queue: one or more TaggedImages can be received on the input queue (by the poll() call) and modified, combined, or multiplexed, and then sent out on the output queue. Dimensional indices (such as ChannelIndex) should be adjusted by the DataProcessor to ensure that each TaggedImage is display in the right place in the MDA data set. The ImageFlipper plugin shows a simple example.

ProcessorStack: This object connects up a list of active DataProcessors. It is designed to received images from the queue returned by IAcquisitionEngine2010.run(…) and produces an output queue that yields processed TaggedImages.

MMImageCache (interface ImageCache): An object that provides a way to store images either in RAM or on disk. The MMImageCache has a pluggable saving mechanism that allows different file formats to be saved (currently two are available: single TIFF and multi-page TIFF). Incoming TaggedImages are automatically indexed according to their “Index” metadata tags (ChannelIndex, FrameIndex, SliceIndex, PositionIndex) and these can be retrieved by calling getImage with the appropriate indices. The ImageCache is wired up to the display components of Micro-Manager so that display settings and comments are also stored in RAM or saved on disk whenever they are adjusted by the user. When the user opens an existing data set, it is loaded into a new MMImageCache. When a disk-based ImageCache is attached to a VirtualAcquisitionDisplay, it essentially operates in “virtual” mode.

TaggedImageStorage: An interface for code to store images and metadata in a multi-dimensional data set, in RAM or on disk (or potentially in a database). Currently there are three implementations: TaggedImageStorageRam holds images in memory, TaggedImageStorageDiskDefault saves individual TIFF files to each image plane (and produces a collection of files compatible with earlier versions of Micro-Manager) and TaggedImageStorageMultipageTiff stores images in one or a very few multipage TIFF files, and reads and writes data substantially faster than the default saving method. The MMImageCache internally uses TaggedImageStorage instances.

VirtualAcquisitionDisplay: The default display class for Micro-Manager. This display module uses ImageJ windows to display multi-dimensional data sets stored in a given ImageCache. The VirtualAcquisitionDisplay object is also connected to the MetadataPanel, which shows the current metadata, comments, and channel settings. The display is designed to adapt to different numbers of channels, slices, positions and frames and show appropriate sliders and colors. In most cases, if color images or (because of DataProcessors) extra channels, slices, positions or frames are detected, the VirtualAcquisitionDisplay will adapt to show the full range of these dimensions. It expects a single height, width, and pixel type, however.

AcquisitionWrapperEngine (interface AcquisitionEngine): the traditional “acquisition engine,” available in the script panel as acq or by calling gui.getAcquisitionEngine(). This wrapper uses a DefaultTaggedImagePipeline to set up the data flow of TaggedImages.

High-level Introduction to Acquisition Engine Programming

In the broadest sense Acquisition Engine takes an acquisition specification (protocol) as an input and produces a data set as the output. The Engine parses the protocol, sequences all required logical steps, converts them to MMCore API commands, collects all resulting images and metadata and stores them to disk, memory or both.

Acquisition Engine is a part of the Micro-manager application and it runs within the Multi-Dimensional Acquisition Window, controlled by the GUI. However, the Acquisition Engine is independent of the GUI operation and can be used as a powerful tool for creating custom automated protocols.

There are three main components involved in running acquisition protocols:

SequenceSettings: protocol specification corresponding to the settings in the Multi-dimensional Acqusition Window in the GUI

PositionList: an array of stage positions (XY or XYZ) that acquisition protocol is going to be applied to

IAcquisitionEngine2010: the actual acquisition engine component defined as an interface. The implementation used in micro-manager is written in Clojure and can be imported as org.micromanager.AcqusitionEngine2010

Reading and Writing Micro-manager Data Sets

Programmatic Control of the Acquisition Engine

Implementing Custom Acquisition Engine