JnJ Logo JAS Image Processing Program:

Medical Image Processing

IMA Fig1 The increasing quantity and diversity of diagnostic information is rapidly outgrowing the traditional film-based display format and requires advanced display services (ADS). The visualization of volumes of information, such as images routinely acquired throughout the entire head, is a difficult task. This is compounded by the increasing availability of additional images containing different forms of information (such as from functional or metabolite methods) (Figure 1). One of the main activities at the Neuroimaging Facility is to develop data presentation formats that more efficiently convey the multiplicity of information to the radiologists and their colleagues.

3D Computer Graphics

The display of rendered 3D objects is a commonly demonstrated task in modern radiology. However, routine use of these methods is far less common due to the hardware and software difficulties that arise in routine production of clinically useful presentations. Current research at the Facility focuses on how to make these presentations more efficient and more practical.

IMA Fig2 High resolution photorealistic rendering typically involves conversion of anatomical structures to "models" consisting of the geometrical description of the structure boundaries (typically by triangles or polygons). (Figure 2) The hardware problems associated with this process mostly center on the speed of model generation and on the size of the geometry models. Currently available workstation computers are not capable of interactive manipulation of these quantities of data and the supercomputing facilities at nearby UNM and SNL are being applied towards this problem in two general ways.

The first aspect is to apply visualization algorithms developed for non-medical purposes to medical imaging data in order to extract detailed models of specific structures. In this case the entire volume of imaging information is processed, the models are prepared, and images produced using the parallel supercomputer abilities to handle very large data sets and numbers of polygons. The second aspect of the joint research is to use specialized processing on these systems in order to manipulate the geometry model to reduce bulk without significantly reducing information content. These more efficient models are then transferred back to Facility computers for image rendering and use with the intraoperative localization systems (see below).

Often the anatomical structures of interest are difficult to identify by automated methods. This software problem is often avoided by employing tedious manual tracing of regions of interest - a procedure not appropriate for routine application. In order to improve capabilities for image "segmentation" or computer identification of specific tissue structures, additional joint research with SNL is evaluating the application of satellite imaging technology to medical images. Specialized image segmentation and analysis algorithms are being applied to MR and CT data to help identify pathology and to modify the image appearance to better convey the information to the reviewing radiologists. This work has established and is expanding a "training" database of patient image data that is used to continually update and improve the algorithm performance.

Data Fusion

IMA Fig3 A number of Facility research activities support the clinical application of advanced display systems and intraoperative image localization. Often the key to both is in the coregistration of imaging data from multiple modalities or from data taken at different times. (Figure 3) New forms of images can be provided that more compactly present information from multiple registered modalities. Improved interpretation can then be performed, for example, to quantitatively assess inter-modality agreement, the extent of disease progression, or response to therapy.

IMA Fig4 Data registration also arises in procedures to relate physical locations on a patient at the time of surgery to locations in the available presurgical image data (Figure 4). Work is ongoing to evaluate the practical characteristics of procedures for registration (e.g., multipoint, limited surface measurements, or some combination) and to extend these procedures to accommodate the more complicated situation of spinal configuration. Whole surface models simultaneously manipulated on the supercomputer are used as a standard for testing of more practical methods. The scope of this effort includes development of hardware that can be used to obtain surface measurements in the limiting environment of the surgical theater and operative field.


Holographic display allows examination and interpretation of a true 3D object rather than requiring mental assembly of 3D structures from a large series of flat 2D image slices. The Facility is working closely with Voxel Inc (Laguna Hills CA) who are developing practical holographic displays for radiology. The Facility continues to be active in assessing the clinical utility of holographic display and are the first of a small number of development and test sites for the camera systems required to produce the holograms on a routine basis.

In addition to using holography for improved anatomical assessment, research is underway to incorporate holographic display into the intraoperative image localization systems. While the hologram often allows improved abilities to assess complicated 3D structures, especially in spinal neurosurgery, systems are being developed to accurately relate positions within the holographic space to actual locations and angles relative to the patient at the time of surgery. The improved data visualization offered by holography is further enhanced by tactile feedback when interacting with the holographic object. Systems for using geometry models registered with the hologram, along with force transducing systems to provide the physical sensations, are being developed at the Facility.

Computer Services

While augmented by those of our partner institutions, very capable computing resources are available in the Facility. A large and expanding network of Sun, DEC, HP, and NT workstation systems, and associated peripheral devices, support both research and clinical activities. Full network and data interchange facilities exist within the Facility and are being extended to incorporate systems at collaborating centers. In order to improve the utilization efficiency of Facility clinical findings, high-speed network systems are being deployed to support audio/video teleconferencing and for simultaneous image data display, manipulation, and evaluation.

Last Modified: May 21, 1999 Copyright 1999 John A Sanders, VAMC