Electro-Optical Remote-Sensing Systems

Engineering and Simulation of Electro-Optical Remote-Sensing Systems

Stephen Cota

In designing remote-sensing systems, performance metrics must be linked to design parameters to flow requirements into hardware specifications. Aerospace has developed tools that comprehensively model the complex interaction of these metrics and parameters.

Electro-optical remote-sensing systems are built to do specific jobs—for example, to make meteorological measurements, to characterize Earth's climate, to track patterns of land use, or to collect high-quality imagery. It is the systems engineer's task to determine what characteristics a proposed system must have to fulfill its mission. To do so, the engineer must "flow down" requirements from the mission level to the sensor as a whole, from the sensor to its components, and from components to subcomponents. Aerospace has developed tools and expertise to facilitate this complex process.

Performance Goals

In most cases, top-level system performance must be expressed in quantitative terms in order to flow down requirements. In the case of a meteorological sensor, performance might be specified in terms of the desired accuracy of surface reflectance or surface temperature measurements. For an imaging sensor, performance might be specified in terms of a quantitative metric such as the National Image Interpretability Rating System (NIIRS), which grades images based on their usefulness in performing analytic tasks.

Once presented with a quantitative top-level requirement, the systems engineer determines which hardware would be suitable based on standard performance metrics. Such metrics are many and varied.

A large class of metrics appropriate to radiometric and imaging systems are those related to signal-to-noise ratio, which must be high enough to confidently distinguish the lowest signal of interest from spurious features caused by electronic noise or the inherent fluctuations of the signal. The signal-to-noise ratio itself may be the preferred metric, or it may be replaced by a "noise-equivalent" quantity, such as noise-equivalent delta reflectance or noise-equivalent temperature difference. A noise-equivalent quantity represents the input signal level required to achieve a signal-to-noise ratio of exactly 1. It is convenient because it presents the noise in the units of the signal. All of these metrics set constraints on the hardware. For example, to maximize the signal, the detectors must be large (to collect the most light possible) and the optics must be highly reflective and fast (i.e., have a low ratio of focal length to aperture); to minimize noise, the detectors must be cooled, stray light must be held to a minimum, and so forth.

Surface plot of the modulation transfer function (top) and the point-spread function (bottom) for a perfect circular aperture. The modulation transfer function is a Fourier-domain representation of how edges are softened by a sensor. The point-spread function describes the blur produced by an infinitely sharp point as imaged by a sensor; it can be computed from the modulation transfer function, which is its Fourier transform.

Another class of metrics are those related to spatial resolution. For a system primarily concerned with the collection of radiometric information, often a simple parameter such as ground-sample distance—the size of a single pixel projected onto the ground—may be adequate. For systems requiring high image quality, other metrics of resolution must be computed, such as the relative edge response or the modulation transfer function. Both metrics characterize how diffraction and other inherent limitations of the optical system blur sharp features in a scene such as coastlines or cloud edges. The relative edge response directly measures how an infinitely sharp edge becomes softened, while the modulation transfer function is a Fourier-domain representation of how all edges are softened, whether inherently sharp or not. Like the signal-to-noise metrics, resolution metrics also place constraints on the hardware—and often, the constraints imposed by resolution oppose those imposed by the signal-to-noise metrics. For example, to achieve a certain ground-sample distance (e.g., 0.5 to 1 meter for the current generation of space-based commercial imagers), the detectors must be made smaller or the effective focal length must be lengthened, to the detriment of the signal-to-noise ratio; to achieve a high relative edge response or modulation transfer function, the optics must be large with minimal obstructions.

Complex Relationships

The relationship between top-level requirements and the standard performance metrics is seldom as simple as it first appears. For example, a system designed for classification of terrain might need to detect a change of 0.05 in surface reflectance in a given visible spectral band with an accuracy of 10 percent. At first glance, it might seem sufficient to start with a signal-to-noise ratio of 10 and divide that into 0.05 to derive a required noise-equivalent delta reflectance of 0.005. This value could then be used in selecting and configuring the hardware. In practice, however, variable atmospheric constituents such as water vapor and aerosols corrupt visible-band measurements—and because the levels of these constituents are not known a priori, they must be estimated using spectral bands in a water-vapor absorption region and in the short-wave infrared before the engineer can correct for them. Thus, the error in the visible band of interest becomes a function not only of its own noise-equivalent delta reflectance but also the noise-equivalent delta reflectances of those bands used to determine water vapor and aerosol levels. And this is just one of many error effects. Others include detector response variations, miscalibration, and band-to-band misregistration, as well as errors introduced by the approximations inherent in any practical water-vapor and aerosol retrieval algorithm. All of these can have a bearing on the ability to detect a change of 0.05 in surface reflectance, and most can't be related to one another via closed-form equations.

Similar problems occur in imaging systems. There are many ways to achieve a NIIRS rating of 5, for example. The designer might simultaneously vary ground-sample distance, relative edge response, and signal-to-noise ratio—to say nothing of using sharpening filter coefficients to emphasize edges and contrast—all of which affect image quality. Variations in detector response and other artifacts such as spectral banding (low-frequency variations in spectral response across a detector array) also affect image quality. For multispectral imagery, band-to-band misregistration and miscalibration must also be considered. Again, in most cases, these effects can not be related to one another via closed-form equations.

Because of the many complications involved in relating mission-level performance to lower-level system parameters, the electro-optical systems engineer must usually build an end-to-end simulation for the sensor. Such simulations typically start with an image of much higher quality than the proposed sensor is expected to produce; they then transform the image by applying models of the sensor's modulation transfer function, noise level, response uniformity characteristics, and calibration accuracy. This produces the expected output of the sensor. Each error source is introduced at that point in the imaging process where it would actually arise, thus replicating any nonlinear interaction between error sources. Finally, the results of the simulation can be fed into algorithms used to extract data from the image to determine actual error levels. The systems engineer can then modify the electro-optical sensor's parameters, either to decrease the error, if the mission-level specification has not been met, or to increase it (while reducing mass and power), if the specification is met with excessive margin.

The Aerospace Corporation has written and maintains several end-to-end simulations, each tailored for a specific class of problems. Those having the broadest applicability include the Visible and Infrared Sensor Trades, Analyses, and Simulations (VISTAS) package; the Physical Optics Code for Analysis and Simulation (PHOCAS); and the Parameterized Image Chain Analysis and Simulation Software (PICASSO). Aerospace has used all three of these tools to complete complex systems engineering tasks for a variety of remote-sensing programs.

PICASSO: A Portrait

PICASSO is the most recent addition to the Aerospace suite of electro-optical systems engineering tools. It was designed to be modular, machine independent, easy to use, and easy to customize.

A PICASSO simulation begins with a set of parameters describing the electro-optical sensor to be simulated. A high-quality image, taken in the sensor's spectral band and at the viewing geometry of interest, is used as the starting point. This input image is meant to represent the real world as seen through a perfect sensor (that is, one with infinite resolution and infinite signal-to-noise ratio); it therefore should have much better signal-to-noise ratio, resolution, and sample spacing than an image from the sensor to be simulated (see sidebar, The Imaging Chain).

In practice, it can be difficult to find such an image because the sensor to be simulated is often intended to surpass existing sensors of its class. The PICASSO analyst can employ a number of strategies to overcome deficiencies in the input imagery. For example, in simulating a space-based sensor, if no high-resolution imagery from space can be found, the analyst might substitute an aircraft image and use an atmospheric modeling code to correct for transmission losses and path radiance effects that would occur between the aircraft's altitude and space. Alternatively, the analyst might take existing space-based imagery of marginally usable resolution and enhance it using standard image restoration techniques. Synthetic images, produced from first-principles physics codes, have effectively infinite signal-to-noise ratio and high resolution, but sometimes appear unrealistic, particularly for vegetation or other natural surface types. Synthetic images have precisely known surface and atmospheric properties, making them attractive for testing algorithms that try to derive these quantities (though such tests are seldom definitive because of the approximations and limitations inherent in the models that translate these properties into observed radiance).

Some imagery will have the requisite resolution and signal-to-noise ratio, and yet be an imperfect match to the spectral passband of the sensor to be simulated. In this case, an atmospheric modeling code can again be used to correct the imagery to the desired passband. Imagery in the desired passband can also be synthesized via a weighted sum of hyperspectral images. Similarly, imagery collected at sun and sensor angles different from those desired can be converted back to reflectance values and then translated to the desired geometry by means of an atmospheric modeling code.

Once a suitable input image has been selected or produced, PICASSO models how the physical limitations of the proposed electro-optical system will affect it. The first step in this process is to degrade the input image's resolution until it matches that of the proposed sensor.

Any practical sensor will have a hard limit on the resolution of its images imposed by the finite size of its optical system. Diffraction off the sensor's primary mirror and other structures lying in the optical path will cause inherently sharp features to diffuse and blur when projected onto the focal plane. The characteristic blur pattern produced by a single, infinitely sharp point as imaged by the sensor is known as the point-spread function. This point-spread function can be convolved with the high-quality input image to model the effects of optical diffraction. Often, it's easiest to compute the point-spread function from the modulation transfer function, which is its Fourier transform.

Optics are not the only sensor elements that degrade image quality. Resolution is also lost through the use of focal-plane arrays to record the image. These arrays are composed of detectors of finite size, and an infinitely sharp feature on the ground can generally appear no smaller in the final image than the size of the detector that collects it. A common type of detector used in visible imagers is the charge-coupled device (CCD)—a monolithic array of silicon detectors, each of which measures light by collecting the charge produced by incident photons. In addition to the resolution lost to the finite size of the CCD detectors, resolution can be lost to the undesired diffusion of charge between detectors. If the sensor moves during its integration period—because of orbital motion, jitter, or scanning motion—the image will smear, just as it does when an ordinary photographer with a handheld camera moves while taking a picture. All of these effects can be described by their own unique modulation transfer functions, and PICASSO accounts for each of them.

After degrading the input image to the sensor's resolution, PICASSO resamples the image, taking a value from the blurred input image at each location where a detector would reside in the sensor's focal plane and mapping it to the output image.

Along with resolution losses, the most significant source of image degradation is noise. PICASSO models several classes of noise. Temporal white noise and flicker are produced by random (often thermal) processes in the sensor's detectors and electronics and by the inherent variability in the signal itself. In addition, the individual elements of a detector array often vary in their response to a uniform source of light, giving rise to fixed-pattern noise. If the sensor employs a two-dimensional focal-plane array, the fixed-pattern noise will probably appear random in a single image, but consistent from one image to the next. If the sensor employs a one-dimensional focal-plane array that is scanned to produce an image, the fixed-pattern noise will give rise to streaks, recognizable as pattern noise even in a single image. Other noise processes include quantization noise, introduced by digitizing the analog signal, and signal distortion, caused by the nonlinearity of the analog-to-digital converter.

Target radiance reaching the sensor is corrupted by atmospheric attenuation due to water vapor, aerosols, and other atmospheric constituents. Measuring the target radiance is further complicated by unwanted radiance reaching the sensor over many paths: In addition to direct and diffuse sunlight reflected from the target, the sensor will receive radiance scattered by the atmosphere, as well as direct and diffuse sunlight first reflected off the target's surroundings and then scattered by the atmosphere into the sensor's field of view. For many applications, it is necessary to correct for both atmospheric attenuation and these so-called path radiance terms.

These steps are usually part of the PICASSO imaging chain regardless of the electro-optical system modeled. The final steps, representing ground processing and data exploitation, vary considerably, depending on the application at hand. For an imaging system, PICASSO will proceed to model various techniques for enhancing resolution, such as the use of sharpening filters or nonlinear image restoration. For a meteorological system attempting to retrieve surface reflectance data, PICASSO would pass the output imagery to an atmospheric compensation algorithm.

The output of PICASSO is a representative image from the simulated sensor, along with one or more figures of merit. The figures of merit—signal-to-noise ratio, NIIRS rating, relative edge response, error in retrieved reflectance, etc.—can be compared with the sensor's mission-level requirements. When they exceed or fall short of requirements, the electro-optical systems engineer may vary the sensor parameters and rerun the simulation, searching for the optimal parameter set. The number of trials required to find an optimal design varies with the complexity of the relationship between the metrics and the sensor parameters upon which they depend.

Sometimes, the standard metrics do not tell the whole story. This is because they measure only particular aspects of sensor performance, and do not reflect the effect that some artifacts and distortions have on performance. Although absent from the metrics, these artifacts and distortions will be present in the simulated imagery, allowing the systems engineer to continue with the optimization process even in cases where the metrics do not reflect the true limitations of the system.

Conclusion

PICASSO and similar end-to-end simulation tools form a vital part of electro-optical systems engineering at Aerospace. These tools have been successfully applied to numerous remote-sensing programs since their inception and can be expected to form the basis of an ongoing robust systems engineering capability.

Acknowledgements

The end-to-end simulation codes discussed here are the product of many years of work. The author would like to acknowledge the efforts of those who helped create them, especially Jabin Bell, Tim Wilkinson, Robert A. Keller, Richard Boucher, Linda Kalman, Mark Vogel, Rose of Sharon Daly, Tom Trettin, Joe Dworak, Terence S. Lomheim, and Mark Nelson.