High quality color reproduction has been obtained from a 256x256 CMOS APS with red, green, and blue polyamid filters using a 3x3 color correction matrix. The matrix coefficients are determined by an error minimization procedure using a color test pattern. The camera signal processing operations of white and black balance, color matrixing, gamma correction, and reducing the effects of color aliasing have been implemented using real-time image processing software. The 0.9 mu CMOS imager has a 20 mu pixel pitch which yields a 5.1mm square active area (see reference [1]). A 1.2mm thick CM-500S glass color balancing filter is used to block the infrared and to attenuate the red wavelengths. While this reduces the red signal, it helps balance the color component signal levels which reduces the gain required to achieve color balance. Even with the filter, the blue response is about half that of the red under fluorescent illumination. With a 12.5mm f/1.4 C-mount lens in typical office illumination, the sensor can be operated at unity gain with a 33ms integration time. The lens is mounted on a printed circuit board that performs analog signal processing and analog to digital conversion. First, correlated double sampling removes pixel reset level variations. A second fixed pattern noise reduction stage removes column to column variations. Next, the signal is fed to three independent offset and gain stages which are used to perform a coarse white and black balance while fully utilizing the range of each of the three eight bit A/D converters. (We have also demonstrated the use of a single A/D converter where color balance is obtained via digital gain.) The color filters are arranged in vertical stripes, and the red, green, and blue pixel data appear in a serial fashion on the sensor's single output. An FPGA produces 24 bit parallel RGB data for each pixel by selectively repeating each pixel component three times. The image data is finally acquired by a frame buffer in a 100Mhz Pentium PC and stored for subsequent processing. The coefficients of the color correction matrix are determined using a procedure similar to that described in reference [2]. First the chromaticity coordinates of 18 color patches on a test pattern are measured and converted to the RGB space assuming NTSC standard phosphors and a D65 white point. Next, an image of the pattern is acquired and regions of pixels are averaged to determine an observed R^'G^'B^' tuple for each patch. The test pattern also includes white and black references which are used for white and black balancing (a digital balancing follows the coarse analog adjustment mentioned above.) Linear least square error minimization is performed in RGB space to yield a 3x3 matrix which, together with three offset terms from the black balancing operation, is used for color correction. A subjective evaluation of the color reproduction is performed by overlaying regions set to the theoretical RGB values of the patches on a corrected image of the test pattern. The PC performs several real-time image processing functions to generate a color-corrected image which is displayed in a window on an SVGA monitor. Automatic white balance is implemented by determining the highest amplitude region of an image and calculating relative gain factors to bring the components into balance. These gain factors can be incorporated in to the columns of the color correction matrix. Color aliasing is a real problem since there are only 85 = 256/3 samples of each color along a given scan line. While it is impossible to remove these effects completely, they can be reduced by interpolation. We use linear interpolation because of its low computational complexity, even though it excessively filters the data. The sensor output is largely linear (in fact the color correction procedure assumes this), and so we apply a gamma correction of 0.45 prior to displaying the image on the SVGA monitor. Using the color correction procedure outlined above, we have obtained color reproduction on par with consumer-grade CCD cameras. Future work would improve the effective spatial resolution using a different color filter array. The software implementation of the camera signal processing algorithms is a required step toward a single chip color camera.

[1] Alex Dickinson, Bryan Ackland, El-Sayed Eid, David Inglis, (AT&T) Eric Fossum (JPL/NASA), "A 256 X 256 CMOS Active Pixel Image Sensor with Motion Detection," ISSCC '95, San Francisco, CA, Feb 14-17, 1995.

[2] R. V. Kollarits and D. C. Gibbon, "Improving the Color Fidelity of Cameras for Advanced Television Systems," High-Resolution Sensors and Hybrid Systems, Proc. SPIE 1656, pp. 19-29, 1992.