1. Imaging System for Mesopic Vision
CHIBA UNIV.
Imaging System for Mesopic Vision
H.Yaguchi, Y.Ushio, K.Kikuchi, D.K.Thahn, J.Shin, and S.Shioiri*
Chiba University and Tohoku University*
Thank you Chairman.
I am going to talk on imaging system for mesopic vision.

2. Introduction Cone Rod Rod and cone Mesopic vision Photopic vision
Human visual system covers
Illuminance range from
10-3 lx to 105 lx.
Cone
Photopic vision　　　　　
Rod
Scotopic vision
Rod and cone
Mesopic vision
Human visual system covers a wide illuminance range from lx to a hundred thousand lx. (return)
This wide dynamic range is achieved with two kinds of photoreceptors in the retina, rods and cones. (return)
There are illuminance levels where both rods and cones are active.
The vision at these levels is called mesopic vision.

3. How does color appearance change with illuminance ?
Color appearance in mesopic vision differs from that in photopic or scotopic vision.
Imaging systems on the market such as digital cameras and photographic films, however, are based on the human color vision in photopic vision.
So, it is hard to reproduce image that would be seen in mesopic vision by the current imaging system. (return)
In order to predict a color appearance in mesopic vision, we have to consider the interaction between cone and rod at levels of mesopic vision.
To predict color appearance in mesopic vision from the photopic image.

4. Outline Experiment of the haploscopic color matching.
A mesopic color appearance model.
A imaging system for mesopic color appearance.
Examples of mesopic color images.
We carried out haploscopic color matches to investigate the color appearance under various illuminance levels, ranging from photopic to scotopic via mesopic levels.
Using these experimental data, we developed a color appearance model applicable in mesopic vision.
Finally, we propose an imaging system that provides color appearance of any natural scene under any illuminance level.
Then I show you some examples of mesopic color images.

5. Color appearance in mesopic vision
•Haploscopic color matching technique lx lx lx lx lx lx
In our experiment, we used a haploscopic color matching technique to measure the corresponding color of test color chips under various illuminance levels.
The observer saw a test color chip with his/her left eye and adjusted the appearance of the color presented on the CRT display with the right eye so as to match the appearance of the test color.
The test color chip was presented in the left room of the booth under one of the illuminance levels of 1000 (return)., 100 (return)., 10 (return)., 1 (return)., 0.1 (return)., and 0.01 lx.

6. Color appearance in mesopic vision
•Stimuli
Test field
48 Munsel color chips
…chromatic: 45，achromatic: 3
Matching field
21” CRT display (SONY Multiscan G500)
Controlled by VSG
(15 bits color, Cambridge System)
Stimulus size… 1010º
The test stimulus was one of 48 color chips including 3 achromatic color chips. (return)
The matching stimulus was generated on a CRT display set behind the booth so that the observer saw the color through an aperture.
The illuminance level of the matching field was fixed at 1000 lx.

7. Results: Hue and Chroma
Chroma reduces continuously with decrease of the illuminance level until 0.01 lx.
The loci of matching color on the a*-b* diagram are not straight for many test color chips, indicating that hue shifts with the change in illuminance level.
This figure show the color appearance of 10 test chips approximately equally spaced in hue angle in the CIELAB a*-b* diagram as a function of illuminance level.
These plots are mean data for all observers. (return)
Chroma reduced continuously with decrease of the illuminance level until 0.01 lx. (return)
The loci of matching color on the a*-b* diagram were not straight for many test color chips, indicating that hue shift with the change in illuminance level.

8. Results: Chroma
Reddish and yellowish color (Y, YR, R, RP, P)：Chroma rapidly decreases from 100 to 1 lx, and constant below 0.1 lx.
Greenish and bluish color (PB, B,BG, G, GY): Chroma rapidly decreases from 1000 to 1 lx, does not change below 1 lx.
These graphs show change of chroma as a function of illuminance.
Reddish and yellowish color (Y, YR, R, RP, P)：Chroma rapidly decreases from 100 to 1 lx, and constant below 0.1 lx.
Greenish and bluish color (PB, B,BG, G, GY): Chroma rapidly decreases from 1000 to 1 lx, does not change below 1 lx.

9. Results: Lightness
Reddish and yellowish color: Lightness gradually reduces from 10 to 0.01 lx.
Greenish and bluish color: Minimum lightness is observed around 10 to 1 lx.
These graphs show change of lightness as a function of illuminance.

10. Results: Achromatic Lightness
This figure shows lightness change of achromatic color as a function of illuminance.
Perceived lightness range is getting small with decreasing illuminance. This phenomenon is called the Stevens effect.
The Stevens effect: Perceived lightness range is reduced with decreasing illuminance.

11. Correlation between perceived lightness at 0
Correlation between perceived lightness at 0.01 lx and photopic and scotopic luminance
These two graphs show correlation between perceived lightness at the lowest illuminance of 0.01 lx and the CIE luminance.
As you see, the correlation between perceived lightness and the CIE photopic luminance is very poor shown in left graph.
On the other hand, correlation is very good for the CIE scotopic vision.
Furthermore, you can notice that the perceived lightness in low luminance is nonlinearly related to the perceived lightness in photopic vision.
Photopic luminance
Scotopic luminance

12. A Color Appearance Model in Mesopic Vision
Luminance channel: L+M
Red/green opponent-color: L-2M
Yellow-blue opponent-color: L+M-S
We applied the Boynton’s two-stage color-vision model8 to our mesopic color vision model.
The model assumes that opponent-color process converts signals from L, M and S cones to red/green and yellow/blue opponent-color and luminance responses, that is, L - 2M, L + M - S and L + M.
Our model puts a rod intrusion in luminance channel and each of the opponent channels.
The amount of rod signal varied depending on the illuminance levels and on the channels.
The amount of the cone signals also varied depending on the illuminance levels and on the channels.

13. Luminance as a function of illuminance
A(E) = (E)100 ((Lp+Mp) / (Lp+Mp)w)+(E) 78.4 (Y ’ / Y ’w)
Lp , Mp , Sp: cone outputs at photopic condition
Y’ : scotopic luminance factor
(Lp+Mp)w , Y’w : each output of luminance
channels for white (being 100 = Kw)
 (E),  (E) :Weighting coefficients for functions of illuminance E
This graph shows the model of the luminance channel.
Lp, Mp and Sp represent cone stimulus values at photopic condition, and A(E) represents outputs of the luminance channel at illuminance level of E.
Y’ represents the scotopic luminance factor, which can be regarded as rod output.
Y’ is calculated for each test chip by the CIE spectral luminous efficiency V’(). (return)
This coefficient takes account for the Stevens effect, and this gamma is come from the nonlinearity between phtopic lightness and scotopic lightness.

14. Change of cone- and rod-signal as a function of illuminance
A(E) = (E)100 ((Lp+Mp) / (Lp+Mp)w)+(E) 78.4 (Y ’ / Y ’w)
The weighting coefficients, (E) and (E) indicate contribution amounts of photopic and scotopic luminance as a function of illuminance E.
The weight of cone signals decreases with decrease in illuminance while the weight of rod signals increase.

15. Red/green- and yellow/blue opponent channel as a function of illuminance
Outputs of r/g and y/b channels at illuminance level of E
r/g(E) = l(E)(Lp-2Mp) + (E) Y ’
y/b(E) = m(E)(Lp+Mp-Sp) + (E) Y ’
Lp , Mp , Sp: cone outputs at photopic condition
Y’ : scotopic luminance factor
l (E),  (E) and m (E), (E) :Weighting coefficients for functions of illuminance E
This slide shows the model of the two opponent-color channel.
r/g(E) and y/b(E) represent outputs of the luminance, red/green and yellow/blue channels at illuminance level of E.
The weighting coefficients, l(E) and a(E) indicate contribution amounts of the photopic red/green process (red/green signal made of L and M cone outputs) and rod to the red/green channel.
The weighting coefficients, m(E) and b(E) indicate contribution amounts of the photopic yellow/blue process (yellow/blue signal made of L, M and S cone outputs) and rod to the yellow/blue channel. (return)
These weighting coefficients vary with illuminance level of E to express the change of contribution amounts of the factors among different illuminance levels as shown in this graph.

16. Comparison between Experimental Results and Prediction: Hue and Chroma
This graph shows a comparison between experimental results and the predicted value by the model.
The model can well fit the experimental data of hue and chroma.

17. Comparison between Experimental Results and Prediction: Lightness
This graphs show the prediction of luminance.
Again, the model can well fit the experimental data.

18. A Color Appearance Model in Mesopic Vision
Chromatic components decrease with decreasing illuminance.
Hue shifts are predicted by introducing a different process for red/green and yellow/blue opponent-color channel.
So, our model can predict these two color appearance in mesopic vision.(return)
Chromatic components decrease with decreasing illuminance.(return)
Hue shifts are predicted by introducing a different process for red/green and yellow/blue opponent-color channel.

19. Performance of a model Average E*ab in mesopic range is around 3.
In order to estimate how accurately the model predicts the experimental results, we calculated the CIE color differences between the experimental data and model predictions obtaining L*, a* and b* coordinates using the Judd modified color matching functions.
The average E*ab value is about 3 at the illuminance below 1 lx and about 4 at 10 lx.
The error of the prediction is large for the purple region.
Average E*ab in mesopic range is around 3.

20. Imaging System for Mesopic Vision
Now, let me introduce our imaging system for mesopic vision.
An actual scene is taken by a digital camera at the photopic illuminance condition, then applied the mesopic color appearance model, finally we get the image which would be seen in any illuminance level on a CRT monitor or print.

21. Imaging System for Mesopic Vision
This slide shows a flow chart of the imaging system.
At first, the scene of interest is taken by a digital camera, and get data of Rcamera, Gcamera and Bcamera for each pixel.
Secondly, the pixel data are transformed to the X, Y, Z tristimulus values and the scotopic luminance factor, Y’ using the camera model, where the polynomial function of R, G, B are used to obtain X, Y, Z and Y’.
Thirdly, the X, Y, Z tristimulus values are converted to the cone stimulus values at a photopic level (Lp, Mp, Sp).
Then, in order to obtain output values of the opponent state at a given illuminance E, Lp, Mp, and Sp are input to the mesopic color appearance model with the weighting coefficients which depend on illuminance E.
From the tristimulus values, the model provides outputs of the opponent channel (Lum(E), r/g(E) and y/b(E)) at the illuminance E, which in turn are transformed to tristimulus values.
Finally, using the output device colorimetric model, X, Y, Z tristimulus values are transformed to Routput, Goutput and Boutput digital unit for each pixel of the scene.

22. To obtain XYZ and Y’ from Digital Camera RGB
Predicted Value
Measured Value Xp
Yp’ Zp Yp X R G B
Camera
model Y
Minimize error Z Y’
This slide show how to obtain the XYZ tristimulus value from the camera output RGB.
Color samples are taken by the camera, and the colorimetric values of the same color samples are measured by the spectral photometer.
We develop a camera model to minimize the color difference between the predicted by the camera model and the measured colorimetric values

23. Calibration of Camera Color Sample・・・Macbeth Color Checker
and GretagMacbeth ColorCheker
Digital Camera・・・Canon,EOS1-Dｓ
We used Macbeth Color checker and Gretag Macbeth color checker for digital camera as color samples
Canon EOS1-Ds is used for the camera model.
・Image size　　4064×2704 pixels
・Quantization　　RGB 12 bits

24. Camera Model R3 G3 B3 R2G R2B ・ BR Xp a1 b1 c1 ・・・・・・・・ r1 s1 t1 R Yp１ Xp Yp Zp
Yp’
a1 b1 c1 ・・・・・・・・ r1 s1 t1
a2 b2 c2 ・・・・・・・・ r2 s2 t2
a3 b3 c3 ・・・・・・・・ r3 s3 t3
a4 b4 c4 ・・・・・・・・ r4 s4 t4 =
We used the trinomial equation to obtain the tristimulus value and the scotopic luminance from the camera RGB value.
Coefficients are obtained by the pseudo-inverse method.
Coefficients a, b,,,t are obtained by
the pseudo-inverse method.

25. Camera performance as a colorimeter
Average color difference　 ⊿E*ab = 2.23
Maximum color difference ⊿Eab = 9.03
Average color difference between the measured colorimetric value and the predicted value by a camera is 2.23 for Macbeth color checker. Maximum error is about 9 in delta E.
Original
Macbeth Color Checker
Predicted by a Camera
Macbeth Color Checker

26. Prediction of scotopic luminance Y’ by camera RGB
Predicted value
This graph shows the prediction of scotpic luminance by the camera RGB.
Measured value
Coefficient of Correlation
Slope
0.998
1.000 :

27. Examples of mesopic color image
100(lx)
10(lx)
Original
1(lx)
0.1(lx)
0.01(lx)
This slide shows the examples of mesopic color image.
Macbeth color checker was taken by the camera, and the color appearance in any illuminance is reproduced by this imaging system.