Mobile Phone Imaging Module with Extended Depth of Focus Based ...
Mobile Phone Imaging Module with Extended Depth of Focus Based on Axial Irradiance Equalization Phase Coding Hsin-Yueh Sung*a,b, Po-Chang Chenb, Chuan-Chung Changb, Chir-Weei Changb, Sidney S. Yanga, Horng Changb a Institute of Photonics Technologies, Department of Electric Engineering, National Tsing Hua University, Hsinchu 300, Taiwan b Industrial Technology Research Institute, Electronics and Opto-Electronics Research Lab., Hsinchu 310, Taiwan ABSTRACT This paper presents a mobile phone imaging module with extended depth of focus (EDoF) by using axial irradiance equalization (AIE) phase coding. From radiation energy transfer along optical axis with constant irradiance, the focal depth enhancement solution is acquired. We introduce the axial irradiance equalization phase coding to design a twoelement 2-megapixel mobile phone lens for trade off focus-like aberrations such as field curvature, astigmatism and longitudinal chromatic defocus. The design results produce modulation transfer functions (MTF) and phase transfer functions (PTF) with substantially similar characteristics at different field and defocus positions within Nyquist pass band. Besides, the measurement results are shown. Simultaneously, the design results and measurement results are compared. Next, for the EDoF mobile phone camera imaging system, we present a digital decoding design method and calculate a minimum mean square error (MMSE) filter. Then, the filter is applied to correct the substantially similar blur image. Last, the blur and de-blur images are demonstrated. Keywords: mobile phone lens, extended depth of focus, phase coding, axial irradiance equalization, computational imaging, digital decoding, image restoration, minimum mean square error filter
1. INTRODUCTION In recent years, many researchers have been devoted to design and develop extended depth of focus or field (EDoF) imaging systems [1-5]. The EDoF imaging systems posses an advantage capability of implement with fixed lens and no moving parts, hence their sizes can keep compact and small. The way to achieve the goal of EDoF is to use a so-called computational imaging technique. The technique is inherently an integrated design, which employs phase coding in optical design and digital decoding in image signal processor. The phase coding is utilized to substantially increase the depth of focus (DoF) of the imaging system by means of reducing the point spread function (PSF) variance in the range near the focal plane, but the PSF might be in a special form of a big size. Such PSF often results in some image blur. The digital decoding (also referred to image restoration, de-convolution or software lens compensation) is thus applied to restore the blurry image. A typical approach of phase coding is the wave-front coding method proposed by Dowski and Cathy [1]. By introducing an asymmetrically cubic phase mask into lens design, the PSF of the lens will be more insensitive to defocus compared with the traditional lens. However, for the asymmetrically cubic phase coding, the phase transfer function is not substantially invariant in the EDoF focal region and therefore the artifact of restored image is inevitable by using only a single restoration filter [6]. On the contrary, a special symmetric surface form, designed by George and Chi [2], increase in the depth of field was achieved by using a logarithmic asphere. The logarithmic asphere is a circularly symmetric surface and it is designed by equaling pupil area and applying Fermat’s principle. The logarithmic asphere have been applied to the mobile phone lens design [3] with EDoF range from 300 mm to infinity and the extension factor in the range of 2 to 3 is achieved. More recently, Mouroulis [4] proposed the idea that spherical aberration is applied to a low-power microscope by means of equalizing the MTF across different focus positions in order to achieve the extended depth of field. Afterwards, Robinson [5] uses the spherical coding further to increase lens speed and improve manufacturing yield. *[email protected]; phone 886-3-5731170; fax 886-3-5751113
Digital Photography VII, edited by Francisco H. Imai, Feng Xiao, Jeffrey M. DiCarlo, Nitin Sampat, Sebastiano Battiato, Proc. of SPIE-IS&T Electronic Imaging, SPIE Vol. 7876, 787606 © 2011 SPIE-IS&T · CCC code: 0277-786X/11/$18 · doi: 10.1117/12.872260 Proc. of SPIE-IS&T/ Vol. 7876 787606-1 Downloaded from SPIE Digital Library on 08 Feb 2011 to 192.17.192.96. Terms of Use: http://spiedl.org/terms
In this paper, we propose a mobile phone imaging module with extended depth of focus by using AIE phase coding and MMSE digital decoding. In section 2, we introduce the AIE phase coding and apply the phase coding to design a twoelement 2-megapixel mobile phone lens for trade off focus-like aberrations such as field curvature, astigmatism and longitudinal chromatic defocus. Section 3 show the measure results of AIE EDoF lens and make a comparison with the design results. In section 4, we give de-blurred images by means of using MMSE digital decoding. Section 5 concludes the paper.
2. EDOF MOBILE PHONE LENS DESIGN USING AIE PHASE CODING In the section, we design an EDoF mobile phone lens by means of using AIE phase coding. First, let us consider a circularly symmetric multi-focal lens from exit pupil. In the multi-focal lens, f0 is the paraxial focal length, Rmax is the maximum exit pupil radius, Δz is the fully desired depth of focus with distances measured along the Z axis as positive to the right and negative to the left of the paraxial focal plane. Besides, in most imaging systems, the exit pupil is round and the irradiance is the same as the irradiance from a uniform Lambertian disk [10]. On this condition, from radiation energy transfer along optical axis with constant irradiance, the multi-focal lens f(r) is described by [7]
f (r ) =
f0 3 −
3 3
3
2
3[2 f 0 + 27Cr 2 + 18 f 0 r 2 + 4(3r 2 − f 0 ) 3 + (2 f 0 + 27Cr 2 + 18 f 0 r 2 ) 2 ]1/ 3 3
+
2
2 (3r 2 − f 0 ) 2 3
3
.
(1)
[2 f 0 + 27Cr + 18 f 0 r + 4(3r − f 0 ) + (2 f 0 + 27Cr + 18 f 0 r ) ] 2
2
2
2
2 2 1/ 3
3(3 2 )
Where C is defined by
C≡
Δz[( Rmax ) 2 + ( f 0 + Δz ) 2 ] . ( Rmax ) 2
(2)
Due to no any paraxial approximation, the Eq. (1) is not only suitable for small aperture and small depth of focus but also suitable for large aperture and large depth of focus. Next, we will use the Eq. (1) to design a two-element 2-megapixel mobile phone lens in order to alleviate focus-like aberrations such as field curvature, astigmatism and longitudinal chromatic defocus and reduce the modulation transfer function (MTF) variance in the range near the focal plane. Detail design specifications of the EDoF mobile phone lens are listed in Table 1. The EDoF mobile lens consists of only two E48R plastic elements and is f = 3.4 mm, f/# = 3.0, 66 degree field of view, polychromatic wavelength band from 0.470 μm to 0.643 μm and default object distance at 300 mm. Because the field of view is so large, the major defocus aberration is field curvature. In order to minimize the field curvature, firstly we choice Petzval lens type as paraxial layout design. However, it is not sufficient to achieve to 2-megapixel mobile phone lens performance and the field curvature is still too large. Thus, the large depth of focus is required in order to alleviate the effect of field curvature. Secondly we introduce the AIE phase coding to enlarge the depth of focus. In this case, we choice the desired depth of focus Δz = -0.2mm to against focus-like aberrations. Next, according to the condition f0 = 3.4 mm, Rmax = 0.566 mm (f0/ 2Rmax = F/# = 3.0), Δz = -0.2 mm and Eq. (1), we can acquire the desired AIE multi-focal lens and describe the multifocal lens in spherical wave-front aberration representations [11-14]. The AIE phase coding W(ρ) is written by:
W ( ρ ) = (2.3013λ ) ρ 4 + (0.2062λ ) ρ 6 + (0.0238λ ) ρ 8 + (0.0074λ ) ρ 10 ,
(3)
Where wavelength unit λ is 0.55 μm, normalized radius ρ is from 0 to 1 and the spherical wave-front aberration expansions are accurate to ±0.0001λ.
Proc. of SPIE-IS&T/ Vol. 7876 787606-2 Downloaded from SPIE Digital Library on 08 Feb 2011 to 192.17.192.96. Terms of Use: http://spiedl.org/terms
Table1. The specifications for EDoF mobile phone lens
Parameter Effective Focal Length F/Number Design Wavelength FOV Maximum Chief Ray Angle Optical Distortion Relative Illumination Lens Construction CMOS Sensor Resolution Pixel Size
Value 3.4 mm 3.0 0.47μm ~ 0.643μm ±33° 22.5°±1.5° < 2% > 50% 2P ; [+|-] 1600*1200 Bayer Pattern 2.2μm
In the reminder of this section, we applied the AIE spherical wave-front aberration Eq. (3) to design the 2-megapixel EDoF mobile phone lens. The design degrees of freedom include the central aperture stop location, the back focal distance, the lens thickness, curvatures and standard aspheric coefficients of r4, r6, r8, r10. And further, by adding a penalty function on the spherical wave-front aberration coefficient of Eq. (3) to a traditional root mean square wave-front error merit function [15,16], we can minimize the other spatially varying optical aberrations such as field curvature, astigmatism, and even longitudinal chromatic aberration at the expense of AIE phase coding. The final design results of EDoF mobile lens at object distance 30cm are shown in Fig. 1, including (a) 2D lens layout, (b) ray aberration curve, (c) modulation transfer function (MTF) curve at normalized field 0.0, 0.2, 0.4, 0.6, 0.8, and 1.0, and (d) chief ray angle (CRA) at image surface. Excluding MTF at field 1.0, the other MTF are substantially invariant and no PTF reversal issue below 112 lp/mm. It is suitable for using MMSE digital decoding in Section 4. Besides, from viewpoint of aberration theory, AIE phase coding is just an especially longitudinal spherical aberration. The fully depth of focus Δz is also the distance of longitudinal spherical aberration from paraxial focus to marginal focus. Thus, in order to make EDoF effectively, not only the distance of longitudinal spherical aberration must be larger than the sag distance of field curvature but also the preferred focal plane is about at the middle point between paraxial and marginal focal point. The Fig. 1 (b) ray aberration curve show these results under the AIE phase coding condition.
Fig. 1(a) 2D Lens Layout
Proc. of SPIE-IS&T/ Vol. 7876 787606-3 Downloaded from SPIE Digital Library on 08 Feb 2011 to 192.17.192.96. Terms of Use: http://spiedl.org/terms
Fig. 1(b) Ray Aberration Curve
Fig. 1(c) Modulation Transfer Function Curve
Fig. 1(d) Chief Ray Angle at Image Surface Fig.1. EDOF mobile phone lens performance at object distance 30cm, (a) 2D lens layout, (b) MTF curve at normalized field 0.0, 0.2, 0.4, 0.6 0.8 and 1.0, (c) ray aberration curve, (d) chief Ray angle at image surface.
Proc. of SPIE-IS&T/ Vol. 7876 787606-4 Downloaded from SPIE Digital Library on 08 Feb 2011 to 192.17.192.96. Terms of Use: http://spiedl.org/terms
3. MEASUREMENT RESULTS FOR AIE EDOF LENS The designed EDoF lens was manufacture by local optical manufacturer, and then some of optical properties were measured by commercial equipment (ImageMasterHR, TRIOPTICS) [14]. The results of MTF, CRA and distortion are shown in Fig. 2. 1.0 On Axis _ T 0.9
On Axis _ S
0.8
0.77mm_T 0.77mm_S
0.7
1.35mm_T 1.35mm_S
MTF
0.6
1.93mm_T 1.93mm_S
0.5 0.4 0.3 0.2 0.1 0.0 0
10
20
30
40
50
60
70
80
90
100
110
4.5
5.0
Frequency (lp/mm)
Fig. 2(a) MTF of Manufactured EDoF Lens 2.5
Image height (mm)
2.0
1.5
1.0
0.5
0.0 0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
4.0
Optical Distortion (%)
Fig. 2(b) Distortion of Manufactured EDoF Lens 30.00
20.00
CRA (deg.)
10.00
0.00
-10.00
-20.00
-30.00 -2.5
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
2.5
Image height (mm)
Fig. 2(c) CRA of Manufactured EDoF Lens
Fig.2. Measurement results of AIE EDoF mobile phone lens. (a) MTF of Manufactured EDoF Lens, (b) Distortion of manufactured EDoF lens, (c) CRA of manufactured EDoF lens.
Proc. of SPIE-IS&T/ Vol. 7876 787606-5 Downloaded from SPIE Digital Library on 08 Feb 2011 to 192.17.192.96. Terms of Use: http://spiedl.org/terms
In Fig. 2(a), the measured MTF is higher than design result, especially for maximum off-axis condition (for example: image height with 1.92 mm). This cause from extra mechanical vignettting during lens manufacturing. And the extra mechanical vignetting will lead to increase optical distortion in our designed EDoF lens. In Fig. 2(b), the optical distortion is about 3%, which is much higher than design result. In Fig. 2(c), the measured CRA shows the EDoF lens is well satisfied with design requirement which represents our EDoF lens will be suitable for a CMOS image sensor which we used.
4. DEBLUR RESULTS FOR AIE PHASE CODING LENS In this section, we use a method for designing restoration filter with no need of the optical PSF information [8, 9]. The design flow and restoration processing are shown in Figs. 3 and 4 respectively. The imaging system first captures an image of a well-designed test chart. Due to the special-form PSF of the AIE phase coding lens, this image is blurred. The blur image is then corrected by using the perspective transformation. We use both the original image of the test chart and the corrected blur image to calculate a MMSE filter, so that blur image processed by the restoration filter can be very alike to the test chart image.
Fig. 3 Design Flow of MMSE Filter
Fig. 4 Imaging Restoration Processing
Proc. of SPIE-IS&T/ Vol. 7876 787606-6 Downloaded from SPIE Digital Library on 08 Feb 2011 to 192.17.192.96. Terms of Use: http://spiedl.org/terms
The experiment is performed with Micron’s compact camera module development board as shown in Fig. 5. The board equips with a Micron UXGA CMOS image sensor (MT9D112). To restore the special-purpose blur introduced by the AIE phase coding lens, the MMSE filter is used. Three filter masks Wr , Wg and Wb all with dimension 5x5 are designed respectively for the red, greed and blue channels. The processing of the blur image B is as follows 5
5
Iˆc (i, j ) = ∑∑ Bc (i + k − 2, j + l − 2)Wc (k , l ) ,
(4)
k =1 l =1
where Iˆ is the output of the filter and the suffix ( ⋅)c is the color index. Since the masks satisfy the MMSE criterion: 2 5 5 ⎞ ⎪⎫ ⎪⎧⎛ Wc = arg min E ⎨⎜ I c (i, j ) − ∑∑ Bc (i + k − 2, j + l − 2)Wc ( k , l ) ⎟ ⎬ k =1 l =1 ⎠ ⎪⎭ ⎩⎪⎝
(5)
The error between the output Iˆ and the ideal image I can be minimized. In order to show the efficacy of the proposed design in depth of focus extension, two cameras are used. One camera (Camera 1) is with the coding lens as described in Section 2 and 3, while the other one (Camera 2) is with Micron’s kit lens. The scene for experiment is shown in Fig. 6, and the distances of the objects are labeled thereon.
Development Board
Image Restoration Software
80 cm
65 cm
50 cm
25 cm
8 cm Camera 1
Fig.5 Experiment Platform
Camera 2
Fig.6 Demo Scene
In Fig. 7, we show the intermediately blurry image for our designed AIE phase coding lens. The blurry degree is very slight. In Fig. 8, we also show the de-blurred image by means of using MMSE filter kernel. The original image for kit lens in Micron demo board is shown in Fig. 9. It is easy to observe from Fig. 8 and Fig. 9 that the proposed design enlarges the depth of focus without too much loses of image quality at far object distances and wide angle view. Hence, it is able to provide good EDoF performance compared with the conventional counterpart.
Proc. of SPIE-IS&T/ Vol. 7876 787606-7 Downloaded from SPIE Digital Library on 08 Feb 2011 to 192.17.192.96. Terms of Use: http://spiedl.org/terms
Fig.7 Intermediately blurry Image for our Designed AIE EDoF Lens
Fig. 8 De-blurred Image for our Designed AIE EDoF Lens by MMSE Filter Kernel
Fig. 9 Image for Kit Lens in Micron Demo Board
Proc. of SPIE-IS&T/ Vol. 7876 787606-8 Downloaded from SPIE Digital Library on 08 Feb 2011 to 192.17.192.96. Terms of Use: http://spiedl.org/terms
5. CONCLUSIONS In this paper, we present a mobile phone imaging module with extended depth of focus by using axial irradiance equalization phase coding. We introduce the axial irradiance equalization phase coding to design a two-element 2megapixel mobile phone lens for trade off focus-like aberrations such as field curvature, astigmatism and longitudinal chromatic defocus. The design results produce modulation transfer functions and phase transfer functions with substantially similar characteristics at different field and defocus positions within Nyquist pass band. Besides, for the mobile phone camera imaging system with depth of focus extension, we present a digital decoding design method and calculate a minimum mean square error filter. Then, the filter is applied to correct the substantially similar blur image. Last, the blurry degree of intermediately blurry image is very slight and the de-blurred image is without too much loses of image quality at far object distances and wide angle view.
REFERENCES [1] [2] [3] [4] [5]
[6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17]
E. R. Dowski and W. T. Cathey, “Extended depth of field through wave-front coding,” Appl. Opt. 34(11), 18591866 (1995) W. Chi and N. George, “Electronic imaging using a logarithmic,” Optics Letters 26(12), 875-877 (2001) X. Chen, D. Bakin, C. Liu and N. George, “Optics optimization in high-resolution imaging module with extended depth of field,” Proc. SPIE 7061, 706103 (2008). P. Mouroulis, “Depth of field extension with spherical optics,” Optics Express 16(17), pp. 12995-13004 (2008). M. D. Robinson, G. Feng and D. G. Stork, “Spherical coded imagers: Improving lens speed, depth-of-field and manufacturing yield through enhanced spherical aberration and compensating image processing,” Proc. SPIE 7429, 74290M (2009). M. Demenikov and A. R. Harvey, “Image artifacts in hybrid imaging systems with a cubic phase mask,” Optics Express 18(8), 8207-8212 (2010). H. Y. Sung, S. S. Yang and H. Chang, are preparing a manuscript to be called “Focal depth enhancement by means of axial irradiance equalization.” P. C. Chen, C. H. Liu, C. W. Chang, C. C. Chang and L. Angot, “Digital decoding design for phase coded imaging system”, Proc. SPIE 7443, 74431A (2009) P. C. Chen, Y. L. Chen and H. Y. Sung, “Image restoration based on multiple PSF information with applications to phase-coded imaging system,” Proc. SPIE 7798, 779809 (2010) A. V. Arecchi, T. Messadi and R. J. Koshel, [Field Guide to Illumination], SPIE Press (2007). S. Sinzinger and J. Jahns, [Microoptics], Chap. 2, WILEY-VCH (2003). V. N. Mahajan, [Optical Imaging and Aberrations, Part I. Ray Geometrical Optics], Chap. 4, SPIE Press (1998). V. N. Mahajan, [Optical Imaging and Aberrations, Part II. Wave Diffraction Optics], Chap. 2, SPIE Press (2001). Max Born and Emil Wolf, [Principles of Optics], 6th Edition, Chap. 3, Pergamon Press (1980). OSLO Optics Reference Version 6.1, Lambda Research Corporation (2001). OSLO Program Reference Version 6.3, Lambda Research Corporation (2004). Image Master HR User’s Manual, TRITOPICS (2007).
Proc. of SPIE-IS&T/ Vol. 7876 787606-9 Downloaded from SPIE Digital Library on 08 Feb 2011 to 192.17.192.96. Terms of Use: http://spiedl.org/terms
1. INTRODUCTION In recent years, many researchers have been devoted to design and develop extended depth of focus or field (EDoF) imaging systems [1-5]. The EDoF imaging systems posses an advantage capability of implement with fixed lens and no moving parts, hence their sizes can keep compact and small. The way to achieve the goal of EDoF is to use a so-called computational imaging technique. The technique is inherently an integrated design, which employs phase coding in optical design and digital decoding in image signal processor. The phase coding is utilized to substantially increase the depth of focus (DoF) of the imaging system by means of reducing the point spread function (PSF) variance in the range near the focal plane, but the PSF might be in a special form of a big size. Such PSF often results in some image blur. The digital decoding (also referred to image restoration, de-convolution or software lens compensation) is thus applied to restore the blurry image. A typical approach of phase coding is the wave-front coding method proposed by Dowski and Cathy [1]. By introducing an asymmetrically cubic phase mask into lens design, the PSF of the lens will be more insensitive to defocus compared with the traditional lens. However, for the asymmetrically cubic phase coding, the phase transfer function is not substantially invariant in the EDoF focal region and therefore the artifact of restored image is inevitable by using only a single restoration filter [6]. On the contrary, a special symmetric surface form, designed by George and Chi [2], increase in the depth of field was achieved by using a logarithmic asphere. The logarithmic asphere is a circularly symmetric surface and it is designed by equaling pupil area and applying Fermat’s principle. The logarithmic asphere have been applied to the mobile phone lens design [3] with EDoF range from 300 mm to infinity and the extension factor in the range of 2 to 3 is achieved. More recently, Mouroulis [4] proposed the idea that spherical aberration is applied to a low-power microscope by means of equalizing the MTF across different focus positions in order to achieve the extended depth of field. Afterwards, Robinson [5] uses the spherical coding further to increase lens speed and improve manufacturing yield. *[email protected]; phone 886-3-5731170; fax 886-3-5751113
Digital Photography VII, edited by Francisco H. Imai, Feng Xiao, Jeffrey M. DiCarlo, Nitin Sampat, Sebastiano Battiato, Proc. of SPIE-IS&T Electronic Imaging, SPIE Vol. 7876, 787606 © 2011 SPIE-IS&T · CCC code: 0277-786X/11/$18 · doi: 10.1117/12.872260 Proc. of SPIE-IS&T/ Vol. 7876 787606-1 Downloaded from SPIE Digital Library on 08 Feb 2011 to 192.17.192.96. Terms of Use: http://spiedl.org/terms
In this paper, we propose a mobile phone imaging module with extended depth of focus by using AIE phase coding and MMSE digital decoding. In section 2, we introduce the AIE phase coding and apply the phase coding to design a twoelement 2-megapixel mobile phone lens for trade off focus-like aberrations such as field curvature, astigmatism and longitudinal chromatic defocus. Section 3 show the measure results of AIE EDoF lens and make a comparison with the design results. In section 4, we give de-blurred images by means of using MMSE digital decoding. Section 5 concludes the paper.
2. EDOF MOBILE PHONE LENS DESIGN USING AIE PHASE CODING In the section, we design an EDoF mobile phone lens by means of using AIE phase coding. First, let us consider a circularly symmetric multi-focal lens from exit pupil. In the multi-focal lens, f0 is the paraxial focal length, Rmax is the maximum exit pupil radius, Δz is the fully desired depth of focus with distances measured along the Z axis as positive to the right and negative to the left of the paraxial focal plane. Besides, in most imaging systems, the exit pupil is round and the irradiance is the same as the irradiance from a uniform Lambertian disk [10]. On this condition, from radiation energy transfer along optical axis with constant irradiance, the multi-focal lens f(r) is described by [7]
f (r ) =
f0 3 −
3 3
3
2
3[2 f 0 + 27Cr 2 + 18 f 0 r 2 + 4(3r 2 − f 0 ) 3 + (2 f 0 + 27Cr 2 + 18 f 0 r 2 ) 2 ]1/ 3 3
+
2
2 (3r 2 − f 0 ) 2 3
3
.
(1)
[2 f 0 + 27Cr + 18 f 0 r + 4(3r − f 0 ) + (2 f 0 + 27Cr + 18 f 0 r ) ] 2
2
2
2
2 2 1/ 3
3(3 2 )
Where C is defined by
C≡
Δz[( Rmax ) 2 + ( f 0 + Δz ) 2 ] . ( Rmax ) 2
(2)
Due to no any paraxial approximation, the Eq. (1) is not only suitable for small aperture and small depth of focus but also suitable for large aperture and large depth of focus. Next, we will use the Eq. (1) to design a two-element 2-megapixel mobile phone lens in order to alleviate focus-like aberrations such as field curvature, astigmatism and longitudinal chromatic defocus and reduce the modulation transfer function (MTF) variance in the range near the focal plane. Detail design specifications of the EDoF mobile phone lens are listed in Table 1. The EDoF mobile lens consists of only two E48R plastic elements and is f = 3.4 mm, f/# = 3.0, 66 degree field of view, polychromatic wavelength band from 0.470 μm to 0.643 μm and default object distance at 300 mm. Because the field of view is so large, the major defocus aberration is field curvature. In order to minimize the field curvature, firstly we choice Petzval lens type as paraxial layout design. However, it is not sufficient to achieve to 2-megapixel mobile phone lens performance and the field curvature is still too large. Thus, the large depth of focus is required in order to alleviate the effect of field curvature. Secondly we introduce the AIE phase coding to enlarge the depth of focus. In this case, we choice the desired depth of focus Δz = -0.2mm to against focus-like aberrations. Next, according to the condition f0 = 3.4 mm, Rmax = 0.566 mm (f0/ 2Rmax = F/# = 3.0), Δz = -0.2 mm and Eq. (1), we can acquire the desired AIE multi-focal lens and describe the multifocal lens in spherical wave-front aberration representations [11-14]. The AIE phase coding W(ρ) is written by:
W ( ρ ) = (2.3013λ ) ρ 4 + (0.2062λ ) ρ 6 + (0.0238λ ) ρ 8 + (0.0074λ ) ρ 10 ,
(3)
Where wavelength unit λ is 0.55 μm, normalized radius ρ is from 0 to 1 and the spherical wave-front aberration expansions are accurate to ±0.0001λ.
Proc. of SPIE-IS&T/ Vol. 7876 787606-2 Downloaded from SPIE Digital Library on 08 Feb 2011 to 192.17.192.96. Terms of Use: http://spiedl.org/terms
Table1. The specifications for EDoF mobile phone lens
Parameter Effective Focal Length F/Number Design Wavelength FOV Maximum Chief Ray Angle Optical Distortion Relative Illumination Lens Construction CMOS Sensor Resolution Pixel Size
Value 3.4 mm 3.0 0.47μm ~ 0.643μm ±33° 22.5°±1.5° < 2% > 50% 2P ; [+|-] 1600*1200 Bayer Pattern 2.2μm
In the reminder of this section, we applied the AIE spherical wave-front aberration Eq. (3) to design the 2-megapixel EDoF mobile phone lens. The design degrees of freedom include the central aperture stop location, the back focal distance, the lens thickness, curvatures and standard aspheric coefficients of r4, r6, r8, r10. And further, by adding a penalty function on the spherical wave-front aberration coefficient of Eq. (3) to a traditional root mean square wave-front error merit function [15,16], we can minimize the other spatially varying optical aberrations such as field curvature, astigmatism, and even longitudinal chromatic aberration at the expense of AIE phase coding. The final design results of EDoF mobile lens at object distance 30cm are shown in Fig. 1, including (a) 2D lens layout, (b) ray aberration curve, (c) modulation transfer function (MTF) curve at normalized field 0.0, 0.2, 0.4, 0.6, 0.8, and 1.0, and (d) chief ray angle (CRA) at image surface. Excluding MTF at field 1.0, the other MTF are substantially invariant and no PTF reversal issue below 112 lp/mm. It is suitable for using MMSE digital decoding in Section 4. Besides, from viewpoint of aberration theory, AIE phase coding is just an especially longitudinal spherical aberration. The fully depth of focus Δz is also the distance of longitudinal spherical aberration from paraxial focus to marginal focus. Thus, in order to make EDoF effectively, not only the distance of longitudinal spherical aberration must be larger than the sag distance of field curvature but also the preferred focal plane is about at the middle point between paraxial and marginal focal point. The Fig. 1 (b) ray aberration curve show these results under the AIE phase coding condition.
Fig. 1(a) 2D Lens Layout
Proc. of SPIE-IS&T/ Vol. 7876 787606-3 Downloaded from SPIE Digital Library on 08 Feb 2011 to 192.17.192.96. Terms of Use: http://spiedl.org/terms
Fig. 1(b) Ray Aberration Curve
Fig. 1(c) Modulation Transfer Function Curve
Fig. 1(d) Chief Ray Angle at Image Surface Fig.1. EDOF mobile phone lens performance at object distance 30cm, (a) 2D lens layout, (b) MTF curve at normalized field 0.0, 0.2, 0.4, 0.6 0.8 and 1.0, (c) ray aberration curve, (d) chief Ray angle at image surface.
Proc. of SPIE-IS&T/ Vol. 7876 787606-4 Downloaded from SPIE Digital Library on 08 Feb 2011 to 192.17.192.96. Terms of Use: http://spiedl.org/terms
3. MEASUREMENT RESULTS FOR AIE EDOF LENS The designed EDoF lens was manufacture by local optical manufacturer, and then some of optical properties were measured by commercial equipment (ImageMasterHR, TRIOPTICS) [14]. The results of MTF, CRA and distortion are shown in Fig. 2. 1.0 On Axis _ T 0.9
On Axis _ S
0.8
0.77mm_T 0.77mm_S
0.7
1.35mm_T 1.35mm_S
MTF
0.6
1.93mm_T 1.93mm_S
0.5 0.4 0.3 0.2 0.1 0.0 0
10
20
30
40
50
60
70
80
90
100
110
4.5
5.0
Frequency (lp/mm)
Fig. 2(a) MTF of Manufactured EDoF Lens 2.5
Image height (mm)
2.0
1.5
1.0
0.5
0.0 0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
4.0
Optical Distortion (%)
Fig. 2(b) Distortion of Manufactured EDoF Lens 30.00
20.00
CRA (deg.)
10.00
0.00
-10.00
-20.00
-30.00 -2.5
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
2.5
Image height (mm)
Fig. 2(c) CRA of Manufactured EDoF Lens
Fig.2. Measurement results of AIE EDoF mobile phone lens. (a) MTF of Manufactured EDoF Lens, (b) Distortion of manufactured EDoF lens, (c) CRA of manufactured EDoF lens.
Proc. of SPIE-IS&T/ Vol. 7876 787606-5 Downloaded from SPIE Digital Library on 08 Feb 2011 to 192.17.192.96. Terms of Use: http://spiedl.org/terms
In Fig. 2(a), the measured MTF is higher than design result, especially for maximum off-axis condition (for example: image height with 1.92 mm). This cause from extra mechanical vignettting during lens manufacturing. And the extra mechanical vignetting will lead to increase optical distortion in our designed EDoF lens. In Fig. 2(b), the optical distortion is about 3%, which is much higher than design result. In Fig. 2(c), the measured CRA shows the EDoF lens is well satisfied with design requirement which represents our EDoF lens will be suitable for a CMOS image sensor which we used.
4. DEBLUR RESULTS FOR AIE PHASE CODING LENS In this section, we use a method for designing restoration filter with no need of the optical PSF information [8, 9]. The design flow and restoration processing are shown in Figs. 3 and 4 respectively. The imaging system first captures an image of a well-designed test chart. Due to the special-form PSF of the AIE phase coding lens, this image is blurred. The blur image is then corrected by using the perspective transformation. We use both the original image of the test chart and the corrected blur image to calculate a MMSE filter, so that blur image processed by the restoration filter can be very alike to the test chart image.
Fig. 3 Design Flow of MMSE Filter
Fig. 4 Imaging Restoration Processing
Proc. of SPIE-IS&T/ Vol. 7876 787606-6 Downloaded from SPIE Digital Library on 08 Feb 2011 to 192.17.192.96. Terms of Use: http://spiedl.org/terms
The experiment is performed with Micron’s compact camera module development board as shown in Fig. 5. The board equips with a Micron UXGA CMOS image sensor (MT9D112). To restore the special-purpose blur introduced by the AIE phase coding lens, the MMSE filter is used. Three filter masks Wr , Wg and Wb all with dimension 5x5 are designed respectively for the red, greed and blue channels. The processing of the blur image B is as follows 5
5
Iˆc (i, j ) = ∑∑ Bc (i + k − 2, j + l − 2)Wc (k , l ) ,
(4)
k =1 l =1
where Iˆ is the output of the filter and the suffix ( ⋅)c is the color index. Since the masks satisfy the MMSE criterion: 2 5 5 ⎞ ⎪⎫ ⎪⎧⎛ Wc = arg min E ⎨⎜ I c (i, j ) − ∑∑ Bc (i + k − 2, j + l − 2)Wc ( k , l ) ⎟ ⎬ k =1 l =1 ⎠ ⎪⎭ ⎩⎪⎝
(5)
The error between the output Iˆ and the ideal image I can be minimized. In order to show the efficacy of the proposed design in depth of focus extension, two cameras are used. One camera (Camera 1) is with the coding lens as described in Section 2 and 3, while the other one (Camera 2) is with Micron’s kit lens. The scene for experiment is shown in Fig. 6, and the distances of the objects are labeled thereon.
Development Board
Image Restoration Software
80 cm
65 cm
50 cm
25 cm
8 cm Camera 1
Fig.5 Experiment Platform
Camera 2
Fig.6 Demo Scene
In Fig. 7, we show the intermediately blurry image for our designed AIE phase coding lens. The blurry degree is very slight. In Fig. 8, we also show the de-blurred image by means of using MMSE filter kernel. The original image for kit lens in Micron demo board is shown in Fig. 9. It is easy to observe from Fig. 8 and Fig. 9 that the proposed design enlarges the depth of focus without too much loses of image quality at far object distances and wide angle view. Hence, it is able to provide good EDoF performance compared with the conventional counterpart.
Proc. of SPIE-IS&T/ Vol. 7876 787606-7 Downloaded from SPIE Digital Library on 08 Feb 2011 to 192.17.192.96. Terms of Use: http://spiedl.org/terms
Fig.7 Intermediately blurry Image for our Designed AIE EDoF Lens
Fig. 8 De-blurred Image for our Designed AIE EDoF Lens by MMSE Filter Kernel
Fig. 9 Image for Kit Lens in Micron Demo Board
Proc. of SPIE-IS&T/ Vol. 7876 787606-8 Downloaded from SPIE Digital Library on 08 Feb 2011 to 192.17.192.96. Terms of Use: http://spiedl.org/terms
5. CONCLUSIONS In this paper, we present a mobile phone imaging module with extended depth of focus by using axial irradiance equalization phase coding. We introduce the axial irradiance equalization phase coding to design a two-element 2megapixel mobile phone lens for trade off focus-like aberrations such as field curvature, astigmatism and longitudinal chromatic defocus. The design results produce modulation transfer functions and phase transfer functions with substantially similar characteristics at different field and defocus positions within Nyquist pass band. Besides, for the mobile phone camera imaging system with depth of focus extension, we present a digital decoding design method and calculate a minimum mean square error filter. Then, the filter is applied to correct the substantially similar blur image. Last, the blurry degree of intermediately blurry image is very slight and the de-blurred image is without too much loses of image quality at far object distances and wide angle view.
REFERENCES [1] [2] [3] [4] [5]
[6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17]
E. R. Dowski and W. T. Cathey, “Extended depth of field through wave-front coding,” Appl. Opt. 34(11), 18591866 (1995) W. Chi and N. George, “Electronic imaging using a logarithmic,” Optics Letters 26(12), 875-877 (2001) X. Chen, D. Bakin, C. Liu and N. George, “Optics optimization in high-resolution imaging module with extended depth of field,” Proc. SPIE 7061, 706103 (2008). P. Mouroulis, “Depth of field extension with spherical optics,” Optics Express 16(17), pp. 12995-13004 (2008). M. D. Robinson, G. Feng and D. G. Stork, “Spherical coded imagers: Improving lens speed, depth-of-field and manufacturing yield through enhanced spherical aberration and compensating image processing,” Proc. SPIE 7429, 74290M (2009). M. Demenikov and A. R. Harvey, “Image artifacts in hybrid imaging systems with a cubic phase mask,” Optics Express 18(8), 8207-8212 (2010). H. Y. Sung, S. S. Yang and H. Chang, are preparing a manuscript to be called “Focal depth enhancement by means of axial irradiance equalization.” P. C. Chen, C. H. Liu, C. W. Chang, C. C. Chang and L. Angot, “Digital decoding design for phase coded imaging system”, Proc. SPIE 7443, 74431A (2009) P. C. Chen, Y. L. Chen and H. Y. Sung, “Image restoration based on multiple PSF information with applications to phase-coded imaging system,” Proc. SPIE 7798, 779809 (2010) A. V. Arecchi, T. Messadi and R. J. Koshel, [Field Guide to Illumination], SPIE Press (2007). S. Sinzinger and J. Jahns, [Microoptics], Chap. 2, WILEY-VCH (2003). V. N. Mahajan, [Optical Imaging and Aberrations, Part I. Ray Geometrical Optics], Chap. 4, SPIE Press (1998). V. N. Mahajan, [Optical Imaging and Aberrations, Part II. Wave Diffraction Optics], Chap. 2, SPIE Press (2001). Max Born and Emil Wolf, [Principles of Optics], 6th Edition, Chap. 3, Pergamon Press (1980). OSLO Optics Reference Version 6.1, Lambda Research Corporation (2001). OSLO Program Reference Version 6.3, Lambda Research Corporation (2004). Image Master HR User’s Manual, TRITOPICS (2007).
Proc. of SPIE-IS&T/ Vol. 7876 787606-9 Downloaded from SPIE Digital Library on 08 Feb 2011 to 192.17.192.96. Terms of Use: http://spiedl.org/terms
