sensors Article
Insect-Mimetic Imaging System Based on a Microlens Array Fabricated by a Patterned-Layer Integrating Soft Lithography Process Minwon Seo 1 , Jong-Mo Seo 1,2, *, Dong-il “Dan” Cho 1 and Kyoin Koo 3, * 1 2 3
*
ID
Department of Electrical and Computer Engineering, Seoul National University, Seoul 08826, Korea;
[email protected] (M.S.);
[email protected] (D.C.) Biomedical Research Institute, Seoul National University Hospital, Seoul 03080, Korea Department of Biomedical Engineering, University of Ulsan, Ulsan 44610, Korea Correspondences:
[email protected] (J.-M.S.);
[email protected] (K.K.); Tel.: +82-2-880-1739 (J.-M.S.)
Received: 16 May 2018; Accepted: 21 June 2018; Published: 22 June 2018
Abstract: In nature, arthropods have evolved to utilize a multiaperture vision system with a micro-optical structure which has advantages, such as compact size and wide-angle view, compared to that of a single-aperture vision system. In this paper, we present a multiaperture imaging system using a microlens array fabricated by a patterned-layer integrating soft lithography (PLISL) process which is based on a molding technique that can transfer three-dimensional structures and a gold screening layer simultaneously. The imaging system consists of a microlens array, a lens-adjusting jig, and a conventional (charge-coupled device) CCD image sensor. The microlens array has a light screening layer patterned among all the microlenses by the PLISL process to prevent light interference. The three-dimensionally printed jig adjusts the microlens array on the conventional CCD sensor for the focused image. The manufactured imaging system has a thin optic system and a large field-of-view of 100 degrees. The developed imaging system takes multiple images at once. To show its possible applications, multiple depth plane images were reconstructed based on the taken subimages with a single shot. Keywords: microlens array; insect-mimetic; multiaperture vision system
1. Introduction Most conventional imaging systems adopt a single aperture structure inspired by vertebrates’ vision system. Stacking lenses along the optical axis enables this imaging system to achieve a wide-angle view, a telescopic view, and reduced aberrations. However, the single-aperture vision system has disadvantages, such as the limitation of miniaturization and low time resolution [1]. To overcome these disadvantages, efforts have been made to mimic insects’ vision system. Figure 1 shows a schematic illustration of anatomical structures of the insect compound eye. There are two types of compound eye: an apposition compound eye and a superposition compound eye [2]. The apposition compound eye receives light through hundreds to thousands of facets. Every facet has a pigment to prevent light from entering neighborhood facets. This enables a distinct point pixel instead of a cross-talked white Gaussian noised pixel [3,4]. Oppositely, insects with a superposition compound eye, which are those insects that usually move around at nighttime, do not have the pigment unit among the facets. This pigmentless structure causes light interference on photoreceptors and blurs object images. However, superposed light in the pigmentless facets enables the nocturnal insect to react more sensitively to the nighttime environment [5,6].
Sensors 2018, 18, 2011; doi:10.3390/s18072011
www.mdpi.com/journal/sensors
Sensors 2018, 18, 2011
2 of 9
Sensors 2018, 18, x FOR PEER REVIEW
2 of 9
(a)
(b)
Figure 1. The schematic diagram of (a) the apposition compound eye and (b) the superposition
Figure 1. The schematic diagram of (a) the apposition compound eye and (b) the superposition compound eye. compound eye.
To mimic the structure of the compound eye, a microlens array has been widely used [7–10]. To fabricate microlens array, a thermal refloweye, process is usuallyarray employed because of its reliable To mimicthethe structure of the compound a microlens has been widely used [7–10]. spherical profile as well as its simple fabrication process. To fabricate the microlens array, a thermal reflow process is usually employed because of its reliable several works mimicking the compound eye of Xenos peckii, which envisions spherical There profileare as well as its simple fabrication process. subimages of the object with its clustered eyes [11–13]. To perceive high-quality subimages from the There are several works mimicking the compound eye of Xenos peckii, which envisions subimages compound eye, a light screening layer among the microlenses is indispensable. Morgan et al. [14] of theimplemented object with an its clustered eyes [11–13]. To perceive high-quality subimages from the compound opaque wall to change the color of glasses among microlenses using thermal eye, atreatment. light screening layer among the microlenses is indispensable. Morgan et al. [14] implemented Ichioka et al. [15] inserted optical separators between an image sensor and a microlens an opaque wallmethod to change the color of glasses microlenses treatment. array. This required aligning process among of the optical separatorusing to thethermal microlenses. Kim et Ichioka al. et al.[16] [15]made inserted optical separators between an image andmicrolens a microlens This method a perforated black polymer sheet and coveredsensor a curved arrayarray. with the black sheetaligning to block process the interference of light.separator The abovementioned techniques used a microlens required of the optical to the microlenses. Kim et to al.achieve [16] made a perforated array with a screening layer require manual alignment or high thermal treatment. black polymer sheet and covered a curved microlens array with the black sheet to block the interference In this paper, we presented a compound imaging array systemwith usinga ascreening microlens layer of light. The abovementioned techniques usedeye to mimicking achieve a an microlens array integrated with a gold screening layer fabricated by a patterned-layer integrating soft require manual alignment or high thermal treatment. lithography (PLISL) process. The microlens array integrated with the light screening pattern was In this paper, we presented a compound eye mimicking an imaging system using a microlens adjusted on a conventional charge-coupled device (CCD) image sensor by a lens-adjusting jig. The arraycaptured integrated with a gold screening layer fabricated by a patterned-layer integrating soft lithography image was refocused in three different planes to demonstrate depth perception capability. (PLISL) process. The microlens arraytheintegrated with thePLISL light process, screening pattern was Moreover, in order to demonstrate extensibility of our a microlens arrayadjusted with a on a conventional charge-coupled device (CCD) image sensor by a lens-adjusting jig. The captured image gold screening layer was fabricated on a spherical surface as well.
was refocused in three different planes to demonstrate depth perception capability. Moreover, in order 2. Materials the andextensibility Methods to demonstrate of our PLISL process, a microlens array with a gold screening layer was fabricated on a spherical surface as well. 2.1. Fabrication of a Microlens Array with a Gold Screening Layer
2. Materials and to Methods In order determine the focal length of the microlens, the thickness of the glass protecting the image sensor should be considered. The thickness of the protecting glass, as measured by a caliper,
2.1. Fabrication of aItMicrolens Array with Goldhas Screening Layer was 0.44 mm. was assumed that the aglass a refractive index similar to N-BK7 glass, which has a refractive index of 1.517. Therefore, the focal length of the microlens should be above 0.67 mm. In order to determine the focal length of the microlens, the thickness of the glass protecting the Since the thermal reflow process enables a cylindrical photoresist structure to transform into a image sensor should be considered. The thickness of the protecting glass, as measured by a caliper, microlens structure due to the property of minimizing surface energy [17], the height (H) and was 0.44 mm. It was assumed that the glass has a refractive index similar to N-BK7 glass, which has diameter (2r) of the cylindrical structure decides the curvature of the microlens. The height of a a refractive ofμm 1.517. focal length ofresin the microlens be above 0.67ofmm. cylinderindex with 10 andTherefore, a refractive the index of UV curing is 1.56 [18].should Therefore, the height the Since the thermal reflow process enables a cylindrical photoresist structure to transform microlens can be calculated by Equations (1) and (2). The radius of the microlens should be above 118 into a microlens structure to the property of minimizing energy [17], and diameter μm. In this paper,due the radius of the microlens array wassurface set to 175 μm for thethe thinheight optical(H) system and stable focus adjustability (Seedecides Supplementary Information). (2r) of the cylindrical structure the curvature of the microlens. The height of a cylinder with
10 µm and a refractive index of UV curing resin is ℎ1.56 [18]. Therefore, the height of the microlens can (1) π = (3 + ℎ ) be calculated by Equations (1) and (2). The radius 6 of the microlens should be above 118 µm. In this paper, the radius of the microlens array was set to 175 µm for the thin optical system and stable focus adjustability (See Supplementary Information). πr2 H =
πh 2 3r + h2 6
(1)
Sensors 2018, 18, 2011
3 of 9
Sensors 2018, 18, x FOR PEER REVIEW
h=
√ 3
3 of 9
r2
X− √ 3
X h = √ − √
f or X =
p
9H 2 r4 + r6 + 3Hr2
= 9
+
+3
(2) (2)
r 2 + h2 f= 2h(n +− ℎ 1)
(3)
f =
(3)
2ℎ( to − 1) The soft lithography process is widely used fabricate three-dimensional microstructures. The soft lithography process is widely to fabricate three-dimensional microstructures. Even though the soft lithography process hasused several advantages, including repeatability and Even simplicity, though process has several advantages, including repeatability and simplicity, it it is not easythe tosoft addlithography another patterned layer on the surface of the soft-lithography-processed polymer. is not easy to add another patterned layer on the surface of the soft-lithography-processed polymer. In this investigation, to fabricate a screening layer among the microlenses, a PLISL process was In this investigation, to fabricate a screening layer among the microlenses, a PLISL process was proposed, as shown in Figure 2. Basically, the proposed process was an advanced version of the soft proposed, as shown in Figure 2. Basically, the proposed process was an advanced version of the soft lithography process. A gold film which has weak adhesion to a silicon wafer [19] was used as a light lithography process. A gold film which has weak adhesion to a silicon wafer [19] was used as a light screening material. It was patterned on a silicon substrate without any adhesion-promoting layer screening material. It was patterned on a silicon substrate without any adhesion-promoting layer like like titanium orchrome. chrome. gold prepatterned substrate, a three-dimensional microstructure titanium or OnOn thethe gold prepatterned substrate, a three-dimensional microstructure was was implemented andtransferred transferred to the polymer material by molding. the molding process, implemented and to the polymer material by molding. DuringDuring the molding process, the the prepatterned film was wastransferred transferredand and integrated onto polymer because itsadhesion low adhesion prepatterned gold gold film integrated onto thethe polymer because of itsof low the silicon wafer. to thetosilicon wafer.
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
(i)
(j)
(k) Figure 2. The specific steps of the PLISL process. (a) The gold layer was deposited on the silicon; (b)
Figure 2. The specific steps of the PLISL process. (a) The gold layer was deposited on the silicon; positive PR patterning; (c) gold layer etching; (d) PR removal; (e) negative PR patterning; (f) thermal (b) positive PR patterning; (c) gold layer etching; (d) PR removal; (e) negative PR patterning; (f) thermal reflow; (g) 1st molding with the elastomer; (h) the 1st molded layer; (i) 2nd molding with the UV reflow; (g)(j) 1stthe molding withlayer; the elastomer; (h) the microlens 1st molded layer; 2ndlayer. molding with the UV resin; resin; 2nd molded (k) the fabricated array with(i)gold (j) the 2nd molded layer; (k) the fabricated microlens array with gold layer. The specific steps of the PLISL process are as follows: A silicon wafer was cleaned in a mixture of H 2SO4:H2O2 (3:1) for 30 min at 60 degrees Celsius. After that, a 120-nm thick gold layer [20], which The specific steps of the PLISL process are as follows: A silicon wafer was cleaned in a mixture would serve as the light interference screening layer, was deposited on the cleaned wafer by an Eof H2 SO4 :H2 O2 (3:1) for 30 min at 60 degrees Celsius. After that, a 120-nm thick gold layer [20], gun evaporator (ZZS550-2/D, Maestech Co., Ltd., Pyungtaek, Korea) without any adhesion layer like which would serve as the light interference screening layer, was deposited on the cleaned wafer by titanium or chrome (Figure 2a). A positive photoresist, SS-03A9 (Dongwoo Fine-chem Co., Ltd., an E-gun evaporator (ZZS550-2/D, Maestech Co.,evaporated Ltd., Pyungtaek, Korea) without anyphotoresist adhesion layer Gyeonggi-do, Korea), was patterned on the gold silicon wafer (Figure 2b). The like titanium or chrome (Figure 2a). A positive photoresist, SS-03A9 (Dongwoo Fine-chem patterned wafer was etched by gold etchant, which was a mixture of I2:KI:H2O (1:4:10) (Figure Co., 2c). Ltd., Gyeonggi-do, Korea), was patterned on the gold evaporated silicon wafer (Figure 2b). The photoresist After removing the photoresist on the wafer with acetone, the gold-patterned wafer was coated by patterned wafer was etched gold etchant, which was aIncheon, mixtureKorea), of I2 :KI:H negative photoresist, DNRby (Dongjin Semichem Co., Ltd., about2 O 10(1:4:10) μm by (Figure a spin 2c). and baked on a hot plateon at 95 The second layer was coated the by Aftercoater removing the photoresist thedegrees wafer Celsius. with acetone, the photoresist gold-patterned wafer wasoncoated first photoresist layer and baked on the hot plate at 110 degrees Celsius. The wafer was exposed using negative photoresist, DNR (Dongjin Semichem Co., Ltd., Incheon, Korea), about 10 µm by a spin coater
and baked on a hot plate at 95 degrees Celsius. The second photoresist layer was coated on the first photoresist layer and baked on the hot plate at 110 degrees Celsius. The wafer was exposed using the identical mask with the gold-layer patterning mask to pattern on the gold-free area (Figure 2e).
Sensors 2018, 18, 2011
4 of 9
Sensors 2018, 18, x FOR PEER REVIEW
4 of 9
After the UV exposed wafer was developed, the developed wafer was baked on a hot plate to melt the the identical mask the gold-layer patterning mask to pattern on the gold-free photoresist pattern forwith the spherical structure by the surface tension (Figure 2f). area This(Figure heating2e). step is After the UV exposed wafer was developed, the developed wafer was baked on a hot plate to melt widely known as the thermal reflow process. the photoresist pattern for the spherical structure by the surface tension (Figure 2f). This heating step After the silicon substrate steps, the microlens array pattern and the gold screening layer was is widely known as the thermal reflow process. transferred using polydimethylsiloxane (PDMS, Sylgard 184, Dow Corning, Scottsdale, AZ, USA) After the silicon substrate steps, the microlens array pattern and the gold screening layer was (Figure 2h). To get a convex microlens array with Sylgard a screening a UV-curable epoxy resin (NOA transferred using polydimethylsiloxane (PDMS, 184, layer, Dow Corning, Scottsdale, AZ, USA) 72, Norland Products Inc., Cranbury, NJ, USA) with a refractive index 1.56 was poured on (Figure 2h). To get a convex microlens array with a screening layer, a UV-curable epoxy resin (NOA the concave-patterned PDMSInc., mold. A transparent film was placed 72, Norland Products Cranbury, NJ, USA)polyethylene-terephthalate with a refractive index 1.56 (PET) was poured on the concave-patterned PDMSfor mold. A transparent polyethylene-terephthalate (PET) film was placed on light on the poured epoxy resin flattening and easy handling. The resin was cured by 15 W UV poured epoxy for (Figure flattening2i). and easy handling. The resin resin was cured 15 W UV light witharray with athe wavelength of resin 365 nm The cured and stripped had by a convex microlens a wavelength of 365 nm (Figure 2i). The cured and stripped resin had a convex microlens array integrated with the gold layer, as shown in Figure 2k. Finally, a thin parylene was deposited on the integrated with the gold layer, as shown in Figure 2k. Finally, a thin parylene was deposited on the convex microlens array and the gold screening layer for electrical isolation from the CCD sensor. convex microlens array and the gold screening layer for electrical isolation from the CCD sensor.
2.2. Imaging System Setup Using a 3D Printed Jig
2.2. Imaging System Setup Using a 3D Printed Jig
Figure 3 shows a schematic of the proposed imaging system. It consists of a microlens array, Figure 3 shows a schematic of the proposed imaging system. It consists of a microlens array, a a focus-adjustable jig,jig, and a 6.72-mm CCD P5756,Hanjindata, Hanjindata,GyeonggiGyeonggi-do, focus-adjustable and a 6.72-mm CCDsensor sensor(1920 (1920×× 1080 1080 pixels, pixels, P5756, Korea). jig was designed and three-dimensionally printed for andfocusing focusing of do,The Korea). The jig was designed and three-dimensionally printed forsecure secure positioning positioning and the microlens array.array. The aperture sizesize of the jigjig was Screwsand andsprings springs were used to adjust of the microlens The aperture of the was44mm. mm. Screws were used to adjust a four-point focalfocal distance between the andthe theCCD CCDsensor. sensor. a four-point distance between themicrolens microlensarray array and
(a)
(b)
Figure 3. The schematic diagram of the imaging system setup. (a) The convex part of the microlens
Figure 3. The schematic diagram of the imaging system setup. (a) The convex part of the microlens array looks outside. (b) The convex part of the microlens array faces the CCD sensor. array looks outside. (b) The convex part of the microlens array faces the CCD sensor.
There are two possible directions when integrating the microlens array with the sensor. One is that a convex of the microlens looks outside (Figure 3a). The other is with that the part There are twopart possible directionsarray when integrating the microlens array theconvex sensor. One is the microlens faces the CCD 3b). In the 3a). case The of the microlens facing the that aofconvex part ofarray the microlens arraysensor looks(Figure outside (Figure other is thatarray the convex part of CCD sensor, thefaces goldthe screening layer (Figure is closer3b). to the CCDcase sensor compared with the facing case ofthe theCCD the microlens array CCD sensor In the of the microlens array microlens array looking outside. This closer distance is expected to reduce the interference of the sensor, the gold screening layer is closer to the CCD sensor compared with the case of the microlens entered light and to enhance imaging quality. array looking outside. This closer distance is expected to reduce the interference of the entered light and to 3. enhance Results imaging quality.
3. Results 3.1. Characteristics of the Microlens Array A cylindrical photoresist 3.1. Characteristics of the Microlensstructure Array with a height of 10 μm and a diameter of 350 μm was thermalreflowed. The microlens mold with a curvature of 778.9 μm (rsph was 175 μm and hsph was 19.91 μm)
A cylindrical a height and a implemented diameter ofusing 350 µm was fabricated, photoresist as shown in structure Figure 4a. with The focal lengthof of 10 theµm microlens this was thermal-reflowed. The microlens with a among curvature 778.9 µm µmμm. and hsph thermal-reflowed mold was 1.442mold mm. Spacing everyofmicrolens was(rfabricated as 500 sph was 175 4b shows the concave microlens array4a. andThe the gold thatofwas in PDMS on a was 19.91Figure µm) was fabricated, as shown in Figure focallayer length themolded microlens implemented cover slip glass substrate. By using the UV-imprinting process, the concave pattern was molded in using this thermal-reflowed mold was 1.442 mm. Spacing among every microlens was fabricated NOA 72 on PET film (Figure 4c). as 500 µm. One of the advantages of our PLISL process is that three-dimensional microstructure and Figure 4b shows the concave microlens array and the gold layer that was molded in PDMS on integrated layer is transferred not only on a planar substrate but also on a hemispherical substrate a cover slip glass substrate. By using the UV-imprinting process, the concave pattern was molded in using the widely known negative pressure technique (Figure 5a). Due to the negative pressure, the NOA 72 on PET film (Figure 4c). One of the advantages of our PLISL process is that three-dimensional microstructure and integrated layer is transferred not only on a planar substrate but also on a hemispherical substrate using the widely known negative pressure technique (Figure 5a). Due to the negative pressure, the flexible
Sensors 2018, 18, 2011
5 of 9
Sensors 2018, 18, x FOR PEER REVIEW
5 of 9
Sensors 2018, 18, x FOR PEER REVIEW
5 of 9
PDMS moldPDMS is deformed the hemispherical structure. FigureFigure 5b shows a microlens array on the flexible mold is as deformed as the hemispherical structure. 5b shows a microlens array hemispherical substrate using a three-dimensionally printed chamber. flexible PDMS issubstrate deformed as the hemispherical structure. Figure 5b shows a microlens array Sensors 18, xmold FOR PEER REVIEWusing 5 of 9 on the 2018, hemispherical a three-dimensionally printed chamber. on the hemispherical substrate using a three-dimensionally printed chamber. flexible PDMS mold is deformed as the hemispherical structure. Figure 5b shows a microlens array on the hemispherical substrate using a three-dimensionally printed chamber.
(a) (b) (c) (a) (b) (c) Figure 4. The fabricated microlens array (a) implemented using thermal reflow process, (b) firstly Figure 4. The fabricated microlens array (a) implemented using thermal reflow process, (b) firstly Figure The PDMS fabricated microlens array (a)(c)implemented usingwith thermal reflowresin process, (b) film. firstly molded4.with on glass substrate and secondly molded UV curing on PET (b)molded with UV curing resin(c) molded with PDMS(a) on glass substrate and (c) secondly on PET film. molded with PDMS on glass substrate and (c) secondly molded with UV curing resin on PET film. Figure 4. The fabricated microlens array (a) implemented using thermal reflow process, (b) firstly molded with PDMS on glass substrate and (c) secondly molded with UV curing resin on PET film.
(a) (a)
(b) (b)
Figure 5. (a) The fabrication schematic for the hemispherical substrate. (b) The fabricated microlens Figure 5. (a)the The fabrication schematic for the hemispherical array with screening layer on the hemispherical substrate.substrate. (b) The fabricated microlens Figure 5. (a) The fabrication (a) schematic for the hemispherical substrate. (b) (b) The fabricated microlens array with the screening layer on the hemispherical substrate. array with the screening layer on the hemispherical substrate. Figure 5. Imaging (a) The fabrication schematic for the hemispherical substrate. (b) The fabricated microlens 3.2. Integrated System with a Conventional CCD Image Sensor 3.2. Integrated Imaging Systemlayer with on a Conventional CCDsubstrate. Image Sensor array with the screening the hemispherical
Figure 6 shows the assembled imaging system microlens array adjusted by the 3D 3.2. Integrated Imaging System with a Conventional CCD with Imagethe Sensor
Figure shows and the bolts. assembled imaging system the microlens array adjusted the 3D printed jig, 6springs, Dueato the springs andwith the bolts, the distance between theby microlens 3.2. Integrated Imaging System with Conventional CCD Image Sensor Figure 6jig, shows thesensor assembled imaging system withthe thebolts, microlens array adjusted by microlens the 3D printed printed springs, and bolts. to theto springs and theimage. distance the array and the CCD was Due calibrated focus every microlens’ Thebetween light-entering window Figure 6CCD shows assembled system with the microlens array adjusted the 3D and array the sensor was every microlens’ image. The light-entering window of theand 3D printed jig isthe a square shapeimaging ofto 4 ×focus 4the mm. Eighteen microlenses receive light fromby the lightjig, springs, and bolts. Due to thecalibrated springs and bolts, the distance between the microlens array printed jig, springs, and bolts. Due to the springs and the bolts, the distance between the microlens of the 3D printed jig is a square shape of 4 × 4 mm. Eighteen microlenses receive light from the lightentering window. The photo sensing part of the CCD sensor is a rectangular shape that is 5.37 × 4.04 the CCD sensor was calibrated to focus every microlens’ image. The light-entering window of the 3D array and the ×CCD sensor was calibrated to the focus every microlens’ image. The light-entering window entering The photo part of CCD sensor is a rectangular shape thatthe is 5.37 × 4.04 mm and 1920 1080 pixels. printed jig iswindow. a square shape of sensing 4 × 4 mm. Eighteen microlenses receive light from light-entering of the printed jigpixels. is a square shape of 4 × 4 mm. Eighteen microlenses receive light from the lightmm and3D 1920 × 1080 window. The photo sensing part of the CCD sensor is a rectangular shape that is 5.37 × 4.04 mm and entering window. The photo sensing part of the CCD sensor is a rectangular shape that is 5.37 × 4.04 1920 × 1080 mm andpixels. 1920 × 1080 pixels.
(a) (a)
(b) (b)
Figure 6. (a) The fabrication schematic for the hemispherical substrate. (b) The fabricated microlens Figure 6. (a)the The fabrication schematic for the hemispherical array with screening layer on the hemispherical substrate.substrate. (b) The fabricated microlens (a) (b) array with the screening layer on the hemispherical substrate. Figure 6. (a) The fabrication schematic for the hemispherical substrate. (b) The fabricated microlens Figure 6. (a) The fabrication schematic for the hemispherical substrate. (b) The fabricated microlens array with the screening layer on the hemispherical substrate.
array with the screening layer on the hemispherical substrate.
Sensors 2018, 18, 2011
6 of 9
Sensors 2018, 18, x FOR PEER REVIEW Sensors 2018, 18, x FOR PEER REVIEW
6 of 9 6 of 9
Figure presents images imagesofofa aprinted printedUSAF USAF 1951 pattern captured by two different setups: Figure 77 presents 1951 pattern captured by two different setups: the the microlens-facing-outside setup (Figure 7a) and the microlens-facing-inside setup (Figure 7b). Figure 7 presents images of a printed USAF 1951 pattern captured by two different setups: the microlens-facing-outside setup (Figure 7a) and the microlens-facing-inside setup (Figure 7b). The The facing-inside setup shows better contrast around rim thanthe thelooking-outside looking-outside setup. We We could microlens-facing-outside setup (Figure 7a) and the microlens-facing-inside setup (Figure 7b).could The facing-inside setup shows better contrast around thethe rim than setup. assume that the apertures of the facing-inside setup play a better role because the gold screening layer facing-inside setup shows better contrast around the rim than the looking-outside setup. We could assume that the apertures of the facing-inside setup play a better role because the gold screening of theof facing-inside setup setup isof closer to the to image sensor by a thickness of the NOA substrate. assume thatfacing-inside the apertures the facing-inside setupsensor play aby better role because the gold screening layer the is closer the image a thickness of the NOA substrate. layer of the facing-inside setup is closer to the image sensor by a thickness of the NOA substrate.
(a) (a)
(b) (b)
Figure 7. 7. The distance of 10 cm cm by by our our Figure The captured captured images images of of the the printed printed USAF USAF 1951 1951 test test pattern pattern at at a a distance of 10 Figure 7.system The captured images of the printed USAF setup 1951 test pattern at a distance of 10 cm by our imaging with (a) the microlens-facing-outside and (b) the microlens-facing-inside setup. imaging system with (a) the microlens-facing-outside setup and (b) the microlens-facing-inside setup. imaging system with (a) the microlens-facing-outside setup and (b) the microlens-facing-inside setup.
Field-of-view (FOV) of the assembled imaging system was measured using a 3D-printed halfField-of-view (FOV) ofofthe imaging system was measured using a 3D-printed half-circle Field-of-view theassembled assembled imaging measured using a83D-printed circle structure and(FOV) a paper-printed goniometer, assystem shownwas in Figure 8a. Figure shows thathalfthe structure and a paper-printed goniometer, as shown Figure Figure shows 8that the leftmost circle structure andcaptured a paper-printed goniometer, as in shown in 8a. Figure 8a.8Figure shows the leftmost microlens −50 degrees and the rightmost microlens took +50 degrees. Thisthat means microlensmicrolens capturedcaptured −50 degrees and the rightmost microlens took +50took degrees. This means that our leftmost degrees and theofrightmost microlens +50the degrees. that our imaging system has−50 about 100 degrees FOV, which is higher than FOV ofThis themeans single imaging system has about 100 degrees of FOV, which is higher than the FOV of the single microlens that our imaging system has about 100 degrees of FOV, which is higher than the FOV of the single microlens (80 degrees). (80 degrees). microlens (80 degrees).
(a) (a)
(b) (b)
Figure 8. The experimental setup for measuring the FOV of the assembled imaging system. (a) The Figure 8. The experimental setup for theThe assembled imaging system. (a) The captured represents about 100measuring degrees ofthe theFOV FOV. (b) captured images of (upper) the Figure 8. image The experimental setup for measuring the of FOV of the assembled imaging system. captured image represents about 100 degrees of the FOV. (b) The captured images of (upper) the leftmost microlens, (middle) the middle and the microlens. (a) The captured image represents aboutmicrolens, 100 degrees of (lower) the FOV. (b)rightmost The captured images of (upper) leftmost microlens, (middle) the the middle microlens, andand (lower) the the rightmost microlens. the leftmost microlens, (middle) middle microlens, (lower) rightmost microlens.
Every subimage represents different perspective views of the identical object. Due to this Every subimage post-image represents processing different perspective views of imaging the identical object. Due diverse to this plentiful information, of the multiaperture system generates plentiful information, post-image processing of the multiaperture imaging system generates diverse images including higher-resolution images [21] and multifocusing images [22]. Figure 9 shows images including higher-resolution images [21] and multifocusing images [22]. Figure 9 shows
Sensors 2018, 18, 2011
7 of 9
Every subimage represents different perspective views of the identical object. Due to this plentiful information, post-image processing of the multiaperture imaging system generates diverse images Sensors 2018, 18, x FOR PEER REVIEW 7 of 9 including higher-resolution images [21] and multifocusing images [22]. Figure 9 shows multifocusing images. Using images. the 18 subimages of one single shot (Figure differently focused images were multifocusing Using the 18 subimages of one single9a), shotthree (Figure 9a), three differently focused reconstructed, as shown in Figure 9b–d. The characters “SNU”, “UoU”, and “EFE LAB” are apart 1 cm, images were reconstructed, as shown in Figure 9b–d. The characters “SNU”, “UoU”, and “EFE LAB” 2 cm, and13cm, cm 2from system, respectively. are apart cm, the andimaging 3 cm from the imaging system, respectively.
(a)
(b)
(c)
(d)
Figure 9. (a) The captured 18 subimages. subimages. (b) “EFE LAB” LAB” apart apart Figure 9. (a) The captured 18 (b) The The reconstructed reconstructed image image focusing focusing the the “EFE 33 cm from the imaging system. (c) The reconstructed image focusing the “UoU” apart 2 cm from cm from the imaging system. (c) The reconstructed image focusing the “UoU” apart 2 cm from the the imaging system.(d) (d)The Thereconstructed reconstructed image focusing the “SNU” 1 cmthe from the imaging imaging system. image focusing the “SNU” apartapart 1 cm from imaging system. system.
4. Discussion & Conclusions 4. Discussion & Conclusions The imaging system mimicking an insect’s compound eye was developed. Due to the light The imaging system mimicking an insect’s compound eye was developed. Due to the light screening layer among the microlenses fabricated by our PLISL process and the adjustable jig, all of screening layer among the microlenses fabricated by our PLISL process and the adjustable jig, all of the 18 subimages were clearly focused. A simple change in the mask for the microlens patterning can the 18 subimages were clearly focused. A simple change in the mask for the microlens patterning can increase the number of the subimages, which means that there is higher angular resolution and depth increase the number of the subimages, which means that there is higher angular resolution and depth resolution in reconstructed images. Although the field-of-view of our imaging system is wider than resolution in reconstructed images. Although the field-of-view of our imaging system is wider than other types of microlens array imaging systems and a conventional 35 mm camera with a compact lens other types of microlens array imaging systems and a conventional 35 mm camera with a compact system, it should be increased for recent emerging demands such as virtual reality and unmanned lens system, it should be increased for recent emerging demands such as virtual reality and vehicles. Currently, we are researching a microlens array on hemispherical light sensors to meet unmanned vehicles. Currently, we are researching a microlens array on hemispherical light sensors these requirements. to meet these requirements. Supplementary Materials: The following are available online at www.mdpi.com/xxx/s1. Author Contributions: Conceptualization, J.-M.S. and K.K.; Methodology, M.S.; Software, M.S.; Validation, K.K.; Formal Analysis, M.S.; Investigation, M.S.; Resources, J.-M.S., D.C., and K.K.; Data Curation, M.S. and K.K.; Writing-Original Draft Preparation, M.S.; Writing-Review & Editing, J.-M.S. and K.K.; Visualization, M.S.;
Sensors 2018, 18, 2011
8 of 9
Supplementary Materials: The following are available online at http://www.mdpi.com/1424-8220/18/7/2011/ s1. Author Contributions: Conceptualization, J.-M.S. and K.K.; Methodology, M.S.; Software, M.S.; Validation, K.K.; Formal Analysis, M.S.; Investigation, M.S.; Resources, J.-M.S., D.C., and K.K.; Data Curation, M.S. and K.K.; Writing-Original Draft Preparation, M.S.; Writing-Review & Editing, J.-M.S. and K.K.; Visualization, M.S.; Supervision, J.-M.S. and K.K.; Project Administration, J.-M.S., D.C. and K.K.; Funding Acquisition, J.-M.S., D.C. and K.K. Funding: This research was funded by Defense Acquisition Program Administration, and by Agency for Defense Development (UD160027ID). Acknowledgments: This research was supported by a grant to Bio-Mimetic Robot Research Center Funded by Defense Acquisition Program Administration, and by Agency for Defense Development (UD160027ID). Conflicts of Interest: The authors declare no conflict of interest.
References 1. 2. 3. 4.
5. 6. 7.
8. 9.
10.
11. 12. 13.
14. 15.
16.
Ahuja, N.A.; Bose, N.K. Design of large field-of-view high-resolution miniaturized imaging system. EURASIP J. Adv. Signal Process. 2007, 2007, 1–10. [CrossRef] Land, M.F. Compound eyes: Old and new optical mechanisms. Nature 1980, 287, 681–686. [CrossRef] [PubMed] Horridge, G.A. The separation of visual axes in apposition compound eyes. Philos. Trans. R. Soc. B Biol. Sci. 1978, 285, 1–59. [CrossRef] Snyder, A.W. Physics of vision in compound eyes. In Comparative Physiology and Evolution of Vision in Invertebrates: A: Invertebrate Photoreceptors; Autrum, H., Bennett, M.F., Diehn, B., Hamdorf, K., Heisenberg, M., Järvilehto, M., Kunze, P., Menzel, R., Miller, W.H., Snyder, A.W., et al., Eds.; Springer: Berlin/Heidelberg, Germnay, 1979; pp. 225–313. McIntyre, P.; Caveney, S. Graded-index optics are matched to optical geometry in the superposition eyes of scarab beetles. Philos. Trans. R. Soc. B Biol. Sci. 1985, 311, 237–269. [CrossRef] Rafael, N.; Nicolas, F. On image quality of microlens arrays in diurnal superposition eyes. Pure Appl. Opt. J. Eur. Opt. Soc. Part A 1998, 7, L69. Park, H.S.; Hoskinson, R.; Abdollahi, H.; Stoeber, B. Compact near-eye display system using a superlensbased microlens array magnifier. In Proceedings of the 2015 28th IEEE International Conference on Micro Electro Mechanical Systems (MEMS), Estoril, Portugal, 18–22 January 2015; pp. 952–955. Brückner, A.; Duparré, J.; Dannberg, P.; Bräuer, A.; Tünnermann, A. Artificial neural superposition eye. Opt. Express 2007, 15, 11922–11933. [CrossRef] [PubMed] Stollberg, K.; Brückner, A.; Duparré, J.; Dannberg, P.; Bräuer, A.; Tünnermann, A. The gabor superlens as an alternative wafer-level camera approach inspired by superposition compound eyes of nocturnal insects. Opt. Express 2009, 17, 15747–15759. [CrossRef] [PubMed] Floreano, D.; Pericet-Camara, R.; Viollet, S.; Ruffier, F.; Bruckner, A.; Leitel, R.; Buss, W.; Menouni, M.; Expert, F.; Juston, R.; et al. Miniature curved artificial compound eyes. Proc. Natl. Acad. Sci. USA 2013, 110, 9267–9272. [CrossRef] [PubMed] Buschbeck, E.K. The compound lens eye of strepsiptera: Morphological development of larvae and pupae. Arthropod Struct. Dev. 2005, 34, 315–326. [CrossRef] James, M.; Nandamuri, S.P.; Stahl, A.; Buschbeck, E.K. The unusual eyes of Xenos peckii (strepsiptera: Xenidae) have green- and uv-sensitive photoreceptors. J. Exp. Biol. 2016, 219, 3866–3874. [CrossRef] [PubMed] Keum, D.; Jeon, D.S.; Kim, M.H.; Jeong, K.H. Artificial compound eye inspired by imaging principle of xenos peckii. In Proceedings of the 2015 Transducers—2015 18th International Conference on Solid-State Sensors, Actuators and Microsystems (TRANSDUCERS), Anchorage, AK, USA, 21–25 June 2015; pp. 403–406. Borrelli, N.F.; Morse, D.L.; Bellman, R.H.; Morgan, W.L. Photolytic technique for producing microlenses in photosensitive glass. Appl. Opt. 1985, 24, 2520–2525. [CrossRef] [PubMed] Tanida, J.; Kumagai, T.; Yamada, K.; Miyatake, S.; Ishida, K.; Morimoto, T.; Kondou, N.; Miyazaki, D.; Ichioka, Y. Thin observation module by bound optics (tombo): Concept and experimentalverification. Appl. Opt. 2001, 40, 1806–1813. [CrossRef] [PubMed] Song, Y.M.; Xie, Y.; Malyarchuk, V.; Xiao, J.; Jung, I.; Choi, K.-J.; Liu, Z.; Park, H.; Lu, C.; Kim, R.-H.; et al. Digital cameras with designs inspired by the arthropod eye. Nature 2013, 497, 95–99. [CrossRef] [PubMed]
Sensors 2018, 18, 2011
17. 18. 19.
20. 21. 22.
9 of 9
Daly, D.; Stevens, R.F.; Hutley, M.C.; Davies, N. The manufacture of microlenses by melting photoresist. Meas. Sci. Technol. 1990, 1, 759. [CrossRef] Norland Optical Adhesive 72. Available online: https://www.norlandprod.com/adhesives/noa%2072.html (accessed on 23 March 2018). Ikjoo, B.; Anthony, W.C.; Beomjoon, K. Transfer of thin au films to polydimethylsiloxane (pdms) with reliable bonding using (3-mercaptopropyl)trimethoxysilane (MPTMS) as a molecular adhesive. J. Micromech. Microeng. 2013, 23, 085016. Axelevitch, A.; Apter, B.; Golan, G. Simulation and experimental investigation of optical transparency in gold island films. Opt. Express 2013, 21, 4126–4138. [CrossRef] [PubMed] Sung Cheol, P.; Min Kyu, P.; Moon Gi, K. Super-resolution image reconstruction: A technical overview. IEEE Signal Process. Mag. 2003, 20, 21–36. [CrossRef] Kim, J.; Jung, J.-H.; Jeong, Y.; Hong, K.; Lee, B. Real-time integral imaging system for light field microscopy. Opt. Express 2014, 22, 10210–10220. [CrossRef] [PubMed] © 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
