Photon arrival timing with sub-camera exposure time ... - OSA Publishing

Photon arrival timing with sub-camera exposure time resolution in wide-field time-resolved photon counting imaging Zdenˇek Petrásˇek1,3 and Klaus Suhling2,4 1

Biotechnologisches Zentrum, Technische Universität Dresden, Tatzberg 47/49, 01307 Dresden, Germany 2 Department of Physics, King’s College London, Strand, London WC2R 2LS, UK 3 [email protected] 4 [email protected]

Abstract: We demonstrate that an ultra-fast CMOS camera combined with a photon counting image intensifier can be used to determine photon arrival times well below the exposure time of the camera. We can obtain a time resolution down to around 1% of the exposure time, i.e. of the order of 40 ns with microsecond exposure times. This is achieved by exploiting the invariant phosphor decay of the image intensifier’s phosphor screen: Developing a suitable mathematical framework, we show that the relative intensities of the phosphor decay in successive frames following the photon detection uniquely determine the photon arrival time. This approach opens a way to measuring fast luminescence decays in parallel in many pixels. Possible applications include oxygen and ion concentration imaging using probes with luminescence lifetimes in the range of 100 ns to microseconds. © 2010 Optical Society of America OCIS codes: (030.5260) Photon counting; (040.3780) Low light level; (180.2520)Fluorescence microscopy; (230.0040) Detectors.

References and links 1. K. Suhling, P. M. W. French, and D. Phillips, “Time-resolved fluorescence microscopy,” Photochem. Photobiol. Sci. 4, 13–22 (2005). 2. F. Festy, S. M. Ameer-Beg, T. Ng, and K. Suhling, “Imaging proteins in vivo using fluorescence lifetime microscopy,” Mol. Biosyst. 3, 381–391 (2007). 3. M. Peter and S. M. Ameer-Beg, “Imaging molecular interactions by multiphoton FLIM,” Biol. Cell 96, 231–236 (2004). 4. J. A. Levitt, D. R. Matthews, S. M. Ameer-Beg, and K. Suhling, “Fluorescence lifetime and polarization-resolved imaging in cell biology,” Curr. Opin. Biotechnol. 20, 28–36 (2009). 5. D. V. O’Connor and D. Phillips, Time-Correlated Single Photon Counting (Academic Press, 1984). ISBN 0125241402. 6. W. Becker, Advanced Time-Correlated Single Photon Counting Techniques, Springer Series in Chemical Physics (Springer, 2005). ISBN 3540260471. 7. F. E. W. Schmidt, M. E. Fry, E. M. C. Hillman, J. C. Hebden, and D. T. Delpy, “A 32-channel time-resolved instrument for medical optical tomography,” Rev. Sci. Instrum. 71, 256–265 (2000). 8. G. S. Buller, R. D. Harkins, A. McCarthy, P. A. Hiskett, G. R. MacKinnon, G. R. Smith, R. Sung, A. M. Wallace, R. A. Lamb, K. D. Ridley, and J. G. Rarity, “Multiple wavelength time-of-flight sensor based on time-correlated single-photon counting,” Rev. Sci. Instrum. 76, 083112 (2005). 9. X. Michalet, O. H. W. Siegmund, J. V. Vallerga, P. Jelinsky, J. E. Millaud, and S. Weiss, “Detectors for singlemolecule fluorescence imaging and spectroscopy,” J. Mod. Opt. 54, 239–281 (2007). 10. G. Hungerford and D. J. S. Birch, “Single-photon timing detectors for fluorescence lifetime spectroscopy,” Meas. Sci. Technol. 7, 121–135 (1996).

#133298 - $15.00 USD Received 11 Aug 2010; revised 22 Sep 2010; accepted 23 Sep 2010; published 15 Nov 2010

(C) 2010 OSA

22 November 2010 / Vol. 18, No. 24 / OPTICS EXPRESS 24888

11. X. Michalet, R. A. Colyer, J. Antelman, O. H. W. Siegmund, A. Tremsin, J. V. Vallerga, and S. Weiss, “Singlequantum dot imaging with a photon counting camera,” Curr. Pharm. Biotechnol. 10, 543–558 (2009). 12. Y. Prokazov, E. Turbin, M. Vitali, A. Herzog, B. Michaelis, W. Zuschratter, and K. Kemnitz, “Reborn quadrant anode image sensor,” Nucl. Instrum. Methods Phys. Res., Sect. A 604, 221–223 (2009). 13. J. A. Spitz, R. Yasukuni, N. Sandeau, M. Takano, J. J. Vachon, R. Meallet-Renault, and R. B. Pansu, “Scanningless wide-field single-photon counting device for fluorescence intensity, lifetime and time-resolved anisotropy imaging microscopy,” J. Microsc. 229, 104–114 (2008). 14. Z. Petrásˇek, H. J. Eckert, and K. Kemnitz, “Wide-field photon counting fluorescence lifetime imaging microscopy: application to photosynthesizing systems,” Photosynth. Res. 102, 157–168 (2009). 15. O. Jagutzki, A. Cerezo, A. Czasch, R. Dörner, M. Hattaß, M. Huang, V. Mergel, U. Spillmann, K. UllmannPfleger, T. Weber, H. Schmidt-Böcking, and G. D. W. Smith, “Multiple Hit Readout of a Microchannel Plate Detector With a Three-Layer Delay-Line Anode,” IEEE Trans. Nucl. Sci, 49, 2477–2483 (2002). 16. A. S. Tremsin, O. H. W. Siegmund, J. V. Vallerga, R. Raffanti, S. Weiss and X. Michalet, “High speed multichannel charge sensitive data acquisition system with self-triggered event timing” IEEE Trans. Nucl. Sci. 56, 1148–1152 (2009). 17. D.-U. Li, J. Arlt, J. Richardson, R. Walker, A. Buts, D. Stoppa, E. Charbon, and R. Henderson, “Real-time fluorescence lifetime imaging system with a 32 × 32 0.13μ m CMOS low dark-count single-photon avalanche diode array,” Opt. Express 18, 10527–102692 (2010). 18. C. Niclass, C. Favi, T. Kluter, M. Gersbach, and E. Charbon, “A 128 × 128 Single-Photon Image Sensor With Column-Level 10-Bit Time-to-Digital Converter Array” IEEE J. Sol.-St. Circ. 43, 2977–2989 (2008). 19. S. Tisa, F. Guerrieri, F. Zappa, “Monolithic arrayof 32 SPAD pixels for single-photon imaging at high frame rates” Nucl. Instrum. Methods Phys. Res., Sect. A 610, 24–27 (2009). 20. G. Vereb, E. Jares-Erijman, P. R. Selvin, and T. M. Jovin, “Temporally and spectrally resolved imaging microscopy of lanthanide chelates,” Biophys. J. 74, 2210–2222 (1998). 21. G. Marriott, R. M. Clegg, D. J. Arndt-Jovin, and T. M. Jovin, “Time resolved imaging microscopy - phosphorescence and delayed fluorescence imaging,” Biophys. J. 60, 1374–1387 (1991). 22. A. C. Mitchell, J. E. Wall, J. G. Murray, and C. G. Morgan, “Direct modulation of the effective sensitivity of a ccd detector: a new approach to time-resolved fluorescence imaging,” J. Microsc. 206, 225–232 (2002). 23. D. S. Elson, I. Munro, J. Requejo-Isidro, J. McGinty, C. Dunsby, N. Galletly, G. W. Stamp, M. A. A. Neil, M. J. Lever, P. A. Kellett, A. Dymoke-Bradshaw, J. Hares, and P. M. W. French, “Real-time time-domain fluorescence lifetime imaging including single-shot acquisition with a segmented optical image intensifier,” N. J. Phys. 6, 180 (2004). 24. J. B. Hutchings, J. Postma, D. Asquin, and D. Leahy, “Photon event centroiding with UV photon-counting detectors,” Publ. Astron. Soc. Pac. 119, 1152–1162 (2007). 25. H. W. Kröger, G. K. Schmidt, and N. Pailer, “Faint object camera - european contribution to the Hubble Space Telescope,” ACTA Aeronaut. Astronaut. Sinica 26, 827–834 (1992). 26. P. W. A. Roming, T. E. Kennedy, K. O. Mason, J. A. Nousek, L. Ahr, R. E. Bingham, P. S. Broos, M. J. Carter, B. K. Hancock, H. E. Huckle, S. D. Hunsberger, H. Kawakami, R. Killough, T. S. Koch, M. K. Mclelland, K. Smith, P. J. Smith, J. C. Soto, P. T. Boyd, A. A. Breeveld, S. T. Holland, M. Ivanushkina, M. S. Pryzby, M. D. Still, and J. Stock, “The swift ultra-violet/optical telescope,” Space Sci. Rev. 120, 95–142 (2005). 27. C. L. Joseph, “UV image sensors and associated technologies,” Exp. Astron. 6, 97–127 (1995). 28. J. E. Lees and G. W. Fraser, “Efficiency enhancements for MCP-based beta autoradiography imaging,” Nucl. Instrum. Methods Phys. Res., Sect. A 477, 239–243 (2002). 29. P. D. Read, M. K. Carter, C. D. Pike, R. A. Harrison, B. J. Kent, B. M. Swinyard, B. E. Patchett, R. M. Redfern, A. Shearer, and M. Colhoun, “Uses of microchannel plate intensified detectors for imaging applications in the X-ray, EUV and visible wavelength regions,” Nucl. Instrum. Methods Phys. Res., Sect. A 392, 359–363 (1997). 30. K. Suhling, G. Hungerford, R. W. Airey, and B. L. Morgan, “A position-sensitive photon event counting detector applied to fluorescence imaging of dyes in sol-gel matrices,” Meas. Sci. Technol. 12, 131–141 (2001). 31. N. A. Sharp, “Millisecond time resolution with the kitt peak photon-counting array,” Publ. Astron. Soc. Pac. 104, 263–269 (1992). 32. M. H. V. Werts, “Making sense of lanthanide luminescence,” Sci. Prog. 88, 101–131 (2005). 33. J. C. G. Bunzli, “Lanthanide luminescence for biomedical analyses and imaging,” Chem. Rev. 110, 2729–2755 (2010). 34. S. W. Botchway, M. Charnley, J. W. Haycock, A. W. Parker, D. L. Rochester, J. A. Weinstein, and J. A. G. Williams, “Time-resolved and two-photon emission imaging microscopy of live cells with inert platinum complexes,” Proc. Natl. Acad. Sci. USA 105, 16071–16076 (2008). 35. Y. Yamaguchi, K. Hashino, M. Ito, K. Ikawa, T. Nishioka, and K. Matsumoto, “Sodium dodecyl sulfate polyacrylamide slab gel electrophoresis and hydroxyethyl cellurose gel capillary electrophoresis of luminescent lanthanide chelate-labeled proteins with time-resolved detection,” Anal. Sci. 25, 327–332 (2009). 36. T. Nishioka, J. L. Yuan, Y. Yamamoto, K. Sumitomo, Z. Wang, K. Hashino, C. Hosoya, K. Ikawa, G. L. Wang, and K. Matsumoto, “New luminescent europium(III) chelates for DNA labeling,” Inorg. Chem. 45, 4088–4096 (2006).


(C) 2010 OSA


37. I. Hemmilä and V. Laitala, “Progress in lanthanides as luminescent probes,” J. Fluoresc. 15, 529–542 (2005). 38. P. R. Selvin, “Principles and biophysical applications of lanthanide-based probes,” Annu. Rev. Biophys. Biomol. Struct. 31, 275–302 (2002). 39. W. K. Young, B. Vojnovic, and P. Wardman, “Measurement of oxygen tension in tumours by time-resolved fluorescence,” Br. J. Cancer 74, S256–S259 (1996). 40. M. P. Coogan, J. B. Court, V. L. Gray, A. J. Hayes, S. H. Lloyd, C. O. Millet, S. J. A. Pope, and D. Lloyd, “Probing intracellular oxygen by quenched phosphorescence lifetimes of nanoparticles containing polyacrylamideembedded [Ru(dpp(SO3 Na)2 )3 ]Cl2 ,” Photochem. Photobiol. Sci. 9, 103–109 (2010). 41. N. A. Hosny, D. A. Lee, and K. M. M., “Extracellular oxygen concentration mapping with a confocal multiphoton laser scanning microscope and TCSPC card,” Proc. SPIE 7569, 756932 (2010). 42. J. G. Mainprize and M. J. Yaffe, “The effect of phosphor persistence on image quality in digital x-ray scanning systems,” Med. Phys. 25, 2440–2454 (1998). 43. K. Suhling, R. Airey, and B. Morgan, “Optimisation of centroiding algorithms for photon event counting imaging,” Nucl. Instrum. Methods Phys. Res., Sect. A 437, 393–418 (1999). 44. K. Suhling, R. W. Airey, and B. L. Morgan, “Minimization of fixed pattern noise in photon event counting imaging,” Rev. Sci. Instrum. 73, 2917–2922 (2002). 45. G. Bub, M. Tecza, M. Helmes, P. Lee, and P. Kohl, “Temporal pixel multiplexing for simultaneous high-speed, high-resolution imaging,” Nat. Methods 7, 209–211 (2010). 46. Z. Bajzer, A. Zelić, and F. G. Prendergast, “Analytical approach to the recovery of short fluorescence lifetimes from fluorescence decay curves,” Biophys. J. 69, 1148–1161 (1995). 47. A. J. Berglund, M. D. McMahon, J. J. McClelland, and J. A. Liddle, “Fast, bias-free algorithm for tracking single particles with variable size and shape,” Opt. Express 16, 14064–14075 (2008). 48. Z. Petrásˇek and P. Schwille, “Fluctuations as a source of information in fluorescence microscopy,” J. R. Soc. Interface 6, S15–S25 (2009). 49. J. Siegel, K. Suhling, S. Leveque-Fort, S. E. D. Webb, D. M. Davis, D. Phillips, Y. Sabharwal, and P. M. W. French, “Wide-field time-resolved fluorescence anisotropy imaging (TR-FAIM): Imaging the rotational mobility of a fluorophore,” Rev. Sci. Instrum. 74, 182–192 (2003).

1.

Introduction

Fluorescence and luminescence imaging techniques are powerful tools in the biological and biomedical sciences, mainly because of their high sensitivity, high labeling specificity and minimal invasiveness [1,2]. Fluorescence can be characterized not only by the intensity of its emission, but also by its wavelength, lifetime and polarization. All of these parameters have been analyzed and employed as a source of contrast in imaging. Using the fluorescence lifetime as contrast in an image is particularly advantageous because it allows probe concentration and quenching effects to be separated, since, at low probe concentrations, the decay is independent of the probe concentration [1–3]. It also allows imaging multi-exponential decay kinetics which are inaccessible via intensity-based imaging. Moreover, polarization-resolved lifetime imaging allows mapping of viscosity or homo-FRET (Förster Resonance Energy Transfer) [4]. Lifetime on the nanosecond scale is measured either with time-domain or with frequencydomain methods [1]. Here we focus on the time-domain, which is particularly advantageous in situations when the intensity levels are low, and photon counting operation has to be employed. With very low signals, time-correlated single photon counting (TCSPC) is a preferred method, since all detected photons are timed with high picosecond accuracy. Conventional single-detector TCSPC is a mature, precise and most reliable technique [5, 6]. It records the arrival time of single photons after an excitation pulse, and is used not only for time-resolved fluorescence spectroscopy or confocal or multiphoton excitation scanning Fluorescence Lifetime Imaging (FLIM) [1–4], but also for photon time-of-flight measurements in optical tomography [7] and lidar [8]. Microchannel plate (MCP) detectors for TCSPC timing routinely achieve picosecond timing resolution [9, 10], but wide-field TCSPC is less common. Spatial resolution can be achieved by different read-out architectures for photon counting imaging detectors, such as crossed-delay line anodes [11], wedge and strip anodes or quadrant anodes [12–14]. Generally, conventional photon pile-up restrictions apply and they are limited to timing only #133298 - $15.00 USD Received 11 Aug 2010; revised 22 Sep 2010; accepted 23 Sep 2010; published 15 Nov 2010

(C) 2010 OSA


one single photon per excitation cycle in the entire field of view. However, multiple hit readouts [15] have been developed, as well as crossed strip anodes with parallel processing electronics with MHz event rates and nanosecond timing resolution [16]. Another recent very promising development are two-dimensional single photon avalanche diode (SPAD) arrays [19]. They are single photon sensitive, have picosecond timing resolution and have been demonstrated for lidar applications [18] and FLIM [17]. SPAD arrays combine the advantages of TCSPC detection with parallel pixel acquisition and allow GHz count rates. However, at present they have a low fill factor due to the timing electronics associated with each pixel (2% in reference [17], 6% in reference [18]) and they have short fixed time windows, typically below 100ns [18] - but this may improve in the future. Long lifetime luminescent probes are difficult to image in a time-resolved manner using confocal or multiphoton scanning systems, because a long pixel dwell time is needed to collect a sufficient number of photons from each pixel. Despite the ease with which scanning systems can be combined with time-resolved detection such as TCSPC, this approach results in an unfeasibly large acquisition time. Therefore, time-resolved wide field imaging is usually carried out in the time domain with gated image intensifiers [20, 21] or directly gated CCD-cameras [22]. However, the light level has to be high enough so that sufficient signal is detected in individual gates. Also, sequential acquisition of the time gates is vulnerable to excitation or emission intensity fluctuations which can affect the accuracy of the decay analysis. Another disadvantage of gated detection is that the signal outside the gate is lost, compromising sensitivity. Thus, the available photon budget is spent unwisely and biological samples have to be subjected to prolonged high intensity excitation with its concomitant detrimental phototoxic effects. The loss of data in time-gating could be avoided if all delayed images following one excitation pulse could be recorded which has been demonstrated with a segmented image intensifier [23]. Photon counting imaging is a well-established low light level imaging technique, particularly in astronomy, both on the ground [24] and in space [25,26]. It is based on the collection of many individual photon events to yield an image. Linearity, high dynamic range, high sensitivity, zero read-out noise, large area, and well-defined Poisson statistics are particular strengths of the photon counting imaging approach [27]. Applications of photon counting imaging to diverse fields such as autoradiography [28], bioluminescence [29] and fluorescence imaging [30] have also been reported. Photon counting imaging has another distinct advantage over common intensity-based CCD imaging: the ability to time the arrival of photons. Conventionally, photon events on the phosphor screen of the image intensifiers operating in photon counting mode are imaged with a camera at video frame rates, and many frames are accumulated to build up an image. The time resolution is given by the frame rate of the camera, typically video frame rates (60 Hz), which yields a millisecond time resolution [31]. Faster processes such as molecular diffusion and transport, luminescence decay and other photophysical processes are thus beyond observation, although the image intensifier could in principle allow very much faster timing (with picosecond resolution [9, 10]) than a video-rate camera can provide. Labeling biological samples with long lifetime probes such as lanthanides [32, 33], platinum [34] or palladium compounds has some distinct advantages over using fluorescence dyes. Lanthanide transitions are much slower than molecular fluorescence, typically in the millito microsecond time range [33, 35, 36], which allows for easy discrimination from cellular autofluorescence [37,38]. Long-lifetime ruthenium dyes used as oxygen sensors provide an important read-out of the metabolic state of cells [39–41]. Luminescent Resonance Energy Transfer (LRET) assays which are used in some commercial multiwell plate readers [37] with lanthanide ˚ has been reported [37]) which eases the donors can benefit from a large Förster radius R0 (90 A restrictions on labeling sites of large proteins. Moreover, due to the long excited state lifetime, randomization of the acceptor orientation during this time will reduce errors related to the gen-


(C) 2010 OSA


erally unknown orientation factor κ [38]. In addition, long lifetime decays also allow probing cellular processes occurring on a slower time-scale than is possible with fluorescence, e.g., by time-resolved fluorescence anisotropy [9]. Here, we present a method that allows simultaneous timing of many photon events with uncertainty of the order of 40 ns. It is based on determination of the photon arrival time from the decay of the phosphor of the image intensifier recorded by an ultra-fast CMOS camera. This approach combines ultra-fast wide-field imaging with single photon sensitivity and parallel arrival timing in each pixel. The method is suitable for low light applications down to a single molecule level. 2.

Theory

We consider a common experimental configuration where the photocathode of the image intensifier is placed in the image plane of the microscope or other imaging device, and the phosphor screen of the intensifier, where the intensified image is formed, is imaged by a camera [30]. When a photon hits the photocathode, the resulting photoelectron is multiplied by the MCP producing an electron cloud that hits the phosphor and generates a bright scintillation, a photon event. The intensity of the photon event depends on the statistical variation of the secondary electron multiplication process along the MCP channels. Due to a short transit time, the transit time spread is also short (a few tens of picoseconds), so the arrival time information is preserved. The decay of the phosphor luminescence is strongly non-exponential, and extends over 1 − 100 μ s, or even longer. When a camera with a sufficiently short repetition period T is used, the phosphor decay of one event can be recorded by monitoring the phosphor intensity decay over a number of image frames. In conventional photon counting imaging applications, this phosphor persistence is not desired [42]. However, if the phosphor decay is independent of the gain, and if the time course of the phosphor decay is known, the arrival time of the photon can be determined from the phosphor decay measured with the camera. The timing resolution thus achieved considerably exceeds the camera repetition rate (Fig. 1). For example, consider a photon arriving towards the end of the camera exposure period. Then the intensity I1 in the frame in which the event is first detected will be very low, possibly much lower than the intensity I2 in the following frame. If, on the other hand, the photon arrives closer to the beginning of the camera exposure period, the intensity in the first frame will be higher than in the second frame. Thus, the photon arrival time with respect to the start of the camera exposure determines the ratio between the intensities I1 and I2 in the first two frames, or generally, the relative intensities Im in all frames. Conversely, the experimentally determined intensities Im can be used to determine the arrival time relative to the beginning of the camera exposure period. With pulsed excitation locked to the camera shutter, timing relative to the excitation pulses can thus be achieved. Suppose that the phosphor decay is described by a function f (t), and that there is additionally a dead time (due to camera readout) within the time interval (0, zT ) occupying a fraction z of the full period T , during which no light is detected (Fig. 1). In general, the phosphor decay can be approximated to an arbitrary precision by a sum of exponentials: f (t) = ∑ a j e−t/τ j .

(1)

j

The intensity I1 in the first frame (0 < t < T ) where the event is detected is then given by: I1 =

T β

f (t − xT ) dt = ∑ a j τ j exT /τ j e−β /τ j − e−T /τ j ,

(2)

j


(C) 2010 OSA


I1

0

zT

I2

xT

T

t

I3

2T

3T

Fig. 1. The detection timing diagram. The camera exposure period T , determined by the shutter frequency 1/T , consists of the dead time (0, zT ) and the active time (zT, T ). The photon event at time xT relative to the beginning of the exposure period, together with the phosphor decay f (t), determines the intensities Im detected in the frames m, (m ≥ 1). x thus determines the sub-exposure arrival time of a photon event, and can be calculated from the intensities of the photon event in subsequent frames. The exposure time in our case is 4μ s.

where x (x ∈ (0, 1)) determines the photon arrival time, and β is the higher of the two values xT and zT : β = max(xT, zT ). The intensities Im in the subsequent frames (m > 1) are given by: Im =

mT (m−1)T +zT

f (t − xT ) dt = ∑ a j τ j exT /τ j e−((m−1)T +zT )/τ j − e−mT /τ j .

(3)

j

It turns out that due to the thresholding needed to identify the events in the experimental data (see below) the first frame is sometimes missed for events close to the end of the exposure period, because of the very low intensity in that frame. For these events, x has then a small negative value. To account for this effect, one additionally considers the intensity I0 in the frame nominally preceding the first frame (m = 0): I0 =

0 β

f (t − xT ) dt = ∑ a j τ j exT /τ j e−β /τ j − 1 ,

(4)

j

where β = max(−T + zT, xT ) and x < 0. Equations (2), (3) and (4) can be combined into one expression: Im =

β2 β1

f (t − xT ) dt = ∑ a j τ j exT /τ j e−β1 /τ j − e−β2 /τ j ,

(5)

j

where β1 = max((m − 1)T + zT, xT ), β2 = max(mT, xT ), and m ≥ 0. In principle, two intensity values (m = 2) would suffice to determine the photon arrival time. Including more intensity values from the later frames improves the precision of the determination of x. In the present work we use m = 22. The position x of the photon event is determined from the experimental decay dm by minimizing the difference between the theoretical decay Im and the experimental decay dm with


(C) 2010 OSA


respect to x. Assuming Gaussian noise, this leads to minimizing the value of χ 2 defined as follows:

χ 2 = ∑ (dm − AIm )2 ,

(6)

m

where A is a scaling factor accounting for the variations in the phosphor event size. The χ 2 is minimized with respect to both x and A using standard methods, in our case implemented via the in-built fitminbnd() function of Matlab. The procedure described above assumes that it is possible to identify individual events in the images, that is, that there are no spatial overlaps between different events. This requirement sets a limit on the maximum photon density at which events can be separated and timed. Assuming randomly spatially distributed events with a minimum center-to-center distance d of two events that can still be separated by the identification algorithm, the probability that there is another event closer than the distance d to the selected event is determined by Poisson distribution: p = 1 − exp(−ρπ d 2 nT ), where ρ is the spatial event density per unit of time (light intensity, determined by the sample brightness), T is the camera exposure period, and n is the number of frames after which the phosphor decays to the background level. When the tolerable probability of spatial overlap is p (p 1), the maximum event intensity ρ is: p − ln(1 − p) ≈ (7) 2 π d nT π d 2 nT This condition could be further relaxed. While it may not be possible to separate two events spaced by a distance lower than d that arrive within the same frame, they could still be separated if their arrival times are one or more frames apart. A more complex identification algorithm would have to be developed, that inspects the event shapes in all frames along the decay, and takes advantage of the fact that before the arrival of the second event the first event can be precisely localized.

ρ=

3.

Experimental details

We used a Photron camera Fastcam SA1.1 model 675K-M1. The camera operated at 250 kHz frame rate, i.e. had an exposure time of 4 μ s. The image size was 80 × 128 pixels, with a pixel size of 20 μ m, and 256 (8 bit) grey values. The image intensifier is a 40 mm-diameter Photek dual proximity-focused, three-MCP device, operating in photon counting mode. The cathode was at ground potential, 150 V applied between the photocathode and the first MCP, 800 V between the first and the second MCP, 2.35 kV between the second and the third MCP, and 4.5 kV between the third MCP and the P20 phosphor screen, as described previously [30, 43, 44]. The phosphor screen of the intensifier was imaged using a 50 mm focal length Canon photographic lenses (F=1/1.2), and the multiexponential P20 phosphor decay time (to 1/10 of its peak value) is quoted as 250 μ s by the manufacturer. A Hamamatsu PLP-10 470 pulsed diode laser with an optical pulse width of 90 ps at 470 nm was used for illuminating a scattering sample on a inverted microscope (Nikon Eclipse TE2000E). A Nikon 10x, NA=0.30 objective was used to collect the signal which was then detected by the image intensifier. Data processing was done in Matlab (Natic, USA). 4. 4.1.

Results Event identification and phosphor decay extraction

In the experiments described below, the image intensifier was illuminated with low intensity at a frequency of 20 kHz. The camera frame rate was 250 kHz, and the two frequencies were not locked to each other. #133298 - $15.00 USD Received 11 Aug 2010; revised 22 Sep 2010; accepted 23 Sep 2010; published 15 Nov 2010

(C) 2010 OSA


Prior to determining the photon arrival time, the photon event has to be identified in the sequence of image frames, and the intensity decay over a number of frames has to be extracted. The frames were processed sequentially. In every frame, the pixels with intensity above the threshold value (4 counts), lying above the dark count background, were identified. Starting with the brightest pixel, a surrounding area of 3 × 3 pixels, which was found to be the typical photon event size, was used to determine the event intensity in the given frame. The intensity from the same area in several preceding and following frames, typically 3 frames before the photon event and 22 after the photon event, respectively, was extracted and defined the decay of that event. Then, the photon event was marked (so it would not be analyzed again) by essentially setting the brightness values in the pixels corresponding to this event to zero. This procedure was repeated, until all pixels above the threshold value were exhausted.

A

B

I II 0

1

2

3

C

4

5

6

sum

D 50

event I event II

70 60

mean mean ± σ

40

scaled count

count

50 40 30

30

20

20 10 10 0

0 0

5

10

time channel

15

20

0

5

10

15

20

time channel

Fig. 2. Event identification and decay extraction. A: An example of two events arriving in frame 1 as imaged in frames 0 to 6. The first event, in sequence I, reaches the maximum intensity in the first frame that is above the threshold level, while the second event, in sequence II, reaches its maximum in the second frame. B: The sum of frames 0 to 22 of the two events shown in A, together with the centroid position marked by a red cross. C: The intensity of the two events extracted from a 3 × 3 pixel region in every frame. Each time channel corresponds to 4 μ s. D: An average decay of all events in one measurement, together with the standard deviation σ in every data point. The intensities in each set of frames in A and in both images in B have been scaled to use the whole gray scale for better visibility.

Two typical photon events and their decays are shown in Fig. 2. In both cases the threshold #133298 - $15.00 USD Received 11 Aug 2010; revised 22 Sep 2010; accepted 23 Sep 2010; published 15 Nov 2010

(C) 2010 OSA


used to identify the events was first exceeded in the frame numbered 1. Note, that the first event, in sequence I, reaches the maximum intensity in the first frame that is above the threshold level, while the second event, in sequence II, reaches its maximum in the second frame. This is because the first event arrived near the beginning of the exposure period, and the second event towards the end of the exposure of frame 1. In Fig. 2(D) an average of many decays together with the standard deviation of intensity in every time channel (image frame) are shown. Prior to averaging, the decays were scaled using the intensity values in channels starting from channel 2 onwards, to compensate for a different amplification factor for individual photon events by the image intensifier. Note, that while the standard deviations of the intensities in the frames 2 and later are very similar, the standard deviation in the first channel is much larger. The large spread of intensity values in the channel 1 is caused by the variations in the photon arrival time relative to the start camera exposure, and provides the basis for timing the events. 4.2.

Photon Event timing

In order to determine the arrival time of the event x by the procedure described above, the phosphor decay was estimated from the data, and approximated by a sum of three exponentials [Eq. (1)] with the following parameter values: τ1 = 1.32 μ s, τ2 = 7.24 μ s, τ3 = 36.4 μ s, a1 = 0.701, a2 = 0.250, and a3 = 0.049. The phosphor decay was estimated from the data using the following procedure: For an initial estimate, a one- or two-exponential approximation of the phosphor decay f (t) [Eq. (1)] was calculated from the experimental phosphor decay as shown in Fig. 2 by simply fitting the data points to Eq. (1). The values obtained, a j and τ j , were then used to calculate the timing values x for all events, resulting in a plot similar to that in Fig. 2. This revealed that the events can be divided into two groups depending on in which frame they arrived, and also the drift of the x value with the measurement time due to the slight deviation of the frame rate to the laser frequency ratio from 12.5, as discussed below. To further refine the estimation of the phosphor decay, we noted that for a constant ratio of the frame rate to the laser frequency the ideal event timing values x of a whole measurement should lie along lines: (8) x = cn f + b, where n f is the frame number, the gradient c is the same for both groups of events (c1 = c2 ), and the offset b varies by 0.5 between the two groups of events (b2 = b1 − 0.5, where index 1 refers to the photon events plotted in blue in Fig. 3 and index 2 to those plotted in red). The phosphor decay parameters a j and τ j were determined by minimizing the value of χ 2 given by Eq. (9): 2 χ 2 = ∑ ∑ dm,k − Ak Im,k , (9) k m

where the first sum is over all events in one measurement, indexed by k, dm,k are the experimental photon event decays, Im,k were calculated from Eq. (5) with x determined by Eq. (8), where n f are known for each event. The parameters to optimize were c, b, and Ak for each photon event, in addition to a j and τ j . The χ 2 minimization method was the same as with Eq. (6), as described above. The phosphor decay parameters obtained in this way were robust and consistent between different measurements. Using four-exponential fits resulted in insignificant changes in the recovered phosphor decay, and the event timing values x. In Fig. 3, the relative arrival time x within the frame of all events is plotted as a function of the frame number. Since a laser excitation pulse was applied every 12.5 camera frames, there are two relative times x at which the pulses arrive, separated by Δx = 0.5. All events can #133298 - $15.00 USD Received 11 Aug 2010; revised 22 Sep 2010; accepted 23 Sep 2010; published 15 Nov 2010

(C) 2010 OSA


thus be separated into two groups: those arriving within frames which are even multiples of 12.5, and those arriving within frames that are odd multiples of 12.5, plotted as blue and red dots, respectively, in Fig. 3. If the camera frame rate were exactly evenly divisible by the laser frequency, all event arrivals would be characterized by the same value of x.

1

0.8

x

0.6

0.4

0.2

0

−0.2 0

1000 2000 3000 4000 5000 6000 7000 8000

frame number Fig. 3. Timing of photon events with the camera exposure period T = 4 μ s, and the excitation frequency 20 kHz. The events occurring in even and odd frames are displayed in blue and red, respectively. The drift of x vs. the frame number, marked by solid lines, is caused by the ratio between the camera frame rate and the laser trigger not being exactly 12.5. The horizontal line at x = 0.174 marks the end of the dead time.

Further, one can observe that the event arrival time x decreases during the measurement. This is caused by the fact that the exact ratio between the frame rate and laser frequency deviates slightly from 12.5. Using the slope of the dependence x vs. the frame number, it can be calculated as 12.50085. Locking the laser frequency to the camera frame rate would eliminate this drift. We can, however, take advantage of the slight drift between the camera shutter and the laser trigger in our data to estimate the uncertainty of the event timing as a function of x. It can be seen in Fig. 3 that the spread of the arrival times x around the mean times marked by fitted lines is smallest for events that arrived close to the end of the exposure (x → 1), and increases as x decreases. The dead time period (0, zT ) extends in this case within the interval x ∈ (0, 0.174), corresponding to the dead time of zT = 0.174T = 0.7 μs. The timing for events within this period and immediately afterwards, is least precise. It can be seen, that the algorithm preferentially positions the events at the start (z = 0) or the end (z = 0.174) of the dead time period. In spite of the fact that significant information is lost by the arrival within the dead time, a large fraction of events is still timed within this period. For a small fraction of events the arrival time x is slightly less than zero, meaning that the event occurred towards the end of the frame numbered 0. The reason for this effect is, that by setting a threshold for the event identification in the image frames there is a small probability, #133298 - $15.00 USD Received 11 Aug 2010; revised 22 Sep 2010; accepted 23 Sep 2010; published 15 Nov 2010

(C) 2010 OSA


that an event arriving towards the end of the frame will have intensity below this threshold, is therefore ignored in this frame, and is identified only in the following frame. This, however, poses no problem for event timing, since it is only the arrival time relative to the start of the exposure period (mathematically expressed as a remainder of x divided by 1, or x mod 1) that is significant. As the last step of the timing procedure we therefore replace x with x mod 1. 4.3.

Timing uncertainty

For a better estimation of the timing uncertainty, a running standard deviation σx of the arrival time x (Fig. 3, x replaced with x mod 1) over 50 events was calculated after the linear trend of decrease of x with time was subtracted (Fig. 4). The line in Fig. 4 is a fit to the events with x > 0.6 (blue dots) only. This shows that the timing uncertainty decreases approximately linearly as the arrival time x decreases. The timing uncertainty for events arriving within the dead time and shortly after the shutter opening is somewhat worse than indicated by the linear trend. The best timing uncertainty, at around x ≈ 1, approaches a remarkable 1% of the exposure period, that is, 40 ns in our case. 0.18 0.16 0.14 0.12

σx

0.1 0.08 0.06 0.04 0.02 0 0

0.2

0.4

0.6

0.8

1

x Fig. 4. The experimentally determined dependence of the timing uncertainty σx (standard deviation of x) on the position x of the photon event relative to the beginning of the camera exposure period. The line is a fit to the events with x > 0.6.

The timing uncertainty is related to the relative change of I1 (and, to a lesser degree, the relative change of Im , m > 1) with the arrival time x, which depends on both the arrival time x and the phosphor decay shape f (t). For events near x = 1 the value of I1 is small, and a small change in x produces relatively large change in I1 , explaining the high timing uncertainty for late events. On the other hand, for early events a small change of x results in a small relative change in I1 , both because I1 is large, and because the phosphor has decayed to a fraction of its initial intensity by the time the exposure period ends at t = T . For events arriving during the dead time, a fraction of signal is lost, further worsening the timing uncertainty. The variation of amplification in the image intensifier changes only the photon event intensity


(C) 2010 OSA


but does not have a direct effect on the timing uncertainty, since it is only the relative values of Im that determine the arrival time x. 5.

Discussion

We have shown that by exploiting the invariant phosphor decay of a photon counting image intensifier we can time photon arrival well below the exposure time of the camera. We can obtain a time resolution below 15% of the camera exposure period T , in the best case around 1% of the exposure time, i.e. of the order of 40 ns. Our novel approach thus allows ultra-fast time-resolved wide-field imaging with single photon sensitivity. This is not possible with conventional CCD or CMOS cameras alone, because they cannot photon count at 250 kHz. However, in principle sub-exposure time resolution could be achieved, to some degree, by temporal pixel multiplexing, i.e. trading spatial resolution for time resolution [45]. In single point detector TCSPC, it has been shown previously that is possible to measure fluorescence decays with lifetimes shorter than the width of the instrument response function (IRF) [46]. In our case, the width of the instrument response is determined by the event timing uncertainty. This approaches in the best case 40 ns, and luminescence decays of the order of 100 ns could potentially be measured by this method. While we have used 22 frames to demonstrate the basic idea, in practise two successive frame should suffice to obtain a similar result. Since the phosphor decay affects the timing uncertainty, the measurement of decay times within a given range can be optimized by choosing a phosphor with a matching decay characteristics. Moreover, the phosphor decay on the output of the intensifier could be mapped to account for any possible spatially non-uniform temporal responses, to be taken into account for lower uncertainty in the photon timing algorithm. Furthermore, the maximum count rate at one position is limited by the phosphor decay, in our case it is on the order of 20 kHz. However, we note that the frame rate of the ultra-fast Photron SA 1.1 camera can be up to 500,000 frames per second (at a reduced number of pixels), and if a suitable short decay time phosphor, e.g. P46, is used, the local count rate could be significantly higher. This temporal approach is analogous to photon event centroiding to sub-CCD pixel accuracy, which recovers the spatial resolution lost in the detector by amplifying and reading out the photon signal [24, 26, 43, 44]. However, instead of sub-pixel spatial resolution increase, we obtain sub-exposure time photon arrival information here. When measuring an unknown luminescence decay, the camera exposure trigger would ideally be locked to the excitation trigger of the pulsed light source, to prevent the drift of x seen in Fig. 3. The fixed delay x0 T between the excitation and the camera exposure should be adjusted so that most events arrive within that part of the interval (0, T ) where the timing uncertainty is highest. This means that the measured luminescence decay should be positioned towards the end of the exposure period as shown in Fig. 5. The variation of the timing uncertainty σx with x effectively means that the IRF of this method varies with x. This does not, however, pose a major problem for the decay measurements, since the IRF can be determined experimentally for all delays x0 , and the dependence of IRF on x can be implemented into standard fitting algorithms, as known from TCSPC, in a rather straightforward manner. The data presented were acquired with an intensity of 105 photons per second, which is a common count-rate in low-light applications. The corresponding event intensity ρ is 10 photons per pixel per second, resulting in only 2% probability of spatial overlap between two events, according to Eq. (7). In order to allow for higher intensities while keeping the event overlap probability low, the magnification between the image intensifier and the camera could be adjusted to decrease the event size, or the size of the image (in pixels) could be increased.


(C) 2010 OSA


dead time

0

zT

t

x0 T

T

Fig. 5. For the highest timing uncertainty, the optimum delay between the camera shutter and the excitation trigger should be chosen. By adjusting the delay x0 T between the camera shutter and the excitation trigger, the luminescence decay (blue curve) to be measured is positioned towards the end of the exposure period (0, T ) where the timing uncertainty is highest (red line).

The number of pixels per image is limited by the frame rate, and in turn limits the resolution. However, by calculating the center of mass of each event, images with much higher pixel resolution can be constructed [24, 26, 43, 44]. The position of the center of the event on the camera chip can be determined with uncertainty better than 5% of the camera pixel size, depending on exact conditions [47]. Event centroiding is demonstrated in Fig. 2(B), where the center of the two events, determined with a sub-pixel resolution, is marked by a red cross. It should be noted that by this procedure the optical diffraction limit of spatial resolution is not exceeded, since it is the position of the photon event on the camera chip and not that of the single emitter that is determined. Event centroiding recovers the resolution lost due to the amplification and camera pixelization, but it does not overcome the optical diffraction limit of the microscope. Even though the size of one image in the presented data was limited to 80 × 128 pixels, the high frame rate means that enormous amounts of data are produced at high rates: 2.5 GB/s with the 250 kHz frame rate used here. Consequently, relatively short acquisition periods must be followed by longer periods of data transfer between the camera and the storage device. An ideal solution would therefore be to reduce the data volume by performing the first steps of analysis, thresholding, event identification and decay extraction, or even the event timing, in real time, and to store only the result: the event decay or the event arrival time x. The prospective applications of this method include luminescence lifetime imaging on the timescales 100 ns–100 μ s at low light levels. The major advantage in comparison to time-gating techniques is that the available photon budget from the sample is fully utilized, thus maximizing the information that can be obtained from the experiment [48]. In addition, it is straightforward to combine this approach with polarization-resolved lifetime measurements [1, 4, 49], leading to time-resolved luminescence anisotropy imaging. Acknowledgements We would like to thank the UK’s EPSRC Engineering Instrument Loan Pool, particularly Adrian Walker, for the loan of the Photron camera and the EPSRC Life Science Interface programme for funding.


(C) 2010 OSA


It has been brought to our attention that in the field of ion velocity map imaging, a similar principle based on a double exposure intensified CCD has been used to image the decay of a P46 phosphor at a repetition rate of 25 Hz: L. Dinu, A. T. J. B. Eppink, F. Rosca-Pruna, H. L. Offerhaus, W. J. van der Zande, and M. J. J. Vrakking, “Application of a time-resolved event counting technique in velocity map imaging,” Rev. Sci. Instrum. 73, 4206–4213 (2002).


(C) 2010 OSA
