Super-multiplexed fluorescence microscopy via ... - OSA Publishing

Vol. 9, No. 7 | 1 Jul 2018 | BIOMEDICAL OPTICS EXPRESS 2943

Super-multiplexed fluorescence microscopy via photostability contrast ANTONY ORTH,1,* RICHIK N. GHOSH,2 EMMA R. WILSON,1 TIMOTHY DOUGHNEY,1,3 HANNAH BROWN,4 PHILIPP REINECK,1 JEREMY G. THOMPSON,4 AND BRANT C. GIBSON1 1

ARC Centre of Excellence for Nanoscale BioPhotonics, School of Science, RMIT University, Melbourne, VIC 3001, Australia 2 Thermo Fisher Scientific, 100 Technology Drive, Pittsburgh, PA 15219, USA 3 Defence Science and Technology Group, Cyber and Electronic Warfare Division, Edinburgh, SA 5111, Australia 4 ARC Centre of Excellence for Nanoscale BioPhotonics, Robinson Research Institute, Institute for Photonics and Sensing, Adelaide Medical School, The University of Adelaide, Adelaide, SA 5005, Australia *[email protected]

Abstract: Fluorescence microscopy is widely used to observe and quantify the inner workings of the cell. Traditionally, multiple types of cellular structures or biomolecules are visualized simultaneously in a sample by using spectrally distinct fluorescent labels. The wide emission spectra of most fluorophores limits spectral multiplexing to four or five labels in a standard fluorescence microscope. Further multiplexing requires another dimension of contrast. Here, we show that photostability differences can be used to distinguish between fluorescent labels. By combining photobleaching characteristics with a novel unmixing algorithm, we resolve up to three fluorescent labels in a single spectral channel and unmix fluorescent labels with nearly identical emission spectra. We apply our technique to organic dyes, autofluorescent biomolecules and fluorescent proteins. Our approach has the potential to triple the multiplexing capabilities of any digital widefield or confocal fluorescence microscope with no additional hardware, making it readily accessible to a wide range of researchers. © 2018 Optical Society of America under the terms of the OSA Open Access Publishing Agreement OCIS codes: (170.2520) Fluorescence microscopy; (110.4234) Multispectral and hyperspectral imaging.

References and links 1. 2.

J. Pawley, Handbook of Biological Confocal Microscopy (Springer, 2006). A. Orth and K. B. Crozier, “High throughput multichannel fluorescence microscopy with microlens arrays,” Opt. Express 22(15), 18101–18112 (2014). 3. B. Kraus, M. Ziegler, and H. Wolff, “Linear fluorescence unmixing in cell biological research,” Mod. Res. Educ. Top. Microsc. 2, 863–873 (2007). 4. A. Orth, M. J. Tomaszewski, R. N. Ghosh, and E. Schonbrun, “Gigapixel multispectral microscopy,” Optica 2(7), 654–662 (2015). 5. L. Gao, R. T. Kester, N. Hagen, and T. S. Tkaczyk, “Snapshot Image Mapping Spectrometer (IMS) with high sampling density for hyperspectral microscopy,” Opt. Express 18(14), 14330–14344 (2010). 6. P. Favreau, C. Hernandez, A. S. Lindsey, D. F. Alvarez, T. Rich, P. Prabhat, and S. J. Leavesley, “Thin-film tunable filters for hyperspectral fluorescence microscopy,” J. Biomed. Opt. 19(1), 011017 (2013). 7. L. Song, E. J. Hennink, I. T. Young, and H. J. Tanke, “Photobleaching kinetics of fluorescein in quantitative fluorescence microscopy,” Biophys. J. 68(6), 2588–2600 (1995). 8. P. Reineck, A. Francis, A. Orth, D. W. M. Lau, R. D. V. Nixon-Luke, I. D. Rastogi, W. A. W. Razali, N. M. Cordina, L. M. Parker, V. K. A. Sreenivasan, L. J. Brown, and B. C. Gibson, “Brightness and Photostability of Emerging Red and Near-IR Fluorescent Nanomaterials for Bioimaging,” Adv. Opt. Mater. 4(10), 1549–1557 (2016). 9. L. Gao, A. Garcia-Uribe, Y. Liu, C. Li, and L. V. Wang, “Photobleaching imprinting microscopy: seeing clearer and deeper,” J. Cell Sci. 127(2), 288–294 (2014). 10. J. Yao, L. Wang, C. Li, C. Zhang, and L. V. Wang, “Photoimprint photoacoustic microscopy for threedimensional label-free subdiffraction imaging,” Phys. Rev. Lett. 112(1), 014302 (2014).

#331803 Journal © 2018

https://doi.org/10.1364/BOE.9.002943 Received 16 May 2018; revised 24 May 2018; accepted 26 May 2018; published 6 Jun 2018


11. D. T. Burnette, P. Sengupta, Y. Dai, J. Lippincott-Schwartz, and B. Kachar, “Bleaching/blinking assisted localization microscopy for superresolution imaging using standard fluorescent molecules,” Proc. Natl. Acad. Sci. U.S.A. 108(52), 21081–21086 (2011). 12. M. J. Rust, M. Bates, and X. Zhuang, “Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy (STORM),” Nat. Methods 3(10), 793–796 (2006). 13. S. T. Hess, T. P. Girirajan, and M. D. Mason, “Ultra-high resolution imaging by fluorescence photoactivation localization microscopy,” Biophys. J. 91(11), 4258–4272 (2006). 14. K. Braeckmans, L. Peeters, N. N. Sanders, S. C. De Smedt, and J. Demeester, “Three-dimensional fluorescence recovery after photobleaching with the confocal scanning laser microscope,” Biophys. J. 85(4), 2240–2252 (2003). 15. B. L. Sprague, R. L. Pego, D. A. Stavreva, and J. G. McNally, “Analysis of Binding Reactions by Fluorescence Recovery after Photobleaching,” Biophys. J. 86(6), 3473–3495 (2004). 16. G. J. Brakenhoff, K. Visscher, and E. J. Gijsbers, “Fluorescence bleach rate imaging,” J. Microsc. 175(2), 154– 161 (1994). 17. R. I. Ghauharali, R. Van Driel, and G. J. Brakenhoff, “Structure-orientated fluorescence photobleaching analysis: a method for double fluorescent labelling studies,” J. Microsc. 185(3), 375–384 (1997). 18. D. Rönnlund, L. Xu, A. Perols, A. K. B. Gad, A. Eriksson Karlström, G. Auer, and J. Widengren, “Multicolor Fluorescence Nanoscopy by Photobleaching: Concept, Verification, and Its Application To Resolve Selective Storage of Proteins in Platelets,” ACS Nano 8(5), 4358–4365 (2014). 19. C. Lawson and R. Hanson, Solving Least Squares Problems, Classics in Applied Mathematics (Society for Industrial and Applied Mathematics, 1995). 20. M. W. Berry, M. Browne, A. N. Langville, V. P. Pauca, and R. J. Plemmons, “Algorithms and applications for approximate nonnegative matrix factorization,” Comput. Stat. Data Anal. 52(1), 155–173 (2007). 21. M. Guizar-Sicairos, S. T. Thurman, and J. R. Fienup, “Efficient subpixel image registration algorithms,” Opt. Lett. 33(2), 156–158 (2008). 22. C. J. G. Bakker and J. Vriend, “Multi-exponential water proton spin-lattice relaxation in biological tissues and its implications for quantitative NMR imaging,” Phys. Med. Biol. 29(5), 509–518 (1984). 23. Y. Lu, J. Lu, J. Zhao, J. Cusido, F. M. Raymo, J. Yuan, S. Yang, R. C. Leif, Y. Huo, J. A. Piper, J. Paul Robinson, E. M. Goldys, and D. Jin, “On-the-fly decoding luminescence lifetimes in the microsecond region for lanthanide-encoded suspension arrays,” Nat. Commun. 5(1), 3741 (2014). 24. T. Niehörster, A. Löschberger, I. Gregor, B. Krämer, H.-J. Rahn, M. Patting, F. Koberling, J. Enderlein, and M. Sauer, “Multi-target spectrally resolved fluorescence lifetime imaging microscopy,” Nat. Methods 13(3), 257– 262 (2016). 25. P. H. R. Orth, W. R. Falk, and G. Jones, “Use of the maximum likelihood technique, for fitting counting distributions,” Nucl. Instrum. Methods 65(3), 301–306 (1968). 26. A. Grinvald and I. Z. Steinberg, “On the analysis of fluorescence decay kinetics by the method of least-squares,” Anal. Biochem. 59(2), 583–598 (1974). 27. I. Isenberg and R. D. Dyson, “The Analysis of Fluorescence Decay by a Method of Moments,” Biophys. J. 9(11), 1337–1350 (1969). 28. D. G. Gardner, J. C. Gardner, G. Laush, and W. W. Meinke, “Method for the analysis of multicomponent exponential decay curves,” J. Chem. Phys. 31(4), 978–986 (1959). 29. F. Chen, P. W. Tillberg, and E. S. Boyden, “Optical imaging. Expansion microscopy,” Science 347(6221), 543– 548 (2015). 30. P. W. Tillberg, F. Chen, K. D. Piatkevich, Y. Zhao, C. C. Yu, B. P. English, L. Gao, A. Martorell, H. J. Suk, F. Yoshida, E. M. DeGennaro, D. H. Roossien, G. Gong, U. Seneviratne, S. R. Tannenbaum, R. Desimone, D. Cai, and E. S. Boyden, “Protein-retention expansion microscopy of cells and tissues labeled using standard fluorescent proteins and antibodies,” Nat. Biotechnol. 34(9), 987–992 (2016). 31. G. H. Patterson and D. W. Piston, “Photobleaching in Two-Photon Excitation Microscopy,” Biophys. J. 78(4), 2159–2162 (2000). 32. L. Wei, Z. Chen, L. Shi, R. Long, A. V. Anzalone, L. Zhang, F. Hu, R. Yuste, V. W. Cornish, and W. Min, “Super-multiplex vibrational imaging,” Nature 544(7651), 465–470 (2017).

1. Introduction Fluorescence microscopy is widely used in biological research for high contrast imaging with unrivalled specificity [1]. In a typical assay, 2-4 cellular targets are each labeled by fluorescent species with distinct spectral emission [2]. Combinations of different excitation sources and emission filters are then used to provide spectral contrast between the fluorescent species. In practice, there is inevitable mixing between fluorescent channels because fluorophores are excited and emit over a finite bandwidth. Fluorescent emission from a given fluorescent species contributes to the signal in multiple emission channels. This spectral cross-talk can be significant when using more than 3 fluorescent probes, necessitating unmixing algorithms for successful separation [3,4]. Generally, spectral unmixing procedures


work well, however, one cannot isolate more fluorescent probes than one has spectral channels. High-end filter-based fluorescence microscopes have at most five excitation/emission filter combinations and can therefore spectrally separate up to 5 fluorescent species. Alternatively, spectrometer-based imaging systems can in principle acquire dozens of spectral channels, at the added cost of a large filter set or spectral imaging module. Unfortunately, even with a large number of spectral channels it is extremely challenging to identify more than 4-5 fluorescent species in a sample, regardless of the spectral measurement scheme [5,6]. This is because of the wide spectral width of each fluorophore (~40-50 nm with broad tails), the need for excitation windows (~20 nm minimum per excitation source) and the finite spectral bandwidth of the visible spectrum (300 nm). However, in many areas of biology there is a growing need to separate more objects or structures or to perform several fluorescence-based sensing experiments simultaneously to increase the information content derived from fluorescence microscopy assays. Here we introduce a method for expanding the number of fluorescent species that can be simultaneously multiplexed in an image, without resorting to large filter sets or spectral detectors. We show that one may discriminate between fluorescent species by leveraging a photophysical process inherent in all organic fluorophores – photobleaching. Instead of using spectral signatures to distinguish fluorescent species, we use their photobleaching behaviour as an identifying property. This technique, which we call bleaching-assisted multichannel microscopy (BAMM), can be applied either by itself or in conjunction with spectral filters and is even capable of discriminating between fluorophores with nearly identical emission spectra. Fluorescence photobleaching refers to a decrease in emission intensity of a fluorescent sample over time under illumination. This decrease is the result of chemical reactions between optically excited fluorescent molecules and the surrounding medium [7]. Bleached fluorophores are irreversibly “turned off” and are no longer able to emit light. The emission intensity (I) of an ensemble of fluorophores decreases exponentially over time ( t ) according to I ( t ) = I ( 0 ) e − kt . The bleaching constant k depends on a myriad of experimental and environmental parameters in addition to the electronic structure of the fluorophore itself [1]. For example, excitation power, excitation wavelength, oxygen concentration, and a fluorophore’s energy level structure all affect its bleaching rate [7,8]. This means that spectrally identical fluorophores can have very different bleaching rates. A variety of microscopy techniques rely on photobleaching. For example, photo-imprint microscopy can increase resolution beyond the diffraction limit in both in-plane and axial dimensions [9,10]. Another super-resolution method called bleaching/blinking assisted localization microscopy (BALM) uses discrete photobleaching events to localize single molecules [11]. The resulting images are similar to that created by stochastic optical reconstruction microscopy (STORM) or photoactivated localization microscopy (PALM) [12,13], but sample preparation is greatly simplified because of the universal occurrence of fluorescence photobleaching. Yet another technique, fluorescence recovery after photobleaching (FRAP), has become a standard tool for investigating diffusion kinetics in living cells [14,15]. In the mid-1990s, photostability was briefly investigated as a contrast mechanism for multi-probe fluorescence microscopy [16,17], but suffered due to a lack of sophisticated unmixing algorithms and inadequate computing power. Though multicolor photobleachingenabled super resolution imaging has recently seen interest [18], these photobleaching-based approaches have yet to be generalized and combined with modern unmixing techniques. We take advantage of modern developments in non-negative unmixing approaches [19,20], and introduce a specialized non-negative matrix factorization algorithm for photobleaching data. Moreover, we show that multiplexing can be further increased by combining spectral and photostability dimensions. Our work establishes photostability as a bona fide optical


dimension for extended multiplexing without specialized sample preparation or additional microscope hardware. 2. Methods 2.1. Timelapse acquisition The first step in BAMM is to record a timelapse of sample bleaching. The sample is repeatedly imaged with one or multiple lasers (or light emitting diodes, LEDs), causing the sample to fade. We record a timelapse movie, followed by background subtraction and drift compensation pre-processing steps [21]. We use unmodified commercial confocal and widefield fluorescence microscopes (confocal: Nikon AR1, Olympus FV1200; widefield: Thermo Fisher Scientific CellInsight CX7 High Content Analysis (HCA) platform) for timelapse acquisition. Sample-specific imaging parameters and filter combinations are given in the Results section. 2.2. Unmixing 2.2.1 Non-negative least-squares Assuming that several fluorescent species are present in a sample, they can all potentially contribute to the signal collected in a given pixel. In principle, determining the relative abundances of these fluorophore species via bleaching can be achieved by fitting the amplitudes of a multi-exponential decay at each pixel. This basic problem occurs in many arenas from magnetic resonance imaging [22] to fluorescence lifetime imaging [23,24] and nuclear physics [25]. There are a wide range of computational approaches to this challenge, such as maximum likelihood estimation [25], the method of least-squares [26], method of moments [27] and the Gardner Transform [28]. However, our problem is more general as it can include both spectral and bleaching information. Accordingly, we extract the spectral and photobleaching characteristics from the data set itself. This self-calibrated approach avoids using physical models of photobleaching that may not be consistent with real world samples. We will refer to the spectral-bleaching characteristic of a fluorophore as its spectral-bleaching fingerprint. An analogous quantity has recently been used to unmix fluorescent probes based on their fluorescence lifetime and emission spectra [24]. Though the combination of spectral and photostability information is the most general form of our technique, we note that we can also use photostability information by itself, as will be seen in the Results section. To separate pixel-by-pixel contributions of each fluorescent label, we use the MATLAB non-negative least-squares (NNLS) function lsqnonneg to solve the relevant linear unmixing problem [19]: N

I k ( x, y ) = ai ( x, y )vik , k = 1, 2, …, T

(1)

i =1

where I k ( x, y ) is the (measured) intensity at pixel ( x, y ) for frame index k (spectrally concatenated if spectral information is included), ai ( x, y ) ≥ 0 is the relative scalar abundance of fluorophore species i at pixel ( x, y ) and vik is the k th entry of the T-element (spectral-) bleaching fingerprint for fluorophore type i . Note that the system is overdetermined when T > N (N is the number of fluorescent labels in the sample). The extra information in this overdetermined system is crucial for noise suppression. The non-negativity prior for ai prevents the unmixed abundances from reaching nonphysical negative values and improves unmixing fidelity. The i th image - the abundance map of fluorophore type i - is given by ai ( x, y ) and ideally contains only signal from the i th fluorescent species.


2.2.2 Non-negative matrix factorization If there are regions of the sample where each fluorescent probe exists in isolation, then the (spectral-) bleaching fingerprints vik can be manually identified. However, for many samples, this is not the case. For such situations (typically cellular samples), the bleaching fingerprint of each fluorophore along with their pixel-by-pixel abundances, are simultaneously estimated by nonnegative matrix factorization (NMF) [20]. This algorithm attempts to find non-negative bleaching traces and fluorophore abundances that solve the mixing problem of Eq. (1) in the least squared sense, using the alternating least squares (ALS) procedure [20]. The nonnegativity constraint restricts possible solutions for the bleaching traces and the abundance maps to those with only positive values. As with NNLS above, this guarantees that the result is consistent with the fact that the intensity is a non-negative quantity. We perform NMF by using the built-in MATLAB non-negative matrix factorization function nnmf. We supply the nnmf function with the principal components of photobleaching curves as the initial estimates of the bleaching characteristics. For added robustness, we take the optimal solution out of three replicates – the first seeded with principal components as initial solutions and the following two with random initial guesses. Typical results required no more than 25 iterations. 2.2.3. Non-increasing non-negative matrix factorization For highly multiplexed samples, the NMF approach above may converge to unphysical solutions where bleaching traces increase in intensity over time, yielding incorrect unmixing results. This can be mitigated by imposing additional restrictions to the solution space. To this end, we implement a modification to the standard ALS NMF procedure, which we call nonincreasing NMF (NI-NMF). At each iteration, if the value of a bleaching trace estimate B (t ) at time t = tn +1 exceeds the value of the bleaching trace estimate at time tn , then we set

B ( tn +1 ) = B (tn ) . This is modification is performed for each time point in succession so that

the value of the bleaching trace is monotonically decreasing. We run NI-NMF for 25 replicates, with the first one being seeded with the principal components, and select the result with the lowest mean-squared error. For all NI-NMF and NMF procedures it was found to be beneficial to exclude dim pixels to reduce the influence of noise and decrease computation time. We typically exclude all pixels with that are dimmer than 1-10% of the brightest pixel in the data set. Once the estimate for the bleaching curves is found, the least squares solution for the abundances at every pixel in the image is found by using the timelapse data and the estimated bleaching curves together with the MATLAB backslash operator (a QR solver), and then setting negative abundances to zero. This is equivalent to including all pixels on the final iteration of the ALS (NI-) NMF algorithm. 3. Results 3.1. Working principle In this section, we demonstrate the working principle of BAMM using a model system consisting of five different types of fluorescent beads. The beads have peak emission wavelengths ranging from 500 - 700 nm, and are imaged simultaneously in yellow (570 - 620 nm) and red (663 - 738 nm) spectral channels using a confocal microscope (Fig. 1(a)). Bead types I-IV are Spherotech “Sky Blue”, “Blue”, “Purple”, and “Yellow”, respectively. Bead type V are “Chromeon 642” beads from Sigma Aldrich. Bead types I-IV are all > 2μm in diameter and are therefore easily resolvable by the confocal microscope. However, bead type V is only 80nm in diameter, and therefore appears as a dim amorphous red background since the beads themselves are spatially unresolvable.


The bleaching timelapse for this sample was acquired using a Nikon AR1 confocal microscope, equipped with continuous wave (CW) excitation lasers at λ = 405, 488, 561 and 640 nm. Images were acquired using a 40x, 0.95 NA microscope objective, with a pixel dwell time of 1 μs and 2x line averaging. Figure 1(a) shows the first frame of the bleaching timelapse. This image is color coded with yellow and red channels corresponding to emission windows 570-620 nm and 663-738 nm, respectively. All emission channels are recorded simultaneously, and all 4 lasers excite the sample simultaneously with laser powers of (0.20, 1.10, 1.20, 0.09) mW for (405, 488, 561, 640) nm lasers, respectively. The confocal pinhole was set to 80% of the diffraction limit at 640 nm.

Fig. 1. BAMM with beads. a) The first frame of a 30-frame photobleaching experiment, with each frame consisting of a yellow/red dual spectral channel image of a mixture of 5 different fluorescent beads. Examples of different bead types are boxed and labeled as I-V. The red channel is gamma corrected to enhance dim pixels (for display only). Boxes II, IV and V have increased brightness for visibility. Scale bar is 50 μm. b) Time traces of beads I-V in (a), during the photobleaching experiment. Concatenated frames # 1-30 correspond to frames 1-30 in the yellow channel (570-620 nm) of image (a). Concatenated frames #31-60 correspond to frames 1-30 from the red channel (663-738 nm) of image (a). Each bead type has a unique spectral-bleaching “fingerprint”. c) False-coloured unmixed image of all five bead types. Abundance maps for bead types I-V are coloured red, yellow, purple, green and blue, respectively. d) The cross-talk matrix of the unmixing process. Cross-talk is generally low across all bead types except for bead type II into channel 1 (bead type I). This is due to the similarity in their bleaching rates, and low signal from type II beads (see (a) and (b)). For all other bead types, more than 97% of the bead signal is unmixed into the correct channel.


To supply the spectral-bleaching fingerprints of each bead, we manually identify 5 pixels in Fig. 1(a), each of which contains signal from only one of each of the fluorophore types in the sample. At each pixel, we have an associated 30-frame bleaching curve in each of the two emission channels, for a total of 60 data points across spectral and temporal dimensions. The photobleaching traces for the two spectral channels are concatenated into the 60-frame spectral-bleaching fingerprint curves shown in Fig. 1(b): concatenated frames 1-30 originate from the yellow spectral channel, concatenated frames 31-60 are from the red spectral channel. The abundance maps are subsequently obtained as described in Methods – Unmixing – Non-negative least-squares. The result is a 5-channel image consisting of 5 independent fluorophore abundance maps (Fig. 1(c)), each of which indicates the relative concentration of given fluorescent species. Abundance maps for bead types I-V are colored red, yellow, purple, green and blue, respectively. Cross-talk between abundance maps is shown in Fig. 1(d). Unmixing fidelity is generally high (