A Short Course on Bioimage Informatics
Lecture 2: Tracking and Analysis of Spatiotemporal Cell Dynamics Ge Yang (杨戈) Department of Biomedical Engineering & Department of Computational Biology Carnegie Mellon University
August 12, 2015
1
Outline • Background • Image registration and related techniques • Single particle detection • Single particle tracking • Computational analysis of particle behavior • Summary
2
• Background • Image registration and related techniques • Single particle detection • Single particle tracking • Computational analysis of particle behavior • Summary
3
Tracking & Analysis of Cell Dynamics: Work Flow
4
A Simple Classification of Biological Images
Single Particle Images Continuous Region Images
5
Airy Disk • Airy (after George Biddell Airy) disk is the diffraction pattern of a point feature under a circular aperture. • It has the following form
2J1 x y x
2
J1(x) is a Bessel function of the first kind.
• The PSF is the smallest image feature. 6
A More Realistic Example of Airy Disks
7
Single Particle Images: Example I
8
Axonal Transport in Drosophila Larvae
9
Axonal Transport in Drosophila Larvae
Bill Saxton, UC Santa Cruz
axonal transport of human APP-YFP vesicles 10 frames/sec
10 m
10
Single Particle Images: Example II
11
What is a single particle image? • Image features are particles, which cannot be differentiated from an Airy disk. • Each particle can be fully characterized by its position and intensity. • Here “single particle” means “individual particle”, not “one particle”.
12
Continuous Region Images: Example I
Rho GTPase sensor in mouse embryonic fibroblast, Machacek et al, Nature, 2009
13
Continuous Region Images: Example II
14
What is a continuous region image? • Image features are regions, which can be differentiated from an Airy disk. • To represent a region, we need to characterize its position, shape (i.e. boundary) as well as internal intensity distribution. • Some images can have both particle and region features.
15
• Background • Image registration and related techniques • Single particle detection • Single particle tracking • Computational analysis of particle behavior • Summary
16
Example I: Global Image Registration
http://www.cs.cmu.edu/~kangli/code/Image_Stabilizer.html
17
Example II: Local Image Registration
Raw
Aligned Cropped Yang et al, Journal of Cell Biology, 182:631-639, 2008.
18
Example III: Local Image Registration
19
Concepts of Image Registration/Alignment • "Image registration" and "image alignment" are often used interchangeably. • The main goal of image registration is to establish pixel-bypixel correspondence between two images. I x, y J u , v
x, y u , v
• Another way to look at registering image I(x, y) to J(x,y) is to find a transformation T(·) such that the transformed image T(I(x,y)) or its features best matches J(x,y) or its features. - What is the transformation? - What is the meaning of "best match"? 20
Classification of Image Transformations (I) • Transformation - rigid - affine - projective - curved • Domain of transformation - Local - Global
21
Classification of Image Transformations (II) • Rigid transformation Preferred, because no artificial movement is introduced in image alignment - only translation and rotation are used. - distances are preserved • Affine transformation - parallel lines are mapped into parallel lines • Projective transformation - lines are mapped into lines • Curved (elastic) - lines are mapped into curves 22
Image Registration in Bioimage Analysis • Image registration is particularly important to the analysis of live cell imaging data because cells often move. • Image registration is often the first step in bioimage analysis. • Common sources of sample drift: - Sample movement - Thermo drift of microscope stages in long-term imaging
• Other important applications include the alignment of light microscopy and electron microscopy data. 23
References [1] B. Zitova & J. Flusser, Image registration methods: a survey, Image and vision computing, vol. 21, pp. 977-1000, 2003. [2] J. B. A. Maintz & M. A. Viergever, A survey of medical image registration, Medical Image Analysis, vol. 21, pp. 1-36, 1998. [3] S. Baker & I. Matthews, Lucas-Kanade 20 years on, International Journal of Computer Vision, vol. 56, pp. 221-255, 2004.
24
• Background • Image registration and related techniques • Single particle detection • Single particle tracking • Computational analysis of particle behavior • Summary
25
Pixel Resolution Detection
26
Point Feature Detection • What information is extracted from feature detection: - point position: sub-pixel resolutions are often required. - point intensity: may contain information about the number of molecules within the diffraction limit.
• The main purpose of point detection, and bioimage analysis in general, is to get accurate and precise measurements.
27
What is a Particle? • ONE perspective: A point/particle is a local intensity maximum whose level is substantially higher than its local background intensity level.
28
Basic Work Flow of Particle Detection Raw image
Low pass filter for noise suppression
Local maximum Imax search
Local background IBG search
Is Imax significantly higher than IBG
NO
YES Record a detected particle
29
Basic Concept of Image Filtering
Gonzalez & Woods, DIP 3/e
30
Image Preprocessing: Image Filtering • Image filtering for noise suppression
original
noise added
σ=2
σ=5
σ=1
σ=10 31
Step 1: Low Pass Filter (I) • The Fourier transform of a Gaussian kernel is Gaussian. 2
x 2 2
σ LPF
2 2
1 e 2 e 2 2 • Impact of selection
F
- A small allows weaker features to be picked up but at the expense of more false positives. - A large selects strong features but at the expense of more true positives. 32
Step 1: Low Pass Filter (II) • Impact of selection
- Applying a that is too large will cause substantial shifting and merging of features.
- Applying a that is too small can not effectively suppress noise.
• Using a small is usually preferred. • A commonly used strategy of selecting is to set it to be the Rayleigh limit. 0.61 3 NA
A. Witkin, Scale-space filtering, ICASSP 1984.
33
Step 2: Local Maximum Detection •
A local maxima has an intensity that is no smaller than those of its neighbors.
•
Large masks give more stable results but lower detection resolution. 3X3 mask 5X5 mask
34
Step 3: Local Background Detection • A local minima has an intensity level that is no higher than those of its neighbors. • Local background is detected through detection of local intensity minima.
3X3 mask
5X5 mask
35
Delaunay Triangulation
Delaunay triangulation
Voronoi diagram
S. Arya, D.M. Mount, N.S.Netanyahu, R. Silverman and A. Wu, An optimal algorithm for approximate nearest neighbor searching, Journal of the ACM, 45(6):891-923, 1998. http://en.wikipedia.org/wiki/Delaunay_triangulation
36
Step 3: Establishing Corresponding Between Local Maxima and Local Minima • Different approaches can be used to establish correspondence between local maxima and local minima.
Local intensity maxima Local intensity minima
- Nearest neighbor - Delaunay triangulation
37
Step 4: Statistical Selection of Features Intensity
ΔI; σΔI
I max I BG Q I ? Q: selection quantile
38
Feature Intensity Measurement • Intensity calculation with background subtraction
I net
1 I max N
Local intensity maxima Local intensity minima
N
i I BG i 1
N: number of local minima used to calculate background Inet: net intensity
39
References • A. Ponti et al, Computational analysis of F-actin turnover in cortical actin meshworks using fluorescent speckle microscopy, Biophysical Journal, 84:3336-3352, 2003. • Moore et al, Introduction to the practice of statistics, 6th ed., W. H. Freeman, 2009.
40
Particle Detection Demo
41
Sub-Pixel Resolution Detection
42
A Simple Example
43
Pixel Resolution Limit in Point Detection
(2, 2)
(2, 2)
44
45
The Gaussian Kernel • Gaussian kernel in 1D & 2D G x ;
G x, y; x ,
1 e 2 y
x2 2 2
1 2 x y
e
x2 y2 2 x 2 2 y 2
• A Gaussian kernel provides a good approximation of an Airy disk.
46
Sub-pixel Detection by Gaussian Fit • Fit using a Gaussian kernel, which represents the ideal image of a point x x0 2 y y0 2 K x, y; x0 , y0 K x x0 , y y0 A exp B
• Problem formulation: to minimize the difference between the translated kernel and the image min I x, y K x, y; x0 , y0
x0 ,y0 R 2
47
Gaussian Fitting Implementation Details • How to set the Gaussian kernel - By fitting the Airy Disk using a Gaussian
0.61 /3 NA
- By measuring PSF (often using beads) then fitting with a Gaussian
• Spatial sampling: three-times oversampling of Airy disk Airy disk radius 3 pixel size
• Under high SNR, spatial sampling may be relaxed to 2~2.5. 48
Stochastic Optical Reconstruction Microscopy (STORM)
Huang et al, Three-Dimensional Super-Resolution Imaging by Stochastic Optical Reconstruction Microscopy, Science, 319:810813, 2008
M. J. Rust et al, Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy (STORM). Nature Methods, 10:793-795, 2006.
• Background • Image registration and related techniques • Single particle detection • Single particle tracking • Computational analysis of particle behavior • Summary
50
Single Particle Images: Example II
51
Basic Concepts of Single Particle Tracking (I) • The goal is to fully recover the trajectory of each point feature, i.e. to determine the position of each point in each frame in which it exists. For particle k , its trajectory is the sequence of its position coordinates in each frame within its total lifetime of N , i.e.
x , y , x 1 k
1 k
2 k
, yk2 , xkN , ykN
If we take into account particle intensity, it would be
x , y ; I , x 1 k
1 k
1 k
2 k
, yk2 ; I k2 , xkN , ykN ; I kN
52
Basic Concepts of Single Particle Tracking (II) • Different cases - Constant number of features - Feature appearance - Feature disappearance
• Cases of feature appearance & disappearance - Moving in or out of field of view - Moving in or out of the focal plane - Assembly/disassembly - Feature merging/splitting
53
A Simple Example (I)
Frame i-1 Frame i Frame i+1 54
A Simple Example (II)
Frame i-1 Frame i Frame i+1 55
A Simple Example (III)
Accumulated evidence from multiple frames makes tracking more reliable.
Frame i-1 Frame i Frame i+1 56
Discussion: Different Tracking Strategies •
Strategy I: If the point correspondence between each pair of frames can be determined, the point correspondence over the entire image sequence is defined. - Advantages: relatively simple to implement - Disadvantages: a greedy approach, inadequate information to make a decision.
•
Strategy II: to establish point correspondence based on information from multiple frames. - Advantages: decision making is more reliable. - Disadvantages: computationally intractable in most cases.
• Solution: to find a solution in between strategy I and II 57
Linear Assignment Based Particle Tracking • Formulation of the tracking problem as a bipartite graph assignment min
a k i, j wk i, j
iGk jGk 1
st.
a i, j 1 a i, j 1 i
a i, j 0 ,1
j
• There are efficient numerical algorithms to solve large scale assignment problems. • Why not use a tripartite graph? - Optimal assignment of tripartite graph is NP-complete. - Difficult to resolve conflicts between two tripartite assignments.
58
Commonly Used Assignment Weights • Distance Nearest neighbor
c k i, j x kj 1 xik • Smooth motion Smooth motion k k 1 k 1 k xik xlk 1 x kj 1 xik x x x x i l j i w2 1 2 k c k i, j w1 1 k k 1 k 1 k xi xl x j xi xi xlk 1 x kj 1 xik
• Mahalanobis distance, where the prediction comes from typically a Kalman filter c i, j x ˆxik S xik k
k i
T
1
x
k i
ˆxik
59
Summary of Single Particle Tracking Techniques (I) • Method I: simple nearest neighbor tracking - Each particle is assigned to its nearest neighbor - Often a search radius is adopted - Rarely used; applicable only when particles are well separated
• Method 2: global nearest neighbor tracking - Association cost between a pair of particles is their distance
wk i, j x kj 1 xik 60
Single Particle Tracking: Cells
Courtesy of Lee Weiss & Takeo Kanade 61
References on Linear Assignment •
Schrijver A., Combinatorial optimization, vol. A, Chapter 17: Weighted bipartite matching and the assignment problem, pp.285292, Springer, 2003.
•
Burkard R., Amico M. D., Martello S., Assignment problems, SIAM, 2009.
•
Burkard R., Cela E., Linear assignment problems and extensions, pp.75-149, in Handbook of Combinatorial Optimization, D.-Z. Du & P. M. Pardalos (Eds.), Kluwer Academic Publishers, 1999. (Downloadable from http://ccdl.compbio.cmu.edu/BME42_731/Burkard_LAP_review.pdf).
62
Summary of Single Particle Tracking Techniques (II) • Method 3: global smooth motion tracking - Association cost between a pair of particles is their motion smoothness
63
• Background • Image registration and related techniques • Single particle detection • Single particle tracking • Computational analysis of particle behavior • Summary
64
Application of the Microscopic Theory (I) Object
Distance diffused 1 μm
100 μm
1 cm
1m
K+
0.25ms
2.5s
2.5104s (7 hrs)
2.5108s (8 yrs)
Protein
5ms
50s
5.0105s (6 days)
5.0109s (150 yrs)
Organelle
1s
104s (3 hrs)
108s (3 yrs)
1012s (31710 yers)
K+: Radius = 0.1nm, viscosity = 1mPa·s-1; T = 25°C; D=2000 μm2/sec Protein: Radius = 3nm, viscosity = 0.6915mPa·s-1; T = 37; D = 100 μm2/sec Organelle: Radis = 500nm, viscosity = 0.8904mPa·s-1; T = 25°C; D = 0.5 μm2/sec Jonathon Howard, Mechanics of motor proteins and the cytoskeleton, Sinauer, 2001
65
Application of the Microscopic Theory (II) Mean square displacement <X2(t)>
Diffusion with external flow Pure diffusion
Diffusion in a cage
t H. Qian, M. P. Sheetz, E. L. Elson, Single particle tracking: analysis of diffusion and flow in two-dimensional systems, Biophysical Journal, 60(4):910-921, 1991.
66
• Background • Image registration and related techniques • Single particle detection • Single particle tracking • Computational analysis of particle behavior • Summary
67
Summary • There are a variety of image alignment methods. Rigid body transformation is preferred because image transformation does not introduce artificial movement. • Single particle detection techniques are becoming rather mature. • A variety of single particle tracking techniques are available. Many of them have comparable performance. For a comprehensive comparison, see (Chenouard et al, Nature Methods, vol. 11, no. 3, pp. 281-289, 2014)
68
Thank You! Questions?
69