an k. ot C e P C op ct o op ed st yr yr e ig .F rB ig ht ht 1 a 00 pr n pr k. ot 0 ot C ec Po C ec op o t e s p te yr te d. yr d. ig rB ig F1 ht F h an tp 00 pr k. ot ro 0 ec Po C te o ct te st py ed d. er r i .F F1 gh Ba 10 t 0 n
op yr ig
Po st er B
C op yr ig ht pr ot ec te d. F
op yr ig ht
C
er B Po st 10 00
te d. F
.F 10 00 te d te c
ht pr o rig op y
pSTS
SL > SS + SM RL > RS + RM
TL > TS
450
500
SL = loud normal speech, SS = soft normal speech, SM = mute normal speech, RL = loud reversed speech, RS = soft reversed speech, RM = mute reversed speech, TS = soft tones, TL = loud tones.
rIPL
c)
IPL
IFG
MTG
S > R (S > T)
SS > SL + SM
RL > SL
Hypotheses testing with SPM. a) Loud stimuli activated the auditory cortex more than other sound levels. b) Normal speech activated the left pSTS; soft speech in addition to the right IPL. c) Reversed speech activated more than normal speech the right MTG, IFG and IPL, as well as the auditory cortex bilaterally.
rig ht
op y C
k.
Conclusions
► Left-lateralized sensorimotor network, including the IFG (Broca’s region) is involved in audiovisual speech comprehension, when one strives to understand the message. ►The signal increase of the right-lateralized network at each block may be related to working memory update. ► The strongest activity of the auditory cortex by unintelligible reversed speech most likely reflects increased attentional demands. ► Combined ISC–ICA analysis is feasible in sorting the ICs according to their stimulus relateness. It is especially useful in facilitating IC selection, when hemodynamic response is complex or difficult to model.
an k.
10 00 te d. F
ec
b)
400
er B
0
350
st
.F ct ed
IC10
0 10.3
pr ot e
yr ig ht op
300
Po
00 10
0
ht
a)
Po st er 0
12.4
IC9
pr
0 11.0
rig
250
Time (s)
0
IC8
te d ot
ec
IC3 IC4 IC5
200
7.9
13.5
C an k. te rB
os
150
Ba
IC7
.F
IC2
0
op y
Po
► 41 ICs estimated with group ICA.
100
ICs 1 (red), 6 (green) and 10 (blue) and their time courses. IC1 covers auditory cortices bilaterally: the louder the stimuli the stronger the response with the strongest response for reversed speech. IC6 covers the left pSTS, left aSTG, left IFG, left BA6 and right IPL. It reacts to normal speech and occasionally stronger for soft than loud stimulus. IC10 occupies the right MTG, IFG and IPL. It reacts to every stimulus block and strongest to reversed speech.
pr
8.9
9.2
0
te c
ig 0
10
9.2
50
an
0
0 −3.1
er B
R
0
−3.1 2.8
.C
R P/L
0
Po st
L
61
nk
A
00
IC6
IC1
P
8.9
Po
A
ec t pr ot
ht
st er
F1 00
ed .
13.8
51
ht pr o
Signal change (z score)
op yr
,
bin x, y,z
18
−1.8 2.8
.C
0
x, y,z
)
Ba nk
i
bin i x, y,z
The 10 ICs with highest SP were the most stimulusrelated components (z-scores).
(Calhoun et al., Hum Brain Mapp 14:140–151, 2001)
Po st er
pr ot ec te d.
ht op yr ig
BOTTOM
C
bin x, y,z
x, y,z
ig
yr
op
.C
nk
st er
(Golland et al., Cer Cortex 17:766–777, 2007)
10 00
TOP
Ba nk .
Po st er
ot ec
pr
ht Ba
► ISC map formed by voxel-by-voxel correlations.
.F
ed .
tp ro te ct
op
C
nk .
Ba st er 00
10
.F te d
∑ ISC ( IC SP = λr ( ISC , IC ) ⋅ (1 − λ ) ∑ ISC
yr ig
C
k.
an
rB
st e
Po
BACK
R
z=9
SS RL RM TL TS 2.3 SL SM RS
where λ = weighting factor, r = correlation coefficient, bin = binarized image.
► Data preprocessing with SPM2, including realignment, normalization into MNI space and smoothing (6-mm FWHM).
0
FRONT
► Sorting parameter (SP) for the i:th IC i
op
► 8.5 min continuous audiovisual speech. The loudness of the speech either loud, soft (just audible, –18 db lower than loud) or silent; occasionally time-reversed or interrupted by tone pips.
ct ed
LEFT
Po
te c
ro
tp
ig h
yr
op
C k.
an
F1 00 0
er B Po st
10
.F
te d
RIGHT
A
P/L
ISC map showed brain areas related to audiovisual speech processing.
► 3-T fMRI with TR = 3 s, 3 x 3 x 3 mm3, 44 slices, 170 volumes + anatomical images.
rB
Results
►ICs were sorted according to their spatial overlap with the ISC map.
00
pr ot ec
ht
ig
Methods
00
yr ig h
an k.
Po st 00 10
.F
te d
Component ordering
►To find out whether the involvement of the sensorimotor network (including Broca’s region) is enhanced during audiovisual speech comprehension. ► To find out the most stimulus-related independent components (ICs), by combining intersubject correlation (ISC) and independent component analysis (ICA).
F1
Ba nk .
Sanna Malinen and Riitta Hari
Brain Research Unit of Low Temperature Laboratory and Advanced Magnetic Imaging Centre, Helsinki University of Technology, Espoo, Finland
Objectives
► 10 subjects.
F1 00 0
er B
an k. C
Trying to comprehend audiovisual speech
260.3/W11