Kullback Proximal Algorithms for Maximum Likelihood
INSTITUT NATIONAL DE RECHERCHE EN INFORMATIQUE ET EN AUTOMATIQUE
Kullback Proximal Algorithms for Maximum Likelihood Estimation St´ephane Chr´etien, Alfred O. Hero
N ˚ 3756 Aoˆut 1999 ` THEME 4
ISSN 0249-6399
apport de recherche
!#"$&%(') +* ,- ./ 01$ 243 5 %6","!7 8 96:?A@CB=ED ?GF = ; :H =)BJIAKMLONPFQ=SRUTWV!XM=)FZY\[ ]J^`a_ bca,d z egfShib _ jlkimonphrqSsMantqvulnphwbchixpm)nyhwqvs axy{lxpn aQbc q)aaxtn|7hwqSxbc u}kra~aQx m)uuq n z a a| ^ a | ^ as)ooeqj} n )v eZcu}movax
4G^ z })) ¡ s n ^ hixu}m)uCa ¢£ axyn¤j z { n ^ a|^ qvsS¥)z a vas}|aAqo¦m,sl]Ja ^ £ |7kim)x¤xq)^ ¦}¦§m)z xpnGmv s z xpnpmv¨lz kratxpa©jlasSnphwm)kvqSulnyhibchrªQmonyhwqvs bc°#´¶»«a¾n µ¿q hibcxu}¦«q kraQb |7qvaQsSb¬nyu}a zj}ny£ hislhwn ^ b¬m m~lÀhwjlb kwkwj}¨bm)|#kwvhioÁOnPa{lkwh uCaÂqqu qZa~lxphwnyb¬hib¬m)k}mo¦§npj}axs}® |nyhwqvs\aQ®!xyaJà bc^ aan sMq n ^ x aÄm u atqZ~¨}himvb¬xya mok qSas vjlm,ki¯±m °phw²ª³vmo´¶npµ¸hrqS·s¬¹)¯lu}²om ´¶ ºm)»Abc·a¹ ¼Snpa²½ hi xÅaSxpjlanÅkwm ny q¬hwªj}m)slnyhwhwqvnP{¬sqvu}slm a mvqvbc¨ln¤amonyahis}x ¢n a^ kiaÂm~l|akwmvz xpxp¥)hia|moxpkChwqva7~}s}uCxÂaqo|7¦ n¤moÇÅnpÉ-hrqSs4m b¬ acmqv~l¨}hib npmohwªhislm)a nyzhwqvs £^ÆÈÇ hi| ÉE^ Ê |m)moskw)q ^ mZhw¥)n ^acbb ®Gj}Ë| q^ ¦§q)m)nxp^nya a |7¥qvmosSkijl¥vaa xÅ )qoa¦sn |7^ a)a ® ¡ sÌum nyhiz |jlkim ¢ hr¦An ^ a aSjlkwm hwªm)nyhwqvsu}m m)bcanya ¥^m)slhix ^ ax, mon1^ hisÍCs}hrnP{ ¢ m4^ xpj}uCa kwhwslaQm kr{ |7qSsS¥)az )hisl2m) krvq hrn ^ bÎhix qShwkr¨lkij}npxpm)n hwslmoa nphrqS®Ws4Ãn ^ a au bcaaxpn a^ sSq n z mvhisÏxJm)hib¬u}u}ulkrkrhwaQa bcz anysSq2npm¬m)nyslhwqvqSsÏsÁPq)©S¦jmn z a mom)nphwkr|Âvq hwsShr¥vn a bÐxyaÄju xy hiqvsl¨lEkwan b aE£ »Èhw°#n Ñ^Ò#»Ä qv°pÓÔhwx¤¼)xy´§qS²sºWz j}hiu xpn mohw¨}nya j}nyxpa n z moz nyam))npm{v®® ËCq Õ ÖS×`ØpÙ Ú ÛG)¢ mv||7a kra m)nya z Ç É^ moz kw)q hwn ^ b ¢ Àjlkwkw¨m)|# avjlkim hrªQmonphrqSs ¢ u qZ~lhwb¬mok
F
:9DC ;E< >
,
@?BA
@?BA
G
#"%$7H
UT
+V
PSR
-X
?
JILK MONQPSR
ZT[
[V
H
YX
UW
PSR
?
ZT[
]
\T
8
D^`_
a L
cb
f
hg`
d
jV
kd
+l
#"%$7H
o]o
eb
id
÷ #
kp q2rs
ILK M
N
PSn
?
mV
V
H
l
hd
d
C»Ó ¯l¾}·oº±Ó6¾°Ó »È´§Óº ¹ °pÓ
Ó°p² £ hwn ^ Æ Ê krqv Æ Ê#®fShis}|7a 6Æ `Êtn ^ a z aQs}xphrnP{ 2q)¦ hix akimonya z npq¬n ^ a z aQs}xphrnP{ qo¦ Ðn ^} qvjl ^ ±Æ Ê Æ Ê !Æ `Ê ÆÈdÊ ¦«#q " m)Æ s $ &m)%ouÊAu n ^ qSau xphwq)m)kinyjla nyhwbcqvs}axtm)x¤q)j ¦ ÆOa!SÊÅqm S s a xy qvkw® j}Ânyhwsqvs}z axt q)|¦Gqvn s ^ z a hwnyq hw qvhrsEvhwsÆÈdmoÊtk`Émvs ! z a¦«qxpny hib¬m¸movnphrhr¥vqSasMsMu qv ¨}qvxp¨}a kr aQ¥)b a z Æyx¤m)Ê®b¬¡ ulskwa n ^ hw¢ x¦«xpq a npnym)hisSsl{ ¢ n ^ hws a ¡z n m)np m aQbc4m)m hws a x£ |nym)q krkwaz ]Jz am)^ ´¶k º £ ²ohwµ1n ^ ¯±¹rn¢ Ó7^ »PaMÓ^ ¦§)m)·o|»Ôn · n£^ ^ moa n aQn ^m)z xtaMn |^ qva b¬z ulmokwn¤amnp^ a Ìz m zm) npaÄm | mokwkr|7a q z p a²ox¤µ1uqS¯±s ¹rÓ7z »PhiÓ sl)·ny»PqE·® mÌvhr¥vasWqv¨xya ¥vz a z xpmvb¬ulkra ^ qSmÆÈĘ́}a¸xyÉEa j} fSs}¥vÇJazÄslÊÅq z qom)¦
Zg] A
Zg] A
H
Zg`
Zg]
H
Zg
H
T[
;
l
Zg
T
@?BA
%>
H
?(A
>
T
"
"
5P R
Z
j^`_
c^`_
[
@?BA
>
Z
.
@?BA
U^`_
I
H
PSR
P R
I
A
7
7^
_
>
I
@? A
0
i
I
c0
\PSR
I
I
I
j
[
@? A
>
4P R
>
m
@?BA
hg`
P R
@?BA
>
QX
ZT[
Zg]
U
PSR
? A
[i>
E
>
>
A ? A
H
@?(A
0
U
m
[
+
a
k
;
H
JILK M
;
PSR
>
?=
H
0
.>
;
?=
?=
>
=
PSR
>
;
>
PSR
0
PSR
,P R
;
PSR
N
;
?=
?
[
?
>
P R
[
?=
=
ST]o?S
Ñl¹¶¹ · t°p²³v´¶µ¸·¹ ¹ ¼v²o°#´¶»¶¾lµ Ò 7²o° E·³v´¶µ Ñlµ !´ vÓ¹ ´r¾}²Q² tÒ»O´¶µ¸·»O´§²º
#,
. 1 $"
Ëh xpn £ a| ^ m mv|7npa hrªa1n ^ aÄÍ}~la z uqvhwsSn¤xtqo¦n ^ a À Å m)krvq hrn ^ b® E Ú 5 Ú B F«B F HÚ G _´ ^Ó7º Q 7 +Q 0 t»«¾}Ó h
9
?=
;E< >
Y9
5
;=< >
[ D9
9%PSR
0
\
;
= 9
?
9;=
@?
9%P R
C ;=< >
"%$
"%$
"%$
C P R 9
?
"%$
9%;=
9
H
"D$
"D$
?
"%$
"D$
"%$
"D$
?
ST]o?S
Ñl¹¶¹ · t°p²³v´¶µ¸·¹ ¹ ¼v²o°#´¶»¶¾lµ Ò 7²o° E·³v´¶µ Ñlµ !´ vÓ¹ ´r¾}²Q² tÒ»O´¶µ¸·»O´§²º
cÆ !B Ê 2 ÆO odÊ 8 °p²oµ »«¾}Ó )ÓOhAºC´¶ºS¼4°pÓ¹r·»O´§²º Q#R :]T2»«¾}Ó ´¶»ÔÓ°p·o»ÔÓ 78= Ò·»O´i_Ò hÓ#Ò Æ 78= Ê B Q 7 ±Æ 78= 7 Ê 2 ÆO vvÊ \² A»P· o´¶ºS¼ 78= ´¶º QPS >TM·oº ¸ÑÒ#´¶º9¼ Q S: T L Ó ² 7»P·´¶º Q 7 ±Æ 78= 97 Ê B ±Æ 978= Ê > Æ ÊÆ 78= B Ê B Q 7 ±Æ ÊÆ 78= B Ê > Æ 78= B Ê2 K ¾ÑNÒ Q ±Æ 7-= 7 Ê B ±Æ 78= Ê B Æ 78= B Ê ( ( 7 Ê#Æ 7-= B Ê B Q 7 CÆ Ê#Æ 7-= B Ê ( 2 ÆO )SÊ ( Æ tº»¶¾Ó ²»«¾}Ó°J¾·º ²º±Ó vÓ Ñ Ó#Ò °p²µ ÄÒÒÑlµÂ¯C»O´§²º: ·oÒ#º Ñ ¾ »«¾}K·o»¾7Ó7²oº ° AÒ#·´¶¹¶º¹ 8Ó $ 7 %·´iº Ò ¤²oÑlº ´¶ºvÓ A »¶¾Ó7°¤Ó Ó¤³v´iÒ»ÈÒ Ò R · QO´Tc²Ñl»¶¾º ·)» Ó ¬ÒÓ»±Æ ¤²oÊ1º´i»PÒ1·´¶¹rºC² ´¶ºS·¼¹¶¹iN $ !7 ´ ¯C%Ò ·¤¾lº ´¶4»_a · ´¶h ºº»«¾}´¶»PÓÓ ^Z·o²°#º´§Ò#· »Ô·o¹rºÓ#Ò » & B CÆ Ê ( , ( B ( > ( B ( 2 ( ±Æ Ê Ò´¶º¼2»«¾}ӻȰ#´§·oºS¼)¹rÓ ´¶º±]Ó \Ñ·¹ ´¶P» N4·º 2»«¾´iÒ ¹r·Ò»°pÓ7Ò#Ñl¹ » dQ S ( Æ B Q 7 CÆ Ê Æ 78= B Ê ( 2 ÆO S)Ê >² L ²ºÒ´ vÓ7°c·¼S·´¶ºE»¶¾Ó ²Ñlº )Ó ¬ÒÓ» ¤²oº»P·´¶ºC´¶ºS¼ $ 7 % GÓ» (+* ·º 1 vÓ7º`²»PÓ »¶¾Ó µc´¶º´¶µ¸· ( * bchws V B $ & ( * X "D$
I
I
"%$
:P R
H
"%$
.9%P R
H
5#"%$
0
0
?
"D$
9;E
H
"D$
0
9
;E< >
@?=#"%$
9
C ;E< >
,"%$4? #"D$
9%PSR
;
9%P R
;
H
;
?=
;
;
A
A
?=
H
"%$
?= "%$
\
;
#"D$
?
i
@?BA
[
9;E
? A
;
#"D$
.
;
#"%$
L
[
A
C
[
5#"%$
A
c
C
;
"D$
-
H
¾±´·>¶º ^Ó Ó ´¶7µc²oµ¸°MÓ ·oº ´§N ·»PÓ7Ò]N¹ N2µc»¶µ¸¾·Ó7» »O°#´ 2µ¸·»O°#´ ³@
IK
( (
C
I
1 bc= ? his $ &3/21 =@? I
5 "%$
[
ILK
H
#"%$
C PSR
9
?
"%$
C ;E< >
9
I
I
2
´iÒ¸¹r²[L Ó7° ²oÑlº )Ó UN»«¾}Ó4µ ¶´ ºC´¶µ Ñlµ Ó7´ ¼SÓº^·¹ ÑÓ ² !@6L Ó B Æ 1 Ê > Q 7 1 Æ Ê Æ 78= B Ê ( ( G (+* > Q 7 978= B ÆO SÊ ( ( 2 ,ÒÒ#Ñlµ1¯±»È´§²oº}Ò R w´¶´¶´t·º R w´_^ ( * > Q 7 1 ·º Å· »PÓ° Ò#Ñ #Ò#»O´¶»ÈÑl»O´§²º² +Q S T2´¶ºC»Ô9² QPS !T L Óc² »Ô·o´¶º Q 7 7 B 78= B Ê( G ( ( > ( *( cÆ 1 > Q 7 97-= B ÆO )SÊ ( (
g 1@+g
Z
9
C P R
"D$
g
[
;
0
5#"%$
i
#"%$
[
?
"D$
C ;E< P
9
C
5
÷ #
kp q2rs
E
;
"D$
5 "%$
C
C
E
/
;
=
#"D$
;
o]o
H
[
N
;
@? A
;=< >
`9
5 "%$
#"D$\? #"D$
C ;E< >
9
;
[
5 "%$
A
[
Z
;
"%$
9;=
-
0
H
C ;=< >
9
9;E
;
c
? #"D$
5 "%$
?
#"D$
C PSR
;
5#"%$
9;=
9
;
;=< >
9
"%$
C P R
;
#"%$
;=< >
9
5#"%$
#"D$
9;=
9
#"%$
C PSR
9
;
D
H
"%$
?
C»Ó ¯l¾}·oº±Ó6¾°Ó »È´§Óº ¹ °pÓ
Ó°p²
0
Y
7²o° ·¹¶¹
K
K !
¾}Ó°pÓ 7²°pÓ8 ²o¹¶¹rÓ 7»O´¶ºS¼M»PÓ7°#µ Ò,´¶º h
Q S T
(+* > Q 7 1 B ( cÆ 7878= = B B Ê ( ( 78=7 B B ( 2 ÆÈ SÊ ( ( ( ( ²>L\°pÓ ·¹¶¹S»¶¾·»&$ 7 %Ä´iÒ ²º ^ZӰȼSÓ7ºC» K ¾lÑÒ lkwhwb 7 7 B " ¬ · º 1 Ò Ñ 7 Ò ] Ó 7 \ Ñ Ó º O » i ¹ N vkwhib 7 7 " ÑÓ»P²»¶¾ Ó ´¶º)ÓO^hA² oº´¶´¶ºS»O´§¼ ²º »«¾}²ÓE4·Z»«Ò¾}Ò#ÓMÑlµ1°pÓ¯±µ¬»È´§·o²o´¶º}º Ò vÓ7² ° G MÓµ C´µ¸¶º · (Ó N²´§º Ó7^Z¹ ÓҰȼSZE(Ó7ºkrhi¤b Ó ² $ Q 7 %Ï·¹òÒkw²hwb ´¶µÂ ¯C ¹ ´§Ó7ÒÆ »«¾}·o7 »JB krhib 7 7 " " Ê ( 7 &B
7 7 7 ( ( K ¾}Ó°p(Ó 7²°pÓ8t·Ò ( * ÅÓ]\7Ñ·o»È´§²oº Q T ¼o´_^Ó#Ò k7 w hwb ( 78=7 B B ( ( ( ·oº ¸»«¾}Ó¯C°¤²² ² ÄÒ#ÑZ¯}Ó7°#¹ ´¶º±Ó·° ²º ^ZӰȼSÓ7º Ó´iÒ ²µÂ¯C¹rÓ»PÓ 9
4Ö /l 6 K ¾Ó ²º^ӰȼvÓº Ó K ¾}Ó²o°pÓ7µ Ò RW·º .SEµ¸· vÓÑÒÓ ² ²oº ·>^Q´¶»ON ² Æ Ê·oº ²º^Ó¤³)´¶»PN² ±Æ Ê Q 7
0
;
;
H
5 "%$
[
I I
=
5 "%$
H
#"D$
E
c
[
c
0
I
I
;
H
;
=
0
0
0
[
E :
h
H
U
#"%$
"%$
#"%$
=
0
;
#"D$
Q
+
;
G
H
0
?
Q²´§· º ^Z,ÓҰȼSÒ#ÑlÓ7º µ1¯±Ó»È´§°p²oÓ7º}Ò#ÑlÒ ¹ »R w´¶·o´¶´Jº ·º Ó Ó7R9Ò#»Ô w´_· ^ 7 ¹ ´i Òy¾²[Ó L 2ÓUÑl^º Ó°N) Ó°Ä7²oÒ°»O°pÒ#²oµ¬ºS¼S²QÓ7²°»«¾Ì·ZÒº±Ò#²ÑlºCµÂ½ ¯C²»Oº´§²^º}Ó¤³ÒÄÒ#Ñl´¶µcº 7´¶¹r»O·´§°²º»PÒ ²¸·»«¾}º ²·ÒÓº`·´¶¹rº²¤¼v ²o8Ñ Ò ¹r² ·¹ Æ 7 ÊÆ !B 7 Ê > Æ !B 7 Ê Æ Ê#Æ DB 7 Ê 2 mvs z ± Æ 97 Ê Æ !B 97 Ê ±Æ 97 97 ÊÆ !B 97 Ê 2 ¡ sn ^ aĦ«q)kwkrq £ hisl ¢£ a m z qSulnJn ^ as}q)npm)nyhwqvs}x Æ 7 Ê 7 @ 7 Æ 7Ê mvs z ±Æ 7 7 Ê 2 7 PSR
H
P R
P R
@?
H
5
9
H
9%PSR
X
>
H
H
9
@?
5
!9%P R
5
>
>
9
C ;=< >
9
C PSR
@?=
C ;=< >
C P R
?=
ST]o?S
H
Ñl¹¶¹ · t°p²³v´¶µ¸·¹ ¹ ¼v²o°#´¶»¶¾lµ Ò 7²o° E·³v´¶µ Ñlµ !´ vÓ¹ ´r¾}²Q² tÒ»O´¶µ¸·»O´§²º ) ]J^ amvu}u qZ~lhwb¬monpa1À Å m)krvq hrn ^ b hix z a7ͱsla z m)x 7-= m vb¬m~ >V SÆ 7 Ê > 7 Æ !B 7 Ê > ÆÈZÊ Q Æ !B 7 Ê @ 7 Æ !B 7 Ê B 7 Æ !B 7 Ê 7 Æ !B 7 Ê X tmvu}n u n ^ qZhw~x hib¬uqvmohwnpsSanxynphwn auhwxÆO}hwb¬ZÊ!uChwq xÅ anpm)©sSjln hw¥npmoqÌkwasSbcn mvnpoq a¸m xpa Â¥)a a £ m)npk!qvs2|7qvxpb¬nyaQbcu\a® sS¡ n¤n xhi® x £ q)akwnpk`hw|Sa¬s}qÍ £ xps¬nÄn n ^ ^ m)m)n n 1¦«q a £ Q nyqS7 s xbc¢ an0 ^ q z ¢ mo kixyq 2-22 sl¢ q n £ ^ s a mhivslx!hwnyËhimohwx kw^hrªa a z xp® |q ]J hw^ sla Ä a7his¸¦«q xy n¤a mo¢ nphwmoxpn¸nyhi|kwax mv¢Sxy^ n¬m)¦«xAq x¤ j}uCxpb¬a mokrhikwslka¥m mo kijlm)aQxpx¬{lb¬qo¦Äulnyn qv^ nya hi|J |7aqvkimsS~}¥vmoa np hr)qSas s|7ua m m)m)nybcaa¨}npjla nÅ bcQ mZ{¢ n z ^ hw¥)aÌa m) )u}aJu hr¦qZ~lslhiq)bcnÅm)u ny a qSu a kw { mQ)krvq hrn ^ b n ^ a2bcnymZa {E b ¦§m)hrBktnyqW Æ |7LBqSsS¥)a 7 )Ê a2¦«q Æ LBn ^ a7 x¤ÊÄm)bcuCaa s}m)akrhwmvªxyaQqSx s}nx ^ mva xz ¦«hiq xyn¤m) Âs}|a a¸£ npqoqv¦Js n ^x¬aMb slaa7n ~l^ nq z hwny® a 7 m)nysÏa n ^ 78a= qvnny^qa n ^ ^ aMm)|s j z ¤¢ aQ¦«qsvn hw Âny7a a £ m)nynyqSa smo7 kw® )q  ahwn s}^ |b a ¢6£ £ hra¸n ^ |qvmvjls nÂhixpsSmvny| a h u ÍC7 |ahwnÄs}¸n ^ hrn¤hwx x1npx¤aj} ub a kwm)hix slam m am)Sxpjl{kwb¬m ulhwªnpq)m)npnyhwhw| qvs |qv£sS^¥)hwa | ^ vaxps}np|m)a ¨}hr kwmohrªnpaa)xÄ® n ^ { a2m)uuqSu xpxp hiqS¨lu kr{ hwm)z nyhra ¥va| ^ )q)aQhisv|7an q) ¦ $ Q 7 %¸n ^ a¸hw nya m)nya ]J^ 78= z |m)sE¨ac¦«q |a z nyq aQ b¬mohis hism avhrqSs m qvz js z 7 q¥) a ,££^ hw| ^ n ^ a¬^ ©j}m ]Gz monyhi|¬ bcq z ak npaÆ| ^ Ê slhihi©xcjlm)a | |j l monyav )® Ô® hix h aQmÌhix ¥va {EuCqvujlkwm his qSulnyhibchrªQmonyhwqvs/m)s hixc¨anynya Ss}q sÏm)x n a jxyn a)hwqvs 4Ö /l #º/µ¸·º N ·ZÒÓ#ÒM · \Ñ· °¤·»O´ ·#¯)¯±°p²³v´¶µ¸·»O´§²º ² 4·ÌÒ#´¶ºS¼)¹rÓ ²º`ÓM² 2»«¾}Ó4O» L ²Ì»ÔÓ°#µ Ò Æ Ê4²o° ±Æ 7 Ê ´iµ¸Ò ·Ò#»O Ñ ´§²ºW7´§´iÓÒ1º¯±»°p»ÔÓ ²7Ó7°p² · 7»P·¹wÓ¬´¶º »Ô²M· »«¾}¹rÓ¸²Ò7·#Ó ¯)¯C7°¤²²°#³vµ ´¶µ¬µ¸·o»È·´§²o³vº´¶µc² b´ a·o»È²´§²o»«¾ º »ÔQ#ÓR °#>µ TÒ ´i·Ò 7»OÑlÑ°pÒ·Ò¹¶Ó ¹ N · L¾}^ÓÓ º 78 Ó¤·²oÒ#° ´ 7¹r²8Ó º7Ò°¤Ñ Ó7»P¾Ó7º`·2Ó#Ò°pÒNÓ toÑ´¶ºE¤Ó »«M¾}Ó ·¯)Ò¯±Ó]°p\²Ñ³vÓ7´¶¹ ½ ·o·o¹¹ ¼v»¶¾²o²°#Ñ)´¶»¶¼Z¾l¾Eµ+²´¶Ñlµ1° ¯±°p¹rÓ7Ó7Ò#µ¸Ñl¹Ó»§ºÒ1»P¾}Ó ²o¹ L!´¶7»«²¾ ° »«»¶¾}¾Ó Ó°pÑlÓ ¹¶o¹Ñ O» L ¤Ó ²o½P»Ô·Ó¯v°#¯Cµ °p²·³)¯)´¶¯±µ¸°p·²»O³v´§´¶²µ¸ºÏ··o»O´§¹òÒ²²º ²L ºCÓ1i¹ N¯± °p>² ^Ó ¤²oº ^Ó7°O¼vÓº Ó 7²°»«¾}Ó1¯±°p²³v´¶µ¸·¹}¯}²´¶ºC»
;
H
JIK M
N
PSR
aX
5
5
@>
H
H
0
?
?
8
;
>
0
C
&
;
P R
+
2E
6
h
7
Z
=
E %
!
%
%
L
cP R
Z
D
%
.>
@?
[
Q
-
# !
#,
E
#
]J^ a 1 ]aSaQs ¢) ²9a ny a E® )¢ ½ % :>S s¸Q!jR xy aq)¦\T Ç É mokw)q hwn ^ b¦«q uaQs}mokwhwªa z krhioakrh ^ qq z axpnyhibcm)nyhwqvs\® ¢ c 4C»P·» &\² \Ó7-°
o !EC ®\®\ËCK a°px¤·oxyº}kwa Ò- m)C´s ¼)z º±·o¹ ® t°p²® ÂÓ#Òa Ò# ´¶q ºS¢ ¼ ¢ ¥vfq)u}kOmv®l|7dSa ÁÔm)¢ krunpu\a ®s} omovnpohwsld}4 o)a)s} a ¢ movkwhroªda z® a~luCa|npm)nyhwqvsÁPb¬m~lhwbchixpm)nyhwqvsÌm)krvq hrn ^ b ¢ v cÂ' Ó#® ÒÅÓ·akw° a¤j6¾ ~ c¢ ÓPf±¯}® ² °#»^} a nyhwa< s ¢ Ëtc ®SË q 6¨C¾la»Èx »r¢¯ mvs zLLL ®S wÉ ´¶ºC °#´§^ · m z ° h c¢ c pUc |7K qv)b¬¯CÑ uCqv¹ sl´ aQ·sS»On´§Á²£ ºhwÒ#xp½ aJ°pÇ · É ¾l»Èµcm)kr¹ v¢ q  hrjln ^ ®6b )¦«q v }®bchr~n¤j ax ¢
/#
Z
=
@)
#
0
#0#m&
8
&
0/
/
L
j
E
m
/
/
Z
/
,
[
m)
#
#&
Y3
.
j
0
[
8
kk
&
o]o
#
8
,
÷ #
kp q2rs
Z
1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
0
10
20
Ëhwvj a¬ hw)his}m)k`u ^ m)sSnyqSb
50
60
70
ËhrSj a t kwj ¤ a z u ^ m)sSnpqvb
50
60
70
30
)
8
40
5
4.5
4
3.5
3
2.5
2
1.5
1
0.5
0
0
10
20
30
40
@)
Q
C»Ó ¯l¾}·oº±Ó6¾°Ó »È´§Óº ¹ °pÓ
Ó°p² Y
34
Plain EM Trust Region True Likelihood
32
30
28
26
24
22
20
18
16
14
0
20
ËhrSj a !
Kullback Proximal Algorithms for Maximum Likelihood Estimation St´ephane Chr´etien, Alfred O. Hero
N ˚ 3756 Aoˆut 1999 ` THEME 4
ISSN 0249-6399
apport de recherche
!#"$&%(') +* ,- ./ 01$ 243 5 %6","!7 8 96:?A@CB=ED ?GF = ; :H =)BJIAKMLONPFQ=SRUTWV!XM=)FZY\[ ]J^`a_ bca,d z egfShib _ jlkimonphrqSsMantqvulnphwbchixpm)nyhwqvs axy{lxpn aQbc q)aaxtn|7hwqSxbc u}kra~aQx m)uuq n z a a| ^ a | ^ as)ooeqj} n )v eZcu}movax
4G^ z })) ¡ s n ^ hixu}m)uCa ¢£ axyn¤j z { n ^ a|^ qvsS¥)z a vas}|aAqo¦m,sl]Ja ^ £ |7kim)x¤xq)^ ¦}¦§m)z xpnGmv s z xpnpmv¨lz kratxpa©jlasSnphwm)kvqSulnyhibchrªQmonyhwqvs bc°#´¶»«a¾n µ¿q hibcxu}¦«q kraQb |7qvaQsSb¬nyu}a zj}ny£ hislhwn ^ b¬m m~lÀhwjlb kwkwj}¨bm)|#kwvhioÁOnPa{lkwh uCaÂqqu qZa~lxphwnyb¬hib¬m)k}mo¦§npj}axs}® |nyhwqvs\aQ®!xyaJà bc^ aan sMq n ^ x aÄm u atqZ~¨}himvb¬xya mok qSas vjlm,ki¯±m °phw²ª³vmo´¶npµ¸hrqS·s¬¹)¯lu}²om ´¶ ºm)»Abc·a¹ ¼Snpa²½ hi xÅaSxpjlanÅkwm ny q¬hwªj}m)slnyhwhwqvnP{¬sqvu}slm a mvqvbc¨ln¤amonyahis}x ¢n a^ kiaÂm~l|akwmvz xpxp¥)hia|moxpkChwqva7~}s}uCxÂaqo|7¦ n¤moÇÅnpÉ-hrqSs4m b¬ acmqv~l¨}hib npmohwªhislm)a nyzhwqvs £^ÆÈÇ hi| ÉE^ Ê |m)moskw)q ^ mZhw¥)n ^acbb ®Gj}Ë| q^ ¦§q)m)nxp^nya a |7¥qvmosSkijl¥vaa xÅ )qoa¦sn |7^ a)a ® ¡ sÌum nyhiz |jlkim ¢ hr¦An ^ a aSjlkwm hwªm)nyhwqvsu}m m)bcanya ¥^m)slhix ^ ax, mon1^ hisÍCs}hrnP{ ¢ m4^ xpj}uCa kwhwslaQm kr{ |7qSsS¥)az )hisl2m) krvq hrn ^ bÎhix qShwkr¨lkij}npxpm)n hwslmoa nphrqS®Ws4Ãn ^ a au bcaaxpn a^ sSq n z mvhisÏxJm)hib¬u}u}ulkrkrhwaQa bcz anysSq2npm¬m)nyslhwqvqSsÏsÁPq)©S¦jmn z a mom)nphwkr|Âvq hwsShr¥vn a bÐxyaÄju xy hiqvsl¨lEkwan b aE£ »Èhw°#n Ñ^Ò#»Ä qv°pÓÔhwx¤¼)xy´§qS²sºWz j}hiu xpn mohw¨}nya j}nyxpa n z moz nyam))npm{v®® ËCq Õ ÖS×`ØpÙ Ú ÛG)¢ mv||7a kra m)nya z Ç É^ moz kw)q hwn ^ b ¢ Àjlkwkw¨m)|# avjlkim hrªQmonphrqSs ¢ u qZ~lhwb¬mok
F
:9DC ;E< >
,
@?BA
@?BA
G
#"%$7H
UT
+V
PSR
-X
?
JILK MONQPSR
ZT[
[V
H
YX
UW
PSR
?
ZT[
]
\T
8
D^`_
a L
cb
f
hg`
d
jV
kd
+l
#"%$7H
o]o
eb
id
÷ #
kp q2rs
ILK M
N
PSn
?
mV
V
H
l
hd
d
C»Ó ¯l¾}·oº±Ó6¾°Ó »È´§Óº ¹ °pÓ
Ó°p² £ hwn ^ Æ Ê krqv Æ Ê#®fShis}|7a 6Æ `Êtn ^ a z aQs}xphrnP{ 2q)¦ hix akimonya z npq¬n ^ a z aQs}xphrnP{ qo¦ Ðn ^} qvjl ^ ±Æ Ê Æ Ê !Æ `Ê ÆÈdÊ ¦«#q " m)Æ s $ &m)%ouÊAu n ^ qSau xphwq)m)kinyjla nyhwbcqvs}axtm)x¤q)j ¦ ÆOa!SÊÅqm S s a xy qvkw® j}Ânyhwsqvs}z axt q)|¦Gqvn s ^ z a hwnyq hw qvhrsEvhwsÆÈdmoÊtk`Émvs ! z a¦«qxpny hib¬m¸movnphrhr¥vqSasMsMu qv ¨}qvxp¨}a kr aQ¥)b a z Æyx¤m)Ê®b¬¡ ulskwa n ^ hw¢ x¦«xpq a npnym)hisSsl{ ¢ n ^ hws a ¡z n m)np m aQbc4m)m hws a x£ |nym)q krkwaz ]Jz am)^ ´¶k º £ ²ohwµ1n ^ ¯±¹rn¢ Ó7^ »PaMÓ^ ¦§)m)·o|»Ôn · n£^ ^ moa n aQn ^m)z xtaMn |^ qva b¬z ulmokwn¤amnp^ a Ìz m zm) npaÄm | mokwkr|7a q z p a²ox¤µ1uqS¯±s ¹rÓ7z »PhiÓ sl)·ny»PqE·® mÌvhr¥vasWqv¨xya ¥vz a z xpmvb¬ulkra ^ qSmÆÈĘ́}a¸xyÉEa j} fSs}¥vÇJazÄslÊÅq z qom)¦
Zg] A
Zg] A
H
Zg`
Zg]
H
Zg
H
T[
;
l
Zg
T
@?BA
%>
H
?(A
>
T
"
"
5P R
Z
j^`_
c^`_
[
@?BA
>
Z
.
@?BA
U^`_
I
H
PSR
P R
I
A
7
7^
_
>
I
@? A
0
i
I
c0
\PSR
I
I
I
j
[
@? A
>
4P R
>
m
@?BA
hg`
P R
@?BA
>
QX
ZT[
Zg]
U
PSR
? A
[i>
E
>
>
A ? A
H
@?(A
0
U
m
[
+
a
k
;
H
JILK M
;
PSR
>
?=
H
0
.>
;
?=
?=
>
=
PSR
>
;
>
PSR
0
PSR
,P R
;
PSR
N
;
?=
?
[
?
>
P R
[
?=
=
ST]o?S
Ñl¹¶¹ · t°p²³v´¶µ¸·¹ ¹ ¼v²o°#´¶»¶¾lµ Ò 7²o° E·³v´¶µ Ñlµ !´ vÓ¹ ´r¾}²Q² tÒ»O´¶µ¸·»O´§²º
#,
. 1 $"
Ëh xpn £ a| ^ m mv|7npa hrªa1n ^ aÄÍ}~la z uqvhwsSn¤xtqo¦n ^ a À Å m)krvq hrn ^ b® E Ú 5 Ú B F«B F HÚ G _´ ^Ó7º Q 7 +Q 0 t»«¾}Ó h
9
?=
;E< >
Y9
5
;=< >
[ D9
9%PSR
0
\
;
= 9
?
9;=
@?
9%P R
C ;=< >
"%$
"%$
"%$
C P R 9
?
"%$
9%;=
9
H
"D$
"D$
?
"%$
"D$
"%$
"D$
?
ST]o?S
Ñl¹¶¹ · t°p²³v´¶µ¸·¹ ¹ ¼v²o°#´¶»¶¾lµ Ò 7²o° E·³v´¶µ Ñlµ !´ vÓ¹ ´r¾}²Q² tÒ»O´¶µ¸·»O´§²º
cÆ !B Ê 2 ÆO odÊ 8 °p²oµ »«¾}Ó )ÓOhAºC´¶ºS¼4°pÓ¹r·»O´§²º Q#R :]T2»«¾}Ó ´¶»ÔÓ°p·o»ÔÓ 78= Ò·»O´i_Ò hÓ#Ò Æ 78= Ê B Q 7 ±Æ 78= 7 Ê 2 ÆO vvÊ \² A»P· o´¶ºS¼ 78= ´¶º QPS >TM·oº ¸ÑÒ#´¶º9¼ Q S: T L Ó ² 7»P·´¶º Q 7 ±Æ 78= 97 Ê B ±Æ 978= Ê > Æ ÊÆ 78= B Ê B Q 7 ±Æ ÊÆ 78= B Ê > Æ 78= B Ê2 K ¾ÑNÒ Q ±Æ 7-= 7 Ê B ±Æ 78= Ê B Æ 78= B Ê ( ( 7 Ê#Æ 7-= B Ê B Q 7 CÆ Ê#Æ 7-= B Ê ( 2 ÆO )SÊ ( Æ tº»¶¾Ó ²»«¾}Ó°J¾·º ²º±Ó vÓ Ñ Ó#Ò °p²µ ÄÒÒÑlµÂ¯C»O´§²º: ·oÒ#º Ñ ¾ »«¾}K·o»¾7Ó7²oº ° AÒ#·´¶¹¶º¹ 8Ó $ 7 %·´iº Ò ¤²oÑlº ´¶ºvÓ A »¶¾Ó7°¤Ó Ó¤³v´iÒ»ÈÒ Ò R · QO´Tc²Ñl»¶¾º ·)» Ó ¬ÒÓ»±Æ ¤²oÊ1º´i»PÒ1·´¶¹rºC² ´¶ºS·¼¹¶¹iN $ !7 ´ ¯C%Ò ·¤¾lº ´¶4»_a · ´¶h ºº»«¾}´¶»PÓÓ ^Z·o²°#º´§Ò#· »Ô·o¹rºÓ#Ò » & B CÆ Ê ( , ( B ( > ( B ( 2 ( ±Æ Ê Ò´¶º¼2»«¾}ӻȰ#´§·oºS¼)¹rÓ ´¶º±]Ó \Ñ·¹ ´¶P» N4·º 2»«¾´iÒ ¹r·Ò»°pÓ7Ò#Ñl¹ » dQ S ( Æ B Q 7 CÆ Ê Æ 78= B Ê ( 2 ÆO S)Ê >² L ²ºÒ´ vÓ7°c·¼S·´¶ºE»¶¾Ó ²Ñlº )Ó ¬ÒÓ» ¤²oº»P·´¶ºC´¶ºS¼ $ 7 % GÓ» (+* ·º 1 vÓ7º`²»PÓ »¶¾Ó µc´¶º´¶µ¸· ( * bchws V B $ & ( * X "D$
I
I
"%$
:P R
H
"%$
.9%P R
H
5#"%$
0
0
?
"D$
9;E
H
"D$
0
9
;E< >
@?=#"%$
9
C ;E< >
,"%$4? #"D$
9%PSR
;
9%P R
;
H
;
?=
;
;
A
A
?=
H
"%$
?= "%$
\
;
#"D$
?
i
@?BA
[
9;E
? A
;
#"D$
.
;
#"%$
L
[
A
C
[
5#"%$
A
c
C
;
"D$
-
H
¾±´·>¶º ^Ó Ó ´¶7µc²oµ¸°MÓ ·oº ´§N ·»PÓ7Ò]N¹ N2µc»¶µ¸¾·Ó7» »O°#´ 2µ¸·»O°#´ ³@
IK
( (
C
I
1 bc= ? his $ &3/21 =@? I
5 "%$
[
ILK
H
#"%$
C PSR
9
?
"%$
C ;E< >
9
I
I
2
´iÒ¸¹r²[L Ó7° ²oÑlº )Ó UN»«¾}Ó4µ ¶´ ºC´¶µ Ñlµ Ó7´ ¼SÓº^·¹ ÑÓ ² !@6L Ó B Æ 1 Ê > Q 7 1 Æ Ê Æ 78= B Ê ( ( G (+* > Q 7 978= B ÆO SÊ ( ( 2 ,ÒÒ#Ñlµ1¯±»È´§²oº}Ò R w´¶´¶´t·º R w´_^ ( * > Q 7 1 ·º Å· »PÓ° Ò#Ñ #Ò#»O´¶»ÈÑl»O´§²º² +Q S T2´¶ºC»Ô9² QPS !T L Óc² »Ô·o´¶º Q 7 7 B 78= B Ê( G ( ( > ( *( cÆ 1 > Q 7 97-= B ÆO )SÊ ( (
g 1@+g
Z
9
C P R
"D$
g
[
;
0
5#"%$
i
#"%$
[
?
"D$
C ;E< P
9
C
5
÷ #
kp q2rs
E
;
"D$
5 "%$
C
C
E
/
;
=
#"D$
;
o]o
H
[
N
;
@? A
;=< >
`9
5 "%$
#"D$\? #"D$
C ;E< >
9
;
[
5 "%$
A
[
Z
;
"%$
9;=
-
0
H
C ;=< >
9
9;E
;
c
? #"D$
5 "%$
?
#"D$
C PSR
;
5#"%$
9;=
9
;
;=< >
9
"%$
C P R
;
#"%$
;=< >
9
5#"%$
#"D$
9;=
9
#"%$
C PSR
9
;
D
H
"%$
?
C»Ó ¯l¾}·oº±Ó6¾°Ó »È´§Óº ¹ °pÓ
Ó°p²
0
Y
7²o° ·¹¶¹
K
K !
¾}Ó°pÓ 7²°pÓ8 ²o¹¶¹rÓ 7»O´¶ºS¼M»PÓ7°#µ Ò,´¶º h
Q S T
(+* > Q 7 1 B ( cÆ 7878= = B B Ê ( ( 78=7 B B ( 2 ÆÈ SÊ ( ( ( ( ²>L\°pÓ ·¹¶¹S»¶¾·»&$ 7 %Ä´iÒ ²º ^ZӰȼSÓ7ºC» K ¾lÑÒ lkwhwb 7 7 B " ¬ · º 1 Ò Ñ 7 Ò ] Ó 7 \ Ñ Ó º O » i ¹ N vkwhib 7 7 " ÑÓ»P²»¶¾ Ó ´¶º)ÓO^hA² oº´¶´¶ºS»O´§¼ ²º »«¾}²ÓE4·Z»«Ò¾}Ò#ÓMÑlµ1°pÓ¯±µ¬»È´§·o²o´¶º}º Ò vÓ7² ° G MÓµ C´µ¸¶º · (Ó N²´§º Ó7^Z¹ ÓҰȼSZE(Ó7ºkrhi¤b Ó ² $ Q 7 %Ï·¹òÒkw²hwb ´¶µÂ ¯C ¹ ´§Ó7ÒÆ »«¾}·o7 »JB krhib 7 7 " " Ê ( 7 &B
7 7 7 ( ( K ¾}Ó°p(Ó 7²°pÓ8t·Ò ( * ÅÓ]\7Ñ·o»È´§²oº Q T ¼o´_^Ó#Ò k7 w hwb ( 78=7 B B ( ( ( ·oº ¸»«¾}Ó¯C°¤²² ² ÄÒ#ÑZ¯}Ó7°#¹ ´¶º±Ó·° ²º ^ZӰȼSÓ7º Ó´iÒ ²µÂ¯C¹rÓ»PÓ 9
4Ö /l 6 K ¾Ó ²º^ӰȼvÓº Ó K ¾}Ó²o°pÓ7µ Ò RW·º .SEµ¸· vÓÑÒÓ ² ²oº ·>^Q´¶»ON ² Æ Ê·oº ²º^Ó¤³)´¶»PN² ±Æ Ê Q 7
0
;
;
H
5 "%$
[
I I
=
5 "%$
H
#"D$
E
c
[
c
0
I
I
;
H
;
=
0
0
0
[
E :
h
H
U
#"%$
"%$
#"%$
=
0
;
#"D$
Q
+
;
G
H
0
?
Q²´§· º ^Z,ÓҰȼSÒ#ÑlÓ7º µ1¯±Ó»È´§°p²oÓ7º}Ò#ÑlÒ ¹ »R w´¶·o´¶´Jº ·º Ó Ó7R9Ò#»Ô w´_· ^ 7 ¹ ´i Òy¾²[Ó L 2ÓUÑl^º Ó°N) Ó°Ä7²oÒ°»O°pÒ#²oµ¬ºS¼S²QÓ7²°»«¾Ì·ZÒº±Ò#²ÑlºCµÂ½ ¯C²»Oº´§²^º}Ó¤³ÒÄÒ#Ñl´¶µcº 7´¶¹r»O·´§°²º»PÒ ²¸·»«¾}º ²·ÒÓº`·´¶¹rº²¤¼v ²o8Ñ Ò ¹r² ·¹ Æ 7 ÊÆ !B 7 Ê > Æ !B 7 Ê Æ Ê#Æ DB 7 Ê 2 mvs z ± Æ 97 Ê Æ !B 97 Ê ±Æ 97 97 ÊÆ !B 97 Ê 2 ¡ sn ^ aĦ«q)kwkrq £ hisl ¢£ a m z qSulnJn ^ as}q)npm)nyhwqvs}x Æ 7 Ê 7 @ 7 Æ 7Ê mvs z ±Æ 7 7 Ê 2 7 PSR
H
P R
P R
@?
H
5
9
H
9%PSR
X
>
H
H
9
@?
5
!9%P R
5
>
>
9
C ;=< >
9
C PSR
@?=
C ;=< >
C P R
?=
ST]o?S
H
Ñl¹¶¹ · t°p²³v´¶µ¸·¹ ¹ ¼v²o°#´¶»¶¾lµ Ò 7²o° E·³v´¶µ Ñlµ !´ vÓ¹ ´r¾}²Q² tÒ»O´¶µ¸·»O´§²º ) ]J^ amvu}u qZ~lhwb¬monpa1À Å m)krvq hrn ^ b hix z a7ͱsla z m)x 7-= m vb¬m~ >V SÆ 7 Ê > 7 Æ !B 7 Ê > ÆÈZÊ Q Æ !B 7 Ê @ 7 Æ !B 7 Ê B 7 Æ !B 7 Ê 7 Æ !B 7 Ê X tmvu}n u n ^ qZhw~x hib¬uqvmohwnpsSanxynphwn auhwxÆO}hwb¬ZÊ!uChwq xÅ anpm)©sSjln hw¥npmoqÌkwasSbcn mvnpoq a¸m xpa Â¥)a a £ m)npk!qvs2|7qvxpb¬nyaQbcu\a® sS¡ n¤n xhi® x £ q)akwnpk`hw|Sa¬s}qÍ £ xps¬nÄn n ^ ^ m)m)n n 1¦«q a £ Q nyqS7 s xbc¢ an0 ^ q z ¢ mo kixyq 2-22 sl¢ q n £ ^ s a mhivslx!hwnyËhimohwx kw^hrªa a z xp® |q ]J hw^ sla Ä a7his¸¦«q xy n¤a mo¢ nphwmoxpn¸nyhi|kwax mv¢Sxy^ n¬m)¦«xAq x¤ j}uCxpb¬a mokrhikwslka¥m mo kijlm)aQxpx¬{lb¬qo¦Äulnyn qv^ nya hi|J |7aqvkimsS~}¥vmoa np hr)qSas s|7ua m m)m)nybcaa¨}npjla nÅ bcQ mZ{¢ n z ^ hw¥)aÌa m) )u}aJu hr¦qZ~lslhiq)bcnÅm)u ny a qSu a kw { mQ)krvq hrn ^ b n ^ a2bcnymZa {E b ¦§m)hrBktnyqW Æ |7LBqSsS¥)a 7 )Ê a2¦«q Æ LBn ^ a7 x¤ÊÄm)bcuCaa s}m)akrhwmvªxyaQqSx s}nx ^ mva xz ¦«hiq xyn¤m) Âs}|a a¸£ npqoqv¦Js n ^x¬aMb slaa7n ~l^ nq z hwny® a 7 m)nysÏa n ^ 78a= qvnny^qa n ^ ^ aMm)|s j z ¤¢ aQ¦«qsvn hw Âny7a a £ m)nynyqSa smo7 kw® )q  ahwn s}^ |b a ¢6£ £ hra¸n ^ |qvmvjls nÂhixpsSmvny| a h u ÍC7 |ahwnÄs}¸n ^ hrn¤hwx x1npx¤aj} ub a kwm)hix slam m am)Sxpjl{kwb¬m ulhwªnpq)m)npnyhwhw| qvs |qv£sS^¥)hwa | ^ vaxps}np|m)a ¨}hr kwmohrªnpaa)xÄ® n ^ { a2m)uuqSu xpxp hiqS¨lu kr{ hwm)z nyhra ¥va| ^ )q)aQhisv|7an q) ¦ $ Q 7 %¸n ^ a¸hw nya m)nya ]J^ 78= z |m)sE¨ac¦«q |a z nyq aQ b¬mohis hism avhrqSs m qvz js z 7 q¥) a ,££^ hw| ^ n ^ a¬^ ©j}m ]Gz monyhi|¬ bcq z ak npaÆ| ^ Ê slhihi©xcjlm)a | |j l monyav )® Ô® hix h aQmÌhix ¥va {EuCqvujlkwm his qSulnyhibchrªQmonyhwqvs/m)s hixc¨anynya Ss}q sÏm)x n a jxyn a)hwqvs 4Ö /l #º/µ¸·º N ·ZÒÓ#ÒM · \Ñ· °¤·»O´ ·#¯)¯±°p²³v´¶µ¸·»O´§²º ² 4·ÌÒ#´¶ºS¼)¹rÓ ²º`ÓM² 2»«¾}Ó4O» L ²Ì»ÔÓ°#µ Ò Æ Ê4²o° ±Æ 7 Ê ´iµ¸Ò ·Ò#»O Ñ ´§²ºW7´§´iÓÒ1º¯±»°p»ÔÓ ²7Ó7°p² · 7»P·¹wÓ¬´¶º »Ô²M· »«¾}¹rÓ¸²Ò7·#Ó ¯)¯C7°¤²²°#³vµ ´¶µ¬µ¸·o»È·´§²o³vº´¶µc² b´ a·o»È²´§²o»«¾ º »ÔQ#ÓR °#>µ TÒ ´i·Ò 7»OÑlÑ°pÒ·Ò¹¶Ó ¹ N · L¾}^ÓÓ º 78 Ó¤·²oÒ#° ´ 7¹r²8Ó º7Ò°¤Ñ Ó7»P¾Ó7º`·2Ó#Ò°pÒNÓ toÑ´¶ºE¤Ó »«M¾}Ó ·¯)Ò¯±Ó]°p\²Ñ³vÓ7´¶¹ ½ ·o·o¹¹ ¼v»¶¾²o²°#Ñ)´¶»¶¼Z¾l¾Eµ+²´¶Ñlµ1° ¯±°p¹rÓ7Ó7Ò#µ¸Ñl¹Ó»§ºÒ1»P¾}Ó ²o¹ L!´¶7»«²¾ ° »«»¶¾}¾Ó Ó°pÑlÓ ¹¶o¹Ñ O» L ¤Ó ²o½P»Ô·Ó¯v°#¯Cµ °p²·³)¯)´¶¯±µ¸°p·²»O³v´§´¶²µ¸ºÏ··o»O´§¹òÒ²²º ²L ºCÓ1i¹ N¯± °p>² ^Ó ¤²oº ^Ó7°O¼vÓº Ó 7²°»«¾}Ó1¯±°p²³v´¶µ¸·¹}¯}²´¶ºC»
;
H
JIK M
N
PSR
aX
5
5
@>
H
H
0
?
?
8
;
>
0
C
&
;
P R
+
2E
6
h
7
Z
=
E %
!
%
%
L
cP R
Z
D
%
.>
@?
[
Q
-
# !
#,
E
#
]J^ a 1 ]aSaQs ¢) ²9a ny a E® )¢ ½ % :>S s¸Q!jR xy aq)¦\T Ç É mokw)q hwn ^ b¦«q uaQs}mokwhwªa z krhioakrh ^ qq z axpnyhibcm)nyhwqvs\® ¢ c 4C»P·» &\² \Ó7-°
o !EC ®\®\ËCK a°px¤·oxyº}kwa Ò- m)C´s ¼)z º±·o¹ ® t°p²® ÂÓ#Òa Ò# ´¶q ºS¢ ¼ ¢ ¥vfq)u}kOmv®l|7dSa ÁÔm)¢ krunpu\a ®s} omovnpohwsld}4 o)a)s} a ¢ movkwhroªda z® a~luCa|npm)nyhwqvsÁPb¬m~lhwbchixpm)nyhwqvsÌm)krvq hrn ^ b ¢ v cÂ' Ó#® ÒÅÓ·akw° a¤j6¾ ~ c¢ ÓPf±¯}® ² °#»^} a nyhwa< s ¢ Ëtc ®SË q 6¨C¾la»Èx »r¢¯ mvs zLLL ®S wÉ ´¶ºC °#´§^ · m z ° h c¢ c pUc |7K qv)b¬¯CÑ uCqv¹ sl´ aQ·sS»On´§Á²£ ºhwÒ#xp½ aJ°pÇ · É ¾l»Èµcm)kr¹ v¢ q  hrjln ^ ®6b )¦«q v }®bchr~n¤j ax ¢
/#
Z
=
@)
#
0
#0#m&
8
&
0/
/
L
j
E
m
/
/
Z
/
,
[
m)
#
#&
Y3
.
j
0
[
8
kk
&
o]o
#
8
,
÷ #
kp q2rs
Z
1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
0
10
20
Ëhwvj a¬ hw)his}m)k`u ^ m)sSnyqSb
50
60
70
ËhrSj a t kwj ¤ a z u ^ m)sSnpqvb
50
60
70
30
)
8
40
5
4.5
4
3.5
3
2.5
2
1.5
1
0.5
0
0
10
20
30
40
@)
Q
C»Ó ¯l¾}·oº±Ó6¾°Ó »È´§Óº ¹ °pÓ
Ó°p² Y
34
Plain EM Trust Region True Likelihood
32
30
28
26
24
22
20
18
16
14
0
20
ËhrSj a !
