Shrinkage Estimator Generalizations of Proximal Support Vector ...
Shrinkage Estimator Generalizations of Proximal Support Vector Machines Deepak K. Agarwal AT&T Labs-Research 180 Park Avenue Florham Park, NJ 07932, USA
[email protected]
m*( Y 9u0& 8 X j@G 9 ( 9Q #%& G ( )* ~%& 2 )[3-',60F78X + o F ) O [ [O .
!" $#%& "'$() * +!,-. ( )* / %& 2 ) G3'$,60&7/H^T2 !%62 9C N 9 / 10& 2 43/5'$,60&75 8 9: ??8@A ) B .
w`( OWJY2 x9 o #%& / C3D) 7E'$() * +!,- 50F 2 G3'$,60&7/H w( ( X C j/2) 1Y'$,6 0`( Z %6OWJSNWg + E K Mc 9Q 6 4 O x N . 4 A2 F-',60I( J 1 6K)L 69 ( 1 [NW& +`( +9d CR/2)k9 o) )x%6 %6n 9 4 ML 1N O 9 .P + Q%62 9RK STRR2 N)%&HA^G2 1 +( [ T/2) k2 [ U F O x 1 . +/ U %6%V( XWY Z &%[ +\2 ]2 XW]W H_^ 2 . \ 2 %hST/2J %6 #XWQ +%6X 1/ e2 k r `( a Z %b/2) c/ /() / 6 +%& §XHrY2 p< ( + X x) %6 9 XHr}-# 12 [ +[ - ( . !K MS5 +2 $Sw-',60] `( ) E U TiL W +X ( ~ 8 + ~ +/ 2 Z S< 9A 9[( 9UST2[#c %6 %6 9 9 o )942 ( 2d 6p< () O A 42 V 9 9 H j 6ST2\2 1K ~9 %[ l2 1 X e Z( r2 F + HR^T2 A 2W8 X A:2 6 l NWr2 -* + c 9 /-9 N)( Ö-7X× ØÉUØ7ZÙ$·*Ú*3+ÍÛ 3Z·&Í ³\7/¹jÉ[Ü!Û3Z·lÍl³\77M¨ST2 CÚ[ a2 Þ 7 N )U( T/ U %6. ( G 9 G o 9) a) HEêr L2 ~SG WV 9 F8 Z % Q#/ U%& #%V() \/ & %[)( 12 9 ) X )H qr 9 ( + 9A M¨SL l 2l2 ¢õõõ!¢ ©? Hl¬5( 6 V( VN*C +V A ( We2 2 15'$,60%6 9 B/2) a H 5.1 Ionosphere Data
0
q
10
r o
20
s
30
t
40
50 6
-0.4
p
8
10
ridge estimates 0.0 0.2 0.4
uridge trace plot for linear terms for ionosphere data
k
2
p
0
10
q
20
o
r
30
s
40
t
0
-0.4
ridge estimates 0.0 0.2 0.4
4
ridge trace plot for quadratic terms for ionosphere data
50
k
vGwV ] 4RxS UzyBWV R{Q |R®OH[^T2 9 US< l + d Wn9RY§5( J'( 8 C C . V%& 2 H5¬( c 8 +) X N)-2 T0[} =j,6M%69 O 2 ( +- HE^T2 -9j " ) !@>! () ( 9
9l C) %[ ( 9l 9 a 9Q!ôrq6'[SG2 2F 9 . 2 V #%6XW4 H®© « 5'$,60F06yX| ¥ §H¦©« ¥>H ¤8® «
vGwV ] 4R Um c 4R w[ RV *W WR ^RCcXR/[) -ZX*Ldc Y $)5 )! l *$w *c)-!E VWR *dR 6X`WlR * XR 4 [ )*d 8$dR xxWR ^R ! W4V *WR El ) ^R?XZ i X`WUR |R !Y *w3Z? ¢ ?87
U %& C %6 9e d5'$,60F² VST2!ôrq6')H"^G2)1/ 0F'}: C'$,6ýRS< a a8 9BNW\§E Z T %6 jHG'( . WMS5cN8/c8 Z %& L 2 L/ +c c +2 K 2 +* ) S< 9A?8H
/ 912 r9 N +G + + V -©?"®L 9 5 A %6 V3Z¤?8@~ 9 7M$) 9j / V %[)3§? 9 7~ 9C %6 V3>©G 9 7j -&§XH!q ) /( ( - 9 5S 9\ T?l 9B + 94NW1 G 9 9k9)O jHc\ o o ® E !¤"©!9 !8 ST2A®~ ( ( 9 r 94 - jo 9V( +9U 2 r9 L B>G +/ LSG2x@¤8©V 9x>??V + 5 8WH -o 9COS a%6 9 c T2 j9 +$M5'$,60]SG24 ap< ( + X K - 9F-',60F06yX|]ST2x 2 6® 9 562 T
9l L +$H!^j( [ %6 TSLB " +94NW %6 ~ j - %& ¨) 9j / V +$HcyXU N ¥¥ / r j .P %6W 2 %6 & 9= O M ^ #c x9 ( Ê@¦¦® HÏ^T2 Q %& W ()1 T Z %& e U2 = O A q =5 +. J3= q6= 7HrqT$ 2 9g : 2Ô2 ( +\= X Y R ( ( -2 + 1 9w 9 9 + HC^T2 & 9 9 + S 9 9x N) k( r 2: O ( 9)A 9& ( 9 C 2B2) ( © ¤8¦ H ?8>? > ©¥ H ?8@© @ ®> H ?8@ § § H ??8© § H ??8@ > ¥
H@¦ ® H>@ ¥ H>¤" H>¥ © H@ ¥ @H
¥©>H¥¥ ¦>H@¦ @¤8>H©§ ¤¤8H©® @®H© @©H@
Q) R¥ U 7M 2 4X + o Q N)%I 6 + 9RNW %6 %6 Y¶\>.PX + V N)%6 HFY2e 9 L 3DH 8lH Hr F : 1 8 r 5Nj 9 M) / + / W
9 9j7~2 6X + o \) N %_ ( 9&N8[ 9\( +)
4 9 ! +* + %6 9)3Z B 9l Z G2 7H qr 2 U +( l 6 + x)9 k x ed2 4 1 9) j C5'$,60F06yX|6H "( 4W/2) R2 k A )9R ??8@H @?2 ¯H m*9 %& M^UHô6 M" 9QýaH^TN +2 XH q 9 9 + q + ESu jN8 8H à ¹^º » ´ ´ µ^¶  ±\· À ¸wµ^¸Á¾ Âg¸w¾ ¿/Â+ME§3>7\¥ 8¥¤8Mqr +G>???"H @BpCHm*() & 9e¬ H!0& 8 X jHE8 #%& c ( 8 G%& 2 ) O +o HyXbËC³±d¿gººgÏ^¾ ´ ÆdÂm±\· ¸Á¹^º À ºdÐ/ºd´ ¸w¹ » ÉeÄÚÀ È/ÛDÜ × ×âÈF´ ¸Áºd³\´ µ^¸w¾±^´!µ¶lÉe±^´4·9ºd³º/´ ¿º±^´ Üm´ ±^Þ½¶ ºgÏ^ÆdºI× ¾Â¿g±^Ðdº/³gÑ,µ^´ Ïà× µ|¸wµÄá¾ ´ ¾ ´ ÆMj U¥\¥ 8§®H ' 4m* 8ME!q6M H6'$q6M¨qr( () C>??8@ H @¤2 ¯H"pC K k 9Q0FH!pN*OH"= a%6) 6ST2l + 9 () k %6 X 8N G ( ) HyXLË ³±d¿gººgÏ^¾ ´ Æd±\· ¸Á¹^º$À ºdÐ/ºd´ ¸Á¹ » ÉeÄÚÀ È4ÛCÜm× ×ÝÈF´ ¸wº/³\´!µ^¸Á¾ ±^´=µ^¶DÉe±^´4·9º/³ºd´ ¿º ±^´&Üm´!±^Þ½¶ßºÏ^Æ/º,× ¾Â¿g±^Ðdº/³gÑ,µ^´ Ïà× µ|¸wµÄá¾ ´ ¾ ´ ÆMj §g¥ 8¦ ®H"' lm* 8ME!q6M H6'$q6Mqr( ( +U>??8@ H @©®g¥ 8> ®§ME@¦¥© H >?6' H4! +1 9¯ H) lôr (S Hcýj9) %& A V H » Ö^ÖM¶ß¾ ºgÏÀ ¸wµ^¸Á¾ Âg¸w¾¿dÂM¤8@ @¦g@ 8>?8@M @¦¦> H >@rq6H¨ã!B 9\0&Hc'$) )( Hcýj9) %& Al + ½H Ée± ¼I¼ ²=´!¾¿µ^¸Á¾ ±^´  ¾´À ¸Áµ^¸w¾ Âg¸Á¾ ¿dÂgÎ%Ë µ^³\¸ ä-å^À ¾ ¼ ²=¶ µ^¸Á¾ ±^´jµ^´!Ïbɱ ¼ ÖM²¸Áµ^¸w¾±^´ Mc@¥@> g@ 8@>©¥ME@¦§§H >> æ6H¨ã!B 9\¬> g¥ 8>§Hêr 2 ôr O 9R"( N 2) x %6 W3Z}5 7/M q %[ +/+9 %&M @¦¦> H >®6EHc0[() 2Wx )9R= Hqr2 H Hk!y* + WB j%& 2) k9j N H STSGSUH H ( OH 9)W( N Ñ-%6 W N06ãLýj8 WH 2/%6OH >¥6iLH!'$ 9 Hc¬A 9 WB9 B n9l a%[ 9 -H ɱ ¼I¼ ²´ ¾ ¿gµ^¸Á¾ ±^´  ¾ ´ À ¸Áµ^¸w¾ Âg¸Á¾ ¿dÂgÎ ËCµ^³\¸ » å à ¹^º±^³\Ñ µ^´ ÏÄ&ºd¸w¹^±/Ï^Â+M >@>>>g¥ 8>>¤"®M!@¦¦>H >§6pCHc'$%62R 9lmaH"T %6 N*OHq`( C + %[
[email protected]
m*( Y 9u0& 8 X j@G 9 ( 9Q #%& G ( )* ~%& 2 )[3-',60F78X + o F ) O [ [O .
!" $#%& "'$() * +!,-. ( )* / %& 2 ) G3'$,60&7/H^T2 !%62 9C N 9 / 10& 2 43/5'$,60&75 8 9: ??8@A ) B .
w`( OWJY2 x9 o #%& / C3D) 7E'$() * +!,- 50F 2 G3'$,60&7/H w( ( X C j/2) 1Y'$,6 0`( Z %6OWJSNWg + E K Mc 9Q 6 4 O x N . 4 A2 F-',60I( J 1 6K)L 69 ( 1 [NW& +`( +9d CR/2)k9 o) )x%6 %6n 9 4 ML 1N O 9 .P + Q%62 9RK STRR2 N)%&HA^G2 1 +( [ T/2) k2 [ U F O x 1 . +/ U %6%V( XWY Z &%[ +\2 ]2 XW]W H_^ 2 . \ 2 %hST/2J %6 #XWQ +%6X 1/ e2 k r `( a Z %b/2) c/ /() / 6 +%& §XHrY2 p< ( + X x) %6 9 XHr}-# 12 [ +[ - ( . !K MS5 +2 $Sw-',60] `( ) E U TiL W +X ( ~ 8 + ~ +/ 2 Z S< 9A 9[( 9UST2[#c %6 %6 9 9 o )942 ( 2d 6p< () O A 42 V 9 9 H j 6ST2\2 1K ~9 %[ l2 1 X e Z( r2 F + HR^T2 A 2W8 X A:2 6 l NWr2 -* + c 9 /-9 N)( Ö-7X× ØÉUØ7ZÙ$·*Ú*3+ÍÛ 3Z·&Í ³\7/¹jÉ[Ü!Û3Z·lÍl³\77M¨ST2 CÚ[ a2 Þ 7 N )U( T/ U %6. ( G 9 G o 9) a) HEêr L2 ~SG WV 9 F8 Z % Q#/ U%& #%V() \/ & %[)( 12 9 ) X )H qr 9 ( + 9A M¨SL l 2l2 ¢õõõ!¢ ©? Hl¬5( 6 V( VN*C +V A ( We2 2 15'$,60%6 9 B/2) a H 5.1 Ionosphere Data
0
q
10
r o
20
s
30
t
40
50 6
-0.4
p
8
10
ridge estimates 0.0 0.2 0.4
uridge trace plot for linear terms for ionosphere data
k
2
p
0
10
q
20
o
r
30
s
40
t
0
-0.4
ridge estimates 0.0 0.2 0.4
4
ridge trace plot for quadratic terms for ionosphere data
50
k
vGwV ] 4RxS UzyBWV R{Q |R®OH[^T2 9 US< l + d Wn9RY§5( J'( 8 C C . V%& 2 H5¬( c 8 +) X N)-2 T0[} =j,6M%69 O 2 ( +- HE^T2 -9j " ) !@>! () ( 9
9l C) %[ ( 9l 9 a 9Q!ôrq6'[SG2 2F 9 . 2 V #%6XW4 H®© « 5'$,60F06yX| ¥ §H¦©« ¥>H ¤8® «
vGwV ] 4R Um c 4R w[ RV *W WR ^RCcXR/[) -ZX*Ldc Y $)5 )! l *$w *c)-!E VWR *dR 6X`WlR * XR 4 [ )*d 8$dR xxWR ^R ! W4V *WR El ) ^R?XZ i X`WUR |R !Y *w3Z? ¢ ?87
U %& C %6 9e d5'$,60F² VST2!ôrq6')H"^G2)1/ 0F'}: C'$,6ýRS< a a8 9BNW\§E Z T %6 jHG'( . WMS5cN8/c8 Z %& L 2 L/ +c c +2 K 2 +* ) S< 9A?8H
/ 912 r9 N +G + + V -©?"®L 9 5 A %6 V3Z¤?8@~ 9 7M$) 9j / V %[)3§? 9 7~ 9C %6 V3>©G 9 7j -&§XH!q ) /( ( - 9 5S 9\ T?l 9B + 94NW1 G 9 9k9)O jHc\ o o ® E !¤"©!9 !8 ST2A®~ ( ( 9 r 94 - jo 9V( +9U 2 r9 L B>G +/ LSG2x@¤8©V 9x>??V + 5 8WH -o 9COS a%6 9 c T2 j9 +$M5'$,60]SG24 ap< ( + X K - 9F-',60F06yX|]ST2x 2 6® 9 562 T
9l L +$H!^j( [ %6 TSLB " +94NW %6 ~ j - %& ¨) 9j / V +$HcyXU N ¥¥ / r j .P %6W 2 %6 & 9= O M ^ #c x9 ( Ê@¦¦® HÏ^T2 Q %& W ()1 T Z %& e U2 = O A q =5 +. J3= q6= 7HrqT$ 2 9g : 2Ô2 ( +\= X Y R ( ( -2 + 1 9w 9 9 + HC^T2 & 9 9 + S 9 9x N) k( r 2: O ( 9)A 9& ( 9 C 2B2) ( © ¤8¦ H ?8>? > ©¥ H ?8@© @ ®> H ?8@ § § H ??8© § H ??8@ > ¥
H@¦ ® H>@ ¥ H>¤" H>¥ © H@ ¥ @H
¥©>H¥¥ ¦>H@¦ @¤8>H©§ ¤¤8H©® @®H© @©H@
Q) R¥ U 7M 2 4X + o Q N)%I 6 + 9RNW %6 %6 Y¶\>.PX + V N)%6 HFY2e 9 L 3DH 8lH Hr F : 1 8 r 5Nj 9 M) / + / W
9 9j7~2 6X + o \) N %_ ( 9&N8[ 9\( +)
4 9 ! +* + %6 9)3Z B 9l Z G2 7H qr 2 U +( l 6 + x)9 k x ed2 4 1 9) j C5'$,60F06yX|6H "( 4W/2) R2 k A )9R ??8@H @?2 ¯H m*9 %& M^UHô6 M" 9QýaH^TN +2 XH q 9 9 + q + ESu jN8 8H à ¹^º » ´ ´ µ^¶  ±\· À ¸wµ^¸Á¾ Âg¸w¾ ¿/Â+ME§3>7\¥ 8¥¤8Mqr +G>???"H @BpCHm*() & 9e¬ H!0& 8 X jHE8 #%& c ( 8 G%& 2 ) O +o HyXbËC³±d¿gººgÏ^¾ ´ ÆdÂm±\· ¸Á¹^º À ºdÐ/ºd´ ¸w¹ » ÉeÄÚÀ È/ÛDÜ × ×âÈF´ ¸Áºd³\´ µ^¸w¾±^´!µ¶lÉe±^´4·9ºd³º/´ ¿º±^´ Üm´ ±^Þ½¶ ºgÏ^ÆdºI× ¾Â¿g±^Ðdº/³gÑ,µ^´ Ïà× µ|¸wµÄá¾ ´ ¾ ´ ÆMj U¥\¥ 8§®H ' 4m* 8ME!q6M H6'$q6M¨qr( () C>??8@ H @¤2 ¯H"pC K k 9Q0FH!pN*OH"= a%6) 6ST2l + 9 () k %6 X 8N G ( ) HyXLË ³±d¿gººgÏ^¾ ´ Æd±\· ¸Á¹^º$À ºdÐ/ºd´ ¸Á¹ » ÉeÄÚÀ È4ÛCÜm× ×ÝÈF´ ¸wº/³\´!µ^¸Á¾ ±^´=µ^¶DÉe±^´4·9º/³ºd´ ¿º ±^´&Üm´!±^Þ½¶ßºÏ^Æ/º,× ¾Â¿g±^Ðdº/³gÑ,µ^´ Ïà× µ|¸wµÄá¾ ´ ¾ ´ ÆMj §g¥ 8¦ ®H"' lm* 8ME!q6M H6'$q6Mqr( ( +U>??8@ H @©®g¥ 8> ®§ME@¦¥© H >?6' H4! +1 9¯ H) lôr (S Hcýj9) %& A V H » Ö^ÖM¶ß¾ ºgÏÀ ¸wµ^¸Á¾ Âg¸w¾¿dÂM¤8@ @¦g@ 8>?8@M @¦¦> H >@rq6H¨ã!B 9\0&Hc'$) )( Hcýj9) %& Al + ½H Ée± ¼I¼ ²=´!¾¿µ^¸Á¾ ±^´  ¾´À ¸Áµ^¸w¾ Âg¸Á¾ ¿dÂgÎ%Ë µ^³\¸ ä-å^À ¾ ¼ ²=¶ µ^¸Á¾ ±^´jµ^´!Ïbɱ ¼ ÖM²¸Áµ^¸w¾±^´ Mc@¥@> g@ 8@>©¥ME@¦§§H >> æ6H¨ã!B 9\¬> g¥ 8>§Hêr 2 ôr O 9R"( N 2) x %6 W3Z}5 7/M q %[ +/+9 %&M @¦¦> H >®6EHc0[() 2Wx )9R= Hqr2 H Hk!y* + WB j%& 2) k9j N H STSGSUH H ( OH 9)W( N Ñ-%6 W N06ãLýj8 WH 2/%6OH >¥6iLH!'$ 9 Hc¬A 9 WB9 B n9l a%[ 9 -H ɱ ¼I¼ ²´ ¾ ¿gµ^¸Á¾ ±^´  ¾ ´ À ¸Áµ^¸w¾ Âg¸Á¾ ¿dÂgÎ ËCµ^³\¸ » å à ¹^º±^³\Ñ µ^´ ÏÄ&ºd¸w¹^±/Ï^Â+M >@>>>g¥ 8>>¤"®M!@¦¦>H >§6pCHc'$%62R 9lmaH"T %6 N*OHq`( C + %[
