BIAS ROBUST ESTIMATION OF SCALE by - Semantic Scholar
BIAS ROBUST ESTIMATION OF SCALE
by
R. Douglas Martin Ruben H. Zamar
TECHNICAL REPORT No. 214 August 1991
Department of Statistics, GN-22 University of Washington Seattle, Washington 9819S USA
Bias Robust Estimation of Scale R.D. Martin University of Washington Ruben H. Zamar University of British Columbia August, 1991
tun 0, "'It E R. Then (a), (b) and (c) hold uniformly on C. Remark. Suppose that a certain function Xo satisfies (AI) and (A2) and is such that
hxo(s, t) =
-~EFO [x~ (X s
t) xi t] < 0, "'Is> O,t E Rand Xo(x)
1,
Vlxl > xo.
Thenthe set
[(X - t) X- t]}
I C = { X : X'(x):::; X~(x) , V x > 0 and hxCs,t) = -;E Fo X' - s - - s -
satisfies the assumptions of Theorem 1.
3. GENERALIZED BIAS Although the M-estimates of scale with general location introduced in Section 2 are Fisher con~istent at the nominal distribution Fo, they are in general asymptotically biased for F E Fe. Furthermore, the "raw" asymptotic bias Br[S(F)] = S(F) - So can be of two distinct kinds: When F is an outliers generating distribution, the bias Br[S(F)] is positive, and when F is an inliers generating distribution, the bias Br[S(F)] negative. As in Martin and (1989), we consider generalized bias functions which are of positive and negative bias is independently chosen, by allowing the user to put positive and negative bias on different scales. Specifically, we define the bias if 0 < S(F) :::; So if So < :::; 00 are COfltULUoUS, non-negative
o 00.
(9)
are interested
maximum geileraJ.u~ed bias,
B(c) = maxB[S(F)]. FE:Fe
(10)
l,From monotonicity L I and L 2 , it follows that B(X, T) = max {LI[S- / so] , L 2[S+ / so]}, where S- and S+ denote the supremum and the infimum of the functional S(F) as F ranges over Fe. 4. BIAS ROBUST HUBER ESTIMATES
In view of the historical importance and high degree of familiarity of Huber (Proposal 2) estimates we first focus on obtaining bias robust estimates in this class. To emphasize the dependence on X and t/J we use the notation S(F, X, t/J), S+(X,t/J), S+(X,t/J), etc. The first step toward finding the bias robust Huber estimate is deriving the expressions (16) and (17) for S-(X,T1J;) and S+(X,T1J;)' Claims which are made below without proof can be easily verified under (AO)-(A3). The maximum value S+(X, t/J) of the scale functional S(F, X, t/J) is produced by a point mass contamination at infinity, 600 , and such contamination also produces the maximum value of the location estimate T1J;(F). The estimating equations in this limit case are
(1 - c)EFoX[(X - t)/ s1
= b(X)
(11)
and
(1 - c)EFot/J[(X - t)/8] + e = O.
(12)
Let ,At) be the unique solution of (11) for fixed t and let '1'1J;(8) be the unique solution of (12) for fixed s :> O. The fUIletioIlm ,1J;(t) 1'1J;bx(t)] is continuous and non-decreasing. Also, the pair (8*, t*) simultaneously satisfy (11) and (12) if and only if t* mx,1J;(t*) and s" = IX(t*). following lemma characterizes maximum asymptotic bias due to outliers nrovides an algorithm location of
x
IS
For each n ~ 1 let t n m x ,'4t(t n -d , with = inf{t > to : = t}. (a) t n = t'" Sn = IxCt*) = ,(b) the maximum asymptotic bias of the location estimate T( F, x, t/J) is t" and the maximum value of scale functional S(F, X, t/J) is S+(X, = s". LEMA-fA 2. Suppose that to by (13). Sn
UUJln_oo
The minimum value of the scale functional S(F, X, t/J), S-(X, t/J), is produced by a point mass contamination 80 at zero. In this case the estimating equations are
(1
€)EFoX[(X - t)/s]
+ €x(tls)
= b(x)
(14)
+ €t/J( -tis) = o.
(15)
and
(1 - €)EFot/J[(X - t)/s]
By monotonicity of t/J, t = 0 for all s > O. Let 9;1 be the inverse of 9x(s,t) with respect to s, for fixed t, Then, from (14) with t = 0 it follows that (16)
Optimal Centedng The of t/Jhas an effect on the maximum asymptotic bias of the scale estimate by virtue of affecting the bias t* of the location estimate. Observe that since S-(X, t/J) doesn't depend on t/J (see (16)), the optimal choice of t/J must be based S+(X,t/J) alone. It follows from Lemma 2 and (11), with t t", that
(17) Since for all 0 < a < 1 the function 9;1(a) is non-decreasing in t. Therefore, by the Huber (1964) result, ~ to all s > 0 and
X satisfying
dIt1tlcult to 'neorem 2 holds for the class of all M-lestima,tes of scale with cell. teriag functional T(F) navmz the "monotonicity" property
T(F)
~
T[(1 - c)Fo +
(18)
The Minimax-Bias Huber Estimate of Scale By Theorem 2 it suffices to consider S+ (X, 't/Jo) and S- (X, 't/Jo) and the function 't/J can be dropped from the notations. It will be shown that under certain conditions the maximum generalized bias B(X) (see (10)) is minimized by a jump function Xa* (see (6)). For each a > 0, let B(a) = B(Xa) and
b(a)
= b(Xa) = 2[1 -
(19)
Fo(a)].
We begin by showing that given 0 < c < .5, Fo, L l and L 2 there exists a" such that
(20)
Va> O.
Let ao = Fo- l [(1 + c)/2] and al = FO- l [(2 c)/2]. ~From (19), the corresponding values of bare bo b(ao) 1 - c and bl b(al) = c. Hence, letting S- (a) = S- (Xa) and S+(a) = S+(Xa), we have lim S-(a) =
a.....ao
_l
go
rb(a)]
II - c
.
1
-1 [
lim -F a.....ao a o
1 - 2(
b( a)] 1 -1 ) ) = -Fo (.5) = O. (21 1- c ao
and
}1I~\ S+ (a) -
11'm g-:1 [b(a) - c] t" 1_ C
a.....al
lim .!:.F,-1
a.....al a
0
2:: a.....al 11'm g-1 [b(a) - c] 0 1- c
[1 _2(1 b(a) - c] = ..!-FO- l(l) = c) al ---+
ao
0) of that On it mation IS remaraamy more tnas-robustness mnuence curve gross-error-sensitivity as a measure
same nussenee parameters. For approximation to maximal bias curve the impact of the bias of estimation of the nuisance of the Madm bias of the Shorth is unaffected location parameter. Since the by the asymptotic bias of the location estimate, the GES approximation is better in this case.
10. PROOFS OF LEMMAS AND THEOREMS The following lemma is needed to prove Theorem 1.
LEMMA 3. Let 0 < 81 < 82 < 00 as Lemma 1. Suppose that (A-0)-(A-2) hold and also assume that X and hx(s, t) are continuous and hx(s, t) < 0, V s > 0, t E R. Then, for all K > 0, we have: (a) X[(x - t)/s] is uniformly continuous on (x,s,t) E RX[Sl,S2]X[-K,K]. (b) S(F,t) is uniformly continuous on t E R, uniformly on F E:Fe (c) X[(x - t)/S(F,t)] is uniformly continuous on (x,t) E Rx[-K,K], uniformly on FE :F€. Proof. Let 6 > 0 be fixed and let B = [Sl' S2] x [- K, K]. Since X(x) is continuous, even, monotoneon[O, and bounded, it is uniformly continuous. Let ~l > 0 be such that Ix- xII < Al implieslx(x) - x(x')1 < 6. Also, since liml$l->oo X(x) 1, there exists Xo > 0 such that Ixl > Xo and Ix'i > Xo imply Ix(x) - x(x')1 < 8. Let Xl > 0 be such that if Ixl > Xl, then I(x-t)/sl > xofor all (s, t) E B. So X[(x t)/8] is uniformly continuous on {x : Ixl > Xl} x B. If Ixl :::; Xl then, s - $-/ I :::; xII~ ~I +1~1-1;1 and (a) follows. To prove (b) notice that assumptions on hx(s, t)
0)
exists ~
> 0 such that It - t'l < A, ItI :::; K, WI :::; K imply
1
X-t' [ S(F,t) - 6 -
>
1I - 0 there exists 80 > 0 such that (35)
By
lllE:orem
(36)
> (30) tonows
(33),
(35).
) can
way, if 0 '5 x '5 a if a ~ x ~ a if x ~ a 8,
8
proved
a similar
Ss(F,t) = inf {s : EFPs[(x - t)/s]
b(Xa)}
0
(38)
ACKNOWLEDGEMENTS We wish to thank Ricardo Fraiman for some useful comments with regard to establishing uniform consistency of M-estimates of scale. We also thank the referees for a number of very helpful comments and suggestions.
FIGURE 1.Maximum Bias Curves
H95
...
GES(H95)
GES(MADM)=GES{SHORTH)
--- ----- --- ---
--- ---
----- --- ----
_
Q
ci
0.0
0.1
0.2
0.3
EPSILON
0.4
0.5
HUBER, MADM AJ."JD SHORTH. LOG SCALE
I I I I I I I
ISS co
1.1,
Q
I'
I
• SHORTH
I' I'
•
I' (J)
:;!; al
•
I'
co
I'
Q
/
I'
. .. /
/
/
....
/
" ". . /
Q
-:
.
.
•
.
• •
.
....Q q
0.0
0.2
0.1
0.3
0.5
0.4
EPSILON
(a) Maximum Bias Due to Outliers
I
MADMooSHORTH
I
I
'JflE~ /
Iss
/
I
'I
/
'I
co
I'
V
Q
I
I'
I
I I' (J)
:;!;
m
I'
(Q
/
Q
/ I'
/ I'
/
....
/
Q
N
c::i
'"
.- "
.-.-
.-"
/ .- .-
q
0.0
0.1
02
0.3
EPSILON
0.4
0.5
It.::IUMC
} and logarithmic loss function.
#3. Mean-souared-error curves for n=20,
N(0,25) contamination
N(6,1) contamination ~
C\I
rej[.01 ] shorth A-est
t.()
.,...
U1 .,... w
~ .,...
(f)
~
~ .,...
t.()
-"
c.i 0
0
c.i
c.i 0.1
0.2
0.4
0.3
0.0
0.1
0.2
0.3
EPSILON
EPSILON
U(3.5,10) contamination
N(10,1) contamination
o N
~
C\I
U1 .,...
U1 .,...
/ /
/ ~
.,...
~
............
t.()
w
(f)
/
0
.,... t.()
c.i
c.i I
__
0.0
~ 0
0.1
0.2 EPSILON
0.3
0.4
0.0
0.1
0.2 EPSILON
0.3
curves tor n=2U. t-=NlU.l1 ana looantnmlc loss function.
N(O,25) contamination
N(6,1) .,...
.,...
(j)
......-
amole (n=20) and asymptotic maximum bias curve for MAD and SHORTH for F=N(0,1)
N(6,1) contamination
N(0,0.01) contamination
MAD n=inf SHORTH n=inf MAD n=20 SHORTH n=20
C\! .,...
//
o
.~
0.0
0.1
lO
.,...
-
h
co
"."'~~
.f'
~
~,/..~:
--
0.2
.....
h'
h
/
./
by
R. Douglas Martin Ruben H. Zamar
TECHNICAL REPORT No. 214 August 1991
Department of Statistics, GN-22 University of Washington Seattle, Washington 9819S USA
Bias Robust Estimation of Scale R.D. Martin University of Washington Ruben H. Zamar University of British Columbia August, 1991
tun 0, "'It E R. Then (a), (b) and (c) hold uniformly on C. Remark. Suppose that a certain function Xo satisfies (AI) and (A2) and is such that
hxo(s, t) =
-~EFO [x~ (X s
t) xi t] < 0, "'Is> O,t E Rand Xo(x)
1,
Vlxl > xo.
Thenthe set
[(X - t) X- t]}
I C = { X : X'(x):::; X~(x) , V x > 0 and hxCs,t) = -;E Fo X' - s - - s -
satisfies the assumptions of Theorem 1.
3. GENERALIZED BIAS Although the M-estimates of scale with general location introduced in Section 2 are Fisher con~istent at the nominal distribution Fo, they are in general asymptotically biased for F E Fe. Furthermore, the "raw" asymptotic bias Br[S(F)] = S(F) - So can be of two distinct kinds: When F is an outliers generating distribution, the bias Br[S(F)] is positive, and when F is an inliers generating distribution, the bias Br[S(F)] negative. As in Martin and (1989), we consider generalized bias functions which are of positive and negative bias is independently chosen, by allowing the user to put positive and negative bias on different scales. Specifically, we define the bias if 0 < S(F) :::; So if So < :::; 00 are COfltULUoUS, non-negative
o 00.
(9)
are interested
maximum geileraJ.u~ed bias,
B(c) = maxB[S(F)]. FE:Fe
(10)
l,From monotonicity L I and L 2 , it follows that B(X, T) = max {LI[S- / so] , L 2[S+ / so]}, where S- and S+ denote the supremum and the infimum of the functional S(F) as F ranges over Fe. 4. BIAS ROBUST HUBER ESTIMATES
In view of the historical importance and high degree of familiarity of Huber (Proposal 2) estimates we first focus on obtaining bias robust estimates in this class. To emphasize the dependence on X and t/J we use the notation S(F, X, t/J), S+(X,t/J), S+(X,t/J), etc. The first step toward finding the bias robust Huber estimate is deriving the expressions (16) and (17) for S-(X,T1J;) and S+(X,T1J;)' Claims which are made below without proof can be easily verified under (AO)-(A3). The maximum value S+(X, t/J) of the scale functional S(F, X, t/J) is produced by a point mass contamination at infinity, 600 , and such contamination also produces the maximum value of the location estimate T1J;(F). The estimating equations in this limit case are
(1 - c)EFoX[(X - t)/ s1
= b(X)
(11)
and
(1 - c)EFot/J[(X - t)/8] + e = O.
(12)
Let ,At) be the unique solution of (11) for fixed t and let '1'1J;(8) be the unique solution of (12) for fixed s :> O. The fUIletioIlm ,1J;(t) 1'1J;bx(t)] is continuous and non-decreasing. Also, the pair (8*, t*) simultaneously satisfy (11) and (12) if and only if t* mx,1J;(t*) and s" = IX(t*). following lemma characterizes maximum asymptotic bias due to outliers nrovides an algorithm location of
x
IS
For each n ~ 1 let t n m x ,'4t(t n -d , with = inf{t > to : = t}. (a) t n = t'" Sn = IxCt*) = ,(b) the maximum asymptotic bias of the location estimate T( F, x, t/J) is t" and the maximum value of scale functional S(F, X, t/J) is S+(X, = s". LEMA-fA 2. Suppose that to by (13). Sn
UUJln_oo
The minimum value of the scale functional S(F, X, t/J), S-(X, t/J), is produced by a point mass contamination 80 at zero. In this case the estimating equations are
(1
€)EFoX[(X - t)/s]
+ €x(tls)
= b(x)
(14)
+ €t/J( -tis) = o.
(15)
and
(1 - €)EFot/J[(X - t)/s]
By monotonicity of t/J, t = 0 for all s > O. Let 9;1 be the inverse of 9x(s,t) with respect to s, for fixed t, Then, from (14) with t = 0 it follows that (16)
Optimal Centedng The of t/Jhas an effect on the maximum asymptotic bias of the scale estimate by virtue of affecting the bias t* of the location estimate. Observe that since S-(X, t/J) doesn't depend on t/J (see (16)), the optimal choice of t/J must be based S+(X,t/J) alone. It follows from Lemma 2 and (11), with t t", that
(17) Since for all 0 < a < 1 the function 9;1(a) is non-decreasing in t. Therefore, by the Huber (1964) result, ~ to all s > 0 and
X satisfying
dIt1tlcult to 'neorem 2 holds for the class of all M-lestima,tes of scale with cell. teriag functional T(F) navmz the "monotonicity" property
T(F)
~
T[(1 - c)Fo +
(18)
The Minimax-Bias Huber Estimate of Scale By Theorem 2 it suffices to consider S+ (X, 't/Jo) and S- (X, 't/Jo) and the function 't/J can be dropped from the notations. It will be shown that under certain conditions the maximum generalized bias B(X) (see (10)) is minimized by a jump function Xa* (see (6)). For each a > 0, let B(a) = B(Xa) and
b(a)
= b(Xa) = 2[1 -
(19)
Fo(a)].
We begin by showing that given 0 < c < .5, Fo, L l and L 2 there exists a" such that
(20)
Va> O.
Let ao = Fo- l [(1 + c)/2] and al = FO- l [(2 c)/2]. ~From (19), the corresponding values of bare bo b(ao) 1 - c and bl b(al) = c. Hence, letting S- (a) = S- (Xa) and S+(a) = S+(Xa), we have lim S-(a) =
a.....ao
_l
go
rb(a)]
II - c
.
1
-1 [
lim -F a.....ao a o
1 - 2(
b( a)] 1 -1 ) ) = -Fo (.5) = O. (21 1- c ao
and
}1I~\ S+ (a) -
11'm g-:1 [b(a) - c] t" 1_ C
a.....al
lim .!:.F,-1
a.....al a
0
2:: a.....al 11'm g-1 [b(a) - c] 0 1- c
[1 _2(1 b(a) - c] = ..!-FO- l(l) = c) al ---+
ao
0) of that On it mation IS remaraamy more tnas-robustness mnuence curve gross-error-sensitivity as a measure
same nussenee parameters. For approximation to maximal bias curve the impact of the bias of estimation of the nuisance of the Madm bias of the Shorth is unaffected location parameter. Since the by the asymptotic bias of the location estimate, the GES approximation is better in this case.
10. PROOFS OF LEMMAS AND THEOREMS The following lemma is needed to prove Theorem 1.
LEMMA 3. Let 0 < 81 < 82 < 00 as Lemma 1. Suppose that (A-0)-(A-2) hold and also assume that X and hx(s, t) are continuous and hx(s, t) < 0, V s > 0, t E R. Then, for all K > 0, we have: (a) X[(x - t)/s] is uniformly continuous on (x,s,t) E RX[Sl,S2]X[-K,K]. (b) S(F,t) is uniformly continuous on t E R, uniformly on F E:Fe (c) X[(x - t)/S(F,t)] is uniformly continuous on (x,t) E Rx[-K,K], uniformly on FE :F€. Proof. Let 6 > 0 be fixed and let B = [Sl' S2] x [- K, K]. Since X(x) is continuous, even, monotoneon[O, and bounded, it is uniformly continuous. Let ~l > 0 be such that Ix- xII < Al implieslx(x) - x(x')1 < 6. Also, since liml$l->oo X(x) 1, there exists Xo > 0 such that Ixl > Xo and Ix'i > Xo imply Ix(x) - x(x')1 < 8. Let Xl > 0 be such that if Ixl > Xl, then I(x-t)/sl > xofor all (s, t) E B. So X[(x t)/8] is uniformly continuous on {x : Ixl > Xl} x B. If Ixl :::; Xl then, s - $-/ I :::; xII~ ~I +1~1-1;1 and (a) follows. To prove (b) notice that assumptions on hx(s, t)
0)
exists ~
> 0 such that It - t'l < A, ItI :::; K, WI :::; K imply
1
X-t' [ S(F,t) - 6 -
>
1I - 0 there exists 80 > 0 such that (35)
By
lllE:orem
(36)
> (30) tonows
(33),
(35).
) can
way, if 0 '5 x '5 a if a ~ x ~ a if x ~ a 8,
8
proved
a similar
Ss(F,t) = inf {s : EFPs[(x - t)/s]
b(Xa)}
0
(38)
ACKNOWLEDGEMENTS We wish to thank Ricardo Fraiman for some useful comments with regard to establishing uniform consistency of M-estimates of scale. We also thank the referees for a number of very helpful comments and suggestions.
FIGURE 1.Maximum Bias Curves
H95
...
GES(H95)
GES(MADM)=GES{SHORTH)
--- ----- --- ---
--- ---
----- --- ----
_
Q
ci
0.0
0.1
0.2
0.3
EPSILON
0.4
0.5
HUBER, MADM AJ."JD SHORTH. LOG SCALE
I I I I I I I
ISS co
1.1,
Q
I'
I
• SHORTH
I' I'
•
I' (J)
:;!; al
•
I'
co
I'
Q
/
I'
. .. /
/
/
....
/
" ". . /
Q
-:
.
.
•
.
• •
.
....Q q
0.0
0.2
0.1
0.3
0.5
0.4
EPSILON
(a) Maximum Bias Due to Outliers
I
MADMooSHORTH
I
I
'JflE~ /
Iss
/
I
'I
/
'I
co
I'
V
Q
I
I'
I
I I' (J)
:;!;
m
I'
(Q
/
Q
/ I'
/ I'
/
....
/
Q
N
c::i
'"
.- "
.-.-
.-"
/ .- .-
q
0.0
0.1
02
0.3
EPSILON
0.4
0.5
It.::IUMC
} and logarithmic loss function.
#3. Mean-souared-error curves for n=20,
N(0,25) contamination
N(6,1) contamination ~
C\I
rej[.01 ] shorth A-est
t.()
.,...
U1 .,... w
~ .,...
(f)
~
~ .,...
t.()
-"
c.i 0
0
c.i
c.i 0.1
0.2
0.4
0.3
0.0
0.1
0.2
0.3
EPSILON
EPSILON
U(3.5,10) contamination
N(10,1) contamination
o N
~
C\I
U1 .,...
U1 .,...
/ /
/ ~
.,...
~
............
t.()
w
(f)
/
0
.,... t.()
c.i
c.i I
__
0.0
~ 0
0.1
0.2 EPSILON
0.3
0.4
0.0
0.1
0.2 EPSILON
0.3
curves tor n=2U. t-=NlU.l1 ana looantnmlc loss function.
N(O,25) contamination
N(6,1) .,...
.,...
(j)
......-
amole (n=20) and asymptotic maximum bias curve for MAD and SHORTH for F=N(0,1)
N(6,1) contamination
N(0,0.01) contamination
MAD n=inf SHORTH n=inf MAD n=20 SHORTH n=20
C\! .,...
//
o
.~
0.0
0.1
lO
.,...
-
h
co
"."'~~
.f'
~
~,/..~:
--
0.2
.....
h'
h
/
./
