Efficient Mining of Weighted Frequent Patterns over Data Streams
2009 11th IEEE International Conference on High Performance Computing and Communications
! "# $ % { & }'( ))(
Abstract
) '& ) ) '
- 5 ) ' & '' '& !1 !2 !$ !3 !#% ' ) ' ' * ) ( & '
8 ) ' &) ' '
' & ') ' ' ' ' ' (' ' ) 6' & ) ' ) )&)( '
' &) ' ) ' ( *) ) && )' &)
) ' ' 9 ' & & ') ' ' ' '& ' & ) . :9; +. : 9 ;
',- 8 ) ') ' & ) ( & '
' ' & ') - 7 ) 4 & ' ) &&' ) ' & - '' & ( ) ) && & ' & '- ) ' ) &' ' ' & ) ') ( & ' &' & ' & ') '
- 9 ' &' ' & ) & ;< &'' ' &) & ' ) ) ) ' ' ) & ' ) ) )& ' ' ' )
' ) ' & ')
'
- 7 &) ' ' &)
) ' ' ' (' ' )( (
&'' ' ' ' &&'- 8 ) " ') )( - 8 ) / & ' ) '
'- 8 ) 0 4 & '& ' ' &- && ) 1 ) )&' '
-
* + + , -./01 ( +% % , ) 2 ./0 ( +% ) + 3 % % , % % ) 4
3 ( +% %) 5 +
% + + % , ./04 -. / , 0 4 1 % ( +% % ) 3 % + , 6 ./0
)
1. Introduction ! " # !! "$% & ' & ( & ') ) ' ' ) ' ' ) ' *) )' + ', '- . / 0 1 2 $ 3 "2% ' ' ')
( & ) ' ' ) ) & ' & & & ') '- 4
& & & ' '' ' ) & ' & ) ) '&- 5 ) ( & ' & ) ' * & '- ) ' ' ( & ''5 4' ' ) (' & & ' ') ' * ' && & )'' &( '
' !" !/ !0 !1%- '
6' ) ' '
' ) ' ' ) ' - * ' '
' & &4 & & ') '- 7 ) '
' 6 &' - 5 & 978-0-7695-3738-2/09 $25.00 © 2009 IEEE DOI 10.1109/HPCC.2009.36
2. Background 2.1. Frequent pattern mining
' = ) ' '
) ' ) ' ) '400
downward closure Apriori
!" # !" $ % & ' ( )
4!12 + 41" & ) , downward closure : ;a< %6 = ;b< % & ;ab< + . ,- ;ab< ,%6 > %-) ? %= , %= × + ?
;a< %6 × = ? = ;b< % × & ? % 1 ;b< ;ab< 4!12 41" downward closure / 9 1 9 ;a< 9 %6 ;b
LMAXW
Figure 5. Performance on the T10I4D100K dataset
, -&
!
g
> %#0 ? @ ;>A
#?3
/
300
0
4
window2
400
100
-1
-1 , #$
500
200
+ &' ))
+
W=3B, B=10K
600
/
bg
Avg. trans. length 10.1 39.61 8.1
W=3B, B=20K
800
-1 -. 2
No. of distinct items 870 942 41,270
900
Runtime (sec.)
/
No. of trans. 100,000 100,000 990,002
' (
/
Size (MB) 3.83 14.7 30.5
Datasets
4
T10I4D100K
#-
/ #- ---
4. Experimental results
4 @ , 0 * (
window1 window2
/
*35 (
/
'T10I4D100K T40I10D100K ) ( kosarak (
! (
2
. /
%- --- 4 @ ,
6%&7 / #
0 + .
/
!
+ 1
T40I10D100K
60 & . 8 $ %17
4+>5BC
@ , #-D #.D @ , & /
-# -9
5 : ;
404
1400
1900
W=4B, B=15K
WFIM 1700
W=4B, B=10K
1000
Runtime (sec.)
Runtime (sec.)
1200
800 600 400
1300 1100 900 700
200
500
0 5
10 15 20 Minimum threshold (%)
2
25
1600
W=3B, B=100K 1200 1000 800 600 400 3 4 5 Minimum threshold (%)
5
6
."-4
- $ $
4 #
% 5 ."-4 / 5 0 ."6478 " 5 & window1 ."-4 window1 - window1 # &
window1 window2 window2 ."-4 %
window2 " 3 ."-4 ."6478 kosarak . ' (
' ( *!) 2% 6% $ 3 $ ."6478 ."-4 $ % 5
W=3B, B=150K
2
4
Figure 8. Runtime comparison on the kosarak dataset
W=3B, B=200K
1400
3
Minimum threshold (%)
Figure 6. Performance on the T40I10D100K dataset
Runtime (sec.)
WFPMDS
1500
6
Figure 7. Performance on the kosarak dataset kosarak ! " # $ % & kosarak % ' # ' ( !!) *!) !!)+ ' ( , 2% 6% - $ & % % ! . % % - % * ! , * # % / $ %0 T10I4D100K /.(,1 1(!)0 T40I10D100K /.(1 1( *)0 kosarak /.(,1 1(!!)0 & 2 341 3 3,241 2*41 % #% $ % &
% # % % . & 5 ."-4 kosarak 5
5. Conclusions
- # & & # 9 # & 1% $ & ."6478 - & % & % # # 4 # % & % :5 # % & #% $ &
# 405
References
5, 9
'4 !! +& "##%
4 :
- & . /&& , 00&& 0 &1 !/
8$ !!% "##' !! :
2& * *+, & ) - , && !! %%%%% "##'
) !! 6
" ; ; @ 1 8
, /// && ! $ !! %"&%'" "##"
)
, 1 1 / . * . 12 )
,
*+, &
"" / * / ; :0C5 2 6
9 % !! "#'""& "##+
9: 0 5 1 & && , *+, & - !! "'# "### ) 3
& / ; / * . . 3 !
)
2
3
! !! ) && , *+, & - 9 , !! 4',$ "##+
"' / ? :/ ; :;2 6
,
& && "##" )
# > > ) / ?) ( ) 3 (*
/// 3
*+, , /&& 9 $ # !!
"+ (3 ! 8 9
'+$'%" "##4 2 && , *+, & - 9 4 !! 44,%
: 2
3 !
"##$
*+, , &
9 ' !! ",$' "##$
"% - . 3 ! )
" : * *
)
!! ) 3
*+,) ,
9
"# !! ,%&$ "##$
9 ", !!
"$
! "# $ % { & }'( ))(
Abstract
) '& ) ) '
- 5 ) ' & '' '& !1 !2 !$ !3 !#% ' ) ' ' * ) ( & '
8 ) ' &) ' '
' & ') ' ' ' ' ' (' ' ) 6' & ) ' ) )&)( '
' &) ' ) ' ( *) ) && )' &)
) ' ' 9 ' & & ') ' ' ' '& ' & ) . :9; +. : 9 ;
',- 8 ) ') ' & ) ( & '
' ' & ') - 7 ) 4 & ' ) &&' ) ' & - '' & ( ) ) && & ' & '- ) ' ) &' ' ' & ) ') ( & ' &' & ' & ') '
- 9 ' &' ' & ) & ;< &'' ' &) & ' ) ) ) ' ' ) & ' ) ) )& ' ' ' )
' ) ' & ')
'
- 7 &) ' ' &)
) ' ' ' (' ' )( (
&'' ' ' ' &&'- 8 ) " ') )( - 8 ) / & ' ) '
'- 8 ) 0 4 & '& ' ' &- && ) 1 ) )&' '
-
* + + , -./01 ( +% % , ) 2 ./0 ( +% ) + 3 % % , % % ) 4
3 ( +% %) 5 +
% + + % , ./04 -. / , 0 4 1 % ( +% % ) 3 % + , 6 ./0
)
1. Introduction ! " # !! "$% & ' & ( & ') ) ' ' ) ' ' ) ' *) )' + ', '- . / 0 1 2 $ 3 "2% ' ' ')
( & ) ' ' ) ) & ' & & & ') '- 4
& & & ' '' ' ) & ' & ) ) '&- 5 ) ( & ' & ) ' * & '- ) ' ' ( & ''5 4' ' ) (' & & ' ') ' * ' && & )'' &( '
' !" !/ !0 !1%- '
6' ) ' '
' ) ' ' ) ' - * ' '
' & &4 & & ') '- 7 ) '
' 6 &' - 5 & 978-0-7695-3738-2/09 $25.00 © 2009 IEEE DOI 10.1109/HPCC.2009.36
2. Background 2.1. Frequent pattern mining
' = ) ' '
) ' ) ' ) '400
downward closure Apriori
!" # !" $ % & ' ( )
4!12 + 41" & ) , downward closure : ;a< %6 = ;b< % & ;ab< + . ,- ;ab< ,%6 > %-) ? %= , %= × + ?
;a< %6 × = ? = ;b< % × & ? % 1 ;b< ;ab< 4!12 41" downward closure / 9 1 9 ;a< 9 %6 ;b
LMAXW
Figure 5. Performance on the T10I4D100K dataset
, -&
!
g
> %#0 ? @ ;>A
#?3
/
300
0
4
window2
400
100
-1
-1 , #$
500
200
+ &' ))
+
W=3B, B=10K
600
/
bg
Avg. trans. length 10.1 39.61 8.1
W=3B, B=20K
800
-1 -. 2
No. of distinct items 870 942 41,270
900
Runtime (sec.)
/
No. of trans. 100,000 100,000 990,002
' (
/
Size (MB) 3.83 14.7 30.5
Datasets
4
T10I4D100K
#-
/ #- ---
4. Experimental results
4 @ , 0 * (
window1 window2
/
*35 (
/
'T10I4D100K T40I10D100K ) ( kosarak (
! (
2
. /
%- --- 4 @ ,
6%&7 / #
0 + .
/
!
+ 1
T40I10D100K
60 & . 8 $ %17
4+>5BC
@ , #-D #.D @ , & /
-# -9
5 : ;
404
1400
1900
W=4B, B=15K
WFIM 1700
W=4B, B=10K
1000
Runtime (sec.)
Runtime (sec.)
1200
800 600 400
1300 1100 900 700
200
500
0 5
10 15 20 Minimum threshold (%)
2
25
1600
W=3B, B=100K 1200 1000 800 600 400 3 4 5 Minimum threshold (%)
5
6
."-4
- $ $
4 #
% 5 ."-4 / 5 0 ."6478 " 5 & window1 ."-4 window1 - window1 # &
window1 window2 window2 ."-4 %
window2 " 3 ."-4 ."6478 kosarak . ' (
' ( *!) 2% 6% $ 3 $ ."6478 ."-4 $ % 5
W=3B, B=150K
2
4
Figure 8. Runtime comparison on the kosarak dataset
W=3B, B=200K
1400
3
Minimum threshold (%)
Figure 6. Performance on the T40I10D100K dataset
Runtime (sec.)
WFPMDS
1500
6
Figure 7. Performance on the kosarak dataset kosarak ! " # $ % & kosarak % ' # ' ( !!) *!) !!)+ ' ( , 2% 6% - $ & % % ! . % % - % * ! , * # % / $ %0 T10I4D100K /.(,1 1(!)0 T40I10D100K /.(1 1( *)0 kosarak /.(,1 1(!!)0 & 2 341 3 3,241 2*41 % #% $ % &
% # % % . & 5 ."-4 kosarak 5
5. Conclusions
- # & & # 9 # & 1% $ & ."6478 - & % & % # # 4 # % & % :5 # % & #% $ &
# 405
References
5, 9
'4 !! +& "##%
4 :
- & . /&& , 00&& 0 &1 !/
8$ !!% "##' !! :
2& * *+, & ) - , && !! %%%%% "##'
) !! 6
" ; ; @ 1 8
, /// && ! $ !! %"&%'" "##"
)
, 1 1 / . * . 12 )
,
*+, &
"" / * / ; :0C5 2 6
9 % !! "#'""& "##+
9: 0 5 1 & && , *+, & - !! "'# "### ) 3
& / ; / * . . 3 !
)
2
3
! !! ) && , *+, & - 9 , !! 4',$ "##+
"' / ? :/ ; :;2 6
,
& && "##" )
# > > ) / ?) ( ) 3 (*
/// 3
*+, , /&& 9 $ # !!
"+ (3 ! 8 9
'+$'%" "##4 2 && , *+, & - 9 4 !! 44,%
: 2
3 !
"##$
*+, , &
9 ' !! ",$' "##$
"% - . 3 ! )
" : * *
)
!! ) 3
*+,) ,
9
"# !! ,%&$ "##$
9 ", !!
"$
