Cooperating stochastic automata: approximate ... - Semantic Scholar

Cooperating stochastic automata: approximate lumping an reversed process Simonetta Balsamo

Gian-Luca Dei Rossi

Andrea Marin

Dipartimento di Scienze Ambientali, Informatica e Statistica Universit` a Ca’ Foscari, Venezia

ISCIS ’12, Paris, 3-4 October 2012

Context: Cooperating stochastic models

• Models with underlying Continuous Time Markov Chain (CTMC) • Exploitation of compositionality in model definition • Each component is specified in isolation • Semantics of cooperation is defined so that the joint model can be

algorithmically derived • Stochastic automata considered here synchronise on the

active/passive semantics • Performance Evaluation Process Algebra (PEPA) active/passive

synchronisation • Buchholz’s Communicating Markov Processes • Plateau’s stochastic automata networks (SAN) with master/slave

cooperation • ...

Cooperating stochastic automata: approximate lumping an reversed process

2 of 22

Motivation

• In general, the state-space’s cardinality of the joint model grows

exponentially with the number of components • Steady-state analysis becomes quickly unfeasible • Space cost • Time cost • Numerical stability issues • Workarounds • Approximate analysis (e.g. fluid) • Exploitation of the geometry of the state space • Product-form decomposition • Lumping • Approximate lumping

Cooperating stochastic automata: approximate lumping an reversed process

3 of 22

Previous work: Lumping on cooperating automata Definition (Lumping condition) Given active automaton M1 , a set of labels T , and a partition of the states of M1 into N1 clusters C = {C1 , C2 , . . . , CN1 }, we say that C is an exact lumping for M1 if: P 0 ˜1 (Ci → Cj ) not 1 ∀Ci , Cj , Ci 6= Cj , ∀s1 ∈ Ci s01 ∈Ck q1 (s1 → s1 ) = q synchronising label P 0 t ˜1t (Ci → Ck ) 2 ∀t ∈ T , ∀Ci , Cj , ∀s1 ∈ Ci s01 ∈Ck q1 (s1 → s1 ) = q P where ϕt1 (s1 , s˜01 ) = s0 ∈˜s0 q1t (s1 , s01 ). 1

1

• Reduce complexity GBEs’ solution through component-wise lumping • If both automata have a spate-space of cardinality M , time cost reduces from O((M M )3 ) to O((N M )3 ), where N is the number of clusters in the lumping • Intuition: for each synchronising label the original and lumped

automata must behave (in steady-state) equivalently • We treat non-synchronising transitions as a special case • Conditions are stronger than the ones for regular lumpability and

weaker than for PEPA strong equivalence Cooperating stochastic automata: approximate lumping an reversed process

4 of 22

Example

s0

a, 43 λ2

a, λ1

a, λ1 µ1

a, λ2 µ3

a, 14 λ2

µ1 µ2

s2

a, λ2

C0

µ2

C1

a, λ3 s1

a, λ3

µ1 a, λ1

Cooperating stochastic automata: approximate lumping an reversed process

5 of 22

Marginal distribution

Theorem Let M1 and M2 be two cooperating automata, where M2 is passive and M1 active. If: • M2 never blocks M1

˜ 1 is a lumped automaton of M1 • M Then the marginal steady state distribution of M2 in the cooperations ˜ 1 ⊗ M2 are the same. M1 ⊗ M2 and M Note that ergodicity is assumed and the state-space of the joint process is the Cartesian product of the single automata state-spaces.

Cooperating stochastic automata: approximate lumping an reversed process

6 of 22

A trivial example a, λ P1

λ

0

a, λ 1

2 µ1 a, ⊤

µ1 a, ⊤ P2

P1

P2

0

1 µ2

µ2

µ2 P˜1 

π1 (n) =

µ1 a, ⊤ 2

µ2 µ1

a, λ

1−

λ µ1



λ µ1

C0

n

Cooperating stochastic automata: approximate lumping an reversed process

a, λ 

π2 (n) =

1−

λ µ2



λ µ2

n

7 of 22

Another trivial one Q1

λ

λ 0

0

a, µ1 (2) µ1 (2)

0

˜R Q 1

a, λ

C1 a, λ a, ⊤

a, λ a, ⊤ 2

1 µ2

a, µ1 (3) µ1 (3) 2

1 a, λ a, ⊤

Q2

λ 2

1 a, µ1 (1) µ1 (1)

QR 1

µ2

µ1 (n)

λ

Q1

Q2

µ2

Cooperating stochastic automata: approximate lumping an reversed process

µ2

8 of 22

Reversed lumping and product-forms

• Both previous examples allowed for a lumping into a single cluster • First is derived from the forward automaton • Second is derived from the reversed automaton • In both cases we obtain the marginal distribution, but in the latter

we also have product-form! • product-form ⇒ the joint distribution is the product of the marginal

ones

Corollary (Product-forms) A synchronisation is in product-form if the reversed active automaton can be lumped into a single state Note that, in general the marginal steady state distribution of M2 in ˜ R ⊗ M2 ' the one in M ˜ 1 ⊗ M2 , and is equal in product-form models. M 1

Cooperating stochastic automata: approximate lumping an reversed process

9 of 22

Approximation of marginal SSD through aggregation

• With our theorem we can reduce the cost to compute marginal

steady state distributions of a cooperating automaton if we’re able to find an exact lumping of the other one. • What if this is not feasible or even possible? • We could try to find an approximated lumping. • Can be applied also to the reversed process.

• How we evaluate the quality of an approximation? • How we can adapt clustering algorithms to use our definition of

(approximated) exact lumping?

Cooperating stochastic automata: approximate lumping an reversed process

10 of 22

Evaluating the quality of an approximate lumping

How close is an arbitrary state partition W to an exact lumping? • We measure the coefficient of variation of the outgoing fluxes φt1 (s1 ) of the states in s˜1 . • We further refine that measurement.

Cooperating stochastic automata: approximate lumping an reversed process

11 of 22

-error

Definition (-error) ˜1 }, for all Given model M1 and a partition of states W = {˜ 1, . . . , N s˜1 ∈ W and t > 2, we define: P π1 (s1 )φt1 (s1 ) t s1 ∈˜ Ps1 φ1 (˜ s1 ) = s1 ∈˜ s1 π1 (s1 )  v  u 2 X π1 (s1 )(φt (s1 ) − φt (˜ u s )) 1 1 . P1 t (˜ s1 ) = 1 − exp −t s∈˜ s1 π1 (s1 ) s1 ∈˜ s1

where φt1 (s1 ) =

PN1

t 0 s01 =1 q1 (s1 , s1 ).

Cooperating stochastic automata: approximate lumping an reversed process

12 of 22

δ-error

Definition (δ-error) ˜1 }, for all Given model M1 and a partition of states W = {˜ 1, . . . , N s˜1 , s˜01 ∈ W, we define:  0 s˜1 = s˜01 ∧ t = 1 ϕt1 (˜ s1 , s˜01 ) = (Ps1 ∈˜s1 π1 (s1 )ϕt1 (s1 ,˜s01 )) P  otherwise π1 (s1 ) s1 ∈˜ s1

σ t (˜ s1 , s˜01 )


2

=

X π1 (s1 )(ϕt (s1 , s˜0 ) − ϕt (˜ ˜01 ))2 1 s1 , s 1 P 1 s∈˜ s1 π1 (s)

s1 ∈˜ s1

0

δ t (˜ s1 , s˜01 ) = 1 − e−σ(˜s1 ,˜s1 )

where function ϕt1 has been defined in Lumping conditions.

Cooperating stochastic automata: approximate lumping an reversed process

13 of 22

An ideal algorithm Definition (Ideal algorithm) • Input: automata M1 , M2 , T , tolerances  ≥ 0, δ ≥ 0

• Output: marginal distribution π1 of M1 ; approximated marginal

distribution of M2 1

2 3

4

˜ 0 such that there exists a partition Find the minimum N 1 0 ˜ ˜ W = {1, . . . , N1 } of the states of M1 such that ∀t ∈ T , t > 2 and ∀˜ s1 ∈ W (˜ s1 ) ≤  Let W 0 ← W

Check if partition W 0 is such that ∀t ∈ T , ∀˜ s1 , s˜2 ∈ W, s˜1 6= s˜2 , δ t (˜ s1 , s˜01 ) ≤ δ. If this is true then return the marginal distribution of M1 and the approximated of M2 by computing the marginal ˜ 1 ⊗ M2 and terminate. distribution of M

Otherwise, refine partition W to obtain W new such that the number of clusters of W new is greater than the number of clusters in W 0 . W 0 ← W new . Repeat from Step 3

Cooperating stochastic automata: approximate lumping an reversed process

14 of 22

Constructing the approximate lumped automata Definition (Approx. lumped automata) Given active automaton M1 , a set of transition types T , and a partition ˜1 clusters W = {˜ ˜1 }, then we define of the states of M1 into N 1, ˜ 2, . . . , N the automaton M1' as follows: ( ˜ −1 if s˜1 6= s˜2 ϕ11 (˜ s1 , s˜01 )λ 1 0 ˜ E11 (˜ s1 , s˜1 ) = 0 otherwise ˜ 12 = I, E ˜ E1t (˜ s1 , s˜1 ) = ϕt1 (˜ s1 , s˜01 )λ−1 t>2 t where

 ˜t = λ

max

˜1 s˜1 =1,...,N

˜1 N X



 ϕt1 (˜ s1 , s˜01 )

s˜01 =1

are the rates associated with the transition types in the cooperation between M1' and M2 . Cooperating stochastic automata: approximate lumping an reversed process

15 of 22

Initial clustering and refinement phase

Initial clustering: • similarity measure can be Euclidean distance between

(φ31 (s1 ), . . . , φT1 (s1 )) and (φ31 (s01 ), . . . , φT1 (s0 )) • can be implemented using various algorithm • hierarchical clustering • K-means (but number of clusters must be decided a priori...) • ...

Refinement phase: • using the tolerance constant δ • distances between clusters depend on clusters themselves =⇒

K-means cannot be used. • spectral analysis or iterative algorithms

Cooperating stochastic automata: approximate lumping an reversed process

16 of 22

Example

p

Q1

λ1

µ1

Q2

1−p

µ2

λ2 γ(n1 )

where

   0    λ1 γ(n1 ) =    2   λ1

 C1 if n1 ≤   2 C1 if < n1 < C1 2 if n1 = C1

Cooperating stochastic automata: approximate lumping an reversed process



17 of 22

Example 3,λ1 /2 1,λ1 P1

P2

1,λ1

1,λ1 /2 C1 − 1

h

1

0

3,λ1 /2 1,λ1 /2

C1

1,pµ1

1,pµ1

1,pµ1

1,pµ1

3,(1 − p)µ1

3,(1 − p)µ1

3,(1 − p)µ1

3,(1 − p)µ1

3,1

3,1

3,1

2,λ2

2,λ2

2,λ2

0 2,µ2

2,µ2

3,1 2,λ2 C2 − 1

k

1

2,µ2

3,λ1

C2

3,1

2,µ2

Not exactly lumpable. For C1 = 20, C2 = 20, λ1 = 6, λ2 = 1, µ1 = 4, µ2 = 4, p = 0.7,  = 10−13 and δ = 0.95 we could find • L1 = {0}, L2 = {1, . . . , 10}, L3 = {11, . . . , 19} and L4 = {20} on

the forward process

• L1 = {0, . . . , 10}, L2 = {11, 12}, L3 = {13, . . . , 19} and L4 = {20}

on the reversed one

Cooperating stochastic automata: approximate lumping an reversed process

18 of 22

Comparison

KL div. E[N ] Rel. err.

FW-Lump 0.0065 11.62 0.0259

RV-Lump 0.0045 11.55 0.0200

APF 0.0451 9.990 0.1178

FPA 0.0112 11.80 0.0424

Exact 0 11.33 0

Where • APF is the Approximated Product Form of order 4 [Buchholz, 2010] • PFA is the Fixed Point Approximation [Miner et al., 2000]

Cooperating stochastic automata: approximate lumping an reversed process

19 of 22

Conclusion

• Lumping of automata can be applied to other formalisms • Approximate lumpings can be used to derive approximate marginal

distributions • Several examples show that in case of queueing networks lumping

the reversed automata gives better approximations! • Future works: definition of efficient algorithms • The algorithm proposed in [Gilmore et al., 2001] based on strong equivalence can be adapted to consider our notion of lumpability • also for reversed automata • optimality issues

Cooperating stochastic automata: approximate lumping an reversed process

20 of 22

References [Balsamo et al., 2012] Balsamo, S., Dei Rossi, G., and Marin, A. (2012). Lumping and reversed processes in cooperating automata. In LNCS 7314, Proc. of Int. Conf. ASMTA, pages 212–226, Grenoble, FR. Springer. [Buchholz, 2010] Buchholz, P. (2010). Product form approximations for communicating Markov processes. Perf. Eval., 67(9):797 – 815. Special Issue: QEST 2008. [Gilmore et al., 2001] Gilmore, S., Hillston, J., and Ribaudo, M. (2001). An Efficient Algorithm for Aggregating PEPA Models. IEEE Trans. on Software Eng., 27(5):449–464. [Miner et al., 2000] Miner, A. S., Ciardo, G., and Donatelli, S. (2000). Using the exact state space of a markov model to compute approximate stationary measures. In Proc. of ACM SIGMETRICS, pages 207–216, New York, NY, USA. ACM.

Cooperating stochastic automata: approximate lumping an reversed process

21 of 22

Thanks!

Thanks for the attention any question?

Cooperating stochastic automata: approximate lumping an reversed process

22 of 22