XPL: An extended probabilistic logic for probabilistic transition systems
arXiv:1604.06118v1 [cs.LO] 20 Apr 2016
XPL: An extended probabilistic logic for probabilistic transition systems∗ ANDREY GORLIN C. R. RAMAKRISHNAN Department of Computer Science, Stony Brook University, Stony Brook, NY 11794, U.S.A. {agorlin,cram}@cs.stonybrook.edu April 22, 2016
Abstract Generalized Probabilistic Logic (GPL) is a temporal logic, based on the modal mu-calculus, for specifying properties of reactive probabilistic systems. GPL is expressive enough that termination of Recursive Markov Chains (RMCs) can be cast as a model checking problem over GPL. In this paper, we explore XPL, an extension to GPL for expressing properties of transition systems with probabilistic and nondeterministic behavior, such as Recursive Markov Decision Processes (Recursive MDPs or RMDPs). However, since termination of multi-exit RMDPs is undecidable, model checking in XPL is undecidable as well. We provide a model checking procedure for XPL that always terminates: either successfully with the model checking result, or with failure. We define a sufficient condition for success via a syntactically-characterized class, called separable XPL formulae. We show that a large class of problems, including termination of 1-exit RMDPs and PCTL* model checking of MDPs, can be reduced to the model checking of separable XPL formulae.
1
Introduction
For finite-state systems, model checking a temporal property can be cast in terms of model checking in the modal µ-calculus, the so-called “assembly language” of temporal logics. A number of temporal logics have been proposed and used for specifying properties of finite-state probabilistic systems. Two of the notable logics for probabilistic systems based on the µ-calculus are GPL [3] and pLµ [14]. GPL is defined over Reactive Probabilistic Labeled Transition Systems (RPLTSs). The formal definitions of GPL and RPLTSs are recalled in Section 2, mainly following [3]. In an RPLTS, each state has a set of outgoing ∗ This
work was partially supported by NSF grant IIS-1447549.
1
transitions with distinct labels; each transition, in turn, specifies a (probabilistic) distribution of target states. GPL is expressive enough to serve as an “assembly language” of a large number of probabilistic temporal logics. For instance, model checking PCTL* properties over Markov Chains, as well as termination and reachability of Recursive Markov Chains can be cast in terms of GPL model checking [3, 11]. In this paper, we consider an extension to GPL to express properties of probabilistic systems with internal non-deterministic choice. This logic, called Extended Probabilisitic Logic (XPL) is defined over Probabilistic Labeled Transition Systems (PLTSs). In a PLTS, each state has a set of outgoing transitions, possibly with common labels; and each transition specifies a distribution of target states. PLTSs thus exhibit both probabilistic choice and internal non-deterministic choice. Contributions: XPL is a conservative extension to GPL (see Section 3). However, in contrast to RPLTSs and GPL, model checking of PLTSs with XPL is undecidable in general. We give a syntactically-defined subclass, called separable XPL and give a model checking algorithm that always succeeds for this subclass (see Section 4). We show that the separable subset of XPL is sufficiently expressive to encode a wide variety of verification problems. For instance, PCTL* model checking of Markov Decision Processes (MDPs), termination of 1-exit Recursive MDPs, and PTTL model checking of branching MDPs can all be reduced, in linear time, to model checking separable XPL formulae over PLTSs (see Section 5). Related Work: GPL treats conjunction in a traditional manner, retaining the properties that ψ ∧ ¬ψ = ff, and ψ ∧ ψ = ψ for any formula ψ. However, probability value of ψ1 ∧ ψ2 cannot be computed based on the probability values of the conjuncts ψ1 and ψ2 . This makes model checking in GPL more complex, but also contributes to GPL’s expressiveness. In contrast to GPL, the most expressive version of pLµ, denoted pLµ ⊕ [14, 15], defines three conjunction operators and their duals such that their probability values can be computed from the probabilities of the conjuncts. The three conjunctions are defined as minimum, independent product, and truncated cosum. Along the same lines as our XPL encoding, we can encode termination of 1exit RMDP as model checking in pLµ , and RMC termination in pLµ ⊕ . However, PCTL* still appears to be out of reach, because LTL can have conjunctions for which the values cannot be computed directly from the values of the conjuncts. The relationship of our work to earlier papers, is discussed in Section 6.
2
Preliminaries
In this section, we formally define PLTSs which are used to define the semantics of XPL. We also summarize the syntax and semantics of GPL, using the notations from [3]. 2
2.1
Probabilistic Labeled Transition Systems
We define a probabilistic labeled transition system (PLTS) as an extension of [3]’s RPLTS. Definition 1 (PLTS). With respect to fixed sets Act and P rop of actions and propositions, respectively, a PLTS L is a quadruple (S, δ, P, I), where • S is a countable set of states; • δ ⊆ S × Act × N × S is the transition relation; • P : δ → (0, 1] is the transition probability distribution satisfying: P – ∀s ∈ S.∀a ∈ Act.∀c ∈ N. P (s, a, c, s0 ) ∈ {0, 1}, and s0 :(s,a,c,s0 )∈δ
– ∀s ∈ S.∀a ∈ Act.∀c ∈ N. (∃s0 .(s, a, c, s0 ) ∈ δ) ⇒
P (s, a, c, s0 ) = 1;
P
s0 :(s,a,c,s0 )∈δ
• I : S → 2P rop is the interpretation, recording the set of propositions true at a state. A reactive PLTS does not have internal nondeterminism, i.e., having c = 0 for all (s, a, c, s0 ) ∈ δ, in which case we may write elements of δ as triples. This definition is in line with the most general for a PLTS [14, 16], in which, given an action, a probabilistic distribution is chosen nondeterministically. Other equally expressive models include alternating automata, in which labeled nondeterministic ones are followed by silent probabilistic choices. The difference between such models has been analyzed with respect to bisimulation [17]. a a Given L = (S, δ, P, I), a partial computation is a sequence σ = s0 →1 s1 →2 an · · · → sn , where for all 0 ≤ i < n, (si , ai+1 , ·, si+1 ) ∈ δ. Also, fst(σ) = s0 and last(σ) = sn denote, respectively, the first and last states in σ. Each transition of a partial computation is labeled with an action ai ∈ Act. The set of all partial computations of L is denoted by CL , and CL (s) = {σ ∈ CL | fst(σ) = s}. a an a a Composition of partial computations, σ → σ 0 , represents s0 →1 · · · → sn → b1 bm 0 s00 → ··· → sm if (sn , a, ·, s00 ) ∈ δ. Partial computation σ 0 is a prefix of σ if a a 1 σ 0 = s0 → · · · →i si for some i ≤ n. From a set of partial computations, we can build deterministic trees (dtrees). We often denote a d-tree by the set of paths in the tree. Every d-tree is prefix-closed and deterministic. T ⊆ CL is prefix-closed if, for every σ ∈ T and σ 0 a prefix of σ, σ 0 ∈ T . T is deterministic if for every σ, σ 0 ∈ T with a
a
a
a
a
a0
n n σ = so →1 · · · → sn → s · · · and σ 0 = s0 →1 · · · → sn → s0 · · · , either a 6= a0 or s = s0 , i.e., if a pair of computations share a prefix, the first difference cannot involve transitions labeled by the same action. A d-tree T has a starting state, denoted root(T ); if s = root(T ) then T ⊆ CL (s). TL refers to all the d-trees of L, and TL (s) = {T ∈ TL | root(T ) = s}. T 0 is a a a prefix of T if T 0 ⊆ T . T → T 0 means T 0 = {σ | root(T ) → σ ∈ T }. T is finite
3
if |T | < ∞, and maximal if there exists no d-tree T 0 with T ⊂ T 0 . ML and ML (s) are analogous to TL and TL (s), but for maximal d-trees. An outcome is a maximal d-tree. The subsequent concepts apply only to RPLTSs, and we will extend them to PLTSs in Section 3. Intuitively, the probability of some finite prefix is the product of the probabilities of all the edges. Formally, a basic cylindrical subset of ML (s) contains all trees sharing a given prefix. Letting s ∈ S, and T ∈ TL (s) to be finite, BT = {T 0 ∈ ML | T ⊆ T 0 }. The measure of BT is: Y m(BT ) = P (last(σ), a, last(σ 0 )). (σ,a,σ 0 )∈edges(T )
From here, a probability measure ms : Bs → [0, 1] on the smallest field of sets Bs is generated from subsets BT with ms (BT ) = m(BT ).
2.2
GPL Syntax
GPL has two different kinds of formulae. State formulae depend directly only on the given state. Fuzzy formulae depend on outcomes. We give the syntax of GPL, with X ∈ V ar, a ∈ Act, A ∈ P rop, and 0 ≤ p ≤ 1, for state formulae, φ, and fuzzy formulae, ψ, as: φ ::= A | ¬A | φ ∧ φ | φ ∨ φ | Pr>p ψ | Pr≥p ψ, ψ ::= φ | X | ψ ∧ ψ | ψ ∨ ψ | haiψ | [a]ψ | µX.ψ | νX.ψ. Note that only atomic propositions may be negated, but every operator has its dual given in the syntax. The propositional connectives, ∧ and ∨, can be used on both state and fuzzy formulae. Although not stated in the syntax, only closed fuzzy formulae ψ can be used to construct state formulae Pr>p ψ and Pr≥p ψ. These formulae check the probability for a fuzzy formula ψ (Pr>p and Pr≥1−p are duals). Operators µX.ψ and νX.ψ are least and greatest fixed point operators for the “equation” X = ψ, and formulae must be alternation-free. The semantics of GPL is given in terms of RPLTS d-trees. In that interpretation, diamond implies box : haiψ means that there is an a-transition and it satisfies ψ; [a]ψ means that if there is an a-transition, also use a set W it satisfies ψ. We V α ⊆ Act for the modalities, reading hαiψ as haiψ and [α]ψ as [a]ψ. When a∈α
a∈α
we write “−” for α, that represents Act.
2.3
GPL Semantics
We define the semantics of GPL with respect to a fixed RPLTS L = (S, δ, P, I). A function ΘL : Ψ → 2ML , augmented with an extra environment parameter e : V ar → 2ML , returns the set of outcomes satisfying a given fuzzy formula, defined inductively in Table 1. For a given s ∈ S, ΘL,s (ψ) = ΘL (ψ) ∩ ML (s). |=L ⊆ S × Φ indicates when a state satisfies a state formula, and is defined inductively in Table 2. Note that the 4
Table 1: GPL/XPL semantics: fuzzy formulae ΘL (φ)e =
[
ML (s), where φ is a closed formula,
s|=L φ
ΘL (X)e = e(X), a
ΘL (haiψ)e = {T ∈ ML | ∃T 0 : T → T 0 ∧ T 0 ∈ ΘL (ψ)e}, a
ΘL ([a]ψ)e = {T ∈ ML | (T → T 0 ) ⇒ T 0 ∈ ΘL (ψ)e}, ΘL (ψ1 ∧ ψ2 )e = ΘL (ψ1 )e ∩ ΘL (ψ2 )e, ΘL (ψ1 ∨ ψ2 )e = ΘL (ψ1 )e ∪ ΘL (ψ2 )e, ∞ [ Mi , where M0 = ∅ and Mi+1 = ΘL (ψ)e[X 7→ Mi ], ΘL (µX.ψ)e = ΘL (νX.ψ)e =
i=0 ∞ \
Ni , where N0 = ML and Ni+1 = ΘL (ψ)e[X 7→ Ni ].
i=0
Table 2: GPL semantics: state formulae s |=L s |=L s |=L s |=L s |=L s |=L
A ¬A φ1 ∧ φ2 φ1 ∨ φ2 Pr>p ψ Pr≥p ψ
iff iff iff iff iff iff
A ∈ I(s), A∈ / I(s), s |= φ1 and s |= φ2 , s |= φ1 or s |= φ2 , ms (ΘL,s (ψ)) > p, ms (ΘL,s (ψ)) ≥ p.
5
definitions for ΘL and |=L are mutually recursive. There are two properties of GPL fuzzy formulae that are important for the completeness of model checking. First, we have distributivity on box and diamond: Lemma 2 (Distributivity on modal operators ([3] Lemma 1)). Letting ⊕ ∈ {∧, ∨}: ΘL ([a]ψ1 ⊕ [a]ψ2 ) = ΘL [a](ψ1 ⊕ ψ2 ) , ΘL (haiψ1 ⊕ haiψ2 ) = ΘL hai(ψ1 ⊕ ψ2 ) , ΘL ([a]ψ1 ∧ haiψ2 ) = ΘL hai(ψ1 ∧ ψ2 ) , ΘL ([a]ψ1 ∨ haiψ2 ) = ΘL [a](ψ1 ∨ ψ2 ) . Second, we can relate the probability of a conjunction with that of a disjunction [3, Lemma 2]: ms (ΘL,s (ψ1 ∨ ψ2 )) = ms (ΘL,s (ψ1 )) + ms (ΘL,s (ψ2 )) − ms (ΘL,s (ψ1 ∧ ψ2 )). (1) Additionally, although there is no negation operator in the syntax, we can write the negation of a fuzzy formula ψ, neg(ψ), and of a state formula φ, neg(φ), such that, for any RPLTS L and state s ([3, Lemma 3]): ΘL,s (neg(ψ)) = ML (s) − ΘL,s (ψ)
and s |=L neg(φ) ⇐⇒ s 6|=L φ.
The proof involves switching all the operators to their duals.
3
XPL
To resolve the nondeterministic transitions in a PLTS, we additionally require a scheduler. Recall, from Section 2.1, that CL is the set of all partial computations σ of L. Definition 3 (PLTS scheduler). A scheduler for an PLTS L is a function γ : CL × Act → N, such that if an action a is present at last(σ), then γ(σ, a) = c implies that there exists s ∈ S such that (last(σ), a, c, s) ∈ δ. Note that we have defined deterministic schedulers, which are also aware of their relevant histories. Given a scheduler γ for an PLTS L, we have a (countable) RPLTS Lγ . We define a probability distribution: Definition 4 (Combined probability). The probability distribution of a PLTS L with scheduler γ is a function, PL,γ : CL × Act × S → [0, 1], where: PL,γ (σ, a, s) = PL (last(σ), a, γ(σ, a), s).
6
Table 3: XPL semantics: state formulae s |=L s |=L s |=L s |=L s |=L s |=L s |=L s |=L
A ¬A φ1 ∧ φ2 φ1 ∨ φ2 Prmax >p ψ Prmax ≥p ψ Prmin >p ψ Prmin ≥p ψ
iff iff iff iff iff iff iff iff
A ∈ I(s), A∈ / I(s), s |= φ1 and s |= φ2 , s |= φ1 or s |= φ2 , supγ mγs (ΘL,s (ψ)) > p, supγ mγs (ΘL,s (ψ)) ≥ p, inf γ mγs (ΘL,s (ψ)) > p, inf γ mγs (ΘL,s (ψ)) ≥ p.
Following Definition 4, PL,γ (σ, a, s) = 0 for all s when there are no atransitions from last(σ). Recall, from Section 2.1, that the basic cylindrical subset BT contains all maximal d-trees sharing the prefix tree T . For these subsets, we define the probability measure: Definition 5 (Probability measure). For a PLTS L with scheduler γ, the probability measure of a basic cylindrical subset BT is defined by a partial function mγ : 2ML → [0, 1], where: Y mγ (BT ) = PL,γ (σ, a, last(σ 0 )). (σ,a,σ 0 )∈edges(T )
Since mγ may be considered as defined for an RPLTS, we can extend it to a measure mγs as in Section 2.1.
3.1
XPL Syntax
Now we give the XPL syntax: min max max φ ::= A | ¬A | φ ∧ φ | φ ∨ φ | Prmin >p ψ | Pr≥p ψ | Pr>p ψ | Pr≥p ψ,
ψ ::= φ | X | ψ ∧ ψ | ψ ∨ ψ | haiψ | [a]ψ | µX.ψ | νX.ψ. The fuzzy formulae remain the same as in GPL. Prmin and Prmax compare against the infimum and supremum probabilities, respectively, over all schedulers. Note min min max that Prmax >p is the dual of Pr≥1−p . For an RPLTS, Pr./p , Pr./p and GPL’s Pr./p ψ are equivalent, for ./∈ {>, ≥}.
3.2
XPL Semantics
The semantics of XPL changes from GPL only due to the measure of the PLTS outcomes. In particular, we retain the same semantics on diamond and box. The semantics is defined with respect to a fixed PLTS L = (S, δ, P, I). The function 7
sa a b
a
sb
sc
c
Figure 1: Example PLTS with nondeterministic choice on “a” ΘL : Ψ → 2ML remains the same, while |=L ⊆ S × Φ differs for the probabilistic operators. Definition 6 (XPL semantics). The semantics for the state formulae is given in Table 3. For the fuzzy formulae, the semantics are as in Table 1. Note the use of sup and inf in Table 3. We refer to the value supγ mγs (ΘL,s (ψ)) as a probabilistic value and write it as Prmax L,s (ψ) ([4] uses the term capacity). Unlike in GPL, though, we may not always be able to compute it with a model checking algorithm.
3.3
Separability of Fuzzy Formulae
With nondeterminism present, we lose the general relation between conjunctions and disjunctions, as in Eq. (1) (page 6). Instead, since we are maximizing (or minimizing) over schedulers, we would want the relation in Eq. (2). ?
max max max Prmax L,s (ψ1 ∨ ψ2 ) = PrL,s (ψ1 ) + PrL,s (ψ2 ) − PrL,s (ψ1 ∧ ψ2 ).
(2)
We can see that this does not hold with a simple formula ψa = [a]hbitt∨[a]hcitt, for a PLTS L (Figure 1) with {sa , sb , sc } ∈ SL and nondeterministic a-transitions from sa to sb and sc , such that sb has a b-transition, but no c-transitions, and max vice versa for sc . Then, we can easily see that Prmax L,sa (ψa ) = PrL,sa ([a]hbitt) = max max PrL,sa ([a]hcitt) = 1, but PrL,sa ([a]hbitt ∧ [a]hcitt) = 0. However, Lemma 2 still applies with nondeterminism present, and we can still model check this formula, if we delay dealing with the disjunction by rewriting ψa as [a](hbitt ∨ hcitt), as the subformula guarded by a is trivial to check at any state. We generalize this to a syntactic notion of separability, defined below. A subformula of ψ of the form haiψ 0 or [a]ψ 0 is called a modal subformula of ψ. We say that ψ 0 is an unguarded subformula of ψ if there is no modal subformula ψ 00 of ψ such that ψ 0 is a subformula of ψ 00 . The GPL model checking algorithm requires closed formulae and for bound variables to be guarded by actions [3], and we adopt this requirement as well. Definition 7 (Formula Transformations). • The fixed-point expansion of ψ, denoted by FPE (ψ) is a formula ψ 0 obtained by expanding any unguarded subformula of the form σX.ψX to ψX [σX.ψX /X] where σ ∈ {µ, ν}.
8
• We say that a formula is non-probabilistic if it is a state formula, or of the form haiφ and [a]φ for a ∈ Act and φ ∈ {tt, ff}. The purely probabilistic abstraction of a fuzzy formula ψ, denoted by PPA(ψ), is a formula obtained by eliminating unguarded non-probabilistic subformulae. • A grouping of a formula ψ, denoted by GRP (ψ), groups modalities in a formula using distributivity. Formally, GRP maps ψ to a syntactically smallest ψ 0 that is equivalent to ψ based on the equivalences in Lemma 2. At a high level, a necessary condition of separability is that the actions guarding distinct conjuncts and disjuncts of a formula are distinct as well. Definition 8 (Action set). The action set of a formula ψ, denoted by actions(ψ) is the set of actions appearing at unguarded modal subformulae of ψ: • actions(φ) = ∅;
actions(haiψ) = actions([a]ψ) = {a};
• actions(ψ1 ∧ ψ2 ) = actions(ψ1 ∨ ψ2 ) = actions(ψ1 ) ∪ actions(ψ2 ); • actions(µX.ψ) = actions(νX.ψ) = actions(ψ).
We can now define separability based on action sets of formulae as follows. Definition 9 (Separability). The set of all separable formulae is the largest set S such that ∀ψ ∈ S, if ψ 0 = GRP (PPA(FPE (ψ))), then 1. every subformula of ψ 0 is in S, and 2. if ψ 0 = ψ1 ⊕ ψ2 where ⊕ ∈ {∧, ∨}, then action(ψ1 ) ∩ action(ψ2 ) = ∅. A formula ψ is separable if ψ ∈ S.
Below we illustrate separability of formulae. Let ψi for i = 1..4 be all separable and distinct, and also ψ1 ∨ ψ2 and ψ3 ∨ ψ4 are separable. Note that GRP uses only distributivity of the modal operators over “∧” and “∨”, and not the distributivity of the boolean operators themselves. Consequently, a separable formula may be equivalent to a non-separable formula. Example 10 (Separable formula with equivalent non-separable formula). The formula ψs is separable. The DNF version of ψs is not separable since action sets of disjuncts overlap. ψs = [a](ψ1 ∨ ψ2 ) ∧ [b](ψ3 ∨ ψ4 ),
(3)
ψs0 = DNF of ψ ≡ ([a]ψ1 ∧ [b]ψ3 ) ∨ ([a]ψ1 ∧ [b]ψ4 )∨ ∨ ([a]ψ2 ∧ [b]ψ3 ) ∨ ([a]ψ2 ∧ [b]ψ4 ). (4)
9
Example 11 (Non-separable formula). The formula ψe is a subformula of ψs0 (4), is not separable, and has no equivalent separable formula: ψe = ([a]ψ1 ∧ [b]ψ4 ) ∨ ([a]ψ2 ∧ [b]ψ3 ).
(5)
With ψe , we need to satisfy ψ1 or ψ2 following an a action, and likewise for ψ3 or ψ4 following a b action. An equivalent separable formula would thus have to include [a](ψ1 ∨ ψ2 ) and [b](ψ3 ∨ ψ4 ), but this would also be satisfied by, e.g., outcomes satisfying only [a]ψ1 ∧ [b]ψ3 . We say that a formula is entangled at a state if it is not (equivalent to) a separable formula even after considering that state’s specific characteristics. For instance, ψe is entangled only at states with both a and b actions present. Since GRP groups together modal operators with a common action, we have the following important consequence. Remark. All conjunctive formulae and disjunctive formulae are separable.
4
Model Checking XPL Formulae
We outline a model checking procedure for XPL formulae for a fixed PLTS L = (S, δ, P, I), along similar lines to the GPL model checking algorithm in [3, Section 4]. The model checking procedure succeeds whenever the given formula is separable. Definition 12 (Fisher-Ladner closure). Given a formula ψ, its Fisher-Ladner closure, Cl(ψ), is the smallest set such that the following hold: • ψ ∈ Cl(ψ). • If ψ 0 ∈ Cl(ψ), then: – if ψ 0 = ψ1 ∧ ψ2 or ψ1 ∨ ψ2 , then ψ1 , ψ2 ∈ Cl(ψ); – if ψ 0 = haiψ 00 or [a]ψ 00 for some a ∈ Act, then ψ 00 ∈ Cl(ψ); – if ψ 0 = σX.ψ 00 , then ψ 00 [σX.ψ 00 /X] ∈ Cl(ψ), with σ either µ or ν. The core of the model checking algorithm is the construction of a dependency graph Dg(s, ψ), to compute Prmax L,s (ψ). When constructing a dependency graph, in order to divide a formula by actions, we transform it into factored form, in a similar manner to checking separability. If we are unable to transform a formula into a factored form, as can happen when a formula is non-separable, the graph construction terminates with failure. Definition 13 (Factored form). A factored formula ψ with an action set α = actions(ψ) can be trivial, when ψ ∈ {tt, ff} (i.e., α = ∅), or in the action V form, 0 haiψ , when α = {a}. Otherwise, it is conjunctive or disjunctive, i ψi and W ψ , respectively, with each ψ also factored and not the same form as ψ, such i i i that the αi s, with αi = actions(ψi ), form a partition of α. 10
A formula ψ 0 can be transformed into a semantically one ψ 00 equivalent 00 0 0 that is in factored form as: ψ = GRP P E s, FPE (ψ ) . P E(s, ψ ) partially evaluates ψ 0 , by evaluating non-probabilistic subformulae of ψ 0 as well as all unguarded modal subformulae with actions absent at state s, yielding tt or ff for each, and simplifying the result. Let AO(S) represent the set of and-or trees with elements of a set S as leaves. When applying GRP , we will view an action a as a prefix label on the 0 resulting subtree ψa0 ∈ AO(Cl(ψ)), rather than viewing haiψa , which may not 0 00 be in Cl(ψ), as a leaf. (s, ψ ), , (s, ψ ) ∈ E. Definition 14 (Dependency graph). The dependency graph for model checking a formula ψ with respect to a state s in PLTS L, denoted by Dg(s, ψ), is a directed graph (N, E), where node set N ⊆ S × AO(Cl(ψ)), and edge set E ⊆ N × (Act ∪ {, ∧ , ∨ }) × N ; i.e., the edges are labeled from Act ∪ {, ∧ , ∨ }. The sets N and E are the smallest such that: • (s, ψ) ∈ N . • If (s0 , ψ 0 ) ∈ N , ψ 0 is not in factored form: if equivalent ψ 00 in factored form exists, then (s0 , ψ 00 ) ∈ N and ((s0 , ψ 0 ), , (s0 , ψ 00 )) ∈ E. • If (s0 , ψ10 ⊕ ψ20 ) ∈ N then (s0 , ψi0 ) ∈ N for i = 1, 2. Moreover, ((s0 , ψ10 ⊕ ψ20 ), ⊕ , (s0 , ψi0 )) ∈ E for i = 1, 2, and ⊕ ∈ {∧, ∨}. • If (s0 , haiψ 0 ) ∈ N then (s00 , ψ 0 ) ∈ N for each s00 such that (s0 , a, ·, s00 ) ∈ δ. Moreover, ((s0 , haiψ 0 ), a, (s00 , ψ 0 )) ∈ E. A node (s0 , ψ 0 ) ∈ N in has an associated semantics of [[(s0 , ψ 0 )]] = ΘL,s0 (ψ 0 ). If (s0 , ψ 0 ) ∈ N and ψ 0 has no factored form, then dependency graph construction fails. When we transform ψ 0 to the factored form ψ 00 , the semantics does not change, i.e., [[(s, ψ 0 )]] = [[(s, ψ 00 )]]. For the factored formulae, standard XPL semantics apply (Table 1). Note that we can assume action nodes to be of the form (s, haiψ 0 ), as the action a must then be present at state s. From these semantics, we also get the relationships for the probabilistic values. Lemma 15 (Probabilistic values). Fix Dg(s0 , ψ) = (N, E). The probabilistic 0 0 value Prmax L,s (ψ ) for a node (s, ψ ) is as follows: max • Prmax L,s (ff) = 0 and PrL,s (tt) = 1.
• If (s, ψ 0 ) is an and-node, then: Y
0 Prmax L,s (ψ ) =
((s,ψ 0 ),∧ ,(s,ψi0 ))∈E
11
0 Prmax L,s (ψi ).
• If (s, ψ 0 ) is an or-node, then: a
0 Prmax L,s (ψ ) =
0 Prmax L,s (ψi ).
((s,ψ 0 ),∨ ,(s,ψi0 ))∈E
• If (s, ψ 0 ) is an action node, i.e., ψ 0 = haiψa0 , then: X 0 0 Prmax P (s, a, c, s0 ) · Prmax L,s (ψ ) = max L,s0 (ψa ). c∈N
0 ))∈E ((s,ψ 0 ),a,(s0 ,ψa
• The remaining nodes (s, ψ 0 ) have a unique successor (s, ψ 00 ) with max 0 00 Prmax L,s (ψ ) = PrL,s (ψ ). Proof. Most of the cases are straightforward and similar to the GPL model checking algorithm [3, Lemma 8]. The and -node and or -node cases have the product and coproduct, respectively, due to independence. We explain the action node case in more detail. We do not explicitly refer to the schedulers, directly making the nondeterministic choice with max. Instead, this choice is made based c∈N
on a formula, ψa0 , to be satisfied. Given separability of the initial formula ψ, this is well-defined: given ψ and the partial computation, the schedulers we defined can deduce ψa0 . We justify the max choice through a common method, called strategy imc∈N
provement or strategy stealing [10, 14]. Essentially, we assume a scheduler γ 0 that does better than Prmax L,s (ψ) and define γ to make all the choices exactly 0 as γ, except for the current choice. Then, γ will do at least as well as γ, but 0 0 is also bounded from above by Prmax L,s (ψ ) for all s and ψ , contradicting the assumption. 0 We note that, although a particular choice may maximize Prmax L,s (ψ ), a scheduler that makes this choice every time is not necessarily optimal. Indeed, no optimal scheduler may exist, in which case we would only have -optimal schedulers for any > 0 [8, 14]. The probabilistic value may be predicated on making a different choice eventually. The max formulation is consistent with c∈N
this possibility, as are the schedulers. Theorem 16 (Model checking termination). The graph construction of Dg(s, ψ) terminates for any XPL formula ψ and PLTS L. Moreover, if ψ is separable, it will always complete the construction. Proof. Cl(ψ) is finite, so AO(Cl(ψ)) is finite. The number of actions in L and ψ is finite, so the number of factored formulae is finite. This is sufficient to guarantee termination, as we fail when we cannot construct a factored formula. Meanwhile, separability implies that we can always construct a factored formula starting from ψ 0 ∈ AO(Cl(ψ)).
12
b, c s3 a:
1 3
s1 a
b, c
s2 a:
2 3
s4 a:
s5
3 4
a:
1 4
s6
Figure 2: An example PLTS From the completed graph, a system of polynomial equations can be extracted. Equations are readily constructed by considering each Prmax L,s (ψ) as a variable. Note that the graph treats µ and ν nodes exactly the same way. In solving the stratified system, we start from 0 to find the least fixed point, and 1 for the greatest fixed point. The alternation-free restriction ensures that there is no cycle in the graph containing both a µ-node and a ν-node. Theorem 17 (Soundness of computation). The construction of graph Dg(s, ψ) leads to the equation system yielding the correct value of Prmax L,s (ψ). Proof. As our system x = P (x) is on [0, 1]n , the operations in Lemma 15 are monotone. Thus, the entire system is monotone, and, starting with x0 = 0 and iterating with xi+1 = P (xi ), x converges to the least fixed point (likewise for x0 = 1 and the greatest fixed point) [8]. Example 18 (Model Checking). For the PLTS L in Figure 2 and fuzzy formula ψ = µX.[a][b]X ∧ [a][c]X, we have symmetric nondeterministic choices on b and c from state s2 , and the formula is satisfied by all finite d-trees (since both s3 and s4 have a probability greater than 12 of returning to s2 , the infinite d-trees have positive measure on any scheduler). Letting ψbc = [b]ψ ∧ [c]ψ, the dependency min 1 1 graph is shown in Figure 3, and Prmax L,s1 (ψ) = 4 (also, PrL,s1 (ψ) = 9 ). max We find PrL,s1 (ψ) as the least fixed point from the following equations: 1 2 xa1 = xbc xb2 = max(xa3 , xa4 ), xa3 = xbc + xbc , xbc 2 , 5 =1 3 5 3 2 1 3 b c xbc xc2 = max(xa3 , xa4 ), xa4 = xbc + xbc , xbc 2 = x2 · x2 , 6 =1 4 6 4 2 Solving the equations, we get xa1 = 41 . Note that the model checking algorithm can be broken into the following two parts: writing down a polynomial system, and then finding the (approximate) solution. The first part is bounded double-exponentially in the size of the fuzzy formula, as we deal with trees of formulae from the Fisher-Ladner closure. For the second part, value iteration is guaranteed to converge [8], but may be exponentially slow in the number of digits of precision [13]. When the polynomial system is of a specific form, alternative approximation methods have been proven to be efficient [7]. 13
(s1 , ψ) (s1 , [a]ψbc ) a (s2 , [b]ψ) b
∧
(s2 , [b]ψ ∧ [c]ψ)
∧ c
b
(s3 , ψ)
(s2 , [c]ψ) c (s4 , ψ)
a
a
(s3 , [a]ψbc )
(s4 , [a]ψbc )
a
a
(s5 , [b]ψ ∧ [c]ψ)
(s6 , [b]ψ ∧ [c]ψ)
(s5 , tt)
(s6 , tt)
Figure 3: Dependency graph Dg(s1 , ψ) for example in 2
5 5.1
Encoding Other Model Checking Problems Encoding PCTL* over MDPs
The syntax of PCTL* is as follows, where A ∈ P rop, ./∈ {}, and φ and ψ represent state formulae and path formulae, respectively: φ ::= tt | A | φ ∧ φ | ¬φ | Pr./p ψ,
ψ ::= φ | Xψ | ψUψ | ψ ∧ ψ | ¬ψ.
This is similar to the syntax given by [1, Chapter 9], except omitting the bounded until operator. To encode PCTL* in XPL, we can use Prmax for the Pr operator (PCTL* assumes a maximizing scheduler), the neg function for the ¬ operator, haiφ for the next operator, Xφ, and the fixed point formula µX.ψ2 ∨ (ψ1 ∧ haiX) for the until operator, ψ1 Uψ2 . Since PCTL* cannot distinguish between action labels, we translate a Markov Decision Processes (MDPs) to a PLTS by retaining the same transition structure, but renaming all labels to “a”. Note that formulas are trivially separable if they have only one action. Consequently, we can model check PCTL* formulae over MDPs using our encoding and model checking algorithm. To the best of our knowledge, there is no encoding of PCTL* in pLµ ⊕ [15], although it can encode PCTL [12, 14]. We conjecture this is impossible, due to the lack of an operator corresponding to the semantics of conjunction for PCTL* path formulae.
14
A
b 1: B r2
n
en
ex 1 ex 2
1/2 1/4
n
B
1
r1 1/4
e1
c
n
ex'1
1 3/5
n 2/5
en'
u
n
e2
n
b1': A
ex'2 2/3
b2': B
1 1/3
1
z
v
1
Figure 4: Example RMDP with Call, Return, and Exit edges added to “A”.
5.2
Encoding of RMDP Termination
A Recursive MDP [10] is specified as a set of components. A component has an entry node and one or more exit nodes. Components may contain boxes, each box having a call port that represents a procedure call and return ports to represent possible return from the called procedure. Figure 4 shows a recursive MDP with two components, A and B. Any call to A non-deterministically results in either a call to B (via box b1 ) or a transition to u. We can translate an RMDP into a PLTS L with Act = {p, n, c, ri , ei }, with states of the PLTS corresponding to nodes of the RMDP. We retain the RMDPs transitions, labeling them as “n” for actions from a non-deterministic choice and “p” for probabilistic choice. To this basic structure we add three new kinds of edges: • “ei ” for the ith exit node of a component, • “c” edges from a call port to the called component’s entry node, and • “ri ” edges from a call port to each return port in the box. While c edges denote control transfer due to a call, r edges summarize returns from the called procedure. Termination of 1-RMDP can be encoded as the following separable formula: ψ1 = µX.he1 itt ∨ hpiX ∨ hniX ∨ (hciX ∧ hr1 iX).
(6)
Note how branching time, on c and r actions, is used to simulate the recursive RMDP call, and that ψ1 is separable. The corresponding formula for a 2-exit RMDP, is not separable: ψ21 =µ he1 itt ∨ hpiψ21 ∨ hniψ21 ∨ (hciψ21 ∧ hr1 iψ21 ) ∨ (hciψ22 ∧ hr2 iψ21 ), ψ22
5.3
=µ he2 itt ∨
hpiψ22
∨
hniψ22
∨
(hciψ21
∧
hr1 iψ22 )
∨
(hciψ22
∧
(7)
hr2 iψ22 ).
PTTL and Branching Processes
Branching Processes (BPs) are a model for which extinction corresponds to termination of 1-exit Recursive Markov Chains, but with the branching time 15
model where the processes evolve concurrently. Probabilistic Tree Temporal Logic (PTTL) has been introduced as a logic similar to PCTL [12], but for BPs [2]. BPs have also been extended with nondeterminism, yielding Branching MDPs (BMDPs), for which the extinction and reachability problems can be solved [8, 10]. We write the syntax of PTTL [2, Definition 18], where A ∈ P rop, ./∈ {}, and we refer to φ and ψ as state and fuzzy formulae, as for XPL: φ ::= tt | A | ¬φ | φ ∧ φ | Pr./p ψ, ψ ::= AXφ | EXφ | φAUφ | φEUφ | φARφ | φERφ. As for RMDPs, we can readily translate BPs and BMDPs to PLTSs. We give the semantics for PTTL over PLTSs, with Pr./p assuming maximizing schedulers, by encoding it in XPL, as EP T T L (γ). γ ∈ P rop ∪ {tt}, γ, 0 neg(E (γ )), γ = ¬γ 0 , PTTL EP T T L (γ1 ) ∧ EP T T L (γ2 ), γ = γ1 ∧ γ2 , EP T T L (γ) = Prmax E (ψ), γ = Pr./p ψ, PTTL ./p [−]EP T T L (φ), γ = AXφ, µX.EP T T L (φ2 ) ∨ (EP T T L (φ1 ) ∧ [−]X), γ = φ1 AUφ2 , νX.EP T T L (φ2 ) ∧ (EP T T L (φ1 ) ∨ [−]X), γ = φ1 ARφ2 . For EX, EU, and ER, we replace the boxes with diamonds. In this sense, PTTL is a natural extension of PCTL over PLTSs for the case |Act| > 1, and all the formulae in the encoding are separable.
6
Conclusion and Future Work
GPL and pLµ represented two divergent paths in probabilistic extensions of µ-calculus. GPL prioritized probabilistic choices, with the idea that external nondeterminism would be sufficient, while pLµ retained internal nondeterminism, but relegated the probabilistic nature essentially to computation. Although closely related, algorithms to check properties of RMCs (and pPDSs [5]) were developed independently [9]. These were related to algorithms for computing properties of systems such as branching process (BP) extinction and the language probability of Stochastic Context Free Grammars, which were also phrased in terms of solving a set of polynomial equations. The relationship between GPL and these systems was mentioned briefly in [11], but has remained largely unexplored. The logic pLµ is able to support branching time and an intuitive game semantics [14]. The scope of pLµ includes infinite-state systems as well. However, as we showed in Section 5, XPL can be used to encode properties that we are unable to encode in pLµ. Determining the relationship between XPL and pLµ in branching time is an important problem.
16
There has been significant interest in the study of expressive systems with nondeterministic choices, such as RMDPs and Branching MDP (BMDPs) [10]. At the same time, the understanding of the polynomial systems has expanded. In [6], the class of Probabilistic Polynomial Systems (PPS) is introduced, which characterizes when efficient solutions to polynomial equation systems are possible even in the worst case [7]. While [9] did not distinguish the systems arising from single-exit RMCs (1-RMCs) with those from multi-exit RMCs, the PPS class is limited to 1-RMCs. It was also extended for RMDP termination, and later BMDP reachability, both having polynomial-time complexity for min/maxPPSs [6, 8]. Systems producing equations in PPS form show an interesting characteristic: that the properties are expressible as purely conjunctive or purely disjunctive formulae. Recall that such formulae are trivially separable. Characterizing equation systems that arise from separable formulae and investigating their efficient solution is an interesting open problem.
References [1] Christel Baier. On algorithmic verification methods for probabilistic systems. Habilitation thesis, Fakult¨at f¨ ur Mathematik & Informatik, Universit¨at Mannheim, 1998. [2] Taolue Chen, Klaus Dr¨ager, and Stefan Kiefer. Model checking stochastic branching processes. In MFCS, pages 271–282, Berlin, Heidelberg, 2012. Springer. [3] Rance Cleaveland, S. Purushothaman Iyer, and Murali Narasimha. Probabilistic temporal logics via the modal mu-calculus. Theoretical Computer Science, 342(2-3):316–350, 2005. [4] Jose Desharnais, Vineet Gupta, Radha Jagadeesan, and Prakash Panangaden. Weak bisimulation is sound and complete for PCTL*. In CONCUR, volume 2421 of LNCS, pages 355–370. Springer Berlin Heidelberg, 2002. [5] Javier Esparza, Anton´ın Kucera, and Richard Mayr. Model checking probabilistic pushdown automata. In LICS, pages 12–21, 2004. [6] Kousha Etessami, Alistair Stewart, and Mihalis Yannakakis. Polynomial time algorithms for branching Markov decision processes and probabilistic min(max) polynomial Bellman equations. In ICALP, Part I, pages 314–326, Berlin, Heidelberg, 2012. Springer. [7] Kousha Etessami, Alistair Stewart, and Mihalis Yannakakis. Polynomial time algorithms for multi-type branching processes and stochastic contextfree grammars. In STOC, pages 579–588. ACM, 2012. [8] Kousha Etessami, Alistair Stewart, and Mihalis Yannakakis. Greatest fixed points of probabilistic min/max polynomial equations, and reachability for
17
branching Markov decision processes. In ICALP, Part II, pages 184–196, Berlin, Heidelberg, 2015. Springer. [9] Kousha Etessami and Mihalis Yannakakis. Recursive Markov chains, stochastic grammars, and monotone systems of nonlinear equations. J. ACM, 56(1):1:1–1:66, February 2009. [10] Kousha Etessami and Mihalis Yannakakis. Recursive Markov decision processes and recursive stochastic games. J. ACM, 62(2):11:1–11:69, May 2015. [11] Andrey Gorlin, C. R. Ramakrishnan, and Scott A. Smolka. Model checking with probabilistic tabled logic programming. TPLP, 12(4-5):681–700, 2012. [12] Hans Hansson and Bengt Jonsson. A logic for reasoning about time and reliability. Formal Aspects of Computing, 6(5):512–535, 1994. [13] Stefan Kiefer, Michael Luttenberger, and Javier Esparza. On the convergence of Newton’s method for monotone systems of polynomial equations. In STOC, pages 217–226, 2007. [14] Matteo Mio. Probabilistic modal mu-calculus with independent product. In FOSSACS, pages 290–304, Berlin, Heidelberg, 2011. Springer. [15] Matteo Mio. Game semantics for probabilistic modal mu-calculi. PhD thesis, The University of Edinburgh, 2012. [16] Roberto Segala. A compositional trace-based semantics for probabilistic automata. In CONCUR, volume 962 of LNCS, pages 234–248. Springer, 1995. [17] Roberto Segala and Andrea Turrini. Comparative analysis of bisimulation relations on alternating and non-alternating probabilistic models. In QEST, pages 44–53. IEEE Computer Society, 2005.
18
XPL: An extended probabilistic logic for probabilistic transition systems∗ ANDREY GORLIN C. R. RAMAKRISHNAN Department of Computer Science, Stony Brook University, Stony Brook, NY 11794, U.S.A. {agorlin,cram}@cs.stonybrook.edu April 22, 2016
Abstract Generalized Probabilistic Logic (GPL) is a temporal logic, based on the modal mu-calculus, for specifying properties of reactive probabilistic systems. GPL is expressive enough that termination of Recursive Markov Chains (RMCs) can be cast as a model checking problem over GPL. In this paper, we explore XPL, an extension to GPL for expressing properties of transition systems with probabilistic and nondeterministic behavior, such as Recursive Markov Decision Processes (Recursive MDPs or RMDPs). However, since termination of multi-exit RMDPs is undecidable, model checking in XPL is undecidable as well. We provide a model checking procedure for XPL that always terminates: either successfully with the model checking result, or with failure. We define a sufficient condition for success via a syntactically-characterized class, called separable XPL formulae. We show that a large class of problems, including termination of 1-exit RMDPs and PCTL* model checking of MDPs, can be reduced to the model checking of separable XPL formulae.
1
Introduction
For finite-state systems, model checking a temporal property can be cast in terms of model checking in the modal µ-calculus, the so-called “assembly language” of temporal logics. A number of temporal logics have been proposed and used for specifying properties of finite-state probabilistic systems. Two of the notable logics for probabilistic systems based on the µ-calculus are GPL [3] and pLµ [14]. GPL is defined over Reactive Probabilistic Labeled Transition Systems (RPLTSs). The formal definitions of GPL and RPLTSs are recalled in Section 2, mainly following [3]. In an RPLTS, each state has a set of outgoing ∗ This
work was partially supported by NSF grant IIS-1447549.
1
transitions with distinct labels; each transition, in turn, specifies a (probabilistic) distribution of target states. GPL is expressive enough to serve as an “assembly language” of a large number of probabilistic temporal logics. For instance, model checking PCTL* properties over Markov Chains, as well as termination and reachability of Recursive Markov Chains can be cast in terms of GPL model checking [3, 11]. In this paper, we consider an extension to GPL to express properties of probabilistic systems with internal non-deterministic choice. This logic, called Extended Probabilisitic Logic (XPL) is defined over Probabilistic Labeled Transition Systems (PLTSs). In a PLTS, each state has a set of outgoing transitions, possibly with common labels; and each transition specifies a distribution of target states. PLTSs thus exhibit both probabilistic choice and internal non-deterministic choice. Contributions: XPL is a conservative extension to GPL (see Section 3). However, in contrast to RPLTSs and GPL, model checking of PLTSs with XPL is undecidable in general. We give a syntactically-defined subclass, called separable XPL and give a model checking algorithm that always succeeds for this subclass (see Section 4). We show that the separable subset of XPL is sufficiently expressive to encode a wide variety of verification problems. For instance, PCTL* model checking of Markov Decision Processes (MDPs), termination of 1-exit Recursive MDPs, and PTTL model checking of branching MDPs can all be reduced, in linear time, to model checking separable XPL formulae over PLTSs (see Section 5). Related Work: GPL treats conjunction in a traditional manner, retaining the properties that ψ ∧ ¬ψ = ff, and ψ ∧ ψ = ψ for any formula ψ. However, probability value of ψ1 ∧ ψ2 cannot be computed based on the probability values of the conjuncts ψ1 and ψ2 . This makes model checking in GPL more complex, but also contributes to GPL’s expressiveness. In contrast to GPL, the most expressive version of pLµ, denoted pLµ ⊕ [14, 15], defines three conjunction operators and their duals such that their probability values can be computed from the probabilities of the conjuncts. The three conjunctions are defined as minimum, independent product, and truncated cosum. Along the same lines as our XPL encoding, we can encode termination of 1exit RMDP as model checking in pLµ , and RMC termination in pLµ ⊕ . However, PCTL* still appears to be out of reach, because LTL can have conjunctions for which the values cannot be computed directly from the values of the conjuncts. The relationship of our work to earlier papers, is discussed in Section 6.
2
Preliminaries
In this section, we formally define PLTSs which are used to define the semantics of XPL. We also summarize the syntax and semantics of GPL, using the notations from [3]. 2
2.1
Probabilistic Labeled Transition Systems
We define a probabilistic labeled transition system (PLTS) as an extension of [3]’s RPLTS. Definition 1 (PLTS). With respect to fixed sets Act and P rop of actions and propositions, respectively, a PLTS L is a quadruple (S, δ, P, I), where • S is a countable set of states; • δ ⊆ S × Act × N × S is the transition relation; • P : δ → (0, 1] is the transition probability distribution satisfying: P – ∀s ∈ S.∀a ∈ Act.∀c ∈ N. P (s, a, c, s0 ) ∈ {0, 1}, and s0 :(s,a,c,s0 )∈δ
– ∀s ∈ S.∀a ∈ Act.∀c ∈ N. (∃s0 .(s, a, c, s0 ) ∈ δ) ⇒
P (s, a, c, s0 ) = 1;
P
s0 :(s,a,c,s0 )∈δ
• I : S → 2P rop is the interpretation, recording the set of propositions true at a state. A reactive PLTS does not have internal nondeterminism, i.e., having c = 0 for all (s, a, c, s0 ) ∈ δ, in which case we may write elements of δ as triples. This definition is in line with the most general for a PLTS [14, 16], in which, given an action, a probabilistic distribution is chosen nondeterministically. Other equally expressive models include alternating automata, in which labeled nondeterministic ones are followed by silent probabilistic choices. The difference between such models has been analyzed with respect to bisimulation [17]. a a Given L = (S, δ, P, I), a partial computation is a sequence σ = s0 →1 s1 →2 an · · · → sn , where for all 0 ≤ i < n, (si , ai+1 , ·, si+1 ) ∈ δ. Also, fst(σ) = s0 and last(σ) = sn denote, respectively, the first and last states in σ. Each transition of a partial computation is labeled with an action ai ∈ Act. The set of all partial computations of L is denoted by CL , and CL (s) = {σ ∈ CL | fst(σ) = s}. a an a a Composition of partial computations, σ → σ 0 , represents s0 →1 · · · → sn → b1 bm 0 s00 → ··· → sm if (sn , a, ·, s00 ) ∈ δ. Partial computation σ 0 is a prefix of σ if a a 1 σ 0 = s0 → · · · →i si for some i ≤ n. From a set of partial computations, we can build deterministic trees (dtrees). We often denote a d-tree by the set of paths in the tree. Every d-tree is prefix-closed and deterministic. T ⊆ CL is prefix-closed if, for every σ ∈ T and σ 0 a prefix of σ, σ 0 ∈ T . T is deterministic if for every σ, σ 0 ∈ T with a
a
a
a
a
a0
n n σ = so →1 · · · → sn → s · · · and σ 0 = s0 →1 · · · → sn → s0 · · · , either a 6= a0 or s = s0 , i.e., if a pair of computations share a prefix, the first difference cannot involve transitions labeled by the same action. A d-tree T has a starting state, denoted root(T ); if s = root(T ) then T ⊆ CL (s). TL refers to all the d-trees of L, and TL (s) = {T ∈ TL | root(T ) = s}. T 0 is a a a prefix of T if T 0 ⊆ T . T → T 0 means T 0 = {σ | root(T ) → σ ∈ T }. T is finite
3
if |T | < ∞, and maximal if there exists no d-tree T 0 with T ⊂ T 0 . ML and ML (s) are analogous to TL and TL (s), but for maximal d-trees. An outcome is a maximal d-tree. The subsequent concepts apply only to RPLTSs, and we will extend them to PLTSs in Section 3. Intuitively, the probability of some finite prefix is the product of the probabilities of all the edges. Formally, a basic cylindrical subset of ML (s) contains all trees sharing a given prefix. Letting s ∈ S, and T ∈ TL (s) to be finite, BT = {T 0 ∈ ML | T ⊆ T 0 }. The measure of BT is: Y m(BT ) = P (last(σ), a, last(σ 0 )). (σ,a,σ 0 )∈edges(T )
From here, a probability measure ms : Bs → [0, 1] on the smallest field of sets Bs is generated from subsets BT with ms (BT ) = m(BT ).
2.2
GPL Syntax
GPL has two different kinds of formulae. State formulae depend directly only on the given state. Fuzzy formulae depend on outcomes. We give the syntax of GPL, with X ∈ V ar, a ∈ Act, A ∈ P rop, and 0 ≤ p ≤ 1, for state formulae, φ, and fuzzy formulae, ψ, as: φ ::= A | ¬A | φ ∧ φ | φ ∨ φ | Pr>p ψ | Pr≥p ψ, ψ ::= φ | X | ψ ∧ ψ | ψ ∨ ψ | haiψ | [a]ψ | µX.ψ | νX.ψ. Note that only atomic propositions may be negated, but every operator has its dual given in the syntax. The propositional connectives, ∧ and ∨, can be used on both state and fuzzy formulae. Although not stated in the syntax, only closed fuzzy formulae ψ can be used to construct state formulae Pr>p ψ and Pr≥p ψ. These formulae check the probability for a fuzzy formula ψ (Pr>p and Pr≥1−p are duals). Operators µX.ψ and νX.ψ are least and greatest fixed point operators for the “equation” X = ψ, and formulae must be alternation-free. The semantics of GPL is given in terms of RPLTS d-trees. In that interpretation, diamond implies box : haiψ means that there is an a-transition and it satisfies ψ; [a]ψ means that if there is an a-transition, also use a set W it satisfies ψ. We V α ⊆ Act for the modalities, reading hαiψ as haiψ and [α]ψ as [a]ψ. When a∈α
a∈α
we write “−” for α, that represents Act.
2.3
GPL Semantics
We define the semantics of GPL with respect to a fixed RPLTS L = (S, δ, P, I). A function ΘL : Ψ → 2ML , augmented with an extra environment parameter e : V ar → 2ML , returns the set of outcomes satisfying a given fuzzy formula, defined inductively in Table 1. For a given s ∈ S, ΘL,s (ψ) = ΘL (ψ) ∩ ML (s). |=L ⊆ S × Φ indicates when a state satisfies a state formula, and is defined inductively in Table 2. Note that the 4
Table 1: GPL/XPL semantics: fuzzy formulae ΘL (φ)e =
[
ML (s), where φ is a closed formula,
s|=L φ
ΘL (X)e = e(X), a
ΘL (haiψ)e = {T ∈ ML | ∃T 0 : T → T 0 ∧ T 0 ∈ ΘL (ψ)e}, a
ΘL ([a]ψ)e = {T ∈ ML | (T → T 0 ) ⇒ T 0 ∈ ΘL (ψ)e}, ΘL (ψ1 ∧ ψ2 )e = ΘL (ψ1 )e ∩ ΘL (ψ2 )e, ΘL (ψ1 ∨ ψ2 )e = ΘL (ψ1 )e ∪ ΘL (ψ2 )e, ∞ [ Mi , where M0 = ∅ and Mi+1 = ΘL (ψ)e[X 7→ Mi ], ΘL (µX.ψ)e = ΘL (νX.ψ)e =
i=0 ∞ \
Ni , where N0 = ML and Ni+1 = ΘL (ψ)e[X 7→ Ni ].
i=0
Table 2: GPL semantics: state formulae s |=L s |=L s |=L s |=L s |=L s |=L
A ¬A φ1 ∧ φ2 φ1 ∨ φ2 Pr>p ψ Pr≥p ψ
iff iff iff iff iff iff
A ∈ I(s), A∈ / I(s), s |= φ1 and s |= φ2 , s |= φ1 or s |= φ2 , ms (ΘL,s (ψ)) > p, ms (ΘL,s (ψ)) ≥ p.
5
definitions for ΘL and |=L are mutually recursive. There are two properties of GPL fuzzy formulae that are important for the completeness of model checking. First, we have distributivity on box and diamond: Lemma 2 (Distributivity on modal operators ([3] Lemma 1)). Letting ⊕ ∈ {∧, ∨}: ΘL ([a]ψ1 ⊕ [a]ψ2 ) = ΘL [a](ψ1 ⊕ ψ2 ) , ΘL (haiψ1 ⊕ haiψ2 ) = ΘL hai(ψ1 ⊕ ψ2 ) , ΘL ([a]ψ1 ∧ haiψ2 ) = ΘL hai(ψ1 ∧ ψ2 ) , ΘL ([a]ψ1 ∨ haiψ2 ) = ΘL [a](ψ1 ∨ ψ2 ) . Second, we can relate the probability of a conjunction with that of a disjunction [3, Lemma 2]: ms (ΘL,s (ψ1 ∨ ψ2 )) = ms (ΘL,s (ψ1 )) + ms (ΘL,s (ψ2 )) − ms (ΘL,s (ψ1 ∧ ψ2 )). (1) Additionally, although there is no negation operator in the syntax, we can write the negation of a fuzzy formula ψ, neg(ψ), and of a state formula φ, neg(φ), such that, for any RPLTS L and state s ([3, Lemma 3]): ΘL,s (neg(ψ)) = ML (s) − ΘL,s (ψ)
and s |=L neg(φ) ⇐⇒ s 6|=L φ.
The proof involves switching all the operators to their duals.
3
XPL
To resolve the nondeterministic transitions in a PLTS, we additionally require a scheduler. Recall, from Section 2.1, that CL is the set of all partial computations σ of L. Definition 3 (PLTS scheduler). A scheduler for an PLTS L is a function γ : CL × Act → N, such that if an action a is present at last(σ), then γ(σ, a) = c implies that there exists s ∈ S such that (last(σ), a, c, s) ∈ δ. Note that we have defined deterministic schedulers, which are also aware of their relevant histories. Given a scheduler γ for an PLTS L, we have a (countable) RPLTS Lγ . We define a probability distribution: Definition 4 (Combined probability). The probability distribution of a PLTS L with scheduler γ is a function, PL,γ : CL × Act × S → [0, 1], where: PL,γ (σ, a, s) = PL (last(σ), a, γ(σ, a), s).
6
Table 3: XPL semantics: state formulae s |=L s |=L s |=L s |=L s |=L s |=L s |=L s |=L
A ¬A φ1 ∧ φ2 φ1 ∨ φ2 Prmax >p ψ Prmax ≥p ψ Prmin >p ψ Prmin ≥p ψ
iff iff iff iff iff iff iff iff
A ∈ I(s), A∈ / I(s), s |= φ1 and s |= φ2 , s |= φ1 or s |= φ2 , supγ mγs (ΘL,s (ψ)) > p, supγ mγs (ΘL,s (ψ)) ≥ p, inf γ mγs (ΘL,s (ψ)) > p, inf γ mγs (ΘL,s (ψ)) ≥ p.
Following Definition 4, PL,γ (σ, a, s) = 0 for all s when there are no atransitions from last(σ). Recall, from Section 2.1, that the basic cylindrical subset BT contains all maximal d-trees sharing the prefix tree T . For these subsets, we define the probability measure: Definition 5 (Probability measure). For a PLTS L with scheduler γ, the probability measure of a basic cylindrical subset BT is defined by a partial function mγ : 2ML → [0, 1], where: Y mγ (BT ) = PL,γ (σ, a, last(σ 0 )). (σ,a,σ 0 )∈edges(T )
Since mγ may be considered as defined for an RPLTS, we can extend it to a measure mγs as in Section 2.1.
3.1
XPL Syntax
Now we give the XPL syntax: min max max φ ::= A | ¬A | φ ∧ φ | φ ∨ φ | Prmin >p ψ | Pr≥p ψ | Pr>p ψ | Pr≥p ψ,
ψ ::= φ | X | ψ ∧ ψ | ψ ∨ ψ | haiψ | [a]ψ | µX.ψ | νX.ψ. The fuzzy formulae remain the same as in GPL. Prmin and Prmax compare against the infimum and supremum probabilities, respectively, over all schedulers. Note min min max that Prmax >p is the dual of Pr≥1−p . For an RPLTS, Pr./p , Pr./p and GPL’s Pr./p ψ are equivalent, for ./∈ {>, ≥}.
3.2
XPL Semantics
The semantics of XPL changes from GPL only due to the measure of the PLTS outcomes. In particular, we retain the same semantics on diamond and box. The semantics is defined with respect to a fixed PLTS L = (S, δ, P, I). The function 7
sa a b
a
sb
sc
c
Figure 1: Example PLTS with nondeterministic choice on “a” ΘL : Ψ → 2ML remains the same, while |=L ⊆ S × Φ differs for the probabilistic operators. Definition 6 (XPL semantics). The semantics for the state formulae is given in Table 3. For the fuzzy formulae, the semantics are as in Table 1. Note the use of sup and inf in Table 3. We refer to the value supγ mγs (ΘL,s (ψ)) as a probabilistic value and write it as Prmax L,s (ψ) ([4] uses the term capacity). Unlike in GPL, though, we may not always be able to compute it with a model checking algorithm.
3.3
Separability of Fuzzy Formulae
With nondeterminism present, we lose the general relation between conjunctions and disjunctions, as in Eq. (1) (page 6). Instead, since we are maximizing (or minimizing) over schedulers, we would want the relation in Eq. (2). ?
max max max Prmax L,s (ψ1 ∨ ψ2 ) = PrL,s (ψ1 ) + PrL,s (ψ2 ) − PrL,s (ψ1 ∧ ψ2 ).
(2)
We can see that this does not hold with a simple formula ψa = [a]hbitt∨[a]hcitt, for a PLTS L (Figure 1) with {sa , sb , sc } ∈ SL and nondeterministic a-transitions from sa to sb and sc , such that sb has a b-transition, but no c-transitions, and max vice versa for sc . Then, we can easily see that Prmax L,sa (ψa ) = PrL,sa ([a]hbitt) = max max PrL,sa ([a]hcitt) = 1, but PrL,sa ([a]hbitt ∧ [a]hcitt) = 0. However, Lemma 2 still applies with nondeterminism present, and we can still model check this formula, if we delay dealing with the disjunction by rewriting ψa as [a](hbitt ∨ hcitt), as the subformula guarded by a is trivial to check at any state. We generalize this to a syntactic notion of separability, defined below. A subformula of ψ of the form haiψ 0 or [a]ψ 0 is called a modal subformula of ψ. We say that ψ 0 is an unguarded subformula of ψ if there is no modal subformula ψ 00 of ψ such that ψ 0 is a subformula of ψ 00 . The GPL model checking algorithm requires closed formulae and for bound variables to be guarded by actions [3], and we adopt this requirement as well. Definition 7 (Formula Transformations). • The fixed-point expansion of ψ, denoted by FPE (ψ) is a formula ψ 0 obtained by expanding any unguarded subformula of the form σX.ψX to ψX [σX.ψX /X] where σ ∈ {µ, ν}.
8
• We say that a formula is non-probabilistic if it is a state formula, or of the form haiφ and [a]φ for a ∈ Act and φ ∈ {tt, ff}. The purely probabilistic abstraction of a fuzzy formula ψ, denoted by PPA(ψ), is a formula obtained by eliminating unguarded non-probabilistic subformulae. • A grouping of a formula ψ, denoted by GRP (ψ), groups modalities in a formula using distributivity. Formally, GRP maps ψ to a syntactically smallest ψ 0 that is equivalent to ψ based on the equivalences in Lemma 2. At a high level, a necessary condition of separability is that the actions guarding distinct conjuncts and disjuncts of a formula are distinct as well. Definition 8 (Action set). The action set of a formula ψ, denoted by actions(ψ) is the set of actions appearing at unguarded modal subformulae of ψ: • actions(φ) = ∅;
actions(haiψ) = actions([a]ψ) = {a};
• actions(ψ1 ∧ ψ2 ) = actions(ψ1 ∨ ψ2 ) = actions(ψ1 ) ∪ actions(ψ2 ); • actions(µX.ψ) = actions(νX.ψ) = actions(ψ).
We can now define separability based on action sets of formulae as follows. Definition 9 (Separability). The set of all separable formulae is the largest set S such that ∀ψ ∈ S, if ψ 0 = GRP (PPA(FPE (ψ))), then 1. every subformula of ψ 0 is in S, and 2. if ψ 0 = ψ1 ⊕ ψ2 where ⊕ ∈ {∧, ∨}, then action(ψ1 ) ∩ action(ψ2 ) = ∅. A formula ψ is separable if ψ ∈ S.
Below we illustrate separability of formulae. Let ψi for i = 1..4 be all separable and distinct, and also ψ1 ∨ ψ2 and ψ3 ∨ ψ4 are separable. Note that GRP uses only distributivity of the modal operators over “∧” and “∨”, and not the distributivity of the boolean operators themselves. Consequently, a separable formula may be equivalent to a non-separable formula. Example 10 (Separable formula with equivalent non-separable formula). The formula ψs is separable. The DNF version of ψs is not separable since action sets of disjuncts overlap. ψs = [a](ψ1 ∨ ψ2 ) ∧ [b](ψ3 ∨ ψ4 ),
(3)
ψs0 = DNF of ψ ≡ ([a]ψ1 ∧ [b]ψ3 ) ∨ ([a]ψ1 ∧ [b]ψ4 )∨ ∨ ([a]ψ2 ∧ [b]ψ3 ) ∨ ([a]ψ2 ∧ [b]ψ4 ). (4)
9
Example 11 (Non-separable formula). The formula ψe is a subformula of ψs0 (4), is not separable, and has no equivalent separable formula: ψe = ([a]ψ1 ∧ [b]ψ4 ) ∨ ([a]ψ2 ∧ [b]ψ3 ).
(5)
With ψe , we need to satisfy ψ1 or ψ2 following an a action, and likewise for ψ3 or ψ4 following a b action. An equivalent separable formula would thus have to include [a](ψ1 ∨ ψ2 ) and [b](ψ3 ∨ ψ4 ), but this would also be satisfied by, e.g., outcomes satisfying only [a]ψ1 ∧ [b]ψ3 . We say that a formula is entangled at a state if it is not (equivalent to) a separable formula even after considering that state’s specific characteristics. For instance, ψe is entangled only at states with both a and b actions present. Since GRP groups together modal operators with a common action, we have the following important consequence. Remark. All conjunctive formulae and disjunctive formulae are separable.
4
Model Checking XPL Formulae
We outline a model checking procedure for XPL formulae for a fixed PLTS L = (S, δ, P, I), along similar lines to the GPL model checking algorithm in [3, Section 4]. The model checking procedure succeeds whenever the given formula is separable. Definition 12 (Fisher-Ladner closure). Given a formula ψ, its Fisher-Ladner closure, Cl(ψ), is the smallest set such that the following hold: • ψ ∈ Cl(ψ). • If ψ 0 ∈ Cl(ψ), then: – if ψ 0 = ψ1 ∧ ψ2 or ψ1 ∨ ψ2 , then ψ1 , ψ2 ∈ Cl(ψ); – if ψ 0 = haiψ 00 or [a]ψ 00 for some a ∈ Act, then ψ 00 ∈ Cl(ψ); – if ψ 0 = σX.ψ 00 , then ψ 00 [σX.ψ 00 /X] ∈ Cl(ψ), with σ either µ or ν. The core of the model checking algorithm is the construction of a dependency graph Dg(s, ψ), to compute Prmax L,s (ψ). When constructing a dependency graph, in order to divide a formula by actions, we transform it into factored form, in a similar manner to checking separability. If we are unable to transform a formula into a factored form, as can happen when a formula is non-separable, the graph construction terminates with failure. Definition 13 (Factored form). A factored formula ψ with an action set α = actions(ψ) can be trivial, when ψ ∈ {tt, ff} (i.e., α = ∅), or in the action V form, 0 haiψ , when α = {a}. Otherwise, it is conjunctive or disjunctive, i ψi and W ψ , respectively, with each ψ also factored and not the same form as ψ, such i i i that the αi s, with αi = actions(ψi ), form a partition of α. 10
A formula ψ 0 can be transformed into a semantically one ψ 00 equivalent 00 0 0 that is in factored form as: ψ = GRP P E s, FPE (ψ ) . P E(s, ψ ) partially evaluates ψ 0 , by evaluating non-probabilistic subformulae of ψ 0 as well as all unguarded modal subformulae with actions absent at state s, yielding tt or ff for each, and simplifying the result. Let AO(S) represent the set of and-or trees with elements of a set S as leaves. When applying GRP , we will view an action a as a prefix label on the 0 resulting subtree ψa0 ∈ AO(Cl(ψ)), rather than viewing haiψa , which may not 0 00 be in Cl(ψ), as a leaf. (s, ψ ), , (s, ψ ) ∈ E. Definition 14 (Dependency graph). The dependency graph for model checking a formula ψ with respect to a state s in PLTS L, denoted by Dg(s, ψ), is a directed graph (N, E), where node set N ⊆ S × AO(Cl(ψ)), and edge set E ⊆ N × (Act ∪ {, ∧ , ∨ }) × N ; i.e., the edges are labeled from Act ∪ {, ∧ , ∨ }. The sets N and E are the smallest such that: • (s, ψ) ∈ N . • If (s0 , ψ 0 ) ∈ N , ψ 0 is not in factored form: if equivalent ψ 00 in factored form exists, then (s0 , ψ 00 ) ∈ N and ((s0 , ψ 0 ), , (s0 , ψ 00 )) ∈ E. • If (s0 , ψ10 ⊕ ψ20 ) ∈ N then (s0 , ψi0 ) ∈ N for i = 1, 2. Moreover, ((s0 , ψ10 ⊕ ψ20 ), ⊕ , (s0 , ψi0 )) ∈ E for i = 1, 2, and ⊕ ∈ {∧, ∨}. • If (s0 , haiψ 0 ) ∈ N then (s00 , ψ 0 ) ∈ N for each s00 such that (s0 , a, ·, s00 ) ∈ δ. Moreover, ((s0 , haiψ 0 ), a, (s00 , ψ 0 )) ∈ E. A node (s0 , ψ 0 ) ∈ N in has an associated semantics of [[(s0 , ψ 0 )]] = ΘL,s0 (ψ 0 ). If (s0 , ψ 0 ) ∈ N and ψ 0 has no factored form, then dependency graph construction fails. When we transform ψ 0 to the factored form ψ 00 , the semantics does not change, i.e., [[(s, ψ 0 )]] = [[(s, ψ 00 )]]. For the factored formulae, standard XPL semantics apply (Table 1). Note that we can assume action nodes to be of the form (s, haiψ 0 ), as the action a must then be present at state s. From these semantics, we also get the relationships for the probabilistic values. Lemma 15 (Probabilistic values). Fix Dg(s0 , ψ) = (N, E). The probabilistic 0 0 value Prmax L,s (ψ ) for a node (s, ψ ) is as follows: max • Prmax L,s (ff) = 0 and PrL,s (tt) = 1.
• If (s, ψ 0 ) is an and-node, then: Y
0 Prmax L,s (ψ ) =
((s,ψ 0 ),∧ ,(s,ψi0 ))∈E
11
0 Prmax L,s (ψi ).
• If (s, ψ 0 ) is an or-node, then: a
0 Prmax L,s (ψ ) =
0 Prmax L,s (ψi ).
((s,ψ 0 ),∨ ,(s,ψi0 ))∈E
• If (s, ψ 0 ) is an action node, i.e., ψ 0 = haiψa0 , then: X 0 0 Prmax P (s, a, c, s0 ) · Prmax L,s (ψ ) = max L,s0 (ψa ). c∈N
0 ))∈E ((s,ψ 0 ),a,(s0 ,ψa
• The remaining nodes (s, ψ 0 ) have a unique successor (s, ψ 00 ) with max 0 00 Prmax L,s (ψ ) = PrL,s (ψ ). Proof. Most of the cases are straightforward and similar to the GPL model checking algorithm [3, Lemma 8]. The and -node and or -node cases have the product and coproduct, respectively, due to independence. We explain the action node case in more detail. We do not explicitly refer to the schedulers, directly making the nondeterministic choice with max. Instead, this choice is made based c∈N
on a formula, ψa0 , to be satisfied. Given separability of the initial formula ψ, this is well-defined: given ψ and the partial computation, the schedulers we defined can deduce ψa0 . We justify the max choice through a common method, called strategy imc∈N
provement or strategy stealing [10, 14]. Essentially, we assume a scheduler γ 0 that does better than Prmax L,s (ψ) and define γ to make all the choices exactly 0 as γ, except for the current choice. Then, γ will do at least as well as γ, but 0 0 is also bounded from above by Prmax L,s (ψ ) for all s and ψ , contradicting the assumption. 0 We note that, although a particular choice may maximize Prmax L,s (ψ ), a scheduler that makes this choice every time is not necessarily optimal. Indeed, no optimal scheduler may exist, in which case we would only have -optimal schedulers for any > 0 [8, 14]. The probabilistic value may be predicated on making a different choice eventually. The max formulation is consistent with c∈N
this possibility, as are the schedulers. Theorem 16 (Model checking termination). The graph construction of Dg(s, ψ) terminates for any XPL formula ψ and PLTS L. Moreover, if ψ is separable, it will always complete the construction. Proof. Cl(ψ) is finite, so AO(Cl(ψ)) is finite. The number of actions in L and ψ is finite, so the number of factored formulae is finite. This is sufficient to guarantee termination, as we fail when we cannot construct a factored formula. Meanwhile, separability implies that we can always construct a factored formula starting from ψ 0 ∈ AO(Cl(ψ)).
12
b, c s3 a:
1 3
s1 a
b, c
s2 a:
2 3
s4 a:
s5
3 4
a:
1 4
s6
Figure 2: An example PLTS From the completed graph, a system of polynomial equations can be extracted. Equations are readily constructed by considering each Prmax L,s (ψ) as a variable. Note that the graph treats µ and ν nodes exactly the same way. In solving the stratified system, we start from 0 to find the least fixed point, and 1 for the greatest fixed point. The alternation-free restriction ensures that there is no cycle in the graph containing both a µ-node and a ν-node. Theorem 17 (Soundness of computation). The construction of graph Dg(s, ψ) leads to the equation system yielding the correct value of Prmax L,s (ψ). Proof. As our system x = P (x) is on [0, 1]n , the operations in Lemma 15 are monotone. Thus, the entire system is monotone, and, starting with x0 = 0 and iterating with xi+1 = P (xi ), x converges to the least fixed point (likewise for x0 = 1 and the greatest fixed point) [8]. Example 18 (Model Checking). For the PLTS L in Figure 2 and fuzzy formula ψ = µX.[a][b]X ∧ [a][c]X, we have symmetric nondeterministic choices on b and c from state s2 , and the formula is satisfied by all finite d-trees (since both s3 and s4 have a probability greater than 12 of returning to s2 , the infinite d-trees have positive measure on any scheduler). Letting ψbc = [b]ψ ∧ [c]ψ, the dependency min 1 1 graph is shown in Figure 3, and Prmax L,s1 (ψ) = 4 (also, PrL,s1 (ψ) = 9 ). max We find PrL,s1 (ψ) as the least fixed point from the following equations: 1 2 xa1 = xbc xb2 = max(xa3 , xa4 ), xa3 = xbc + xbc , xbc 2 , 5 =1 3 5 3 2 1 3 b c xbc xc2 = max(xa3 , xa4 ), xa4 = xbc + xbc , xbc 2 = x2 · x2 , 6 =1 4 6 4 2 Solving the equations, we get xa1 = 41 . Note that the model checking algorithm can be broken into the following two parts: writing down a polynomial system, and then finding the (approximate) solution. The first part is bounded double-exponentially in the size of the fuzzy formula, as we deal with trees of formulae from the Fisher-Ladner closure. For the second part, value iteration is guaranteed to converge [8], but may be exponentially slow in the number of digits of precision [13]. When the polynomial system is of a specific form, alternative approximation methods have been proven to be efficient [7]. 13
(s1 , ψ) (s1 , [a]ψbc ) a (s2 , [b]ψ) b
∧
(s2 , [b]ψ ∧ [c]ψ)
∧ c
b
(s3 , ψ)
(s2 , [c]ψ) c (s4 , ψ)
a
a
(s3 , [a]ψbc )
(s4 , [a]ψbc )
a
a
(s5 , [b]ψ ∧ [c]ψ)
(s6 , [b]ψ ∧ [c]ψ)
(s5 , tt)
(s6 , tt)
Figure 3: Dependency graph Dg(s1 , ψ) for example in 2
5 5.1
Encoding Other Model Checking Problems Encoding PCTL* over MDPs
The syntax of PCTL* is as follows, where A ∈ P rop, ./∈ {}, and φ and ψ represent state formulae and path formulae, respectively: φ ::= tt | A | φ ∧ φ | ¬φ | Pr./p ψ,
ψ ::= φ | Xψ | ψUψ | ψ ∧ ψ | ¬ψ.
This is similar to the syntax given by [1, Chapter 9], except omitting the bounded until operator. To encode PCTL* in XPL, we can use Prmax for the Pr operator (PCTL* assumes a maximizing scheduler), the neg function for the ¬ operator, haiφ for the next operator, Xφ, and the fixed point formula µX.ψ2 ∨ (ψ1 ∧ haiX) for the until operator, ψ1 Uψ2 . Since PCTL* cannot distinguish between action labels, we translate a Markov Decision Processes (MDPs) to a PLTS by retaining the same transition structure, but renaming all labels to “a”. Note that formulas are trivially separable if they have only one action. Consequently, we can model check PCTL* formulae over MDPs using our encoding and model checking algorithm. To the best of our knowledge, there is no encoding of PCTL* in pLµ ⊕ [15], although it can encode PCTL [12, 14]. We conjecture this is impossible, due to the lack of an operator corresponding to the semantics of conjunction for PCTL* path formulae.
14
A
b 1: B r2
n
en
ex 1 ex 2
1/2 1/4
n
B
1
r1 1/4
e1
c
n
ex'1
1 3/5
n 2/5
en'
u
n
e2
n
b1': A
ex'2 2/3
b2': B
1 1/3
1
z
v
1
Figure 4: Example RMDP with Call, Return, and Exit edges added to “A”.
5.2
Encoding of RMDP Termination
A Recursive MDP [10] is specified as a set of components. A component has an entry node and one or more exit nodes. Components may contain boxes, each box having a call port that represents a procedure call and return ports to represent possible return from the called procedure. Figure 4 shows a recursive MDP with two components, A and B. Any call to A non-deterministically results in either a call to B (via box b1 ) or a transition to u. We can translate an RMDP into a PLTS L with Act = {p, n, c, ri , ei }, with states of the PLTS corresponding to nodes of the RMDP. We retain the RMDPs transitions, labeling them as “n” for actions from a non-deterministic choice and “p” for probabilistic choice. To this basic structure we add three new kinds of edges: • “ei ” for the ith exit node of a component, • “c” edges from a call port to the called component’s entry node, and • “ri ” edges from a call port to each return port in the box. While c edges denote control transfer due to a call, r edges summarize returns from the called procedure. Termination of 1-RMDP can be encoded as the following separable formula: ψ1 = µX.he1 itt ∨ hpiX ∨ hniX ∨ (hciX ∧ hr1 iX).
(6)
Note how branching time, on c and r actions, is used to simulate the recursive RMDP call, and that ψ1 is separable. The corresponding formula for a 2-exit RMDP, is not separable: ψ21 =µ he1 itt ∨ hpiψ21 ∨ hniψ21 ∨ (hciψ21 ∧ hr1 iψ21 ) ∨ (hciψ22 ∧ hr2 iψ21 ), ψ22
5.3
=µ he2 itt ∨
hpiψ22
∨
hniψ22
∨
(hciψ21
∧
hr1 iψ22 )
∨
(hciψ22
∧
(7)
hr2 iψ22 ).
PTTL and Branching Processes
Branching Processes (BPs) are a model for which extinction corresponds to termination of 1-exit Recursive Markov Chains, but with the branching time 15
model where the processes evolve concurrently. Probabilistic Tree Temporal Logic (PTTL) has been introduced as a logic similar to PCTL [12], but for BPs [2]. BPs have also been extended with nondeterminism, yielding Branching MDPs (BMDPs), for which the extinction and reachability problems can be solved [8, 10]. We write the syntax of PTTL [2, Definition 18], where A ∈ P rop, ./∈ {}, and we refer to φ and ψ as state and fuzzy formulae, as for XPL: φ ::= tt | A | ¬φ | φ ∧ φ | Pr./p ψ, ψ ::= AXφ | EXφ | φAUφ | φEUφ | φARφ | φERφ. As for RMDPs, we can readily translate BPs and BMDPs to PLTSs. We give the semantics for PTTL over PLTSs, with Pr./p assuming maximizing schedulers, by encoding it in XPL, as EP T T L (γ). γ ∈ P rop ∪ {tt}, γ, 0 neg(E (γ )), γ = ¬γ 0 , PTTL EP T T L (γ1 ) ∧ EP T T L (γ2 ), γ = γ1 ∧ γ2 , EP T T L (γ) = Prmax E (ψ), γ = Pr./p ψ, PTTL ./p [−]EP T T L (φ), γ = AXφ, µX.EP T T L (φ2 ) ∨ (EP T T L (φ1 ) ∧ [−]X), γ = φ1 AUφ2 , νX.EP T T L (φ2 ) ∧ (EP T T L (φ1 ) ∨ [−]X), γ = φ1 ARφ2 . For EX, EU, and ER, we replace the boxes with diamonds. In this sense, PTTL is a natural extension of PCTL over PLTSs for the case |Act| > 1, and all the formulae in the encoding are separable.
6
Conclusion and Future Work
GPL and pLµ represented two divergent paths in probabilistic extensions of µ-calculus. GPL prioritized probabilistic choices, with the idea that external nondeterminism would be sufficient, while pLµ retained internal nondeterminism, but relegated the probabilistic nature essentially to computation. Although closely related, algorithms to check properties of RMCs (and pPDSs [5]) were developed independently [9]. These were related to algorithms for computing properties of systems such as branching process (BP) extinction and the language probability of Stochastic Context Free Grammars, which were also phrased in terms of solving a set of polynomial equations. The relationship between GPL and these systems was mentioned briefly in [11], but has remained largely unexplored. The logic pLµ is able to support branching time and an intuitive game semantics [14]. The scope of pLµ includes infinite-state systems as well. However, as we showed in Section 5, XPL can be used to encode properties that we are unable to encode in pLµ. Determining the relationship between XPL and pLµ in branching time is an important problem.
16
There has been significant interest in the study of expressive systems with nondeterministic choices, such as RMDPs and Branching MDP (BMDPs) [10]. At the same time, the understanding of the polynomial systems has expanded. In [6], the class of Probabilistic Polynomial Systems (PPS) is introduced, which characterizes when efficient solutions to polynomial equation systems are possible even in the worst case [7]. While [9] did not distinguish the systems arising from single-exit RMCs (1-RMCs) with those from multi-exit RMCs, the PPS class is limited to 1-RMCs. It was also extended for RMDP termination, and later BMDP reachability, both having polynomial-time complexity for min/maxPPSs [6, 8]. Systems producing equations in PPS form show an interesting characteristic: that the properties are expressible as purely conjunctive or purely disjunctive formulae. Recall that such formulae are trivially separable. Characterizing equation systems that arise from separable formulae and investigating their efficient solution is an interesting open problem.
References [1] Christel Baier. On algorithmic verification methods for probabilistic systems. Habilitation thesis, Fakult¨at f¨ ur Mathematik & Informatik, Universit¨at Mannheim, 1998. [2] Taolue Chen, Klaus Dr¨ager, and Stefan Kiefer. Model checking stochastic branching processes. In MFCS, pages 271–282, Berlin, Heidelberg, 2012. Springer. [3] Rance Cleaveland, S. Purushothaman Iyer, and Murali Narasimha. Probabilistic temporal logics via the modal mu-calculus. Theoretical Computer Science, 342(2-3):316–350, 2005. [4] Jose Desharnais, Vineet Gupta, Radha Jagadeesan, and Prakash Panangaden. Weak bisimulation is sound and complete for PCTL*. In CONCUR, volume 2421 of LNCS, pages 355–370. Springer Berlin Heidelberg, 2002. [5] Javier Esparza, Anton´ın Kucera, and Richard Mayr. Model checking probabilistic pushdown automata. In LICS, pages 12–21, 2004. [6] Kousha Etessami, Alistair Stewart, and Mihalis Yannakakis. Polynomial time algorithms for branching Markov decision processes and probabilistic min(max) polynomial Bellman equations. In ICALP, Part I, pages 314–326, Berlin, Heidelberg, 2012. Springer. [7] Kousha Etessami, Alistair Stewart, and Mihalis Yannakakis. Polynomial time algorithms for multi-type branching processes and stochastic contextfree grammars. In STOC, pages 579–588. ACM, 2012. [8] Kousha Etessami, Alistair Stewart, and Mihalis Yannakakis. Greatest fixed points of probabilistic min/max polynomial equations, and reachability for
17
branching Markov decision processes. In ICALP, Part II, pages 184–196, Berlin, Heidelberg, 2015. Springer. [9] Kousha Etessami and Mihalis Yannakakis. Recursive Markov chains, stochastic grammars, and monotone systems of nonlinear equations. J. ACM, 56(1):1:1–1:66, February 2009. [10] Kousha Etessami and Mihalis Yannakakis. Recursive Markov decision processes and recursive stochastic games. J. ACM, 62(2):11:1–11:69, May 2015. [11] Andrey Gorlin, C. R. Ramakrishnan, and Scott A. Smolka. Model checking with probabilistic tabled logic programming. TPLP, 12(4-5):681–700, 2012. [12] Hans Hansson and Bengt Jonsson. A logic for reasoning about time and reliability. Formal Aspects of Computing, 6(5):512–535, 1994. [13] Stefan Kiefer, Michael Luttenberger, and Javier Esparza. On the convergence of Newton’s method for monotone systems of polynomial equations. In STOC, pages 217–226, 2007. [14] Matteo Mio. Probabilistic modal mu-calculus with independent product. In FOSSACS, pages 290–304, Berlin, Heidelberg, 2011. Springer. [15] Matteo Mio. Game semantics for probabilistic modal mu-calculi. PhD thesis, The University of Edinburgh, 2012. [16] Roberto Segala. A compositional trace-based semantics for probabilistic automata. In CONCUR, volume 962 of LNCS, pages 234–248. Springer, 1995. [17] Roberto Segala and Andrea Turrini. Comparative analysis of bisimulation relations on alternating and non-alternating probabilistic models. In QEST, pages 44–53. IEEE Computer Society, 2005.
18
