Hidden Protocols

Embed Size (px)

Citation preview

  • 8/13/2019 Hidden Protocols

    1/10

    Hidden Protocols

    Hans van DitmarschDept. of Logic

    University of SevillaCalle Camilo Jos Cela s/n41018 Sevilla, Spain

    [email protected]

    Sujata GhoshDept. of Artificial Intelligence

    University of GroningenP.O. Box 4079700 AK Groningen,The

    [email protected]

    Rineke VerbruggeDept. of Artificial Intelligence

    University of GroningenP.O. Box 4079700 AK Groningen,The

    [email protected]

    Yanjing WangDept. of Philosophy

    Peking University100871, Beijing, China

    [email protected]

    ABSTRACT

    When agents know a protocol, this leads them to have ex-

    pectations about future observations. Agents can updatetheir knowledge by matching their actual observations withthe expected ones. They eliminate states where they do notmatch. In this paper, we study how agents perceive proto-cols that are not commonly known, and propose a logic toreason about knowledge in such scenarios.

    Categories and Subject Descriptors

    F.4.1 [Mathematical Logic]: Modal Logic; D.4.7 [Organi-zation and Design]: Distributed Systems; I.2.3 [ArtificialIntelligence]: Deduction and Theorem Proving

    General Terms

    Theory

    Keywords

    communication protocols, dynamic epistemic logic, informa-tion hiding

    1. INTRODUCTIONTalking about knowledge and protocols, some questions

    that come foremost to our mind concern the following is-sues. What do we mean by knowing a protocol? How doesthis protocol knowledge affect our knowledge of facts aboutthe world? The existing literature abounds with various

    formal models answering these questions from different an-gles [8, 15, 19, 27, 11]. In some situations, agents have par-tial knowledge of the underlying protocols that guide the

    ACM COPYRIGHT NOTICE. Permission to make digital or hardcopies of all or part of this work for personal or classroom use isgranted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this noticeand the full citation on the first page. To copy otherwise, to re-publish, to post on servers or to redistribute to lists, requires priorspecific permission and/or a fee. Request permissions from PublicationsDept., ACM, Inc., fax +1 (212) 869-0481, or [email protected] 2011, July 12-14, 2011, Groningen, The Netherlands.Copyright c2011 ACM. ISBN 978-1-4503-0707-9, $10.00.

    behaviors of other agents. Based on their incomplete knowl-edge of protocols and their observations, the agents try to

    reason about their epistemic attitudes as well as hard facts.These protocols may occur when agents communicate us-ing full-blown secret codes (see [18] for many intriguing his-torical examples). Our daily communications provide moremundane protocols.

    Consider, as Example 1, a cafe in the 1950s, with threepersons, Kate, Jane and Ann sitting across a table. SupposeKate is gay and wants to know whether any of the othertwo is gay. She wants to convey the right information tothe right person, without the other getting any idea of theinformation that is being communicated. She states, I ammusical, I like Kathleen Ferriers voice. Jane, who is gayherself, immediately realizes that Kate is gay, whereas, forAnn, the statement just conveys a particular taste in music.1

    Coming back to the present day, consider as Example2 a similar cafe scenario with Carl, Ben and Alice. Carland Ben are childhood friends and know each other like theback of their hands. Carl to Ben: On Valentines day Iwent to the pub with Mike and Sara. It was a crazy night!.This immediately catches the attention of Alice who is inlove with Mike. She asks: What happened? Carl winksto Ben and says: Nothing. Knowing Carl very well Benimmediately realizes that nothing has happened, whereasAlice becomes unsure of that, as she saw the wink that Carlhas given to Ben.

    This paper presents a dynamic epistemic logic (DEL, [1,21]) that can suitably describe such scenarios. Knowing aprotocol can mean knowing what to do according to theprotocol [8]. It can also correspond to understanding theunderlying meaning of the actions induced by the proto-col [15]. Here, we follow the latter interpretation, becauseit aptly captures the notion of a protocol in situations weare modeling. Kates making a statement like I am musical,I like Kathleen Ferriers voice corresponds to the fact that

    1This example has been inspired by the interviews in [25],from which it appears that in 1950s Amsterdam, musicalwas indeed a code term for gay, known almost exclusivelyby gay people. The additional mention of singer KathleenFerrier strengthened this gay hint. Among gay women,Ferriers low contralto voice, for example in her performanceas Orfeo in Glucks Orfeo ed Euridice, was widely popular.

  • 8/13/2019 Hidden Protocols

    2/10

    Kate is gay. In the second situation, Nothing (even if ac-companied by a wink) corresponds to the fact that Nothinghas happened.

    Our work is largely inspired by two lines of research: thework relating DEL and ETL [19, 11, 14] and the work onprotocol changes [27, 28]. In [14] protocols are modeled astree compositions, basically equating protocols with plans.In [19, 11], the notion of state-dependent DEL-protocols

    (sets of sequences ofevent models[21]) is proposed in orderto handle protocols that are not common knowledge. Forexample, the model

    s: p(a) 1,2 t: p(b)

    represents an epistemic scenario where the agents are notonly uncertain about the factual state of the world but alsoabout the protocol that can be executed given some factualstate; this is denoted by a state-dependent protocol assign-ing singleton action sets{a} to s and {b}to t, where a andb are actions. A system wherein the protocol can be dif-ferent in any state is clearly more complex than a systemwherein the protocol is a background parameter, and thuscan be assumed common knowledge to all agents. But in our

    example we can still reclaim some form of common knowl-edge of the protocol, namely by describing it as follows: if pthena and if p then b. It seems that in order to discussthe knowledge of protocols formally we need to first fix aprotocol specification language.

    Given a protocol language, how do we obtain such epis-temic models with protocol information from specificationsof conditional protocols, and vice versa? Similar questionsare addressed in [27, 28], presenting a logical frameworkthat incorporates protocol specifications on epistemic mod-els. However, there, protocols are assumed to be commonknowledge. We do not assume that here. Our work is basedon the logic developed in [27] but it uses epistemic modelswith procedural information as in [19, 11] to deal with un-

    certainties of protocols, an agents knowledge of underlyingprotocols, and her current observations affecting factual un-certainty. In our framework, the protocols can be viewed asgiven by nature, so the framework does not cover interest-ing aspects such as how and by whom the protocols havebeen designed and how agents have come to agree to usethem.

    The ingredients of our work are: 1. epistemic modelsencoding state-dependent expected observations; 2. an up-date mechanism for eliminating impossible worlds accordingto the observation of agents and their expectations; 3. a for-mal language for specifying observations and protocols; 4.protocol models that represent agents incomplete informa-tion about the real protocols. 5. an update mechanism forincorporating protocol information (as protocol models) onepistemic (observation) models; 6. a notion of equivalencebetween protocol models; 7. a logic for reasoning aboutknowledge based on protocols.

    The paper is organized as follows. Section 2 introducesthe epistemic observation models and a simple PDL-styleepistemic logic for reasoning about knowledge via matchingthe expectations and observations. Section 3 discusses howwe obtain observation models from protocol models (i.e.,epistemic protocols). We characterize three classes of ob-servation models that can be generated from various epis-temic models. Furthermore we give a characterization ofthe effective-equivalence of epistemic protocols. A logic is

    then given to incorporate the updates of protocols and rea-soning about knowledge and observations. In the end weaddress incorporation of factual change actions in Section 4and point out future work in Section 5.

    2. REASONING VIA EXPECTATION AND

    OBSERVATIONIn this section, we introduce observation models, which

    are Kripke models with expected observations, and proposea dynamic logic style epistemic logic interpreted on suchmodels for reasoning about knowledge via matching obser-vations with expectations.

    2.1 Epistemic Observation ModelsLet I be a finite set of agents, and let P be a finite set

    of propositions describing the facts about the world. LetBool(P) denote the set of all Boolean formulas over P. Toset up the semantics we first define a Kripke model in theusual sense, which models agents epistemic uncertaintiesregarding the actual state of the world.

    Definition 1 (Epistemic model). Anepistemic mod-

    el Me is a triple S, , V: S is a non-empty domain ofstates, stands for the set of accessibility (equivalence) re-lations{i| i I}, V : S P(P) is avaluation assigningto each state a set of propositional variables (those that aretrue in that state).

    We will introduce the concept of epistemic observationmodels based on Kripke models, which captures the ex-pected observations of agents. Agents observe what is hap-pening around them and reason based on these observations.One way of expressing such observations is by means of ac-tions, viz. the action of making statements, going to theright, nodding your head and many others. To this end,we introduce a finite set of actions, viz. . An observa-tion is a finite string of actions, e.g., abcd. Note that anagent may expect different (even infinitely many) potentialobservations to happen at a given a state, e.g. she expectsa . . . a b to happen for any finite sequence ofa preceding theterminating action b. As human beings and computers areessentially finite, we need to denote and talk about such ex-pectations in a finitary way. To this end, we introduce theobservation expressions (as regular expressions over):

    Definition 2 (Observation expressions). The lan-guageLobs ofobservations is given by

    ::= | | a | | + |

    wherea , and andare constants for the empty stringand the empty language respectively.

    The semantics for the observation expressions are givenby sets of observations (strings over ), similar to that forthe regular expressions.

    Definition 3 (Observations). Given an observationexpression , the corresponding set of observations, denotedbyL(), is the set of finite strings over defined as follows.

    L() = {} L() = L(a) = {a}L( ) = {wv | w L() andv L()}L(+) = L() L()L() = {}

    n>0(L(

    n

    ))

  • 8/13/2019 Hidden Protocols

    3/10

    Definition 4 (Epistemic observation model). Anepistemic observation modelMobs is a quadruple

    S, , V, Obs,

    whereS, , V is an epistemic model (the epistemic skele-ton ofMobs) and Obs: S Lobs is an observation functionassigning to each state an observation expression such thatL() = (non-empty set of finite sequences of observations).

    Anepistemic observation stateis a pointed epistemic obser-vation model. Intuitively, Obs assigns to each state a set ofpotential or expected observations.

    Given an epistemic observation model Mobs = S, , V,Obs, note thatS, , Vis an epistemic model in the usualsense. Hence, sometimes, we also denote an observationmodel as (Me, Obs), where Me is the corresponding epis-temic model. An epistemic model Me can be considered asan epistemic observation model Mobs where for all s S,Obs(s) = (shorthand for (a0 + a1 + + ak)

    where{a0, . . . , ak} = ), that is, in an epistemic model the ob-servations possible at each state are not specified; one canobserve anything. In this sense,Me lacks in providing cer-tain information about the world, and Mobsfills up that gap.In what follows we often leave out the subscripts, wheneverthe respective models are clear from the context.

    Example 5 (Dutch or not Dutch). In the Nether-lands, people often greet each other by kissing three times onthe cheek (left-right-left) while in the rest of Europe peopleusually kiss each other only once or twice. We can reasonwhether a person is Dutch-related by observing his behavior.LetpD be the proposition meaning someone is Dutch-related;a and b are two actions denoting kissing the left cheek andkissing the right cheek, respectively. The following model iswhat we expect (reflexive arrows are omitted):

    s: pD(a b a) 1 t: pD(a b)

    The indistinguishability relation above depicts that agent 1does not know whetherpD. The associated observations arethose that the agents might expect in each state. Intuitivelyif we observe someone kissing three times (observation ab a), then we can infer that he or she is Dutch-related.In the next section a simple logic is defined to handle suchreasoning based on actual observations.

    2.2 Public Observation LogicIn this subsection we define a simple dynamic logic with

    knowledge operators to reason about knowledge via the mat-ching of observations and expectations. The idea is similar

    to the one behind public announcement logic where peopleupdate their information by deleting the impossible scenar-ios according to what is publicly announced. Here we relaxthe link between the meaning and public actions (like anannouncement), and assume that when observing an action,people delete some impossible scenarios where they wouldntexpect such an observation to happen.To make such reason-ing formal, we first define the update of observation modelsaccording to some observation w. The idea behind M|w isthat we delete the states where the observed execution couldnot have been performed.

    Definition 6 (Update by observation). Letw be anobservation over , letM = (S, , V, Obs) be an observa-tion model. The updated model M|w = (S

    , , V, Obs).Here, S = {s | L(Obs(s)\w) = }, i = i|SIS , V

    =V|S , and Obs

    (s) =Obs(s)\w, where\w is defined as theregular expression denoting the language{v| wv L()}.

    \w is a regular language [5] and can be axiomatized withthe output function o from the set of regular expressionsover to {, }as follows (cf. [3, 5]):

    \a0. . . an = \a0\a1. . . \an= o() +

    a(a \a)

    \a= \a= b\a= (a =b)a\a= ( )\a= (\a) +o() (\a)(+)\a= \a+\a\a= \a

    o( ) = o() o()o() = o() =o() =o(a) =

    o(+

    ) =o() +o(

    )These are used for the computation of observations in a

    syntactic way.

    We design a logic to reason about the observations, Publicobservation logic (POL):

    Definition 7 (public observation logic). The for-mulas of POL are given by:

    ::= |p | | | Ki | []

    wherep P, i I, and Lobs.

    Definition 8 (Truth definition for POL). Given

    an epistemic observation modelM =(S, , V, Obs), a states S, and a POL-formula , the truth of at s, denotedbyM, s , is defined as follows:

    M, s p p V(s)M, s M, s

    M, s M, s andM, s M, s Ki for all t: s i t = M, s M, s [] for eachw L() : (w init(Obs(s))

    impliesM|w, s )

    where w init() iff v such that wv L() iffL(\w) = .

    Consider the model M in Example 5. If we observe one ortwo kisses, we still cannot tell whether the person is Dutch-related, but if there is one more kiss to follow then we know.Formally, it can be verified that M, s [a b](KpD [a]KpD). More complicated can be used to express (infi-nite) sets of observations, e.g., [ a ]Kisays as longas a is observed at some point,i knows .

    Clearly, standard bisimulation between epistemic modelsis not an invariance of the above logic: POL can reasonabout what may happen at each state. We now define bisim-ulation between observation models, which facilitates char-acterization results in later sections.

  • 8/13/2019 Hidden Protocols

    4/10

    Definition 9 (Observation Bisimulation). A binaryrelation R between the domains of two observation modelsM = (S, , V, Obs) and N = (S, , V , Obs) is called abisimulation if for any s S, s S: (s, s) R impliesthat the following conditions hold:

    Propositional Invariance V(s) = V(s)

    Observational Invariance L(Obs(s)) = L(Obs(s)).

    Zig if s i t in M then there exists a t in N such that

    s i t and tRt.

    Zag if s i t in N then there exists a t inM such that

    s i t and tRt.

    M and N are said to be bisimilar (M o N) if there isa bisimilationR between them. (M, s) and (N, s) are saidto be bisimilar (M, so N, s

    ) if there is a bisimulationRbetween them such that (s, s) R.

    Note that the standard bisimulation (notation) is definedas o without the condition for the invariance for observa-tions. It is not hard to show the following.

    Proposition 10 (Bisimulation Invariance). Forany two finite epistemic observation statesM, s andN, s,the following statements are equivalent:

    M, soN, s

    For any formula POL: M, s N, s

    Proof. [o = POL]: by induction on : Booleanand Ki cases are trivial. Now consider = [] : supposeM, s o N, s

    but M, s [] and N, s []. Thenthere exists a w L() such that w init(Obs(s)) andN |w, s

    .

    By the definition of o, L(Obs(s)) = L(Obs(s)) there-

    fore w init(Obs(s)). Thus M|w, s exists. We now showthat M|w, s o N |w, s. Let R be {(t, t) SM|w SN |w |M, to N, t

    }. Clearly (s, s) R. Note that ifL(Obs(t)) =L(Obs(t)) then L(Obs(t)\w) = L(Obs(t)\w); this provesthe invariance for observations. Based on this invariance, itis not hard to verify that R is indeed an observation bisim-ulation between M|w and N |w.

    SinceM|w, so N |w, s, by induction hypothesis

    M|w, s .

    Clearly, this contradicts the assumption thatM, s [].

    [POL =o]: Let R = {(t, t) SM SN | M, tPOL

    N, t}. We can show that R is an observation bisimula-

    tion. All the conditions are standard and thus can be han-dled by standard techniques except the new clause aboutthe invariance for observations: we need to show tRt =L(Obs(t)) =L(Obs(t)); however, this is trivial since in thelanguage of POL we can express w such that M, t w w L(Obs(t)).

    Intuitively, these observation models can be seen as compactrepresentations of certain epistemic temporal models [15,19]. To make the link more precise, we can relate POLon observation models to the same language on epistemictemporal models with the usual PDL-style interpretation of[] formulas.

    Definition 11. LetM be an epistemic observation modelS, i, V, Obs. TheM-generated epistemic temporal model

    is defined asE T L(M) = H, a, i, V

    where: H= {(s, w) |

    s S, w = orw L(Obs(s))}; (s, w) a (t, v) s =

    t andv = wa, a ; (s, w) i (t, v) s i t andw =v; p V(s, w) p V(s).

    The formula [] is true at a pointed epistemic temporal

    model N, h iff for any w L(), h w h implies N, h

    . The truth definitions for observation-free formulas areas usual. We call this logic EPDL (Epistemic-PDL). Toestablish the precise link between observation models andepistemic temporal models, we can prove the following.

    Proposition 12. Given a pointed POL modelM, s, anda POL formula, it can be shown that:M, s ET L(M), (s, ) EPDL .

    Proof. We need to show for any observation model M, sand any POL formula :

    M, s ET L(M), (s, ) EPDL

    We prove this by induction on .The Boolean case and the Ki case are trivial. Now

    consider the case []. Suppose without loss of general-ity that there is an observation model M, s [] andET L(M), (s, ) []. Then there exists a w L() suchthat ET L(M), (s, w) . By the definition of ET L(M),w Obs(s) thus M|w exists. Based on the definition ofET L(M), it is not hard to show that ET L(M|w), (s, ) isbisimilar (w.r.t. both and ) to ET L(M), (s, w). SinceEPDL is clearly invariant under bisimulation,

    ET L(M|w), (s, ) .

    By induction hypothesis,Mw, s which contradicts theassumption thatM, s [].

    Note that the generated epistemic temporal models can beinfinite, and thus the above result does not give a straightfor-ward model checking procedure for POL. According to thesemantics of []we need to check infinitely manyw L().Fortunately, this can be handled by partitioning L() intoa finite number of regular expressions0. . . k such that forany 0 i k, any w, v L(i), M|w = M|v, providingdecidability of model checking after all (see [28] for detailsin a similar setting).

    3. EXPECTATION COMES FROM PROTO-

    COLSObservation models describe the agents expected obser-

    vations, which in turn influence their reasoning. We inves-tigate how agents acquire and change their expectations, by

    looking at protocols and protocol models as sources for theexpected observations.

    3.1 Protocol modelsA protocol is a rule telling us what we should do under

    what conditions. We can specify protocols in the followinglanguage of protocol expressionsLprot:

    Definition 13 (Protocol expression). The languageLprot of protocols is given by

    ::= | | a | ? | | + |

    where Bool(P).

  • 8/13/2019 Hidden Protocols

    5/10

    The above language of protocol expressions is obtained byadding Boolean tests to observation expressions. For exam-ple, (?love stay) (?love separate) expresses we shouldstay together as long as we are in love. For a discussionon more complicated test scenarios (e.g. considering agentsknowledge) see Section 5.

    In the story of Example 5, there seems to be an underlyingprotocol: ifyou are Dutch thenyou kiss three times andif

    you are non-Dutch thenyou kiss two times. It is the reasonfor the agent to have the corresponding expectations of theobservations. This protocol (call it K) can be expressed as?pD a b a+?pD a b. We would like to generate theobservation model in Example 5 from the protocol K andthe epistemic model

    pD1pD

    Intuitively, the information of the protocol K can be incor-porated by adding to each state the possible observationsallowed by the protocol. We now move on to the technicaldetails.

    To compute the expected observations corresponding toa given protocol we first define the semantics of protocol

    expressions. Intuitively, we associate to each protocol asetLg() ofconditional observationsin the form of

    0a01a1. . . kak

    where eachi P denotes a state of affairs (the basic propo-sitionsp are true while the others are false), encoding theconditions for the later observations to happen. For Booleanformulas, we write if is true under (viewed as avaluation).

    Definition 14. The set of conditional observations cor-responding to is defined as follows:

    Lg() = , Lg() = { | P},Lg(a) = {a | P}, Lg(?) = { | },Lg(1 2) = {w v| w Lg(1), v Lg(2)},Lg(1+ 2) = Lg(1) Lg(2),Lg(

    ) = { | P} n>0(Lg(

    n)),

    where is thefusion product: w v= wv whenw= wandv = v , and not defined otherwise.

    Note that theis in a conditional observation remain un-changed since no factual changeis introduced by the execu-tion of the actions (see Section 4 for a detailed discussionof fact changing actions). In the following we show howto derive the set of observations to be expected under thesame condition according to , by defining the conversionfunction f : Lprot Lobs,

    f() = f() = f(a) = a f(?) = if |=f(

    ) = f() f() if |=

    f(+) = f() +f(

    ) f() = f()

    Proposition 15. For any Lprot,L(f()) = {w| w= a0. . . ak, whereai {} anda0a1. . . ak Lg()}.

    Therefore, every has a normal form

    =

    P

    (? f())

    such thatLg() =Lg(), where is a characteristic for-

    mulas for P (e.g. {p} = p q if P= {p, q}).

    Proof. We first show that L(f()) is equal to

    {w| w= a0. . . ak, ai {}and a0a1. . . ak Lg()}

    . We prove this by induction on LProt.The atomic cases are straightforward. Now we check the

    complex cases:

    = 1+ 2:

    L(f()) = L(f(1+2)) = L(f(1) +f(2))= L(f(1)) L(f(2))= {w| w= a0. . . ak, and a0 . . . ak Lg(1)}

    {w| w= a0. . . ak, and a0. . . ak Lg(2)}(by IH)= {w| w= a0. . . ak, and a0. . . ak Lg(1+2)}

    = 1 2:

    L(f()) = L(f(1 2)) = L(f(1) f(2))= {wv | w L(f(1) and v L(f(2)}= {wv | w= c0. . . cm st. c0. . . cm Lg(1)

    and v = b0. . . bn st. b0. . . bn Lg(2)}(by IH)= {u | u= a0. . . ak, and a0. . . ak Lg(1 2)}

    (by fusion product) = 1 :

    L(f()) = L(f(1)) = L((f(1))

    )= {}

    n>0 L((f(1))

    n)= {u | u= a0. . . ak, and a0. . . ak { | P}

    n>0 Lg(

    n1 )}(by IH)

    = {u | u= a0. . . ak, and a0. . . ak Lg(1)}

    This completes the proof for the following statement :For all in Lprot, for all P, L(f()) = {w | w =a0. . . ak, where ai {}and a0a1. . . ak Lg()}.

    From the result above and the definition ofLg, it follows,

    Lg(f()) = {a0. . . ak

    | P and a0. . . ak

    Lg()}.

    Let G = {a0a1. . . ak | a0a1. . . ak Lg()}, theset of all -guarded expressions in Lg(). Then, by fusionproduct, it follows thatLg(? f()) =G

    . Thus,

    Lg() = Lg(

    P(?f())) =

    P Lg(?f()) =

    P G = Lg().

    From Proposition 15, according to the protocol , the ex-pected observations on a state s in an epistemic model Mcan be computed byfVM(s)().For example,f{p}(?pa+?pb) = a. However, not every observation model can be gen-erated by a single protocol.

    Example 16. Consider the observation model:

    s: p(a) 1,2 t: p(b)

    We cannot associate a protocol to its epistemic skeletonsuch that fVM(s)() = a and fVM(t)() = b, since VM(s) =VM(t). Note that taking?p(a+b) for does not work.

    In this example agents are uncertain about the protocol.Their uncertainty may be seen as follows:

    ?p a 1,2 ?p b

    The following definition formalizes this idea.

  • 8/13/2019 Hidden Protocols

    6/10

    Definition 17 (Epistemic Protocol model). Anepistemic protocol model A is a triple T, , Prot whereT is a domain of abstract objects, stands for a set ofaccessibility (equivalence) relations {i| i I}, and Prot :T Lprotassigns to each domain object a protocol. We calla pointed epistemic protocol model anepistemic protocolanda singleton epistemic protocol model a public protocol.

    Note that public protocols are (implicitly) commonly knownby all the agents.We will now proceed towards our main result in this sec-

    tion, namely that an epistemic observation state uniquelydetermines an epistemic protocol, and that an epistemic pro-tocol and an epistemic state uniquely determine an epistemicobservation state. To show the correspondence, we need onemore semantic operation, that is amodal product operationof an epistemic observation model and a protocol model. Itformalizes the change in possible observations induced by aprotocol. We should see this definition as installing a newprotocol, by means ofnovelobservations, into the epistemicobservation model, and thus completely obliterating thecur-rentexpected observations.

    Definition 18 (Protocol Update). Given an epis-temic observation modelMobs = S, , V, Obs and an epis-temic protocol modelA= T, , Prot. We define the prod-uct (Mobs A) = (S

    , , V , Obs) as follows:

    S = {(s, t) S T : L(fVM(s)(Prot(t))) = };

    (s, t) i (s, t) iffs i s

    inMobs and t i t inA;

    V(s, t) =V(s);

    Obs((s, t)) =fVM(s)(Prot(t)).

    We mentioned that epistemic models can be seen as special

    cases of epistemic observation models, namely with the any-thing goes protocol. Therefore, also in that case the productoperation between an epistemic model and a protocol modelcorresponds to the installation of a protocol.

    We now illustrate the definition by the two example sce-narios of the introduction. In the pictures, assume reflex-ivity and transitivity of access. In the first scenario, at thebeginning neither of Jane or Ann knows the fact g (Kate isgay). However, one of them, Jane, is aware of the protocolthat: if Kate is gay then she will make the statement Iam musical, I like Kathleen Ferriers voice (action a); andif she is not gay, then she will talk about something else(action b). However, Ann has no idea whethera and b cancarry such information. The scenario is modeled as follows,

    where the last model is the observation model resulting fromthe update of the protocol on the first epistemic model (realstates are underlined):

    g()

    Jane,Ann

    ?g a+?g b

    Ann=

    g(a)

    Ann

    Jane,Ann g(b)

    Ann

    g() a+b g(a+b) Jane,Anng(a+b)

    where g denotes the fact that Kate is gay, a denotes theobservation of Kate making the musical statement and bstands for Kate saying something else.

    We now consider the second example. After Carls firstdescription of the night of the Valentines day, Ben and Alicestill do not know what has happened. This prompts Alicesquestion. Now, the wink from Carl creates an uncertaintyin Alice regarding how to interpret Bens statements, whileknowing Carl so well, Ben immediately gets the idea of theprotocol Carl is using. The modeling is as follows:

    p(

    )

    Ben,Alice

    ?p Y+?p N

    Alice=

    p(Y)

    Alice

    Ben,Alice

    p(N)

    Alice

    p() ?p Y+?p N p(Y)Ben,Alicep(N)

    where p denotes the fact that Something has happened in-volving Mike and Sara on V-day night, Y corresponds toCarls saying something in the affirmative to Alices ques-tion, and N the opposite.

    According to our definition, an epistemic protocol modelacts on an epistemic model determining a unique observationmodel. In the rest of this section we will investigate theconverse: whether an arbitrary observation model can begenerated by updating an epistemic model by an epistemicprotocol model.

    Proposition 19. Given an epistemic observation modelM = (N, Obs), there is an epistemic modelN and a pro-tocol modelA such thatMoN

    A.

    Proof. Let N = (S, , V) be the universal ignorancemodel, i.e., S = P(P), for each i, i=S

    S, and V() = P. Given M = (S, , V, Obs), let A = (S, , Prot) suchthatProt(s) =?V(s) Obs(s). where V(s) is the character-istic formula of V(s) P (e.g. p q is a characteristicformula for{p}ifP = {p, q}). Now we showMo N

    Aby proving thatR = {(s, (, s)) | V(s) = } is a bisimulationrelation.

    The invariance conditions are immediate. Now supposes i t in M then (, s) i (V(t), t) in N A by the defini-

    tion of the product. Obviously, tR(, t), where =V(t).Suppose (, s) i (

    , t). ThenV(t) =. Therefores i tand tR(, t).

    This result shows that every observation model is reason-able in the sense that it can be generated from an epistemicmodel by some epistemic protocol model. Note that in theabove proposition, we consider an arbitrary epistemic model.However, it is more intuitive to consider the particular epis-temic model N in M = (N, Obs), and ask if there is aprotocol model A such that N A o M. For singletonprotocol models, we have a characterization result.

    Definition 20. An observation model M is said to beBoolean normal if for any two worlds s, t in it, VM(s) =VM(t) = L(Obs(s)) = L(Obs(t)).

    Theorem 21. Given an epistemic observation modelM=(N, Obs),M is Boolean normal iff there exists a singletonprotocol modelA such thatN AoM, o being a totalbisimulation.

    Proof. : Let s be the Boolean characterization for-mula corresponding toVN(s). LetM =

    s in N?sObs(s).

    Because of the finiteness of P and Boolean normality, Mhas a finite representation. Let AM be the singleton pointed

  • 8/13/2019 Hidden Protocols

    7/10

    protocol model with Protassigning M to the single point.We can verify that N AM oM.

    : suppose M is not Boolean normal then there ares, tinM such that V(s) = V(t) and Obs(s) =Obs(t). Due to thenormal form of protocols, updating a single pointed protocolon s, t will result in the same observations. So there cannotbe any single pointed protocol model to do the job.

    Clearly, not every epistemic observation model is Booleannormal, thus not every observation model can be generatedfrom a public protocol.

    Example 22. Consider the following epistemic observa-tion modelM, we will show thatMcannot be generated byany epistemic protocol on its epistemic skeleton:

    p(b) 1 p(a) 2 p(b)

    Suppose towards contradiction that there is a protocol modelA such that the execution ofA on the epistemic skeleton ofM gives an observation model which is bisimilar to M. Tocompose the middle world in the observation model we needa state t in the protocol model such that Prot(t) allowsa tohappen ifp is true. Thent can be composed with the leftmost

    p world above as well, since the left world and middle worldare Boolean indistinguishable. Therefore there will be ap(a)-world in the resulting model which cannot reach any pworldin one step, due to the definition of (the leftmost stateabove cannot reach anyp world in one step).

    This leads us to consider a subclass of the observationmodels given as follows.

    Definition 23 (Boolean distinguishing). An epist-emic (observation) model M is said to be Boolean distin-guishing if for each state s M, there exists a Booleandistinguishing formula fors, that is, there is a Boolean for-mula which is only true ats and the states inM, related by (o) to s.

    Theorem 24. Given an epistemic observation model M =(N, Obs), ifN is Boolean-distinguishing then there is a pro-tocol modelA such thatN AoM.

    Proof. SupposeN = (W, , V) and let Ns be the Bool-ean distinguishing formula corresponding to s W. LetA = (W, , Prot) whereProt(s) =?Ns Obs(s).We will showthatN AoM.

    Let R W WNA be the binary relation defined bysetting w R(v, t) iffM, wo M, t.We need to show that Ris indeed an observation bisimulation.

    Now suppose wR(v, t). Since Prot(t) =?Nt Obs(t) and(v, t) is in N A then N, v Nt . Since

    Nt is a Boolean-

    distinguishing formula for the worldt, we have that N, vN, t, since a state non-bisimilar to t will not satisfy Nt .Due to the fact that M, w o M, t we have N, w N, t,thus N, w N, v. Therefore the propositional invariancecondition of observation bisimulation holds. Since M, woM, t, Obs(w) =Obs(t) =Obs((v, t)).

    From the fact that M, w o M, t and N, w e N, vthe conditions Zig and Zag of Definition 9 can be verifiedeasily.

    3.2 Equivalence of protocolsWe motivated in the introduction that one observation

    model might be generated in different ways (even based on

    the same epistemic model). For example, consider the fol-lowing model:

    p(b) 1,2 p(a)

    It can be generated from its epistemic skeleton by updatinga public protocol ?p b+?p a or the epistemic protocolmodel:

    ?p b 1,2 ?p aActually, the announcement of ?p b+?p a will alwaysyield the same result as the above epistemic protocol modelon arbitrary epistemic models. On the other hand, the an-nouncement ?p (a+b) has a different update result on thesame epistemic model compared to the update of the follow-ing epistemic protocol:

    ?p a 1,2 ?p b

    Such examples lead us to the following notion of equivalencebetween protocol models.

    Definition 25 (Effective equivalence). Two proto-col modelsA andBare said to be effective-equivalent (nota-tion: A efB) if for any observation modelM : MA oM B.

    Inspired by the idea of action emulation in [24], we char-acterize the notion of effective-equivalence by the followingstructural equivalence. LetL() be L(f()) (cf. Proposi-tion 15).

    Definition 26 (Protocol emulation). Two protocolmodels A = (S, Prot) and B = (T, Prot) are said to beemulated (notation: A B) if there is a binary relationE S T such that wheneversEt we have:

    there exists Psuch thatL(Prot(t)) = L(Prot(s)).

    ifs i s inA then there is a setT T such that:

    1. for any t T: t i t;

    2. for any t T: sEt;

    3. for any P such that L(Prot(s)) = thereexistst T such thatL(Prot(s)) = L(Prot(t))

    if t i t inB then there is a setS Ssuch that:

    1. for anys S: s i s;

    2. for anys S: sEt;

    3. for any Psuch thatL(Prot(t)) = there ex-istss S such thatL(Prot(s)) = L(Prot(t))

    When restricted to public protocols, it is not hard to seethat Lg() = Lg(

    ). In general, we have thefollowing result.

    Theorem 27. For any finite protocol models A and B:A ef B A B .

    Proof. : SupposeA B, we need to show for any ob-servation modelM : M AoM B. We define a binaryrelation between M A and M B as (w, s)R(v, t) w = v,sEt and Obs((w, s)) = Obs((v, t)). Now we verifythe condition Zig of Definition 9 (the invariance conditionis trivial by definition of R). Suppose (w, s) i (w

    , s)

  • 8/13/2019 Hidden Protocols

    8/10

    then w i w in M and s i s

    in A. Since sEt, thereis a t in B such that t i t

    , sEt, and L0(Prot(s)) =L0(Prot(t)) where 0 = V(s

    ). Clearly (w, t) is inM BandObs((w, t)) =Obs((w, s)).Thus we have that (w, t) i(w, t) and (w, s)R(w, t). The condition Zag can be provedin a similar way.

    : Suppose A ef B. It is clear that for a univer-sal ignorance model M (cf. the proof of Proposition 19):

    M A o M B. We set a relation E between thestate spaces of A and B as sEt iff (w, s) o (w, t). We

    can verify that E is a protocol emulation relation. Thefirst (consistency) condition of protocol emulation is im-mediate according to the invariance condition of observa-tion bisimulation. Now we show the second one. Sup-pose s i s

    and sEt. Now consider an arbitrary Psuch that L(Prot(s)) = . Since M is a universal igno-rance model, there is a state w in M such that V(w) = and (w, s) i (w

    , s). Since sEt then by definition of E,(w, s)o (w, t). Thus there is a (v

    , t) in M Bsuch that(w, t) i (v

    , t) and (w, s)o (v, t) (clearlyw =v since

    M is a universal ignorance model). It follows that t i t

    and L(Prot(s)) = L(Prot(t)). Thus for all P suchthatL(Prot(s)) = there is a state t such that t i t

    inB, sEt and L(Prot(s)) = L(Prot(t)). The third condi-tion can be shown similarly.

    We now extend the framework forPOL to provide aDEL-style logical language, describing the installation or changeof protocols, together with the effect of the observations ofagents, based on the current protocol.

    3.3 Epistemic Protocol LogicIn the language of the Epistemic protocol logic (EPL),

    we consider protocol models as first-class citizens, giving aDEL-like language.

    Definition 28 (Language of EPL). Theformulasof EPL are given by:

    ::= |p | | | Ki | [] | [!Ae]

    where p P, i I, Lobs, and Ae is an epistemicprotocol with the designated statee.

    In defining the language we restrict ourselves to finite pro-tocol models. The models for the logic EPL are taken to bethe epistemic observation models M = S, , V, Obs. Thetruth definition is given as follows:

    Definition 29 (Truth definition for EPL). Givenan epistemic observation modelM =S, , V, Obs, a state

    s S, and an EPL-formula , the truth conditions of at s coincide with POL for the formulas that they have incommon. The truth condition for the new formula in EPLis defined as follows:

    M, s [!Ae] L(fV(s)(Prot(e))) = = M A, (s, e)

    Recalling the meaning of the modal product operation, theexpression [!Ae] therefore stands for after installing the

    new epistemic protocol Ae, is true. As an example, letus give the model of Example 1, the observation model in-duced by the epistemic protocol (modelled earlier, call itAe), and the updated model according to observation a (inthe picture, visualized by |a):

    g(a)

    Ann

    Jane,Ann g(b)

    Ann

    |a g()

    Ann

    g(a+b) Jane,Ann g(a+b) g() Jane,Ann g()

    Recall the initial model M:

    g() Jane,Ann g()

    Suppose the actual state (underlined) was the leftmost statein M, (say s), then we can verify:

    M, s [!Ae][a](KJaneg KAnng), andM, s [!Ae][a]KAnn(KJaneg KJaneg).

    The picture corresponding to Example 2 is as follows (Ae

    is the corresponding epistemic protocol modelled earlier):

    p(Y)

    Alice

    Ben,Alicep(N)

    Alice

    |N p()

    Alice

    p(Y) Ben,Alice p(N) p()

    Recall the initial model N:

    p() Ben,Alice p()

    Let the actual state (underlined) be the rightmost state inN (say t), we can verify:

    N, t [!Ae

    ][N](KBenp KAlicep), butN, t [!Ae ][N]KAlice(KBenp KBenp).

    The further investigation of this logicEPL and its relationto DEL is future work.

    4. INCORPORATING FACTUAL CHANGESSo far, we presented information-changing actions, not

    fact-changing actions: recall thatLg() consists of guardedstrings with uniform guards only. This is not so realistic inpractice since many actions used in protocols also changethe facts, e.g., turn on the light if you see the enemy. Fac-tual change can be modelled by assigning to each action afunction which changes the valuation of basic propositions

    (as in [20, 23]). Let us now show how protocols based onfactual changing actions can be incorporated in our setting.

    Definition 30 (factual changing actions). A setof factual changing actions (fc-actions) is a tuple(, )suchthat : P Bool(P).

    Intuitively, after executing action a , p is assigned thetruth value of(a, p) (evaluated before executing a). For ex-ample, let p be the proposition denoting the door is closedthen slamming the door (a) has the post-effect: (a)(p) = .On the other hand, toggling the switch (b) has the post-effects modelled by (b)(p) =p ifp expresses the switch is

  • 8/13/2019 Hidden Protocols

    9/10

    on. Clearly non-factual change actions can b e seen as (, 0)where for any a , 0(a) is the identity function.

    For the ease in proofs, we introduce an alternative way torepresent factual changing actions.

    Definition 31 (factual change system). A- fac-tual change system (fc-system) F is a tuple (Q, ) whereQ= P(P) and:Q Q.

    Clearly, is a deterministic transition function and thusit can be extended to the domain of Q such that(, a0 ak) is the unique state of the fc-system that is reach-able via transitions subsequently labelled bya0, . . . , ak. Weshow that a set of factual change actions can be seen as afactual change system:

    Proposition 32. For each set of fc-actions (, ) thereis a -fc-system such that for each a , P: p(a, p)

    p (a, p) (, a) =

    . For each

    -fc-system there is a set of fc-actions (, ) such that foreach a , P:

    p(a, p)

    p (a, p)

    (, a) =.

    Proof. The first part of the proposition is straightfor-

    ward, just take p(a, p) p (a, p) (, a) = as the definition of the transition function in thefactual change system. For the second part, let (a, p) =

    {| p (, a)}.

    In the sequel, we only work with fc-systems. To interpretobservation expressions w.r.t. a fc-systemF, we only needto revise the definition ofLg as follows:

    LFg(a) = {a | a in F }

    Based on the automaton developed in [12], we can provean analogy of Proposition 15, viz. Proposition 33.

    Proposition 33. Given an fc-system F, every has a

    normal form

    F

    = P(? ) for some Lobs suchthatLFg() = L

    Fg(

    F).

    Proof. (a sketch of the proof) In [12], Kozen gave a gen-eral semantics for guarded expressions (the s in LProt asin our paper), where the only difference concerns the clausefor the atomic a :

    LKg (a) = {a | , P}

    Note that there is no constraint between and in theabove definition. It is not hard to see that given an fc-systemFwe can define a translation tF : LProt LProtby replacingeach a with

    P{? a?

    | a in F }. It follows that

    LKg (tF()) = LFg().It is shown in [12] that guarded regular

    expressions correspond to deterministic guarded automata(finite automata with transitions labelled by atomic actionsand Boolean tests) satisfying the following properties:

    Each state is either a state that only has outgoing ac-tion transitions (action state) or a state that only hasoutgoing test transitions (test state).

    The start state is a test state.

    The outgoing test transitions are deterministic: theyare labelled by characteristic formulas of P andfor each test state q and each , q has one and onlyone -successor.

    Therefore by following the different transitions from thestart state, we can separate the automaton that correspondsto the guarded regular expression into |2P| zones. It is easythen to generate the corresponding regular expressions (ob-servations) for each zone (by ignoring the test transitions).In such a way, the normal form of can be generated.

    Based on the above proposition, we can define the installa-tion of protocols with factual changing actions on observa-tion models, similar to Definition 18.

    5. CONCLUSION AND FUTURE WORKThe information that the actions carry may depend on

    agents knowledge of protocols. In this paper we studiedcases where protocols are not commonly known and pro-posed a logic framework for updating knowledge by obser-vations based on epistemic protocols. We consider variousextensions of our work.

    We only used Boolean tests in the language Lprot. A moreexpressive protocol language includes epistemic tests. Anexample of such a protocol would be (?Kp (a + b)) (?Kp c): as long as you do not knowp, keep choosing an

    a or b action, until you get to know p, and then do c. Asobserved in [7], knowledge-based protocols are much moreinvolved than fact-based protocols. Defining the interpreta-tion and executability of such protocols is a challenge, be-cause, checking epistemic formulas is then non-local. Also,the introduction of knowledge tests may make the satisfia-bility problem of the logic undecidable. For example, theobservations may easily encode iterated public announce-ment, which is known as a source of undecidability in suchlogics [13]. On the positive side, by including more expres-sive tests we expect better matching between observationmodels and epistemic protocols (cf. Theorem 24).

    Another extension is to consider less than radical updatemechanisms for installing new protocols. In our current ap-

    proach, when installing a new protocol, we simply ignore andoverwrite the old expected observations completely. Con-sider a singleton observation epistemic model with observa-tion a +c. Now, when updating with the protocol a +b wesimply replace a+ c by a+ b. Instead, we could integratea + cwitha + b, somehow. For example, such a non-radicalprotocol update with a + b could result in b (intersectedrefinement), or in (b + c) (a + b) (concatenation), or in(b + c) + (a + b) (choice), and so on. See [28] for a discussion.Finally, we can relax the assumption of public observation,e.g., some actions may not be observable to certain agents.

    The subject of hidden protocols is also interesting from thepoint of view of language pragmatics. Speakers who intendto convey information to only some of their listeners in sucha way that others will not understand what is going on, aredeliberately acting against some of Grices maxims of coop-erative conversation [9]. Forms of indirect or uncooperativecommunication, such as veiled bribes and threats, have al-ready been investigated from the perspective of pragmaticsand cognitive science, relating them also to aspects like lackof common knowledge [4, 26, 16, 22]. Our analysis of hiddenprotocols in this paper, by distinguishing between observa-tions and actions, is more fine-grained than the changes instandard dynamic epistemic logic, but could benefit fromtaking such Gricean aspects into account. Thus, in additionto observational powers of the agents, also their assertivepowers may b e modeled. Finally, it would b e interesting to

  • 8/13/2019 Hidden Protocols

    10/10

    investigate the role of the interlocutors goals and intentionswhen they utter a veiled speech act that is part of a hiddenprotocol (cf. [2, 17, 10, 6]).

    Acknowledgments

    We thank the anonymous reviewers whose suggestions helpedus to improve the paper considerably. Hans van Ditmarsch isalso affiliated to IMSc (Institute of Mathematical Sciences,

    Chennai, India), as associated researcher. Part of this re-search was carried out by Hans van Ditmarsch and YanjingWang during their joint stay at the IMSc. Hans van Dit-marsch thanks the Netherlands Organization for ScientificResearch (NWO) for a visiting grant 040.11.177 to RinekeVerbrugge, University of Groningen, of which this publica-tion can be seen as the outcome. Sujata Ghosh acknowledgesNWO research grant 600.065.120.08N201 and Rineke Ver-brugge acknowledges NWO research grants 600.065.120.08N-201 and 227-80-001.

    6. REFERENCES[1] A. Baltag. A logic for suspicious players: Epistemic

    actions and belief-updates in games. Bulletin ofEconomic Research, 54(1):145, 2002.

    [2] M. Bratman. Intention, Plans, and Practical Reason.Harvard University Press, Cambridge (MA), 1987.

    [3] J. A. Brzozowski. Derivatives of regular expressions.Journal of the ACM, 11(4):481494, 1964.

    [4] H. Clark.Using Language. Cambridge UniversityPress, Cambridge, 1996.

    [5] J. H. Conway. Regular Algebra and Finite Machines.Chapman and Hall, 1971.

    [6] F. Dignum, B. Dunin-Keplicz, and R. Verbrugge.Creating collective intention through dialogue. LogicJournal of the IGPL, 9:145158, 2001.

    [7] R. Fagin, J. Y. Halpern, Y. Moses, and M. Y. Vardi.

    Knowledge-based programs.Distributed Computing,10(4):199225, July 1997.

    [8] R. Fagin, J. Y. Halpern, M. Y. Vardi, and Y. Moses.Reasoning about Knowledge. MIT Press, Cambridge,MA, USA, 1995.

    [9] H. P. Grice. Logic and conversation. In P. Cole andJ. L. Morgan, editors, Syntax and semantics,volume 3. New York: Academic Press, 1975.

    [10] B. Grosz and C. Sidner. Plans for discourse. InP. Cohen, J. Morgan, and M. Pollack, editors,Intentions in Communication, pages 417444. MITPress, Cambridge (MA), 1990.

    [11] T. Hoshi.Epistemic Dynamics and ProtocolInformation. PhD thesis, Stanford University, 2009.

    [12] D. Kozen. Automata on guarded strings andapplications. Technical report, Cornell University,Ithaca, NY, USA, 2001.

    [13] J. S. Miller and L. S. Moss. The undecidability ofiterated modal relativization. Studia Logica,79(3):373407, 2005.

    [14] E. Pacuit and S. Simon. Reasoning with protocolsunder imperfect information (abstract). Short paperpresentation at AiML 2010, Moscow, 2010.

    [15] R. Parikh and R. Ramanujam. A knowledge basedsemantics of messages.Journal of Logic, Language andInformation, 12(4):453467, 2003.

    [16] S. Pinker, M. Nowak, and J. Lee. The logic of indirectspeech. Bulletin of Economic Research, 54(1):145,2002.

    [17] A. Rao and M. Georgeff. Modeling rational agentswithin a BDI-architecture. In R. Fikes andE. Sandewall, editors,Proceedings of the SecondConference on Knowledge Representation andReasoning, pages 473484. Morgan Kaufman, 1991.

    [18] S. Singh. The Code Book: The Evolution of Secrecyfrom Mary, Queen of Scots, to QuantumCryptography. Doubleday, New York, NY, USA, 1999.

    [19] J. van Benthem, J. Gerbrandy, T. Hoshi, andE. Pacuit. Merging frameworks for interaction.Journal of Philosophical Logic, 38(5):491526, 2009.

    [20] J. van Benthem, J. van Eijck, and B. Kooi. Logics ofcommunication and change. Information andComputation, 204(11):16201662, 2006.

    [21] H. van Ditmarsch, W. van der Hoek, and B. Kooi.Dynamic Epistemic Logic, volume 337 ofSyntheseLibrary. Springer, 2007.

    [22] H. van Ditmarsch, J. van Eijck, and R. Verbrugge.Common knowledge and common belief. In J. van

    Eijck and R. Verbrugge, editors, Discourses on SocialSoftware, volume 5 ofTexts in Games and Logic.Amsterdam University Press, Amsterdam, 2009.

    [23] J. van Eijck. Perception and change in update logic. InJ. van Eijck and R. Verbrugge, editors, Games,Actions and Social Software, Texts in Logic andGames (FOLLI subseries of LNCS). Springer Verlag,Berlin, 2011.

    [24] J. van Eijck, J. Ruan, and T. Sadzik. Actionemulation, 2008. Unpublished manuscript, available athomepages.cwi.nl/ jve/papers/08/ae/ae.pdf.

    [25] A. van Kooten Niekerk and S. Wijmer. VerkeerdeVriendschap: Lesbisch Leven in de Jaren 1920-1960.Sara, Amsterdam, 1985.

    [26] R. Verbrugge and L. Mol. Learning to apply theory ofmind. Journal of Logic, Language and Information,17:489511, 2008. Special issue on formal models forreal people, edited by M. Counihan.

    [27] Y. Wang.Epistemic Modelling and Protocol Dynamics.PhD thesis, University of Amsterdam, 2010.

    [28] Y. Wang. Reasoning about protocol change andknowledge. In Proceedings of the 4th IndianConference on Logic and its Applications (ICLA2011), LNAI 6521, pages 189203. Springer, 2010.