A Reactive Approach to Explanation - IJCAI
A Reactive A p p r o a c h to Explanation Johanna D. Moore University of California, Los Angeles and U S C / I n f o r m a t i o n Sciences Institute 4676 A d m i r a l t y Way M a r i n a del Rey, CA 90292-6695 Abstract E x p l a n a t i o n is an interactive process, requiri n g a dialogue between advice-giver and adviceseeker. Yet current expert systems cannot part i c i p a t e in a dialogue w i t h users. In p a r t i c u l a r these systems cannot clarify m i s u n d e r s t o o d exp l a n a t i o n s , elaborate on previous e x p l a n a t i o n s , or respond to f o l l o w - u p questions in the cont e x t of the on-going dialogue. In this paper, we describe a reactive approach to explanat i o n - one t h a t can p a r t i c i p a t e in an on-going dialogue and employs feedback f r o m the user to guide subsequent explanations. O u r system plans explanations f r o m a rich set of explanat i o n strategies, recording the system's discourse goals, the plans used to achieve t h e m , and any assumptions made while p l a n n i n g a response. T h i s record provides the dialogue context the system needs to respond a p p r o p r i a t e l y to the user's feedback. We i l l u s t r a t e our approach w i t h examples of d i s a m b i g u a t i n g a f o l l o w - u p question and p r o d u c i n g a c l a r i f y i n g e l a b o r a t i o n in response to a m i s u n d e r s t o o d e x p l a n a t i o n .
1
Introduction
E x p l a n a t i o n requires a dialogue. Users need to be able to ask f o l l o w - u p questions if they do not understand an e x p l a n a t i o n or want f u r t h e r e l a b o r a t i o n . Answers to such questions m u s t take i n t o account the dialogue cont e x t . Studies of advisory consultations between humans bear o u t this observation, showing t h a t e x p l a n a t i o n is an interactive process between explainer and adviceseeker [Pollack et a/., 1982]. S t u d y i n g student-teacher interactions, we f o u n d t h a t advice-seekers frequently did not f u l l y understand the i n s t r u c t o r ' s response. They frequently asked f o l l o w - u p questions requesting clarific a t i o n , e l a b o r a t i o n , or r e - e x p l a n a t i o n . In some cases, f o l l o w - u p questions t o o k the f o r m of a w e l l - a r t i c u l a t e d query; in other cases, the f o l l o w - u p was a vaguely articulated m u m b l e or sentence f r a g m e n t . Often the instrucThe research described in this paper was supported by the Defense Advanced Research Projects Agency (DARPA) under a NASA Ames cooperative agreement number NCC 2-520.
1504
Speech and Natural Language
W i l l i a m R. Swartout U S C / I n f o r m a t i o n Sciences Institute 4676 A d m i r a l t y Way M a r i n a del Rey, CA 90292-6695
tor d i d n o t have m u c h to go o n , b u t s t i l l had to provide an a p p r o p r i a t e response. U n f o r t u n a t e l y , current expert systems cannot p a r t i c i pate in a dialogue w i t h users. In p a r t i c u l a r these systems cannot clarify m i s u n d e r s t o o d e x p l a n a t i o n s , elaborate on previous explanations, or respond to f o l l o w - u p questions in the context of the on-going dialogue. In p a r t , the e x p l a n a t i o n components of current expert systems are l i m i t e d because they are q u i t e simple. However, even the more sophisticated generation techniques employed in c o m p u t a t i o n a l linguistics are inadequate for responding to f o l l o w - u p questions. T h e p r o b l e m is t h a t both expert system e x p l a n a t i o n and n a t u r a l language genera t i o n systems view generating responses as a one-shot process. T h a t is, a system is assumed to have one opp o r t u n i t y to produce a response t h a t the user w i l l f i n d satisfactory. T h i s one-shot approach is clearly inconsistent w i t h analyses of n a t u r a l l y o c c u r r i n g advisory dialogues. Moreover, if a system has o n l y one o p p o r t u n i t y to produce a t e x t t h a t achieves the speaker's goals w i t h o u t over- or u n d e r - i n f o r m i n g , b o r i n g or confusing the listener then t h a t system must have an enormous a m o u n t of detailed knowledge about the listener. T h i s has led to a view t h a t improvements i n e x p l a n a t i o n w i l l come f r o m improvements in the user m o d e l and considerable effort has been expended in representing a detailed model of the user - i n c l u d i n g the user's goals, w h a t the user knows a b o u t the d o m a i n , how i n f o r m a t i o n should be presented to t h a t user, and so f o r t h [ A p p e l t , 1981, M c C o y , 1985, Paris, 1988, Kass and F i n i n , 1989]. However, f o l l o w i n g Sparck Jones [Sparck Jones, 1984], we question whether it w i l l be possible to b u i l d complete and correct user models. Further, by focusing on user models, researchers have ignored the rich source of guidance t h a t people use in p r o d u c i n g explanations, namely feedback f r o m the listener [Ringle and Bruce, 1981]. By t h r o w i n g o u t the one-shot assumption, we can make use of t h a t guidance. T h u s , a reactive approach to e x p l a n a t i o n is required one in w h i c h feedback f r o m the user is an integral p a r t of the e x p l a n a t i o n process. A reactive e x p l a n a t i o n fac i l i t y should include the a b i l i t y t o : 1) accept feedback f r o m the listener, 2) recover if the listener indicates he is not satisfied w i t h the response, 3) answer f o l l o w - u p questions t a k i n g i n t o account previous explanations, not as independent questions, 4) offer f u r t h e r explanations
even if the user does n o t ask a w e l l - f o r m u l a t e d f o l l o w - u p question, a n d 5) use i n f o r m a t i o n in a user model if it exists, b u t n o t require i t .
2
L i m i t a t i o n s of Current Systems
T h e r e are three m a i n reasons w h y current systems cann o t p a r t i c i p a t e in a dialogue w i t h their users. F i r s t , to be able to clarify a m i s u n d e r s t o o d e x p l a n a t i o n or respond to a f o l l o w - u p question in context, a speaker must understand the e x p l a n a t i o n he has produced. U n f o r t u n a t e l y , current expert systems produce explanations by f i l l i n g in templates or using canned t e x t and thus have l i t t l e or no " u n d e r s t a n d i n g " of their o w n explanations. T h e y do not represent the goal of the e x p l a n a t i o n , w h a t rhetorical purposes are served by i n d i v i d u a l clauses in the t e x t , or w h a t assumptions a b o u t t h e listeners knowledge may have been made. Second, current systems always interpret questions in the same way. For e x a m p l e , when a user types " w h y ? " in response to a question, M Y C I N assumes t h a t the user is asking w h a t higher d o m a i n goal gave rise to the current one. Subsequent " w h y s " retrieve i n f o r m a t i o n about s t i l l higher level d o m a i n goals t h a t posted the ones j u s t described. As we describe later, this single interpretat i o n fails to take i n t o account the dialogue context and can be u n n a t u r a l . T h e t h i r d p r o b l e m w i t h current systems is t h a t they t y p i c a l l y have o n l y a single response strategy associated w i t h each question t y p e . M a k i n g oneself understood often requires the a b i l i t y to present the same i n f o r m a t i o n in m u l t i p l e ways or to provide different i n f o r m a t i o n to i l l u s t r a t e the same p o i n t . W i t h o u t m u l t i p l e strategies for responding to a question, a system cannot offer an alt e r n a t i v e response even if it understands why a previous e x p l a n a t i o n was n o t satisfactory. We have b u i l t an e x p l a n a t i o n component for an expert system w h i c h addresses these problems. To provide the capabilities described above, we: 1) p l a n responses such t h a t the i n t e n t i o n a l s t r u c t u r e of the responses is explicit and can be reasoned a b o u t , 2) keep track of conversat i o n a l context by r e m e m b e r i n g not o n l y w h a t the user asks, b u t also the p l a n n i n g process t h a t led to an exp l a n a t i o n , 3) t a x o n o m i z e the types of (follow-up) questions t h a t are asked and understand their relationship to the current c o n t e x t , and 4) provide flexible explanation strategies w i t h m a n y and varied plans for achieving a given discourse g o a l .
3
System Description
O u r e x p l a n a t i o n generation f a c i l i t y is p a r t of the Explainable E x p e r t Systems ( E E S ) f r a m e w o r k [Neches et a/., 1985]. W h e n an expert system is b u i l t in EES, an extensive development h i s t o r y is created t h a t records the d o m a i n goal s t r u c t u r e and design decisions behind the expert system. T h i s s t r u c t u r e is available for use by the explanation facility. We have used E E S to construct a p r o t o t y p e expert syst e m , the P r o g r a m Enhancement A d v i s o r ( P E A ) [Neches et a/., 1985], w h i c h we are using as a testbed for our work on e x p l a n a t i o n generation. P E A is an advice-giving sys-
t e m intended to aid users in i m p r o v i n g their C o m m o n Lisp programs by recommending transformations t h a t enhance the user's code. 1 T h e user supplies P E A w i t h the p r o g r a m to be enhanced. P E A begins the dialogue w i t h the user by asking w h a t characteristics of the prog r a m he would like to improve. T h e user may choose to enhance any c o m b i n a t i o n of readability, m a i n t a i n a b i l i t y , and efficiency. P E A then recommends transformations t h a t would enhance the p r o g r a m along the chosen d i mensions. A f t e r each recommendation is made, the user is free to ask questions about the r e c o m m e n d a t i o n . An overview of the e x p l a n a t i o n generation facility (and its relation to the P E A expert system) is shown in Figure 1. T h e text planner is central to the explanat i o n facility. T h e planner uses a t o p - d o w n hierarchical expansion p l a n n i n g mechansism. W h e n a discourse goal is posted, the text planner searches its l i b r a r y of explanation strategies l o o k i n g for strategies t h a t can achieve i t . A strategy is selected and may in t u r n post subgoal c for the planner to refine. P l a n n i n g continues in this fashion u n t i l the entire plan is refined i n t o p r i m i t i v e operators, i.e., speech acts such as INFORM, RECOMMEND. As the system plans explanations, it keeps track of any assumptions it makes about w h a t the user knows as well as alternative strategies t h a t could have been used to achieve the discourse goals. T h e result is a text plan for achieving the original discourse goal. T h i s text plan is recorded in the dialogue history and passed to the Penman text generation system [ M a n n and Matthiessen, 1983] for t r a n s l a t i o n i n t o E n g l i s h . A discourse goal may be posted as a result of reasoning in the expert system or as a result of a query f r o m the user. User queries must first be interpreted by the query analyzer. Even t h o u g h we assume the user poses queries in a stylized n o t a t i o n , 2 ambiguities may s t i l l arise. An example of an ambiguous follow-up question and the process we use to disambiguate it appears in Section 4 . 1 . I n p u t to the query analyzer may be a f o l l o w - u p question (e.g. " W h y ? " , " W h a t is a generalized-variable?"), an i n d i c a t i o n t h a t the user does not understand the system's response ( " H u h ? " ) , or an i n d i c a t i o n t h a t the user understands and has no follow-up question ( " G o A h e a d " ) . T h e query analyzer interprets this feedback and either returns control to the expert system, or formulates the appropriate discourse goal and passes it to the text planner to produce a response. If the user asks a follow-up question or indicates t h a t he does not understand the e x p l a n a t i o n , the system examines the dialogue history. T h e i n f o r m a t i o n contained there concerning the goal structure of the e x p l a n a t i o n , assumptions made d u r i n g its generation, and alternative strategies, is necessary in d i s a m b i g u a t i n g follow-up questions, selecting perspective when describing or com1
PEA recommends transformations that improve the "style" of the user's code. It does not attempt to understand the content of the user's program. 2 To avoid the myriad problems of parsing English-input, we require that the user's questions be posed in a stylized language. We have also provided a "mouse" interface that allows a user to point to parts of the system's explanations that he doesn't understand or has questions about.
Moore and Swartout
1505
p a r i n g objects, and c l a r i f y i n g misunderstandings.
4
Examples
Consider t h e sample dialogue w i t h our system shown in Figure 2. W h i l e enhancing m a i n t a i n a b i l i t y , the syst e m recommends t h a t the user p e r f o r m an act, namely replace s e t q w i t h s e t f . 3 T h e user, not i m m e d i a t e l y convinced t h a t this replacement should be done, asks " w h y ? " . T h e query analyzer interprets this question and posts the goal: (PERSUADE S H (GOAL H E v e n t u a l l y (DONE H r e p l a c e - 1 ) ) ) where S is the speaker, H is the hearer, and r e p l a c e - 1 is the act of replacing s e t q w i t h s e t f . T h i s is a goal to persuade the user to p e r f o r m r e p l a c e - 1 . Discourse goals are represented in terms of the effects t h a t the speaker intends his utterance to have on the hearer. 4 W h e n a discourse goal is posted, the t e x t planner searches for operators capable of satisfying i t , i.e., all operators whose Effect matches the goal. Each p l a n operator also contains a Constraint list, which l i m i t s the a p p l i c a b i l i t y of the operator; a Nucleus, which is a discourse goal for the m a i n topic to be expressed; and opt i o n a l l y , a list of Satellites which are discourse goals t h a t express a d d i t i o n a l i n f o r m a t i o n needed to achieve the Effect of the operator. One of the p l a n operators t h a t matches the current goal is shown in Figure 3. 5 I n f o r m a l l y , this plan opIn many instances PEA is capable of performing the transformation. In such cases, while the actual replacement is done by the system, the user's approval is required. 4 Following Hovy [Hovy, 1988], we use the terminology for expressing beliefs developed by Cohen and Levesque in their theory of rational interaction [Cohen and Levesque, 1985]. Space limitations prohibit an exposition of their terminology in this paper. We provide English paraphrases where necessary for clarity. 5 (BMB S H x) should be read as "S believes that S and
1506
Speech and Natural Language
erator states t h a t if an act is a step in achieving some d o m a i n goal(s) t h a t the hearer shares, then one way to persuade the hearer to do the act is to m o t i v a t e the act in terms of those goals. More f o r m a l l y , this operator's constraints require t h a t there be a ?domain-goal such t h a t : ?domain-goal is a goal of the expert syst e m , r e p l a c e - 1 is a step in achieving ? domain-goal, and the speaker and hearer m u t u a l l y believe t h a t ?domaingoal is a goal of the hearer. In order to b i n d ?domaingoal, the text planner examines the expert system's goal structure. T h e system assumes t h a t the user shares its top-level goal, e n h a n c e - p r o g r a m , since he is using the system to perform t h a t task. F u r t h e r m o r e , since the system asks w h a t characteristics the user w o u l d like to enhance, the system can assume t h a t the user shares the goal of enhancing those characteristics; in this case, e n h a n c e - m a i n t a i n a b i l i t y . T h e i n f o r m a t i o n t h a t the user shares these t w o domain goals is included in the user model. In order to avoid e x p l a i n i n g parts of the reasoning chain t h a t the user is f a m i l i a r w i t h , the more specific goal is chosen. In this example, once the constraints have been satisfied, the only possible b i n d i n g for the variable ?domain~goal is e n h a n c e - m a i n t a i n a b i l i t y . Once a plan operator has been selected, the planner instantiates it by posting its Nucleus and required Satellites as subgoals to be refined. In this case, since there is o n l y one b i n d i n g for ?domain-goal, the single subgoal (MOTIVATION r e p l a c e - 1 e n h a n c e - m a i n t a i n a b i l i t y ) is posted. One strategy for satisfying this goal, shown in Figure 4, is to i n f o r m the hearer of the goal the system II mutually believe x." Our plan language makes use of Rhetorical Structure Theory [Mann and Thompson, 1988], a descriptive theory characterizing text structure in terms of the relations (e.g. MEANS, MOTIVATION) that hold between parts of a text. A detailed description of the plan language is beyond the scope of this paper, see [Moore and Paris, 1989].
is t r y i n g to achieve (the Nucleus) and then to establish t h a t the act in question is p a r t of the means for achievi n g the goal (the Satellite). These subgoals are eventually refined to speech acts. T h e f i n a l t e x t p l a n , shown in Figure 5, is added to the dialogue history and passed to the generator which produces response (5) in the sample dialogue.
5
D i s a m b i g u a t i n g Follow-up Questions
A f t e r this response is presented, the user asks " w h y ? " a second t i m e . At this p o i n t , there are several possible i n t e r p r e t a t i o n s of this question, i n c l u d i n g : II:
Why are you trying to enhance the maintainability of the program?
12:
Why are you trying to enhance the maintainability of the program by applying transformations that enhance maintainability? (as opposed to enhancing the program via some other method)
13:
Why are you applying transformations that enhance maintainability?
14:
Why is s e t q - t o - s e t f a transformation that enhances maintainability?
Recall t h a t in cases such as this, M Y C I N always assumes t h a t " w h y " is asking why the system is t r y i n g to achieve the higher-level d o m a i n goal, corresponding t o i n t e r p r e t a t i o n I I . T h i s i n t e r p r e t a t i o n i s often inappropriate. Users are frequently asking for j u s t i f i c a t i o n of factual statements made in the e x p l a n a t i o n , corresponding to 14. Even if M Y C I N could recognize the m u l t i p l e interpretations, it could not decide a m o n g t h e m because it does not m a i n t a i n a dialogue history and does not understand the responses it generates. Resolving a m b i g u i t y requires: 1) i d e n t i f y i n g candidate interpretations, and 2) choosing among t h e m . W h i l e these tasks are conceptually d i s t i n c t , our system interleaves t h e m to increase efficiency. It generates the most likely interpretations first, and then uses heuristics to rule t h e m o u t . If an i n t e r p r e t a t i o n cannot be ruled o u t , it is chosen as the i n t e r p r e t a t i o n of the user's question and a response is generated. If the i n t e r p r e t a t i o n is incorrect, the user can s t i l l recover by asking a follow-up
Moore and Swart out 1507
question. O u r s y s t e m uses t h e f o l l o w i n g heuristics t o i d e n t i f y a n d choose a m o n g m u l t i p l e i n t e r p r e t a t i o n s : H i : Follow immediate focus rules (continuing on the same topic is preferred over returning to a previously mentioned topic, [Sidner, 1979, M c K e o w n , 1982].) H2: D o n ' t tell the user things he already knows. H3: D o n ' t tell the user things you've already said. W e have f o u n d t h a t focus o f a t t e n t i o n ( H I ) i s a p o w erful heuristic for ordering the generation of likely i n terpretations. I n t h e c u r r e n t e x a m p l e , t h e m o s t recent focus of a t t e n t i o n ( i n d i c a t e d by the * local-context*) is the statement S e t q - t o - s e t f is a t r a n s f o r m a t i o n t h a t enhances m a i n t a i n a b i l i t y . T h e system thus infers t h a t t h e question concerns t h e r a t i o n a l e b e h i n d t h i s s t a t e m e n t (14), unless H2 or H3 rules t h a t interp r e t a t i o n o u t . For e x a m p l e , t h a t i n t e r p r e t a t i o n could be r u l e d o u t by 112 if t h e user m o d e l i n d i c a t e d t h a t t h e user k n e w w h y s e t q - t o - s e t f enhances m a i n t a i n a b i l i t y . In o u r e x a m p l e , n o t h i n g rules o u t 14, so the s y s t e m exp l a i n s w h y s e t q - t o - s e t f enhances m a i n t a i n a b i l i t y . If t h e f i r s t i n t e r p r e t a t i o n is r u l e d o u t , the system uses t h e n e x t m o s t recent focus o f a t t e n t i o n 6 t o f o r m the n e x t possible i n t e r p r e t a t i o n . I n t h i s e x a m p l e , t h a t focus refers t o t h e m e t h o d t h e s y s t e m i s a p p l y i n g t o achieve 6
T h e text plan records the order in which topics appear in the explanation. This information is used to derive foci of attention in order. In choosing interpretations of " W h y ? " , the system skips over certain rhetorical relations in the text plan t h a t are considered to be purely presentational in nature, e.g., ELABORATE, BACKGROUND.
1508
Speech and Natural Language
a goal (i.e., a p p l y i n g t r a n s f o r m a t i o n s t h a t e n h a n c e m a i n t a i n a b i l i t y ) . T h i s leads t o i n t e r p r e t a t i o n 13. T h i s i n t e r p r e t a t i o n w i l l b e r u l e d o u t b y H 3 because f r o m the semantics of the r h e t o r i c a l r e l a t i o n MEANS, we determ i n e t h a t we have j u s t t o l d t h e user t h a t the system is u s i n g the m e t h o d o f a p p l y i n g t r a n s f o r m a t i o n s t h a t e n h a n c e m a i n t a i n a b i l i t y i n order t o achieve the goal enhance m a i n t a i n a b i l i t y . T h e n e x t m o s t recent focus refers t o t h e means by w h i c h the e n h a n c e m e n t is b e i n g achieved (i.e., by a p p l y i n g t r a n s f o r m a t i o n s t h a t enhance m a i n t a i n a b i l i t y ) . T h i s leads t o i n t e r p r e t a t i o n 12. I f t h a t interp r e t a t i o n were also r u l e d o u t , t h e s y s t e m w o u l d continue i n t h i s fashion u n t i l i t f o u n d a n acceptable i n t e r p r e t a t i o n or reached t h e g l o b a l c o n t e x t , w h i c h , in t h i s case, is t h e t o p node i n the t e x t p l a n . I t i s i n t e r e s t i n g t o n o t e t h a t t h e focus o f t h e user's question is derived f r o m t h e system's s t a t e m e n t , not the user's. T h e user s i m p l y t y p e s " w h y ? " i n his f i r s t t w o queries i n the d i a l o g u e ; c o n t e x t comes f r o m t h e response generated b y t h e s y s t e m . U n t i l n o w , m u c h w o r k has conc e n t r a t e d o n b u i l d i n g discourse m o d e l s t h a t keep t r a c k of t h e user's goals a n d plans - b o t h d o m a i n goals [Carberry, 1983, M c K e o w n et al, 1985] a n d discourse goals [ L i t m a n , 1985]. L i t t l e w o r k has been done o n keeping t r a c k of t h e system's discourse goals a n d t h e p l a n s it uses t o achieve t h e m . 7 A s o u r e x a m p l e i l l u s t r a t e s , conversational context must include the system's statements.
Recent work by Grosz and Sidner addresses this issue for the purposes of analyzing dialogues [Grosz and Sidner, 1986].
Answering a Vaguely Articulated Follow-Up Question
6
T h e user t h e n asks the question, " W h a t is a generalized-variable?". T h e query analyzer interprets the question and posts the discourse goal (BMB S H (KNOW H g e n e r a l i z e d - v a r i a b l e ) ) , i.e., the speaker wishes to achieve the state where the speaker and hearer m u t u a l l y believe t h a t the hearer knows the concept generalized-variable. T h e system has several plan operators for achieving such a goal. It m a y describe a concept by describing its a t t r i b u t e s and its p a r t s , by d r a w i n g an analogy w i t h a s i m i l a r concept, by g i v i n g examples of the concept, or by generalizing a concept the user is f a m i l i a r w i t h . T h e plan operator for the l a t t e r is shown in Figure 6. To choose f r o m a m o n g these candidate plan operators, the planner has several selection heuristics, i n c l u d i n g : SHI: SH2: SH3:
Prefer operators that require making no assumptions about the hearer's beliefs. Prefer operators that make use of a concept the hearer knows. Prefer operators that make use of a concept mentioned in the dialogue history.
In this case, the user m o d e l indicates t h a t the hearer knows the concept s i m p l e - v a r i a b l e . Hence the operator in Figure 6 requires m a k i n g no assumptions about the hearer's knowledge, makes use of a concept the user knows, and uses a concept previously mentioned in the dialogue. T h u s it is ranked highest by the plan selection heuristics. T h e final t e x t plan for this example first describes s i m p l e - v a r i a b l e s and then abstracts this concept t o i n t r o d u c e g e n e r a l i z e d - v a r i a b l e s . T h i s produces the response shown in the sample dialogue. T h e user then indicates t h a t he does not understand t h i s e x p l a n a t i o n w i t h the v a g u e l y - a r t i c u l a t e d f o l l o w - u p , " H u h ? " . F r o m our analysis o f n a t u r a l l y occurring d i alogues, we devised a set of recovery heuristics for res p o n d i n g to such a question. These include: RHl: RII2:
RH3: RH4: 8
If the discourse goal is to describe a concept, give example(s). If the discourse goal is to describe a concept, and there is an analogous entity that the hearer knows, draw an analogy to the familiar concept. Expand any unexpanded optional satellites in previous plan operators. 8 If another plan exists for achieving the discourse goal, try i t .
Plan operators may contain optional satellites which the system may decide to leave unexpanded during planning.
R H 1 and R H 2 apply in the context of a p a r t i c u l a r discourse goal, namely describing a concept, while the other heuristics are more general. T h e system tries to apply its most specific knowledge first. In this case, R H 1 applies and the explainer recovers by g i v i n g examples. As illustrated in Figure 6, constraints on plan operators often refer to the state of the hearer's k n o w l edge. T h e user model includes the d o m a i n concepts and problem-solving knowledge, i.e., d o m a i n goals and plans, assumed to be k n o w n to the current user. However, the system does not require t h a t this model is be either complete or correct. Therefore, the user model may contain concepts the user does n o t actually k n o w or o m i t concepts the user does k n o w . To satisfy a constraint on an operator, the system may assume t h a t a concept is k n o w n to the user even if it is n o t indicated in the user m o d e l . As described above, when such an assumption is made, the selection heuristics give the operator a lower r a t i n g . If the operator is selected, the fact t h a t an ass u m p t i o n was made is recorded in the plan structure. T h e system must keep track of such assumptions because these are likely candidates if a misunderstanding occurs later. T h i s leads to another recovery heuristic: RH5:
If any assumptions were made in planning the last explanation, plan responses to make these assumptions true.
For example, in p r o d u c i n g response (7) in Figure 2 the system assumed (erroneously) t h a t the user knew w h a t generalized-variables were and s i m p l y used it w i t h o u t defining i t . In (8), the user asked e x p l i c i t l y for a definit i o n . If he had j u s t typed " H u h ? " instead, the system would have examined its assumptions and used R H 5 to plan a response defining generalized variables.
7
C u r r e n t Status and Conclusions
T h e expert system and e x p l a n a t i o n f a c i l i t y described are implemented. There are a p p r o x i m a t e l y 75 plan operators, 5 plan selection heuristics, and 5 recovery heuristics. T h e system can produce the t e x t plans necessary to p a r t i c i p a t e in the dialogue shown and several others t h a t are similar. In s u m m a r y , current expert systems f a i l to support explanation as a dialogue. T h e i r u n n a t u r a l , one-shot approach to explanation depends c r i t i c a l l y on the quali t y of the user model and is seriously degraded if t h a t model is incomplete or incorrect. Because they f a i l to support dialogue, these systems cannot clarify misunderstood explanations, elaborate on previous explanations, or respond to f o l l o w - u p questions in the context of the on-going dialogue. As an alternative, we proposed a reactive model of
Moore and Swart out 1509
e x p l a n a t i o n - in w h i c h the system can employ feedback f r o m the user and p a r t i c i p a t e in a dialogue. O u r exp l a n a t i o n generation f a c i l i t y plans explanations f r o m a rich set of strategies, keeping track of the system's discourse goals, the plans used to achieve t h e m , and any assumptions made w h i l e p l a n n i n g a response. O u r syst e m m a i n t a i n s a recorded h i s t o r y of the t e x t plans used in p r o d u c i n g responses so t h a t it can later reason a b o u t its o w n responses when feedback f r o m the listener i n dicates t h a t an e x p l a n a t i o n was not u n d e r s t o o d . O u r system can employ i n f o r m a t i o n in a user m o d e l when it is available, b u t is n o t c r i t i c a l l y dependent on t h a t information.
Acknowledgements T h e authors w o u l d like to t h a n k Cecile Paris for collabo r a t i o n on t h e p l a n language and M a r g o t Flowers, Neil G o l d m a n and J a s m i n a P a v l i n for comments on earlier versions of t h i s paper.
References [ A p p e l t , 1981] Douglas E. A p p e l t . Planning Natural Language Utterances to Satisfy Multiple Goals. PhD thesis, S t a n f o r d University, 1981. [Carberry, 1983] Sandra Carberry. T r a c k i n g user goals in an i n f o r m a t i o n - s e e k i n g e n v i r o n m e n t . In Proceedings of the Third National Conference on Artificial Intelligence, pages 5 9 - 6 3 , W a s h i n g t o n , D.C., A u g u s t 22-26 1983. [Cohen an d Levesque, 1985] P h i l i p R. Coh en and Hect o r J. Levesque. Speech acts and r a t i o n a l i t y . In Proceedings of the Twenty-Third Annual Meeting of the Association for Computational Linguistics, pages 4 9 60, U n i v e r s i t y of Chicago, Chicago, I l l i n o i s , J u l y 8-12 1985. [Grosz and Sidner, 1986] B a r b a r a J. Grosz and C a n dace L. Sidner. A t t e n t i o n , i n t e n t i o n , and the structure of discourse. Computational Linguistics, 12(3): 175— 204,1986. [Hovy, 1988] Ed u a r d H. Hovy. P l a n n i n g coherent m u l tisentential t e x t . In Proceedings of the Twenty-Sixth Annual Meeting of the Association for Computational Linguistics, State U n i v e r s i t y of New Y o r k , Buffalo, New Y o r k , 7-10 June 1988. [Kass and F i n i n , 1989] R o b e r t K ass and T i m F i n i n . M o d e l i n g the user in n a t u r a l language systems. Computational Linguistics Journal, 14:5-22, 1989. Special Issue on User M o d e l l i n g . [ L i t m a n , 1985] Diane L i t m a n . Plan Recognition and Discourse Analysis: An Integrated Approach for Understanding Dialogues. P h D thesis, University of Rochester, 1985. Published by University of Rochester as Technical R e p o r t TR 170. [ M a n n and Matthiessen, 1983] W i l l i a m C. M a n n and C h r i s t i a n Matthiessen. Nigel: A systemic g r a m m a r for t e x t generation. Technical Report, RR-83-105, U S C / I n f o r m a t i o n Sciences I n s t i t u t e , February 1983.
1510
Speech and Natural Language
[ M a n n and T h o m p s o n , 1988] W i l l i a m C . M a n n and Sandra A . T h o m p s o n . R h e t o r i c a l structure theory: Towards a f u n c t i o n a l theory of t e x t organization. TEXT, 8 ( 3 ) : 2 4 3 - 2 8 1 , 1988. [McCoy, 1985] K a t h l e e n F. M c C o y . Correcting ObjectRelated Misconceptions. P h D thesis, University of Pennsylvania, December 1985. Published by University of Pennsylvania as Technical R e p o r t MS-CIS-8557. [McK eown et al., 1985] K a t h l e e n R. M c K e o w n , M y r o n W i s h , and K e v i n M a t t h e w s . T a i l o r i n g explanations for the user. In Proceedings of the Ninth International Joint Conference on Artificial Intelligence, pages 7 9 4 798, Los Angeles, C A , A u g u s t 1985. I J C A I . [ M c K e o w n , 1982] Kathleen R. M c K e o w n . Generating Natural Language Text in Response to Questions About Database Structure. P h D thesis, University of Pennsylvania, 1982. Published by University of Pennsylvania as Technical R e p o r t MS-CIS-82-5. [M
oore and Paris, 1989] J o h a n n a D. Moore and Cecile L. Paris. P l a n n i n g t e x t for advisory dialogues. In Proceedings of the Twenty-Seventh Annual Meeting of the Association for Computational Linguistics, Vancouver, B.C., C a n a d a , June 26-29 1989.
[Neches et al, 1985] R o b e r t Neches, William R. S w a r t o u t , and J o h a n n a D. M o o r e . Enhanced m a i n tenance and e x p l a n a t i o n of expert systems t h r o u g h e x p l i c i t models of their development. IEEE Transactions on Software Engineering, SE-11(11), November 1985. [Paris, 1988] Cecile L. Paris. T a i l o r i n g object descriptions to a user's level of expertise. Computational Linguistics Journal, 14(3), 1988. [Pollack et al., 1982] M a r t h a E, Pollack, Julia Hirschberg, and Bonnie L y n n Webber. User partici p a t i o n in the reasoning processes of expert systems. In Proceedings of the Second National Conference on Artificial Intelligence, P i t t s b u r g h , Pennsylvania, A u gust 18-20 1982. [Ringle and Bruce, 1981] M a r t i n H. Ringle and B e r t r a m C. Bruce. Conversation failure. In Wendy G. Lehnert and M a r t i n H. Ringle, editors, Knowledge Representation and Natural Language Processing, pages 2 0 3 - 2 2 1 . Lawrence E a r l b a u m Associates, Hillsdale, New Jersey, 1981. [Sidner, 1979] Candace L. Sidner. Toward a Computational Theory of Definite Anaphora Comprehension in English Discourse. P h D thesis, Massachusetts I n s t i t u t e of Technology, C a m b r i d g e , Mass., 1979. [Sparck Jones, 1984] K a r e n Sparck Jones. User models and expert systems. Technical R e p o r t No. 6 1 , University of C a m b r i d g e C o m p u t e r L a b o r a t o r y , December 1984.
1
Introduction
E x p l a n a t i o n requires a dialogue. Users need to be able to ask f o l l o w - u p questions if they do not understand an e x p l a n a t i o n or want f u r t h e r e l a b o r a t i o n . Answers to such questions m u s t take i n t o account the dialogue cont e x t . Studies of advisory consultations between humans bear o u t this observation, showing t h a t e x p l a n a t i o n is an interactive process between explainer and adviceseeker [Pollack et a/., 1982]. S t u d y i n g student-teacher interactions, we f o u n d t h a t advice-seekers frequently did not f u l l y understand the i n s t r u c t o r ' s response. They frequently asked f o l l o w - u p questions requesting clarific a t i o n , e l a b o r a t i o n , or r e - e x p l a n a t i o n . In some cases, f o l l o w - u p questions t o o k the f o r m of a w e l l - a r t i c u l a t e d query; in other cases, the f o l l o w - u p was a vaguely articulated m u m b l e or sentence f r a g m e n t . Often the instrucThe research described in this paper was supported by the Defense Advanced Research Projects Agency (DARPA) under a NASA Ames cooperative agreement number NCC 2-520.
1504
Speech and Natural Language
W i l l i a m R. Swartout U S C / I n f o r m a t i o n Sciences Institute 4676 A d m i r a l t y Way M a r i n a del Rey, CA 90292-6695
tor d i d n o t have m u c h to go o n , b u t s t i l l had to provide an a p p r o p r i a t e response. U n f o r t u n a t e l y , current expert systems cannot p a r t i c i pate in a dialogue w i t h users. In p a r t i c u l a r these systems cannot clarify m i s u n d e r s t o o d e x p l a n a t i o n s , elaborate on previous explanations, or respond to f o l l o w - u p questions in the context of the on-going dialogue. In p a r t , the e x p l a n a t i o n components of current expert systems are l i m i t e d because they are q u i t e simple. However, even the more sophisticated generation techniques employed in c o m p u t a t i o n a l linguistics are inadequate for responding to f o l l o w - u p questions. T h e p r o b l e m is t h a t both expert system e x p l a n a t i o n and n a t u r a l language genera t i o n systems view generating responses as a one-shot process. T h a t is, a system is assumed to have one opp o r t u n i t y to produce a response t h a t the user w i l l f i n d satisfactory. T h i s one-shot approach is clearly inconsistent w i t h analyses of n a t u r a l l y o c c u r r i n g advisory dialogues. Moreover, if a system has o n l y one o p p o r t u n i t y to produce a t e x t t h a t achieves the speaker's goals w i t h o u t over- or u n d e r - i n f o r m i n g , b o r i n g or confusing the listener then t h a t system must have an enormous a m o u n t of detailed knowledge about the listener. T h i s has led to a view t h a t improvements i n e x p l a n a t i o n w i l l come f r o m improvements in the user m o d e l and considerable effort has been expended in representing a detailed model of the user - i n c l u d i n g the user's goals, w h a t the user knows a b o u t the d o m a i n , how i n f o r m a t i o n should be presented to t h a t user, and so f o r t h [ A p p e l t , 1981, M c C o y , 1985, Paris, 1988, Kass and F i n i n , 1989]. However, f o l l o w i n g Sparck Jones [Sparck Jones, 1984], we question whether it w i l l be possible to b u i l d complete and correct user models. Further, by focusing on user models, researchers have ignored the rich source of guidance t h a t people use in p r o d u c i n g explanations, namely feedback f r o m the listener [Ringle and Bruce, 1981]. By t h r o w i n g o u t the one-shot assumption, we can make use of t h a t guidance. T h u s , a reactive approach to e x p l a n a t i o n is required one in w h i c h feedback f r o m the user is an integral p a r t of the e x p l a n a t i o n process. A reactive e x p l a n a t i o n fac i l i t y should include the a b i l i t y t o : 1) accept feedback f r o m the listener, 2) recover if the listener indicates he is not satisfied w i t h the response, 3) answer f o l l o w - u p questions t a k i n g i n t o account previous explanations, not as independent questions, 4) offer f u r t h e r explanations
even if the user does n o t ask a w e l l - f o r m u l a t e d f o l l o w - u p question, a n d 5) use i n f o r m a t i o n in a user model if it exists, b u t n o t require i t .
2
L i m i t a t i o n s of Current Systems
T h e r e are three m a i n reasons w h y current systems cann o t p a r t i c i p a t e in a dialogue w i t h their users. F i r s t , to be able to clarify a m i s u n d e r s t o o d e x p l a n a t i o n or respond to a f o l l o w - u p question in context, a speaker must understand the e x p l a n a t i o n he has produced. U n f o r t u n a t e l y , current expert systems produce explanations by f i l l i n g in templates or using canned t e x t and thus have l i t t l e or no " u n d e r s t a n d i n g " of their o w n explanations. T h e y do not represent the goal of the e x p l a n a t i o n , w h a t rhetorical purposes are served by i n d i v i d u a l clauses in the t e x t , or w h a t assumptions a b o u t t h e listeners knowledge may have been made. Second, current systems always interpret questions in the same way. For e x a m p l e , when a user types " w h y ? " in response to a question, M Y C I N assumes t h a t the user is asking w h a t higher d o m a i n goal gave rise to the current one. Subsequent " w h y s " retrieve i n f o r m a t i o n about s t i l l higher level d o m a i n goals t h a t posted the ones j u s t described. As we describe later, this single interpretat i o n fails to take i n t o account the dialogue context and can be u n n a t u r a l . T h e t h i r d p r o b l e m w i t h current systems is t h a t they t y p i c a l l y have o n l y a single response strategy associated w i t h each question t y p e . M a k i n g oneself understood often requires the a b i l i t y to present the same i n f o r m a t i o n in m u l t i p l e ways or to provide different i n f o r m a t i o n to i l l u s t r a t e the same p o i n t . W i t h o u t m u l t i p l e strategies for responding to a question, a system cannot offer an alt e r n a t i v e response even if it understands why a previous e x p l a n a t i o n was n o t satisfactory. We have b u i l t an e x p l a n a t i o n component for an expert system w h i c h addresses these problems. To provide the capabilities described above, we: 1) p l a n responses such t h a t the i n t e n t i o n a l s t r u c t u r e of the responses is explicit and can be reasoned a b o u t , 2) keep track of conversat i o n a l context by r e m e m b e r i n g not o n l y w h a t the user asks, b u t also the p l a n n i n g process t h a t led to an exp l a n a t i o n , 3) t a x o n o m i z e the types of (follow-up) questions t h a t are asked and understand their relationship to the current c o n t e x t , and 4) provide flexible explanation strategies w i t h m a n y and varied plans for achieving a given discourse g o a l .
3
System Description
O u r e x p l a n a t i o n generation f a c i l i t y is p a r t of the Explainable E x p e r t Systems ( E E S ) f r a m e w o r k [Neches et a/., 1985]. W h e n an expert system is b u i l t in EES, an extensive development h i s t o r y is created t h a t records the d o m a i n goal s t r u c t u r e and design decisions behind the expert system. T h i s s t r u c t u r e is available for use by the explanation facility. We have used E E S to construct a p r o t o t y p e expert syst e m , the P r o g r a m Enhancement A d v i s o r ( P E A ) [Neches et a/., 1985], w h i c h we are using as a testbed for our work on e x p l a n a t i o n generation. P E A is an advice-giving sys-
t e m intended to aid users in i m p r o v i n g their C o m m o n Lisp programs by recommending transformations t h a t enhance the user's code. 1 T h e user supplies P E A w i t h the p r o g r a m to be enhanced. P E A begins the dialogue w i t h the user by asking w h a t characteristics of the prog r a m he would like to improve. T h e user may choose to enhance any c o m b i n a t i o n of readability, m a i n t a i n a b i l i t y , and efficiency. P E A then recommends transformations t h a t would enhance the p r o g r a m along the chosen d i mensions. A f t e r each recommendation is made, the user is free to ask questions about the r e c o m m e n d a t i o n . An overview of the e x p l a n a t i o n generation facility (and its relation to the P E A expert system) is shown in Figure 1. T h e text planner is central to the explanat i o n facility. T h e planner uses a t o p - d o w n hierarchical expansion p l a n n i n g mechansism. W h e n a discourse goal is posted, the text planner searches its l i b r a r y of explanation strategies l o o k i n g for strategies t h a t can achieve i t . A strategy is selected and may in t u r n post subgoal c for the planner to refine. P l a n n i n g continues in this fashion u n t i l the entire plan is refined i n t o p r i m i t i v e operators, i.e., speech acts such as INFORM, RECOMMEND. As the system plans explanations, it keeps track of any assumptions it makes about w h a t the user knows as well as alternative strategies t h a t could have been used to achieve the discourse goals. T h e result is a text plan for achieving the original discourse goal. T h i s text plan is recorded in the dialogue history and passed to the Penman text generation system [ M a n n and Matthiessen, 1983] for t r a n s l a t i o n i n t o E n g l i s h . A discourse goal may be posted as a result of reasoning in the expert system or as a result of a query f r o m the user. User queries must first be interpreted by the query analyzer. Even t h o u g h we assume the user poses queries in a stylized n o t a t i o n , 2 ambiguities may s t i l l arise. An example of an ambiguous follow-up question and the process we use to disambiguate it appears in Section 4 . 1 . I n p u t to the query analyzer may be a f o l l o w - u p question (e.g. " W h y ? " , " W h a t is a generalized-variable?"), an i n d i c a t i o n t h a t the user does not understand the system's response ( " H u h ? " ) , or an i n d i c a t i o n t h a t the user understands and has no follow-up question ( " G o A h e a d " ) . T h e query analyzer interprets this feedback and either returns control to the expert system, or formulates the appropriate discourse goal and passes it to the text planner to produce a response. If the user asks a follow-up question or indicates t h a t he does not understand the e x p l a n a t i o n , the system examines the dialogue history. T h e i n f o r m a t i o n contained there concerning the goal structure of the e x p l a n a t i o n , assumptions made d u r i n g its generation, and alternative strategies, is necessary in d i s a m b i g u a t i n g follow-up questions, selecting perspective when describing or com1
PEA recommends transformations that improve the "style" of the user's code. It does not attempt to understand the content of the user's program. 2 To avoid the myriad problems of parsing English-input, we require that the user's questions be posed in a stylized language. We have also provided a "mouse" interface that allows a user to point to parts of the system's explanations that he doesn't understand or has questions about.
Moore and Swartout
1505
p a r i n g objects, and c l a r i f y i n g misunderstandings.
4
Examples
Consider t h e sample dialogue w i t h our system shown in Figure 2. W h i l e enhancing m a i n t a i n a b i l i t y , the syst e m recommends t h a t the user p e r f o r m an act, namely replace s e t q w i t h s e t f . 3 T h e user, not i m m e d i a t e l y convinced t h a t this replacement should be done, asks " w h y ? " . T h e query analyzer interprets this question and posts the goal: (PERSUADE S H (GOAL H E v e n t u a l l y (DONE H r e p l a c e - 1 ) ) ) where S is the speaker, H is the hearer, and r e p l a c e - 1 is the act of replacing s e t q w i t h s e t f . T h i s is a goal to persuade the user to p e r f o r m r e p l a c e - 1 . Discourse goals are represented in terms of the effects t h a t the speaker intends his utterance to have on the hearer. 4 W h e n a discourse goal is posted, the t e x t planner searches for operators capable of satisfying i t , i.e., all operators whose Effect matches the goal. Each p l a n operator also contains a Constraint list, which l i m i t s the a p p l i c a b i l i t y of the operator; a Nucleus, which is a discourse goal for the m a i n topic to be expressed; and opt i o n a l l y , a list of Satellites which are discourse goals t h a t express a d d i t i o n a l i n f o r m a t i o n needed to achieve the Effect of the operator. One of the p l a n operators t h a t matches the current goal is shown in Figure 3. 5 I n f o r m a l l y , this plan opIn many instances PEA is capable of performing the transformation. In such cases, while the actual replacement is done by the system, the user's approval is required. 4 Following Hovy [Hovy, 1988], we use the terminology for expressing beliefs developed by Cohen and Levesque in their theory of rational interaction [Cohen and Levesque, 1985]. Space limitations prohibit an exposition of their terminology in this paper. We provide English paraphrases where necessary for clarity. 5 (BMB S H x) should be read as "S believes that S and
1506
Speech and Natural Language
erator states t h a t if an act is a step in achieving some d o m a i n goal(s) t h a t the hearer shares, then one way to persuade the hearer to do the act is to m o t i v a t e the act in terms of those goals. More f o r m a l l y , this operator's constraints require t h a t there be a ?domain-goal such t h a t : ?domain-goal is a goal of the expert syst e m , r e p l a c e - 1 is a step in achieving ? domain-goal, and the speaker and hearer m u t u a l l y believe t h a t ?domaingoal is a goal of the hearer. In order to b i n d ?domaingoal, the text planner examines the expert system's goal structure. T h e system assumes t h a t the user shares its top-level goal, e n h a n c e - p r o g r a m , since he is using the system to perform t h a t task. F u r t h e r m o r e , since the system asks w h a t characteristics the user w o u l d like to enhance, the system can assume t h a t the user shares the goal of enhancing those characteristics; in this case, e n h a n c e - m a i n t a i n a b i l i t y . T h e i n f o r m a t i o n t h a t the user shares these t w o domain goals is included in the user model. In order to avoid e x p l a i n i n g parts of the reasoning chain t h a t the user is f a m i l i a r w i t h , the more specific goal is chosen. In this example, once the constraints have been satisfied, the only possible b i n d i n g for the variable ?domain~goal is e n h a n c e - m a i n t a i n a b i l i t y . Once a plan operator has been selected, the planner instantiates it by posting its Nucleus and required Satellites as subgoals to be refined. In this case, since there is o n l y one b i n d i n g for ?domain-goal, the single subgoal (MOTIVATION r e p l a c e - 1 e n h a n c e - m a i n t a i n a b i l i t y ) is posted. One strategy for satisfying this goal, shown in Figure 4, is to i n f o r m the hearer of the goal the system II mutually believe x." Our plan language makes use of Rhetorical Structure Theory [Mann and Thompson, 1988], a descriptive theory characterizing text structure in terms of the relations (e.g. MEANS, MOTIVATION) that hold between parts of a text. A detailed description of the plan language is beyond the scope of this paper, see [Moore and Paris, 1989].
is t r y i n g to achieve (the Nucleus) and then to establish t h a t the act in question is p a r t of the means for achievi n g the goal (the Satellite). These subgoals are eventually refined to speech acts. T h e f i n a l t e x t p l a n , shown in Figure 5, is added to the dialogue history and passed to the generator which produces response (5) in the sample dialogue.
5
D i s a m b i g u a t i n g Follow-up Questions
A f t e r this response is presented, the user asks " w h y ? " a second t i m e . At this p o i n t , there are several possible i n t e r p r e t a t i o n s of this question, i n c l u d i n g : II:
Why are you trying to enhance the maintainability of the program?
12:
Why are you trying to enhance the maintainability of the program by applying transformations that enhance maintainability? (as opposed to enhancing the program via some other method)
13:
Why are you applying transformations that enhance maintainability?
14:
Why is s e t q - t o - s e t f a transformation that enhances maintainability?
Recall t h a t in cases such as this, M Y C I N always assumes t h a t " w h y " is asking why the system is t r y i n g to achieve the higher-level d o m a i n goal, corresponding t o i n t e r p r e t a t i o n I I . T h i s i n t e r p r e t a t i o n i s often inappropriate. Users are frequently asking for j u s t i f i c a t i o n of factual statements made in the e x p l a n a t i o n , corresponding to 14. Even if M Y C I N could recognize the m u l t i p l e interpretations, it could not decide a m o n g t h e m because it does not m a i n t a i n a dialogue history and does not understand the responses it generates. Resolving a m b i g u i t y requires: 1) i d e n t i f y i n g candidate interpretations, and 2) choosing among t h e m . W h i l e these tasks are conceptually d i s t i n c t , our system interleaves t h e m to increase efficiency. It generates the most likely interpretations first, and then uses heuristics to rule t h e m o u t . If an i n t e r p r e t a t i o n cannot be ruled o u t , it is chosen as the i n t e r p r e t a t i o n of the user's question and a response is generated. If the i n t e r p r e t a t i o n is incorrect, the user can s t i l l recover by asking a follow-up
Moore and Swart out 1507
question. O u r s y s t e m uses t h e f o l l o w i n g heuristics t o i d e n t i f y a n d choose a m o n g m u l t i p l e i n t e r p r e t a t i o n s : H i : Follow immediate focus rules (continuing on the same topic is preferred over returning to a previously mentioned topic, [Sidner, 1979, M c K e o w n , 1982].) H2: D o n ' t tell the user things he already knows. H3: D o n ' t tell the user things you've already said. W e have f o u n d t h a t focus o f a t t e n t i o n ( H I ) i s a p o w erful heuristic for ordering the generation of likely i n terpretations. I n t h e c u r r e n t e x a m p l e , t h e m o s t recent focus of a t t e n t i o n ( i n d i c a t e d by the * local-context*) is the statement S e t q - t o - s e t f is a t r a n s f o r m a t i o n t h a t enhances m a i n t a i n a b i l i t y . T h e system thus infers t h a t t h e question concerns t h e r a t i o n a l e b e h i n d t h i s s t a t e m e n t (14), unless H2 or H3 rules t h a t interp r e t a t i o n o u t . For e x a m p l e , t h a t i n t e r p r e t a t i o n could be r u l e d o u t by 112 if t h e user m o d e l i n d i c a t e d t h a t t h e user k n e w w h y s e t q - t o - s e t f enhances m a i n t a i n a b i l i t y . In o u r e x a m p l e , n o t h i n g rules o u t 14, so the s y s t e m exp l a i n s w h y s e t q - t o - s e t f enhances m a i n t a i n a b i l i t y . If t h e f i r s t i n t e r p r e t a t i o n is r u l e d o u t , the system uses t h e n e x t m o s t recent focus o f a t t e n t i o n 6 t o f o r m the n e x t possible i n t e r p r e t a t i o n . I n t h i s e x a m p l e , t h a t focus refers t o t h e m e t h o d t h e s y s t e m i s a p p l y i n g t o achieve 6
T h e text plan records the order in which topics appear in the explanation. This information is used to derive foci of attention in order. In choosing interpretations of " W h y ? " , the system skips over certain rhetorical relations in the text plan t h a t are considered to be purely presentational in nature, e.g., ELABORATE, BACKGROUND.
1508
Speech and Natural Language
a goal (i.e., a p p l y i n g t r a n s f o r m a t i o n s t h a t e n h a n c e m a i n t a i n a b i l i t y ) . T h i s leads t o i n t e r p r e t a t i o n 13. T h i s i n t e r p r e t a t i o n w i l l b e r u l e d o u t b y H 3 because f r o m the semantics of the r h e t o r i c a l r e l a t i o n MEANS, we determ i n e t h a t we have j u s t t o l d t h e user t h a t the system is u s i n g the m e t h o d o f a p p l y i n g t r a n s f o r m a t i o n s t h a t e n h a n c e m a i n t a i n a b i l i t y i n order t o achieve the goal enhance m a i n t a i n a b i l i t y . T h e n e x t m o s t recent focus refers t o t h e means by w h i c h the e n h a n c e m e n t is b e i n g achieved (i.e., by a p p l y i n g t r a n s f o r m a t i o n s t h a t enhance m a i n t a i n a b i l i t y ) . T h i s leads t o i n t e r p r e t a t i o n 12. I f t h a t interp r e t a t i o n were also r u l e d o u t , t h e s y s t e m w o u l d continue i n t h i s fashion u n t i l i t f o u n d a n acceptable i n t e r p r e t a t i o n or reached t h e g l o b a l c o n t e x t , w h i c h , in t h i s case, is t h e t o p node i n the t e x t p l a n . I t i s i n t e r e s t i n g t o n o t e t h a t t h e focus o f t h e user's question is derived f r o m t h e system's s t a t e m e n t , not the user's. T h e user s i m p l y t y p e s " w h y ? " i n his f i r s t t w o queries i n the d i a l o g u e ; c o n t e x t comes f r o m t h e response generated b y t h e s y s t e m . U n t i l n o w , m u c h w o r k has conc e n t r a t e d o n b u i l d i n g discourse m o d e l s t h a t keep t r a c k of t h e user's goals a n d plans - b o t h d o m a i n goals [Carberry, 1983, M c K e o w n et al, 1985] a n d discourse goals [ L i t m a n , 1985]. L i t t l e w o r k has been done o n keeping t r a c k of t h e system's discourse goals a n d t h e p l a n s it uses t o achieve t h e m . 7 A s o u r e x a m p l e i l l u s t r a t e s , conversational context must include the system's statements.
Recent work by Grosz and Sidner addresses this issue for the purposes of analyzing dialogues [Grosz and Sidner, 1986].
Answering a Vaguely Articulated Follow-Up Question
6
T h e user t h e n asks the question, " W h a t is a generalized-variable?". T h e query analyzer interprets the question and posts the discourse goal (BMB S H (KNOW H g e n e r a l i z e d - v a r i a b l e ) ) , i.e., the speaker wishes to achieve the state where the speaker and hearer m u t u a l l y believe t h a t the hearer knows the concept generalized-variable. T h e system has several plan operators for achieving such a goal. It m a y describe a concept by describing its a t t r i b u t e s and its p a r t s , by d r a w i n g an analogy w i t h a s i m i l a r concept, by g i v i n g examples of the concept, or by generalizing a concept the user is f a m i l i a r w i t h . T h e plan operator for the l a t t e r is shown in Figure 6. To choose f r o m a m o n g these candidate plan operators, the planner has several selection heuristics, i n c l u d i n g : SHI: SH2: SH3:
Prefer operators that require making no assumptions about the hearer's beliefs. Prefer operators that make use of a concept the hearer knows. Prefer operators that make use of a concept mentioned in the dialogue history.
In this case, the user m o d e l indicates t h a t the hearer knows the concept s i m p l e - v a r i a b l e . Hence the operator in Figure 6 requires m a k i n g no assumptions about the hearer's knowledge, makes use of a concept the user knows, and uses a concept previously mentioned in the dialogue. T h u s it is ranked highest by the plan selection heuristics. T h e final t e x t plan for this example first describes s i m p l e - v a r i a b l e s and then abstracts this concept t o i n t r o d u c e g e n e r a l i z e d - v a r i a b l e s . T h i s produces the response shown in the sample dialogue. T h e user then indicates t h a t he does not understand t h i s e x p l a n a t i o n w i t h the v a g u e l y - a r t i c u l a t e d f o l l o w - u p , " H u h ? " . F r o m our analysis o f n a t u r a l l y occurring d i alogues, we devised a set of recovery heuristics for res p o n d i n g to such a question. These include: RHl: RII2:
RH3: RH4: 8
If the discourse goal is to describe a concept, give example(s). If the discourse goal is to describe a concept, and there is an analogous entity that the hearer knows, draw an analogy to the familiar concept. Expand any unexpanded optional satellites in previous plan operators. 8 If another plan exists for achieving the discourse goal, try i t .
Plan operators may contain optional satellites which the system may decide to leave unexpanded during planning.
R H 1 and R H 2 apply in the context of a p a r t i c u l a r discourse goal, namely describing a concept, while the other heuristics are more general. T h e system tries to apply its most specific knowledge first. In this case, R H 1 applies and the explainer recovers by g i v i n g examples. As illustrated in Figure 6, constraints on plan operators often refer to the state of the hearer's k n o w l edge. T h e user model includes the d o m a i n concepts and problem-solving knowledge, i.e., d o m a i n goals and plans, assumed to be k n o w n to the current user. However, the system does not require t h a t this model is be either complete or correct. Therefore, the user model may contain concepts the user does n o t actually k n o w or o m i t concepts the user does k n o w . To satisfy a constraint on an operator, the system may assume t h a t a concept is k n o w n to the user even if it is n o t indicated in the user m o d e l . As described above, when such an assumption is made, the selection heuristics give the operator a lower r a t i n g . If the operator is selected, the fact t h a t an ass u m p t i o n was made is recorded in the plan structure. T h e system must keep track of such assumptions because these are likely candidates if a misunderstanding occurs later. T h i s leads to another recovery heuristic: RH5:
If any assumptions were made in planning the last explanation, plan responses to make these assumptions true.
For example, in p r o d u c i n g response (7) in Figure 2 the system assumed (erroneously) t h a t the user knew w h a t generalized-variables were and s i m p l y used it w i t h o u t defining i t . In (8), the user asked e x p l i c i t l y for a definit i o n . If he had j u s t typed " H u h ? " instead, the system would have examined its assumptions and used R H 5 to plan a response defining generalized variables.
7
C u r r e n t Status and Conclusions
T h e expert system and e x p l a n a t i o n f a c i l i t y described are implemented. There are a p p r o x i m a t e l y 75 plan operators, 5 plan selection heuristics, and 5 recovery heuristics. T h e system can produce the t e x t plans necessary to p a r t i c i p a t e in the dialogue shown and several others t h a t are similar. In s u m m a r y , current expert systems f a i l to support explanation as a dialogue. T h e i r u n n a t u r a l , one-shot approach to explanation depends c r i t i c a l l y on the quali t y of the user model and is seriously degraded if t h a t model is incomplete or incorrect. Because they f a i l to support dialogue, these systems cannot clarify misunderstood explanations, elaborate on previous explanations, or respond to f o l l o w - u p questions in the context of the on-going dialogue. As an alternative, we proposed a reactive model of
Moore and Swart out 1509
e x p l a n a t i o n - in w h i c h the system can employ feedback f r o m the user and p a r t i c i p a t e in a dialogue. O u r exp l a n a t i o n generation f a c i l i t y plans explanations f r o m a rich set of strategies, keeping track of the system's discourse goals, the plans used to achieve t h e m , and any assumptions made w h i l e p l a n n i n g a response. O u r syst e m m a i n t a i n s a recorded h i s t o r y of the t e x t plans used in p r o d u c i n g responses so t h a t it can later reason a b o u t its o w n responses when feedback f r o m the listener i n dicates t h a t an e x p l a n a t i o n was not u n d e r s t o o d . O u r system can employ i n f o r m a t i o n in a user m o d e l when it is available, b u t is n o t c r i t i c a l l y dependent on t h a t information.
Acknowledgements T h e authors w o u l d like to t h a n k Cecile Paris for collabo r a t i o n on t h e p l a n language and M a r g o t Flowers, Neil G o l d m a n and J a s m i n a P a v l i n for comments on earlier versions of t h i s paper.
References [ A p p e l t , 1981] Douglas E. A p p e l t . Planning Natural Language Utterances to Satisfy Multiple Goals. PhD thesis, S t a n f o r d University, 1981. [Carberry, 1983] Sandra Carberry. T r a c k i n g user goals in an i n f o r m a t i o n - s e e k i n g e n v i r o n m e n t . In Proceedings of the Third National Conference on Artificial Intelligence, pages 5 9 - 6 3 , W a s h i n g t o n , D.C., A u g u s t 22-26 1983. [Cohen an d Levesque, 1985] P h i l i p R. Coh en and Hect o r J. Levesque. Speech acts and r a t i o n a l i t y . In Proceedings of the Twenty-Third Annual Meeting of the Association for Computational Linguistics, pages 4 9 60, U n i v e r s i t y of Chicago, Chicago, I l l i n o i s , J u l y 8-12 1985. [Grosz and Sidner, 1986] B a r b a r a J. Grosz and C a n dace L. Sidner. A t t e n t i o n , i n t e n t i o n , and the structure of discourse. Computational Linguistics, 12(3): 175— 204,1986. [Hovy, 1988] Ed u a r d H. Hovy. P l a n n i n g coherent m u l tisentential t e x t . In Proceedings of the Twenty-Sixth Annual Meeting of the Association for Computational Linguistics, State U n i v e r s i t y of New Y o r k , Buffalo, New Y o r k , 7-10 June 1988. [Kass and F i n i n , 1989] R o b e r t K ass and T i m F i n i n . M o d e l i n g the user in n a t u r a l language systems. Computational Linguistics Journal, 14:5-22, 1989. Special Issue on User M o d e l l i n g . [ L i t m a n , 1985] Diane L i t m a n . Plan Recognition and Discourse Analysis: An Integrated Approach for Understanding Dialogues. P h D thesis, University of Rochester, 1985. Published by University of Rochester as Technical R e p o r t TR 170. [ M a n n and Matthiessen, 1983] W i l l i a m C. M a n n and C h r i s t i a n Matthiessen. Nigel: A systemic g r a m m a r for t e x t generation. Technical Report, RR-83-105, U S C / I n f o r m a t i o n Sciences I n s t i t u t e , February 1983.
1510
Speech and Natural Language
[ M a n n and T h o m p s o n , 1988] W i l l i a m C . M a n n and Sandra A . T h o m p s o n . R h e t o r i c a l structure theory: Towards a f u n c t i o n a l theory of t e x t organization. TEXT, 8 ( 3 ) : 2 4 3 - 2 8 1 , 1988. [McCoy, 1985] K a t h l e e n F. M c C o y . Correcting ObjectRelated Misconceptions. P h D thesis, University of Pennsylvania, December 1985. Published by University of Pennsylvania as Technical R e p o r t MS-CIS-8557. [McK eown et al., 1985] K a t h l e e n R. M c K e o w n , M y r o n W i s h , and K e v i n M a t t h e w s . T a i l o r i n g explanations for the user. In Proceedings of the Ninth International Joint Conference on Artificial Intelligence, pages 7 9 4 798, Los Angeles, C A , A u g u s t 1985. I J C A I . [ M c K e o w n , 1982] Kathleen R. M c K e o w n . Generating Natural Language Text in Response to Questions About Database Structure. P h D thesis, University of Pennsylvania, 1982. Published by University of Pennsylvania as Technical R e p o r t MS-CIS-82-5. [M
oore and Paris, 1989] J o h a n n a D. Moore and Cecile L. Paris. P l a n n i n g t e x t for advisory dialogues. In Proceedings of the Twenty-Seventh Annual Meeting of the Association for Computational Linguistics, Vancouver, B.C., C a n a d a , June 26-29 1989.
[Neches et al, 1985] R o b e r t Neches, William R. S w a r t o u t , and J o h a n n a D. M o o r e . Enhanced m a i n tenance and e x p l a n a t i o n of expert systems t h r o u g h e x p l i c i t models of their development. IEEE Transactions on Software Engineering, SE-11(11), November 1985. [Paris, 1988] Cecile L. Paris. T a i l o r i n g object descriptions to a user's level of expertise. Computational Linguistics Journal, 14(3), 1988. [Pollack et al., 1982] M a r t h a E, Pollack, Julia Hirschberg, and Bonnie L y n n Webber. User partici p a t i o n in the reasoning processes of expert systems. In Proceedings of the Second National Conference on Artificial Intelligence, P i t t s b u r g h , Pennsylvania, A u gust 18-20 1982. [Ringle and Bruce, 1981] M a r t i n H. Ringle and B e r t r a m C. Bruce. Conversation failure. In Wendy G. Lehnert and M a r t i n H. Ringle, editors, Knowledge Representation and Natural Language Processing, pages 2 0 3 - 2 2 1 . Lawrence E a r l b a u m Associates, Hillsdale, New Jersey, 1981. [Sidner, 1979] Candace L. Sidner. Toward a Computational Theory of Definite Anaphora Comprehension in English Discourse. P h D thesis, Massachusetts I n s t i t u t e of Technology, C a m b r i d g e , Mass., 1979. [Sparck Jones, 1984] K a r e n Sparck Jones. User models and expert systems. Technical R e p o r t No. 6 1 , University of C a m b r i d g e C o m p u t e r L a b o r a t o r y , December 1984.
