Towards a Self-Extending Parser - Semantic Scholar
Carnegie Mellon University
Research Showcase @ CMU Institute for Software Research
School of Computer Science
1979
Towards a Self-Extending Parser Jaime G. Carbonell Carnegie Mellon University
Follow this and additional works at: http://repository.cmu.edu/isr
This Conference Proceeding is brought to you for free and open access by the School of Computer Science at Research Showcase @ CMU. It has been accepted for inclusion in Institute for Software Research by an authorized administrator of Research Showcase @ CMU. For more information, please contact [email protected].
T O W A R D S A SELF-EXTENDING PARSER Jaime G. Carbonell Department Of Computer Science Carnegie-Mellon University Pittsburgh, PA 15213
meaning of unknown words. Abstract This p a p e r discusses an approach to incremental learning in natural language processing. The t e c h n i q u e of projecting and integrating semantic c o n s t r a i n t s to learn word definitions is analyzed as Implemented in the POLITICS system. E x t e n s i o n s and improvements of this technique are developed. The problem of generalizing existing word meanings and understanding metaphorical uses of words Is addressed In terms of semantic constraint Integration.
1. Introduction Natural language analysis, like most other subfields of Artificial Intelligence and Computational Linguistics, suffers from t h e f a c t that computer systems are unable to a u t o m a t i c a l l y b e t t e r themselves. Automated learning ia c o n s i d e r e d a v e r y difficult problem, especially when applied t o natural language understanding. Consequently, little effort ha8 b e e n f o c u s e d on this problem. Some pioneering work in Artificial intelligence, such as AM [ I ] and Winston's learning s y s t e m 1"2] s t r o v e to learn or discover concept descriptions in w e l l - d e f i n e d domains. Although their efforts produced i n t e r e s t i n g Ideas and techniques, these techniques do not fully e x t e n d to • domain as complex as natural language analysis. R a t h e r than attempting the formidable task of creating a l a n g u a g e learning system, I will discuss techniques for I n c r e m e n t a l l y Increasing the abilities of a flexible language a n a l y z e r . There are many tasks that can be considered " I n c r e m e n t a l language learning". Initially the learning domain Is r e s t r i c t e d to learning the meaning of new words and g e n e r a l i z i n g e x i s t i n g word definitions. There ere a number of A.I. t e c h n i q u e s , and combinations of these techniques c a p a b l e of exhibiting incremental learning behavior. I first d i s c u s s FOULUP and POLITICS, two programs that exhibit a limited c a p a b i l i t y for Incremental word learning. Secondly, the t e c h n i q u e of semantic constraint projection end Integration, as Implemented in POLITICS, Is analyzed in some detail. Finally, I discuss the application of some general learning t e c h n i q u e s to the problem of generalizing word definitions e n d u n d e r s t a n d i n g metaphors.
2. Learning From Script Expectations Learning w o r d definitions In semantically-rich c o n t e x t s Is p e r h a p s one of the simpler tasks of incremental learning. Initially I confine my discussion to situations where the meaning of a word can be learned from the Immediately surrounding c o n t e x t . Later I relax this criterion to see how global c o n t e x t and multiple examples can help to learn the
The FOULUP program [ 3 ] learned the meaning of some unknown words in the c o n t e x t of applying s script to u n d e r s t a n d a story. Scripts [4, 5] are frame-like knowledge r e p r e s e n t a t i o n s abstracting the important features and c a u s a l s t r u c t u r e of mundane events. Scripts have general e x p e c t a t i o n s of the actions and objects that will be e n c o u n t e r e d in processing a story. For Instance, the r e s t a u r a n t script e x p e c t s to see menus, waitresses, and customers ordering and eating food (at different p r e - s p e c i f l e d times In the story). FOULUP took a d v a n t a g e of these script e x p e c t a t i o n s to c o n c l u d e t h a t Items r e f e r e n c e d in the story, which were part of e x p e c t e d actions, were Indeed names of objects that the s c r i p t e x p e c t e d to see. These expectations were used to form definitions of new words. For instance, FOULUP induced t h e meaning of "Rabbit" in, "A Rabbit veered off the road a n d s t r u c k a t r e e , " to be a self-propelled vehicle. The s y s t e m used information about the automobile accident script t o m a t c h t h e unknown word with the script-role "VEHICLE", b e c a u s e t h e script knows that the only objects that v e e r off r o a d s to smash Into road-side obstructions ere self propelled vehicles.
3 . C o n s t r a i n t P r o j e c t i o n In P O L I T I C S The POLITICS system E6, 7] induces the meanings of u n k n o w n w o r d s b y a one*pass syntactic and semantic c o n s t r a i n t p r o j e c t i o n followed by conceptual enrichment from planning and world-knowledge inferences. Consider how POLITICS p r o c e e d s when It encounters the unknown word "MPLA" In analyzing the sentence: "Russia s e n t massive arms shipments to the MPLA In Angola." Since "MPLA" follows the article '*the N it must be a noun, a d j e c t i v e or adverb. After the word "MPLA", the preposition "in" Is encountered, thus terminating the current p r e p o s i t i o n a l phrase begun with "to". Hence, since all w e l l - f o r m e d prepositional phrases require a head noun, and t h e " t o " phrase has no other noun, "MPLA" must be the head noun. Thus, b y projecting the syntactic constraints n e c e s s a r y for t h e s e n t e n c e to be well formed, one learn8 t h e s y n t a c t i c c a t e g o r y of an unknown word. it Is not always possible to narrow the categorization of a word to a single s y n t a c t i c c a t e g o r y from one example. In such cases, I p r o p o s e Intersecting the sets of possible syntactic c a t e g o r i e s from more then one sample use of the unknown w o r d until the Intersection has a single element. POLITICS learns the meaning of the unknown word by a similar, b u t substantially more complex, application of the same principle of projecting constraints from other parts of t h e s e n t e n c e and subsequently Integrating these constraints t o o o n e t r u o t a meaning representation. In the example
a b o v e , POLITICS analyzes the verb "to send" as either i n ATRANS or s PTRAflS. (Schank [ 8 ] discusses the Conceptual D e p e n d e n c y case frames. Briefly, a PTRANS IS s physical t r a n s f e r of location, and an ATRANS Is an abstract transfer o f ownership, possession or control.) The reason why POLITICS cannot decide on the type of TRANSfer is that it d o e s n o t k n o w w h e t h e r the destination of the transfer (i.e., t h e MPLA) Is s location or an agent. Physical objects, such as w e a p o n s , are PTRANSed to locations but ATRANSed to a g e n t s . The conceptual analysis of the sentence, with MPLA as y e t unresolved, Is diagrammed below: •[CIPSl
< i s > LOC v i i
~qNGOLAe t l
*SUSSIA*
Research Showcase @ CMU Institute for Software Research
School of Computer Science
1979
Towards a Self-Extending Parser Jaime G. Carbonell Carnegie Mellon University
Follow this and additional works at: http://repository.cmu.edu/isr
This Conference Proceeding is brought to you for free and open access by the School of Computer Science at Research Showcase @ CMU. It has been accepted for inclusion in Institute for Software Research by an authorized administrator of Research Showcase @ CMU. For more information, please contact [email protected].
T O W A R D S A SELF-EXTENDING PARSER Jaime G. Carbonell Department Of Computer Science Carnegie-Mellon University Pittsburgh, PA 15213
meaning of unknown words. Abstract This p a p e r discusses an approach to incremental learning in natural language processing. The t e c h n i q u e of projecting and integrating semantic c o n s t r a i n t s to learn word definitions is analyzed as Implemented in the POLITICS system. E x t e n s i o n s and improvements of this technique are developed. The problem of generalizing existing word meanings and understanding metaphorical uses of words Is addressed In terms of semantic constraint Integration.
1. Introduction Natural language analysis, like most other subfields of Artificial Intelligence and Computational Linguistics, suffers from t h e f a c t that computer systems are unable to a u t o m a t i c a l l y b e t t e r themselves. Automated learning ia c o n s i d e r e d a v e r y difficult problem, especially when applied t o natural language understanding. Consequently, little effort ha8 b e e n f o c u s e d on this problem. Some pioneering work in Artificial intelligence, such as AM [ I ] and Winston's learning s y s t e m 1"2] s t r o v e to learn or discover concept descriptions in w e l l - d e f i n e d domains. Although their efforts produced i n t e r e s t i n g Ideas and techniques, these techniques do not fully e x t e n d to • domain as complex as natural language analysis. R a t h e r than attempting the formidable task of creating a l a n g u a g e learning system, I will discuss techniques for I n c r e m e n t a l l y Increasing the abilities of a flexible language a n a l y z e r . There are many tasks that can be considered " I n c r e m e n t a l language learning". Initially the learning domain Is r e s t r i c t e d to learning the meaning of new words and g e n e r a l i z i n g e x i s t i n g word definitions. There ere a number of A.I. t e c h n i q u e s , and combinations of these techniques c a p a b l e of exhibiting incremental learning behavior. I first d i s c u s s FOULUP and POLITICS, two programs that exhibit a limited c a p a b i l i t y for Incremental word learning. Secondly, the t e c h n i q u e of semantic constraint projection end Integration, as Implemented in POLITICS, Is analyzed in some detail. Finally, I discuss the application of some general learning t e c h n i q u e s to the problem of generalizing word definitions e n d u n d e r s t a n d i n g metaphors.
2. Learning From Script Expectations Learning w o r d definitions In semantically-rich c o n t e x t s Is p e r h a p s one of the simpler tasks of incremental learning. Initially I confine my discussion to situations where the meaning of a word can be learned from the Immediately surrounding c o n t e x t . Later I relax this criterion to see how global c o n t e x t and multiple examples can help to learn the
The FOULUP program [ 3 ] learned the meaning of some unknown words in the c o n t e x t of applying s script to u n d e r s t a n d a story. Scripts [4, 5] are frame-like knowledge r e p r e s e n t a t i o n s abstracting the important features and c a u s a l s t r u c t u r e of mundane events. Scripts have general e x p e c t a t i o n s of the actions and objects that will be e n c o u n t e r e d in processing a story. For Instance, the r e s t a u r a n t script e x p e c t s to see menus, waitresses, and customers ordering and eating food (at different p r e - s p e c i f l e d times In the story). FOULUP took a d v a n t a g e of these script e x p e c t a t i o n s to c o n c l u d e t h a t Items r e f e r e n c e d in the story, which were part of e x p e c t e d actions, were Indeed names of objects that the s c r i p t e x p e c t e d to see. These expectations were used to form definitions of new words. For instance, FOULUP induced t h e meaning of "Rabbit" in, "A Rabbit veered off the road a n d s t r u c k a t r e e , " to be a self-propelled vehicle. The s y s t e m used information about the automobile accident script t o m a t c h t h e unknown word with the script-role "VEHICLE", b e c a u s e t h e script knows that the only objects that v e e r off r o a d s to smash Into road-side obstructions ere self propelled vehicles.
3 . C o n s t r a i n t P r o j e c t i o n In P O L I T I C S The POLITICS system E6, 7] induces the meanings of u n k n o w n w o r d s b y a one*pass syntactic and semantic c o n s t r a i n t p r o j e c t i o n followed by conceptual enrichment from planning and world-knowledge inferences. Consider how POLITICS p r o c e e d s when It encounters the unknown word "MPLA" In analyzing the sentence: "Russia s e n t massive arms shipments to the MPLA In Angola." Since "MPLA" follows the article '*the N it must be a noun, a d j e c t i v e or adverb. After the word "MPLA", the preposition "in" Is encountered, thus terminating the current p r e p o s i t i o n a l phrase begun with "to". Hence, since all w e l l - f o r m e d prepositional phrases require a head noun, and t h e " t o " phrase has no other noun, "MPLA" must be the head noun. Thus, b y projecting the syntactic constraints n e c e s s a r y for t h e s e n t e n c e to be well formed, one learn8 t h e s y n t a c t i c c a t e g o r y of an unknown word. it Is not always possible to narrow the categorization of a word to a single s y n t a c t i c c a t e g o r y from one example. In such cases, I p r o p o s e Intersecting the sets of possible syntactic c a t e g o r i e s from more then one sample use of the unknown w o r d until the Intersection has a single element. POLITICS learns the meaning of the unknown word by a similar, b u t substantially more complex, application of the same principle of projecting constraints from other parts of t h e s e n t e n c e and subsequently Integrating these constraints t o o o n e t r u o t a meaning representation. In the example
a b o v e , POLITICS analyzes the verb "to send" as either i n ATRANS or s PTRAflS. (Schank [ 8 ] discusses the Conceptual D e p e n d e n c y case frames. Briefly, a PTRANS IS s physical t r a n s f e r of location, and an ATRANS Is an abstract transfer o f ownership, possession or control.) The reason why POLITICS cannot decide on the type of TRANSfer is that it d o e s n o t k n o w w h e t h e r the destination of the transfer (i.e., t h e MPLA) Is s location or an agent. Physical objects, such as w e a p o n s , are PTRANSed to locations but ATRANSed to a g e n t s . The conceptual analysis of the sentence, with MPLA as y e t unresolved, Is diagrammed below: •[CIPSl
< i s > LOC v i i
~qNGOLAe t l
*SUSSIA*
