Nonmonotonic Model Inferenceï - IJCAI
N o n m o n o t o n i c M o d e l Inference — A
Formalization of Student Modeling
—
M i t s u r u I K E D A , Yasuyuki KONO and Riichiro M I Z O G U C H I I.S.I.R., Osaka University 8-1, Mihogaoka, Ibaraki, 567, Japan {ikeda,kono,miz}@ei.sanken.osaka-u. ac.jp Abstract A student model description language and its synthesis method are presented. The language called SMDL is based on a logic programming language taking 4 truth values such as true, false, unknown and fail. A modeling method called HSMIS is a new nonmonotonic model inference system and has the following major characteristics: (1) Model inference of logic program taking 4 truth values, (2)Treatment of nonmonotonicity of both student's belief and inference process itself. HSMIS incorporates de Kleer's ATMS as a vehicle for formulating the nonmonotonicity. Both SMDL interpreter and HSMIS have been implemented in Common ESP(Extended Self-contained Prolog) and incorporated into a framework for ITS, called FITS.
1
Introduction
Student modeling is one of the most important topics of ITS research, because the behavior of an ITS largely depends on a student model, which represents the snapshot of student's knowledge. This is a reason why many efforts concerning student modeling have been made, for instance, overlay model, buggy model, perturbation model, etc.[Wenger, 1987]. Most of the conventional modeling methods have simple pragmatic structures and have been incorporated into many ITS's. However, all the methods have some limitations and no complete and sound inference procedure for the models is obtained yet. In this paper, we formalize a student modeling problem as an inductive inference problem, that is, a problem to construct a model explaining observed data. In our case, data are student's answers and the model is student's knowledge. In order to make ITS's intelligent, student models have to satisfy the following requirements: 1. A c c u r a c y - c o s t tradeoff: In general, the more accurate the student model becomes, the more effective the behavior of the system becomes. However, there exists trade-off between accuracy of the model and cost to construct it. From a pragmatic viewpoint, we must set up an appropriate representation scheme for student models
by taking the trade-off into considerations. 2. N o n m o n o t o n i c i t y : Tutoring is to guide students toward better understanding of teaching material. This means that the learning process is essentially attained with change of their minds and hence the consistency of student's answers can be easily lost. Therefore, student modeling methods should be able to automatically manage the consistency of student's answers in order to follow the student's mind. However, there is very few attempts to formulate the nonmonotonicity of student modeling process[Burton, 1982][Huang et al., 1991a]. 3. U n k n o w n assertions: When a student fails to deduce her own solution for a problem, she would say to her teacher "I could not solve the problem". Needless to say, this assertion does not mean she does not have any knowledge. The student model module should use this assertion as informative data about her knowledge and construct a model which explains why she cannot deduce the answer from her own knowledge. This requires student model to deduce "unknown" assertions. 4. T h e o r e t i c a l f o u n d a t i o n : Domain-independent and theoretical foundation for the student modeling mechanism should be defined. It contributes to both clarification of the inherent property of student modeling problem and to articulation of the scalability and reusability of the proposed mechanism. To meet these requirements, we have developed a student model description language SMDL and a hypothetical student model inference system HSMIS. SMDL is an extended version of Prolog and takes four truth values including "unknown" to model the student precisely. HSMIS, an extended version of Shapiro's MlS[Shapiro, 1982], is an inductive inference system for SMDL. The second requirement mentioned above suggests that the inference procedure should cope with nonmonotonic modeling process. In HSMIS, ATMS: Assumption-based Truth Maintenance System [de Kleer, 1986] is employed for this purpose. HSMIS has been implemented in Common ESP(Extcnded Self-contained Prolog) on SPARC station.
2
SMDL : A Student Model Description Language
In addition to the above requirements, a student model is required to represent not only students but also sys-
Ikeda, Kono, and Mizoguchi
467
468
Intelligent Tutoring Systems
Figure 1: Block diagram of HSMIS.
Ikeda, Kono, and Mizoguchi
469
F i g u r e 2: E x a m p l e s of the top-level trace a n d the r e f u t a t i o n for a clause.
470
Intelligent Tutoring Systems
and diagnosis. In other words, HSMIS asks questions regardless of their appropriateness in the sense of tutoring. This requires some control mechanism of the HSMIS behavior. This subsection describes several additional mechanisms introduced to augment the HSMIS. V i r t u a l Oracles: Let us discuss the initial model problem. There are two alternative initial models: one is empty which means the teacher does not know anything about the student in advance and the other is complete knowledge (teaching material) which means teacher assumes the students usually understand the material very well. Although the former case is reasonable, the system tends to ask many questions to get a lot of information of how well the student understands the material. On the other hand, the latter case does not require many questions at least for excellent students, since the model can explain their correct behavior. This characteristics is very reasonable in real tutoring. Therefore, we decided to employ the latter. However, a serious problem still remains. One cannot simply put a clause into the student model without any justification. In order to cope with this problem, we devised an Virtual oracle generator, which generates plausible student answers based on the reliability of the current student model instead of asking questions. When a student's behavior is confined within the scope of her teacher's prediction, the teacher asks less questions by replacing the necessary information with correct answers. We call this type of oracle a "virtual oracles".
Ikeda, Kono, and Mizoguchi
471
472
Intelligent Tutoring Systems
should be employed. It seems p r o m i s i n g to a d o p t a belief revision mechanism developed by X. H u a n g e t . a l , w h i c h provides efficient, m i n i m a l revision of belief bases by using a t t e n t i o n ( f o c u s ) in the belief space[Huang et al, 1991b]. We have discussed mechanisms to avoid inconsistency in m o d e l inference in this paper. However, there exist such students w h o have contradictions in their head. To cope w i t h m o d e l i n g of such students, the system may not avoid the inconsistency b u t has to model inconsistent knowledge as it is. T h i s issue w i l l involve the development of more sophisticated control mechanism for student m o d e l i n g . We are c u r r e n t l y engaging in the issue, where student's inconsistent knowledge is m o d eled in m u l t i p l e worlds w h i c h are again supported by A T M S [ K o n o et al, 1992]. A c k n o w l e d g e m e n t s : T h e authors are grateful to reviewers for t h e i r valuable comments.
References
4
C o n c l u d i n g remarks
B o t h S M D L i n t e r p r e t e r and H S M I S have been i m p l e mented i n C o m m o n E S P ( E x t e n d e d Self-contained Prolog) a n d i n c o r p o r a t e d in a f r a m e w o r k for I T S , called F I T S [ M i z o g u c h i a n d I k e d a , 1 9 9 l ] . H S M I S can cope w i t h a variety of teaching materials as far as it can be represented i n P r o l o g . T h e r e f o r e , the a u t h o r s t h i n k t h a t the generality of H S M I S is relatively h i g h . T w o I T S ' s have been b u i l t using F I T S , one is on geography a n d the o t h e r is on chemical reactions. Simple e x a m i n a t i o n of b o t h systems shows they r u n in real t i m e if the oracle c o n t r a d i c t i o n does not occur. W h e n the oracle c o n t r a d i c t i o n occurs a n d there is no heuristics for its r e s o l u t i o n , we f i n d t h a t the cost to resolve the cont r a d i c t i o n is very expensive. T h e c u r r e n t i m p l e m e n t a t i o n of c o n t r a d i c t i o n resolution is based on a somewhat brute-force m e t h o d using d o m a i n - i n d e p e n d e n t heuristics. To improve the efficiency, m o r e powerful d o m a i n - d e p e n d e n t heuristics
[ B u r t o n , 1982] R. R. B u r t o n . In D. H. Sleeman, edit o r , Intelligent Tutoring Systems, pages 157-183, Academic Press, 1982. [de Kleer, 1986] J. de Kleer. An Assumption-based T r u t h M a i n t e n a n c e System. Artificial Intelligence, 28:127 162, 1986. [Huang et al, 1991a] X. H u a n g , G . I . M c C a l l a , J. E. Greer, and E. Neufeld. Revising Deductive K n o w l e d g e and Stereotypical Knowledge in Student Modeling. User Modeling and User-Adapted Interaction, 1:87-115, 1991. [ H u a n g et al, 1991b] X. H u a n g , G . I . M c C a l l a and E. Neufeld. Using A t t e n t i o n in Belief Revision. In Proceedings of the Ninth National Conference on Artificial Intellegeince, pages 275-280, 1991. [Ikeda et al, 1988] M. Ikeda, R. M i z o g u c h i and O. K a k u s h o . A H y p o t h e t i c a l M o d e l Inference Syst e m . Trans, of IEICE Japan, J71-D:1761 1771, 1988 ( i n Japanese). [Ikeda et al, 1989] M. Ikeda, R. M i z o g u c h i a n d O. K a k u s h o . Student M o d e l D e s c r i p t i o n Language S M D L and S t u d e n t M o d e l Inference System. Trans, of IEICE Japan, J 7 2 - D - 1 1 1 1 2 - 1 2 0 , 1989 ( i n Japanese). [Ikeda et al, 1992] M. Ikeda, Y. K o n o a n d R. M i z o g u c h i . N o n m o n o t o n i c M o d e l Inference. Technical Report of Artificial Intelligence Research Group, ISIR Osaka Univ., A L T R - 9 2 - 1 0 , 1992. [ K o n o et al, 1992] Y. K o n o , M. Ikeda and R. M i z o g u c h i . To c o n t r a d i c t is h u m a n — Student m o d e l i n g o f inconsistency — . I n C . Frasson, G . G a u thier, and G. I. M c C a l l a , editors, Intelligent Tutoring Systems, I T S '92, M o n t r e a l , C a n a d a , Proceedings, pages 451-458. L N C S 608, Springer-Verlag, 1992. [Mizoguchi and I k e d a , 1 9 9 l ] R. M i z o g u c h i and M . Ikeda. A Generic F r a m e w o r k for I T S A n d Its Evalu a t i o n . In R. Lewis and S. O t s u k i , editors, Advanced Research on Computers in Education, pages 6 3 - 7 2 . N o r t h - H o l l a n d , 1991. [Shapiro, 1982] E. Y. Shapiro. Algorithmic Program Debugging. M I T Press, 1982. [Wenger, 1987] E. Wenger. Artificial Intelligence and Tutoring Systems. M o r g a n K a u f m a n n Publishers, C a l i f o r n i a , 1987.
Ikeda, Kono, and Mizoguchi
473
Formalization of Student Modeling
—
M i t s u r u I K E D A , Yasuyuki KONO and Riichiro M I Z O G U C H I I.S.I.R., Osaka University 8-1, Mihogaoka, Ibaraki, 567, Japan {ikeda,kono,miz}@ei.sanken.osaka-u. ac.jp Abstract A student model description language and its synthesis method are presented. The language called SMDL is based on a logic programming language taking 4 truth values such as true, false, unknown and fail. A modeling method called HSMIS is a new nonmonotonic model inference system and has the following major characteristics: (1) Model inference of logic program taking 4 truth values, (2)Treatment of nonmonotonicity of both student's belief and inference process itself. HSMIS incorporates de Kleer's ATMS as a vehicle for formulating the nonmonotonicity. Both SMDL interpreter and HSMIS have been implemented in Common ESP(Extended Self-contained Prolog) and incorporated into a framework for ITS, called FITS.
1
Introduction
Student modeling is one of the most important topics of ITS research, because the behavior of an ITS largely depends on a student model, which represents the snapshot of student's knowledge. This is a reason why many efforts concerning student modeling have been made, for instance, overlay model, buggy model, perturbation model, etc.[Wenger, 1987]. Most of the conventional modeling methods have simple pragmatic structures and have been incorporated into many ITS's. However, all the methods have some limitations and no complete and sound inference procedure for the models is obtained yet. In this paper, we formalize a student modeling problem as an inductive inference problem, that is, a problem to construct a model explaining observed data. In our case, data are student's answers and the model is student's knowledge. In order to make ITS's intelligent, student models have to satisfy the following requirements: 1. A c c u r a c y - c o s t tradeoff: In general, the more accurate the student model becomes, the more effective the behavior of the system becomes. However, there exists trade-off between accuracy of the model and cost to construct it. From a pragmatic viewpoint, we must set up an appropriate representation scheme for student models
by taking the trade-off into considerations. 2. N o n m o n o t o n i c i t y : Tutoring is to guide students toward better understanding of teaching material. This means that the learning process is essentially attained with change of their minds and hence the consistency of student's answers can be easily lost. Therefore, student modeling methods should be able to automatically manage the consistency of student's answers in order to follow the student's mind. However, there is very few attempts to formulate the nonmonotonicity of student modeling process[Burton, 1982][Huang et al., 1991a]. 3. U n k n o w n assertions: When a student fails to deduce her own solution for a problem, she would say to her teacher "I could not solve the problem". Needless to say, this assertion does not mean she does not have any knowledge. The student model module should use this assertion as informative data about her knowledge and construct a model which explains why she cannot deduce the answer from her own knowledge. This requires student model to deduce "unknown" assertions. 4. T h e o r e t i c a l f o u n d a t i o n : Domain-independent and theoretical foundation for the student modeling mechanism should be defined. It contributes to both clarification of the inherent property of student modeling problem and to articulation of the scalability and reusability of the proposed mechanism. To meet these requirements, we have developed a student model description language SMDL and a hypothetical student model inference system HSMIS. SMDL is an extended version of Prolog and takes four truth values including "unknown" to model the student precisely. HSMIS, an extended version of Shapiro's MlS[Shapiro, 1982], is an inductive inference system for SMDL. The second requirement mentioned above suggests that the inference procedure should cope with nonmonotonic modeling process. In HSMIS, ATMS: Assumption-based Truth Maintenance System [de Kleer, 1986] is employed for this purpose. HSMIS has been implemented in Common ESP(Extcnded Self-contained Prolog) on SPARC station.
2
SMDL : A Student Model Description Language
In addition to the above requirements, a student model is required to represent not only students but also sys-
Ikeda, Kono, and Mizoguchi
467
468
Intelligent Tutoring Systems
Figure 1: Block diagram of HSMIS.
Ikeda, Kono, and Mizoguchi
469
F i g u r e 2: E x a m p l e s of the top-level trace a n d the r e f u t a t i o n for a clause.
470
Intelligent Tutoring Systems
and diagnosis. In other words, HSMIS asks questions regardless of their appropriateness in the sense of tutoring. This requires some control mechanism of the HSMIS behavior. This subsection describes several additional mechanisms introduced to augment the HSMIS. V i r t u a l Oracles: Let us discuss the initial model problem. There are two alternative initial models: one is empty which means the teacher does not know anything about the student in advance and the other is complete knowledge (teaching material) which means teacher assumes the students usually understand the material very well. Although the former case is reasonable, the system tends to ask many questions to get a lot of information of how well the student understands the material. On the other hand, the latter case does not require many questions at least for excellent students, since the model can explain their correct behavior. This characteristics is very reasonable in real tutoring. Therefore, we decided to employ the latter. However, a serious problem still remains. One cannot simply put a clause into the student model without any justification. In order to cope with this problem, we devised an Virtual oracle generator, which generates plausible student answers based on the reliability of the current student model instead of asking questions. When a student's behavior is confined within the scope of her teacher's prediction, the teacher asks less questions by replacing the necessary information with correct answers. We call this type of oracle a "virtual oracles".
Ikeda, Kono, and Mizoguchi
471
472
Intelligent Tutoring Systems
should be employed. It seems p r o m i s i n g to a d o p t a belief revision mechanism developed by X. H u a n g e t . a l , w h i c h provides efficient, m i n i m a l revision of belief bases by using a t t e n t i o n ( f o c u s ) in the belief space[Huang et al, 1991b]. We have discussed mechanisms to avoid inconsistency in m o d e l inference in this paper. However, there exist such students w h o have contradictions in their head. To cope w i t h m o d e l i n g of such students, the system may not avoid the inconsistency b u t has to model inconsistent knowledge as it is. T h i s issue w i l l involve the development of more sophisticated control mechanism for student m o d e l i n g . We are c u r r e n t l y engaging in the issue, where student's inconsistent knowledge is m o d eled in m u l t i p l e worlds w h i c h are again supported by A T M S [ K o n o et al, 1992]. A c k n o w l e d g e m e n t s : T h e authors are grateful to reviewers for t h e i r valuable comments.
References
4
C o n c l u d i n g remarks
B o t h S M D L i n t e r p r e t e r and H S M I S have been i m p l e mented i n C o m m o n E S P ( E x t e n d e d Self-contained Prolog) a n d i n c o r p o r a t e d in a f r a m e w o r k for I T S , called F I T S [ M i z o g u c h i a n d I k e d a , 1 9 9 l ] . H S M I S can cope w i t h a variety of teaching materials as far as it can be represented i n P r o l o g . T h e r e f o r e , the a u t h o r s t h i n k t h a t the generality of H S M I S is relatively h i g h . T w o I T S ' s have been b u i l t using F I T S , one is on geography a n d the o t h e r is on chemical reactions. Simple e x a m i n a t i o n of b o t h systems shows they r u n in real t i m e if the oracle c o n t r a d i c t i o n does not occur. W h e n the oracle c o n t r a d i c t i o n occurs a n d there is no heuristics for its r e s o l u t i o n , we f i n d t h a t the cost to resolve the cont r a d i c t i o n is very expensive. T h e c u r r e n t i m p l e m e n t a t i o n of c o n t r a d i c t i o n resolution is based on a somewhat brute-force m e t h o d using d o m a i n - i n d e p e n d e n t heuristics. To improve the efficiency, m o r e powerful d o m a i n - d e p e n d e n t heuristics
[ B u r t o n , 1982] R. R. B u r t o n . In D. H. Sleeman, edit o r , Intelligent Tutoring Systems, pages 157-183, Academic Press, 1982. [de Kleer, 1986] J. de Kleer. An Assumption-based T r u t h M a i n t e n a n c e System. Artificial Intelligence, 28:127 162, 1986. [Huang et al, 1991a] X. H u a n g , G . I . M c C a l l a , J. E. Greer, and E. Neufeld. Revising Deductive K n o w l e d g e and Stereotypical Knowledge in Student Modeling. User Modeling and User-Adapted Interaction, 1:87-115, 1991. [ H u a n g et al, 1991b] X. H u a n g , G . I . M c C a l l a and E. Neufeld. Using A t t e n t i o n in Belief Revision. In Proceedings of the Ninth National Conference on Artificial Intellegeince, pages 275-280, 1991. [Ikeda et al, 1988] M. Ikeda, R. M i z o g u c h i and O. K a k u s h o . A H y p o t h e t i c a l M o d e l Inference Syst e m . Trans, of IEICE Japan, J71-D:1761 1771, 1988 ( i n Japanese). [Ikeda et al, 1989] M. Ikeda, R. M i z o g u c h i a n d O. K a k u s h o . Student M o d e l D e s c r i p t i o n Language S M D L and S t u d e n t M o d e l Inference System. Trans, of IEICE Japan, J 7 2 - D - 1 1 1 1 2 - 1 2 0 , 1989 ( i n Japanese). [Ikeda et al, 1992] M. Ikeda, Y. K o n o a n d R. M i z o g u c h i . N o n m o n o t o n i c M o d e l Inference. Technical Report of Artificial Intelligence Research Group, ISIR Osaka Univ., A L T R - 9 2 - 1 0 , 1992. [ K o n o et al, 1992] Y. K o n o , M. Ikeda and R. M i z o g u c h i . To c o n t r a d i c t is h u m a n — Student m o d e l i n g o f inconsistency — . I n C . Frasson, G . G a u thier, and G. I. M c C a l l a , editors, Intelligent Tutoring Systems, I T S '92, M o n t r e a l , C a n a d a , Proceedings, pages 451-458. L N C S 608, Springer-Verlag, 1992. [Mizoguchi and I k e d a , 1 9 9 l ] R. M i z o g u c h i and M . Ikeda. A Generic F r a m e w o r k for I T S A n d Its Evalu a t i o n . In R. Lewis and S. O t s u k i , editors, Advanced Research on Computers in Education, pages 6 3 - 7 2 . N o r t h - H o l l a n d , 1991. [Shapiro, 1982] E. Y. Shapiro. Algorithmic Program Debugging. M I T Press, 1982. [Wenger, 1987] E. Wenger. Artificial Intelligence and Tutoring Systems. M o r g a n K a u f m a n n Publishers, C a l i f o r n i a , 1987.
Ikeda, Kono, and Mizoguchi
473
