An Approach of Student Modelling in a Learning ... - Semantic Scholar
An Approach of Student Modelling in a Learning Companion System Rafael A. Faraco1, Marta C. Rosatelli2, Fernando A. O. Gauthier3 1
Grupo de Sistemas Inteligentes, Universidade do Sul de Santa Catarina Av. José Acácio Moreira, 787, Tubarão-SC, 88704-900, Brazil [email protected] 2 Programa de Pós-Graduação em Informática, Universidade Católica de Santos R. Dr. Carvalho de Mendonça, 144, Santos-SP, 11070-906, Brazil [email protected] 3 Departamento de Informática e Estatística, Universidade Federal Santa Catarina INE-CTC, Cx.P. 476, Campus Universitário, Florianópolis-SC, 88040-900, Brazil [email protected]
Abstract. Nowadays there is an increasing interest in the development of computational systems that provide alternative (to the traditional classroom) forms of education, such as Distance Learning (DL) and Intelligent Tutoring Systems (ITS). Adaptation in the process of interaction with the student is a key feature of ITS that is particularly critical in web-based DL, where the system should provide real-time support to a learner that most times does not rely on other kinds of synchronous feedback. This paper presents the LeCo-EAD approach of student modelling. LeCo-EAD is a Learning Companion System for web-based DL that includes three kinds of learning companions collaborator, learner, and trouble maker - that are always available to interact with and support the remote students. The student modelling approach of LeCoEAD is appropriate to the DL context as it allows updating the student model in order to provide feedback and support to the distant students in real-time.
1 Introduction Nowadays, there is an increasing interest in the development of computational systems that provide alternative forms of education, such as Distance Learning (DL) and Intelligent Tutoring Systems (ITS). ITS are able to (1) modify its knowledge bases by interacting with the students; (2) perceive the students’ characteristics, preferences, and learning pace; and (3) adapt its pedagogical strategies to both individual and groups of students, according to the different learning situations. Adaptation in the process of interaction with the student is a key feature of ITS. In order to provide adapted interactions, such systems need to keep updated about the state of the student knowledge level. This is accomplished by modelling the student. Student modelling and adaptation is particularly critical in web-based DL, where the system should provide real-time support to a learner that most times does not rely on other kinds of synchronous feedback (i.e., where other kinds of support usually take place in an asynchronous mode of interaction).
A kind of system that is considered an evolution of ITS and might be of relevance to DL are Learning Companion Systems (LCS). LCS were initially proposed by [6] and their architecture includes a learning companion (LC) as an additional component. The role of a LC is to be a virtual peer for the human student, interacting with him or her in order to collaborate with the learning process similarly as a real student does. This characteristic of LCS is appropriate to DL as the remote student usually accomplishes the learning activities alone. This paper presents LeCo-EAD student modelling approach. LeCo-EAD is a LCS for web-based DL that includes multiples LCs that are always available to support the remote students. LeCo-EAD student modelling approach is appropriate to DL as it allows updating in order to provide real-time feedback and support to the distant students. The paper is organized as follows. Section 2 discusses related work on student modelling in LCS. Section 3 presents an overview of LeCo-EAD. Section 4 describes LeCo-EAD student modelling components, functionality, and updating mechanism. Section 5 discusses the Artificial Intelligence (AI) technique used for student modelling and reports briefly on a formative evaluation of the system. Finally, Section 6 presents the conclusions.
2 Related Work Among LCS there is no general, widely accepted model, or standard for the definition of the structure and functioning of student models. Usually, these are specially designed and built for each particular system, considering the domain; the number, kinds, and functionality of the LCs; and the computational constraints. For instance, LECOBA [20] is a LCS developed in the domain of Binary Boolean Algebra and is based on the idea of inspectable student models. The system was implemented with the aim to show that less capable LCs could be beneficial for the learning of real students. In this system, the students have the chance to teach the LC, according to the approach of learning by teaching. In order to accomplish this, LECOBA provides a teaching window where the system presents the LC student model to the real student. This window aims to be a reflection tool, promoting thinking about the student knowledge on the domain. He or she is supposed to identify what the LC does not know and teach the LC these topics or concepts. Another example is a reciprocal tutoring system in the domain of Lisp, in which a LC agent assumes the role of tutor and tutee in solving the problems [7]. The agents in the system can be real students or LC. When the LC behaves like a tutee, the system uses an overlay student model, and the agent pretends to learn in a level that is close to the real student level. When the system plays the role of tutor, the LC has a student model that keeps track of the student’s actions. According to Chan and colleagues [8], the student model in LCS can be of tree types: i) a single student model that provides information for the whole system; ii) a single student model and each learning companion has its own interpretation of the model; iii) each learning companion has its own student model.
Devedzic and Harrer [11] consider LCS in agent-based architectures of AI in Education systems that include pedagogical agents, by focussing on the co-learner pattern. The software pattern identified by the authors consists of a generalization of systems that were developed so far and have three main components: tutor, student, and co-learner. In these systems, each co-learner has its own student model. The advantage is a greater independence in the behaviour of the co-learners. The student model in LeCo-EAD is a single one. Each LC has its own interpretation of this model and reacts in a different way, according to its role and/or type. Compared to the first approach pointed out by Chan et al. [8], this (second) type of student model is more flexible as it allows that each LC has its own behaviour, based on the same information about the student. Regarding the software pattern pointed out by [11] the complexity of updating several student models can be considered a constraint in the case of web-based systems like LeCo-EAD, as these might have to handle several students simultaneously connected to it,
3 LeCo-EAD Overview LeCo-EAD is a LCS [9] that includes three kinds of LCs, which are always available to interact with the students. Each LC presents a particular kind of behaviour, handling different teaching strategies that are represented by the following types of LCs: collaborator [5], learner [20, 18], and trouble maker [1]. The LC that will interact with the student is, at first, chosen by the system, based on the student profile, which is identified through a Likert (or attitude) scale [16]. LeCo-EAD is domain-independent, as the contents of the didactic materials and the feedback messages can be easily replaced by the teacher and/or instructional designer to be used in different domains. The system architecture includes the modules of the traditional ITS (namely the domain module, tutor model, student model, and graphical user interface) with the addition of the LCs. Figure 1 presents LeCo-EAD architecture and its components are detailed below. 3.1 Domain Module The domain module includes the contents that are presented to the student on the web pages and is structured as conceptual maps (see Figure 2 for an example). The objective of a conceptual map is to contextualize the subject matter attributing meaning to the topics studied [2, 4]. In LeCo-EAD the concepts included in the map should be studied one by one. A set of exercises is associated to each concept and the students are supposed to work on them. Depending on the student score, other concepts of the map are liberated, according to a pre-requisites structure. A degree of importance (given by a percentage) is attributed to each concept in this structure. The exercises and respective answers also possess relative weights: the knowledge factors (KFs). The KF is based on the certainty factor model [12] and represent the degree of importance of each exercise to a particular concept. That is, the higher the value of an exercise KF, the more representative this exercise is for mastering the related concept.
3.2 Tutor Model LeCo-EAD tutor model handles the pedagogical decisions, which are made in accordance to the individual needs of each student. Thus, based on the pre-requisites structure, the tutor model manages the contents sequencing, deciding which topic should be presented to the student and when, and liberating the concepts represented in the conceptual map as the students reaches a satisfactory score to progress from one level to the next one.
Fig. 1. The LeCo-EAD architecture
3.3 Student Model Amongst the techniques used to represent the student knowledge, LeCo-EAD uses the perturbation or buggy student model [15]. This kind of model is an extension of the overlay model, which includes a subset of the expert’s knowledge as well as possible student misconceptions. These misconceptions are represented by the exercises alternatives that are relative to each problem but are incorrect. The closer the weight of the alternative is to -1.0, the more it indicates a student lack of knowledge if he or she chooses this particular alternative as the exercise response. Thus, for each concept included in the conceptual maps, the system represents (1) if the student knows or does not know it and (2) a quantitative measure of how much he or she knows about it. Hence, besides mapping the correct knowledge, the system handles the student incorrect knowledge through the KF negative values in the various incorrect answers situations. Each student model includes information about the student performance in solving the exercises, the tracing of the student’s accesses to the system, and the kind of LC attributed to the student based on the Likert scale.
3.4 Learning Companions LCs are artificial students that may interact with real students in different ways: guiding, behaving like a co-learner or, yet, provoking the student. In LeCo-EAD each kind of LC (collaborator, learner, or trouble maker) has a graphical representation related with its behaviour and the student actively participates in the process of choosing his or her LC through a Likert scale [16]. The Likert scale indicates the students’ predisposition in relation to objects, people, or events and was elaborated according to the following steps: (1) theory review, elaboration of the initial set of scale items, and reduction of items through a qualitative analysis; (2) collection of a pre-test sample, and pre-test; (3) statistics analysis; and (4) final scale elaboration [13]. The scale consists of an electronic form (questionnaire) that must be filled out by the student in the first time that he or she logs in the system. It aims to identify which kind of LC available in the system is appropriate to a particular student. 10%
20%
10%
is c omposed by Object
Class
send
Message
has allow
is a 10% Structures
10%
Instance 10%
Inheritanc e is a is a
is a
W hole-part Gen-Spec
10%
20%
Concept
Gen-Spec
0.8 Questions
0.1
0.6
0.4
Question Question Question Question 1
2
3
-1.0 Answers
20%
-0.8 A
4
1.0
1.0 0.6 -1.0 B
C
D
A
B
Fig. 2. A conceptual map in the domain of Object-Orientation
To each type of LC is associated a set of feedback messages that are presented to the student in two situations, namely: (1) when the system identifies that the student
needs support to solve a problem, recognized when he or she exceeds a preestablished time limit to present an answer to an exercise; or (2) when the system identifies a misunderstanding in the answer given by the student.
4 Updating the Student Model In LeCo-EAD, the student model is an overlay of the concepts, questions, and answers structure represented in the domain model. The process of updating the student model consists of updating the KFs based on the scores of the exercises presented in association with each concept and solved by the student. Updating the knowledge hierarchy is made by an algorithm that starts with the leaf-nodes (answer situation) and progress to the root (course) of the tree. The manipulation of the KFs is based on rules of the kind If evidence Then hypothesis. There is a KF for the evidence (KF(evidence)) and for the hypothesis to be accepted (KF(hypothesis)). The degree of certainty [12] for rules of the kind evidence/hypothesis is given by equation (1):
KF (evidence / hypothesis) = KF (evidence) * KF ( hypopthesis ) . (1) The rules of the kind evidence/hypothesis with respective KFs are applied to each pair answer/question situation of the contents hierarchy. The rules of the hierarchy presented on Figure 2 are detailed in Table 1. Table 1. Rules of the hierarchy presented on Figure 2
Quest.1
Quest.2
Rule1.1: If the student chooses option “A” => (KF(A) = 1) Then the student almost does not know question => (KF(almost does not know) = - 0.8) KF(Rule1.1) = KF(A)*KF(almost does not know) = -0.8 Rule1.2: If the student chooses option “B” => (KF(B) = 1) Then the student definitely knows question => (KF(definitely knows)= 1) KF(Rule1.2) = KF(B)*KF(definitively knows) = 1 Rule1.3: If the student chooses options “C” => (KF(C) = 1) Then the student probably knows question => (KF(probably knows) = 0.6) KF(Rule1.3) = KF(C)*KF(probably knows) = 0.6 Rule1.4: If the student chooses option “D” => (KF(D) = 1) Then the student definitely does not know question => (KF(definitely does not know) = -1) KF(Rule1.4) = KF(D)*KF(definitively does not know) = -1 Rule2.1: If the student chooses option “A” => (KF(A) = 1) Then the student definitely does not know question => (KF(definitely does not know) = -1) KF(Rule2.1) = KF(A)*KF(definitively does not know) = - 1 Rule 2.2: If the student chooses option “B” => (KF(B) = 1) Then the student definitely knows question => (KF(definitely knows) = 1) KF(Rule2.2) = KF(B)*KF(definitively knows) = 1
The KF values used in the concepts hierarchy follow a linguistic representation according to Table2. Table 2. Values attributed to linguistic meanings
Linguistic Meaning Definitely does not contribute/know Almost does not contribute/know Probably does not contribute Maybe does not contribute No information available Maybe contributes Probably contributes/knows Almost certainly contributes/knows Definitely contributes/knows
Value -1 -0.8 -0.6 -0.4 -0.2 to 0.2 0.4 0.6 0.8 1
As usually there are several questions related to a concept, the value of each rule referring to a particular concept should be incorporated as the student answers its related questions [12]. For handling multiple evidences referring to the same hypothesis, we consider: KF ( KF , KF ) = KF + KF x (1 − KF ) , If KF and KF are > 0 1 2 1 2 1 1 2 KF + KF 1 2 KF ( KF , KF ) = , If KF or KF are < 0 1 2 1 2 1 − min KF , KF 1 2
{
}
KF ( KF , KF ) = KF + KF x (1 + KF ) , If KF and KF are < 0 1 2 1 2 1 1 2 where: KF = Rule 1 knowledge factor; 1 KF = Rule 2 knowledge factor; and 2 KF ( KF , KF ) = Knowledge factor resulting from Rule 1 combined with Rule 2. 1 2 Intuitively, when the KF values are positive, there is an increase in the weight that results from the combination of the two rules. When they have different signs there is a decrease in the resulting weight, tending to zero. When the KF values are negative there is a decrease in the resulting weight, tending to -1.0. In any case, the resulting value of the KFs’ manipulation ranges from -1.0 to 1.0. Still considering the content hierarchy presented on Figure 2, updating the student model regarding the student knowledge about a concept (e.g., concept 1), is as follow: 1. The student model is initialized with zeros, indicating the student performance so far. That is, concerning LeCo-EAD, the student still does not know any concepts as he or she has not interacted with the system yet. 2. When the student chooses option “C” of question 1, Rule 1.3 is fired: KF(question1) = KF(C)*KF(probably knows) = 0.6 KF(question1/concept1) = 0.6*0.8 = 0.048
3. When the student chooses option “A” of question 2, Rule 2.1 is fired: KF(question2 ) = KF(A)*KF(definitely does not know) = -1 KF(question2/concept1) = -1 * 0.1 = -0.1 4. As there are two rules applied to a single concept: KF(question1/concept1) = 0.48 KF(question2/concept1) = -0.1 0 . 48 + ( − 0 . 1) KF ( concept 1) = 1 − min {0 . 48 , 0 . 1}
KF(concept1) = 0.38/0.9 = 0.42 5. The student model registers that the student masters concept 1 (Gen-Spec) with a value of 0.42.
5 Discussion The use of KFs in LeCo-EAD student model is based on the certainty factor model initially proposed for handling uncertainty in [3]. In spite of the fact that this model was criticised for not having a solid mathematical and probabilistic basis, it is computationally simple to implement, intuitive, and easy to manipulate [14]. In order to verify the adequacy of using KFs in LeCo-EAD, we carried out simulations considering the content hierarchies of the domain module with different number of levels. The first hierarchy considered was organized in four levels. The first level corresponded to the course and the levels below were units, concepts, and questions respectively. In the second case, we used a hierarchy that included only concepts and related questions. The certainty factor model did not present a satisfactory behaviour when the content hierarchy included many levels. The bigger the number of levels, the deeper the rules chaining mechanism has to operate. As a result, the final value of the knowledge accumulated for the student was distorted in deeper levels. In order to avoid this problem in LeCo-EAD inference mechanism, we limited the content hierarchy to two levels (as in the second type of contents hierarchy that we tested): concepts and related questions. We believe that more robust techniques, such as Bayesian networks, could be used for modelling and handling the student knowledge. On the other hand, the computational complexity of this particular technique [10] might not be appropriate for a web-based learning system that has to provide real-time feedback to the user. In addition, Bayesian techniques depend strongly on prior probabilities, which are often not available, to initialize the student model [11]. Besides the process of updating the KFs for representing the student performance, LeCo-EAD also keeps track of the students’ following actions within the system: the concepts that he or she studied, the questions answered, the time of access, the time of permanence at each web page, the LC initially indicated by the Likert scale, and if the student requests changing the type of LC. In summary, student modelling in LeCo-EAD generates two main outputs: the first one refers to the student learning represented by the KF values, which vary between
the range of -1.0 (the student did not master any of the concepts) and 1.0 (the student demonstrates that mastered all the concepts). A value next to zero indicates that the student answered the questions with alternatives that do not measure efficiently if he or she knows or does not know a particular concept. Leaving an exercise response blank also has this same effect. The second kind of output regards keeping track of the student’s actions, which prevents that LeCo-EAD repeats the concepts, exercises, and messages that were already presented to the student. Besides, all the interaction sessions of the student with the system are logged. Regarding system evaluation, we carried out an empirical study as a formative evaluation, similarly as [17] and [19]. LeCo-EAD prototype was tested by a group of 5 teachers and 20 students of a Computer Science undergraduate course on Object Oriented Programming in Java. The aim was to test the system functionalities and the student model operation regarding providing the input for the system adaptation. The formative evaluation brought to light some issues related with the system adaptivity. LeCo-EAD mechanism to choose the LC was considered sufficiently clear and direct by the empirical study participants, since the user participation in this process is explicit, either through the Likert scale or the feedback mechanism. This mechanism has a low demand upon the student in terms of the interaction: he or she just fills out the form of the attitude scale once in the beginning of the course, and afterwards controls the acceptance of the LC when closing the messages window. In addition, the conceptual map adaptation based on the student performance was considered appropriate and worked as expected.
6 Conclusions In this paper we presented the student modelling approach of LeCo-EAD. LeCo-EAD is a LCS for DL. It includes three types of LCs that adopt different pedagogical strategies and are always available to interact with the students via the web. The process of modelling the student in LeCo-EAD allows direct and quick updating, what is a desirable feature in web-based learning environments. It aims to monitor the student performance during the interaction with the didactic materials, keeping track of his or her actions. The former, i.e., monitoring the student performance, is carried out by updating the student model based on the inference mechanism of an expert system that uses the certainty factor model. The latter, i.e., keeping track of the student actions, is registered by the student model.
References [1]
[2]
Aimeur, E., Dufort, H., Leibu, D., Frasson, C.: Some Justifications for the Learning by Disturbing Paradigm. In: Du Boulay, B., Mizoguchi, R. (eds.): Proc. of 8th International Conference on AI in Education. IOS Press, Amsterdam (1997) 119-126 Ausubel, D.: Educational Psychology: A Cognitive View. Holt, Rinehart & Winston, New York (1968)
[3] [4]
[5] [6]
[7]
[8]
[9]
[10] [11]
[12] [13]
[14] [15]
[16] [17]
[18] [19] [20]
[21]
Buchanan, B.G., Shortliffe, E.H.: Rule-Based Expert Systems. Addison-Wesley, Reading, (1984) Cañas, A.J., Ford, K.M., Brennan, J., Reichherzer, T., Hayes, P.: Knowledge Construction and Sharing in Quorum. In: J. Greer (ed.): Proc. of 7th World Conference on AI in Education. AACE, Charlottesville (1995) 228-225 Chan, T.W.: Learning Companion Systems, Social Learning Systems, and Global Social Learning Club. International Journal of AI in Education 7(2) (1996) 125-159 Chan, T.W., Baskin, A.B.: Learning Companion Systems. In: Frasson, C., Gauthier G. (eds.): Intelligent Tutoring Systems: At the Crossroads of AI and Education. Ablex Publishing Corporation, New Jersey (1990) Cap. 1 Chan, T.W., Chou, C.Y.: Simulating a Learning Companion in Reciprocal Tutoring Systems. In: Proc. of Computer Support for Collaborative Learning’95 (1995). Available at: http://www-cscl95.indiana.edu/cscl95/chan.html. Chan, T.W., Chou, C.Y., Lin, C.J.: User Modeling in Simulating Learning Companions. In: Vivet, M., Lajoie, S. (eds.): Proc. of 9th International Conference on AI in Education. IOS Press, Amsterdam (1999) 277-284 Chou, C.Y., Chan, T.W., Lin, C.W.: Redefining the Learning Companion: The Past, Present, and Future of Educational Agents. Computers & Education 40(3) (2003) 255269 Coper, G.F.: The Computational Complexity of Probabilistic Inference Using Bayesian Belief Networks. AI Journal 42 (1997) 393-405 Devedzic, V., Harrer, A.: Architectural Patterns in Pedagogical Agents. In: Cerri, S. Gouarderes, G., Paraguaçu, F. (eds.): Intelligent Tutoring Systems. LNCS 2363. Springer-Verlag, Berlin (2002) 91-90 Durkin, J.: Expert Systems: Design and Development. Prentice-Hall, New York (1994) Faraco, R.A., Rosatelli, M.C., Gauthier, F.A.: Adaptivity in a Learning Companion System. In: Proc. of the 4th IEEE International Conference on Advanced Learning Technologies. IEEE Press (2004) in press van der Gaag, L.C.: A Pragmatical View on the Certainty Factor Model. The International Journal of Expert Systems: Research and Applications 7 (3) (1994) 289-300 Holt, P., Dubs, S., Jones, M., Greer, J.: The State of Student Modelling. In: Greer, J., McCalla, G. (eds.): Student Modelling: The Key To Individualized Knowledge-Based Instruction. Springer-Verlag, Berlin (1994) 3-35 Likert, R.: A Technique for the Measurement of Attitudes. Archives of Psychology (1932) Vol. 140 Meizalo, V., Torvinen, C., Suhonen, J., Sutinen, E.: Formative Evaluation Scheme for a Web-Based Course Design. In: Proc. of 7th Annual Conference on Innovation and Technology in Computer Science Education. ACM Press, New York (2002) 130-134 Nichols, D.: Issues in Design Learning by Teaching Systems. AAI-AI-ED Technical Report n. 107. Computing Department, Lancaster University, Lancaster (1994) Tedesco, P.: MArCo: Building an Artificial Conflict Mediator to Support Group Planning Interactions. International Journal of AI in Education 13 (2003) 117-155 Uresti, J.A.R.: Should I Teach My Computer Peer? Some Issues in Teaching a Learning Companion. In: Frasson, C., Gauthier, G., VanLenh, K. (eds.): Intelligent Tutoring Systems. LNCS 1839. Springer-Verlag, Berlin (2000) 103-112 VanLehn, K., Niu, Z., Siler, S. Gertner. A.S.: Student Modeling from Conventional Test Data: A Bayesian Approach without Priors. In: Goettl, B.P, Halff, H.M., Redfield, C.L., Shute, V.J. (eds.): Intelligent Tutoring Systems. LNCS 1452. Springer-Verlag, Berlin (1998) 434-443
Grupo de Sistemas Inteligentes, Universidade do Sul de Santa Catarina Av. José Acácio Moreira, 787, Tubarão-SC, 88704-900, Brazil [email protected] 2 Programa de Pós-Graduação em Informática, Universidade Católica de Santos R. Dr. Carvalho de Mendonça, 144, Santos-SP, 11070-906, Brazil [email protected] 3 Departamento de Informática e Estatística, Universidade Federal Santa Catarina INE-CTC, Cx.P. 476, Campus Universitário, Florianópolis-SC, 88040-900, Brazil [email protected]
Abstract. Nowadays there is an increasing interest in the development of computational systems that provide alternative (to the traditional classroom) forms of education, such as Distance Learning (DL) and Intelligent Tutoring Systems (ITS). Adaptation in the process of interaction with the student is a key feature of ITS that is particularly critical in web-based DL, where the system should provide real-time support to a learner that most times does not rely on other kinds of synchronous feedback. This paper presents the LeCo-EAD approach of student modelling. LeCo-EAD is a Learning Companion System for web-based DL that includes three kinds of learning companions collaborator, learner, and trouble maker - that are always available to interact with and support the remote students. The student modelling approach of LeCoEAD is appropriate to the DL context as it allows updating the student model in order to provide feedback and support to the distant students in real-time.
1 Introduction Nowadays, there is an increasing interest in the development of computational systems that provide alternative forms of education, such as Distance Learning (DL) and Intelligent Tutoring Systems (ITS). ITS are able to (1) modify its knowledge bases by interacting with the students; (2) perceive the students’ characteristics, preferences, and learning pace; and (3) adapt its pedagogical strategies to both individual and groups of students, according to the different learning situations. Adaptation in the process of interaction with the student is a key feature of ITS. In order to provide adapted interactions, such systems need to keep updated about the state of the student knowledge level. This is accomplished by modelling the student. Student modelling and adaptation is particularly critical in web-based DL, where the system should provide real-time support to a learner that most times does not rely on other kinds of synchronous feedback (i.e., where other kinds of support usually take place in an asynchronous mode of interaction).
A kind of system that is considered an evolution of ITS and might be of relevance to DL are Learning Companion Systems (LCS). LCS were initially proposed by [6] and their architecture includes a learning companion (LC) as an additional component. The role of a LC is to be a virtual peer for the human student, interacting with him or her in order to collaborate with the learning process similarly as a real student does. This characteristic of LCS is appropriate to DL as the remote student usually accomplishes the learning activities alone. This paper presents LeCo-EAD student modelling approach. LeCo-EAD is a LCS for web-based DL that includes multiples LCs that are always available to support the remote students. LeCo-EAD student modelling approach is appropriate to DL as it allows updating in order to provide real-time feedback and support to the distant students. The paper is organized as follows. Section 2 discusses related work on student modelling in LCS. Section 3 presents an overview of LeCo-EAD. Section 4 describes LeCo-EAD student modelling components, functionality, and updating mechanism. Section 5 discusses the Artificial Intelligence (AI) technique used for student modelling and reports briefly on a formative evaluation of the system. Finally, Section 6 presents the conclusions.
2 Related Work Among LCS there is no general, widely accepted model, or standard for the definition of the structure and functioning of student models. Usually, these are specially designed and built for each particular system, considering the domain; the number, kinds, and functionality of the LCs; and the computational constraints. For instance, LECOBA [20] is a LCS developed in the domain of Binary Boolean Algebra and is based on the idea of inspectable student models. The system was implemented with the aim to show that less capable LCs could be beneficial for the learning of real students. In this system, the students have the chance to teach the LC, according to the approach of learning by teaching. In order to accomplish this, LECOBA provides a teaching window where the system presents the LC student model to the real student. This window aims to be a reflection tool, promoting thinking about the student knowledge on the domain. He or she is supposed to identify what the LC does not know and teach the LC these topics or concepts. Another example is a reciprocal tutoring system in the domain of Lisp, in which a LC agent assumes the role of tutor and tutee in solving the problems [7]. The agents in the system can be real students or LC. When the LC behaves like a tutee, the system uses an overlay student model, and the agent pretends to learn in a level that is close to the real student level. When the system plays the role of tutor, the LC has a student model that keeps track of the student’s actions. According to Chan and colleagues [8], the student model in LCS can be of tree types: i) a single student model that provides information for the whole system; ii) a single student model and each learning companion has its own interpretation of the model; iii) each learning companion has its own student model.
Devedzic and Harrer [11] consider LCS in agent-based architectures of AI in Education systems that include pedagogical agents, by focussing on the co-learner pattern. The software pattern identified by the authors consists of a generalization of systems that were developed so far and have three main components: tutor, student, and co-learner. In these systems, each co-learner has its own student model. The advantage is a greater independence in the behaviour of the co-learners. The student model in LeCo-EAD is a single one. Each LC has its own interpretation of this model and reacts in a different way, according to its role and/or type. Compared to the first approach pointed out by Chan et al. [8], this (second) type of student model is more flexible as it allows that each LC has its own behaviour, based on the same information about the student. Regarding the software pattern pointed out by [11] the complexity of updating several student models can be considered a constraint in the case of web-based systems like LeCo-EAD, as these might have to handle several students simultaneously connected to it,
3 LeCo-EAD Overview LeCo-EAD is a LCS [9] that includes three kinds of LCs, which are always available to interact with the students. Each LC presents a particular kind of behaviour, handling different teaching strategies that are represented by the following types of LCs: collaborator [5], learner [20, 18], and trouble maker [1]. The LC that will interact with the student is, at first, chosen by the system, based on the student profile, which is identified through a Likert (or attitude) scale [16]. LeCo-EAD is domain-independent, as the contents of the didactic materials and the feedback messages can be easily replaced by the teacher and/or instructional designer to be used in different domains. The system architecture includes the modules of the traditional ITS (namely the domain module, tutor model, student model, and graphical user interface) with the addition of the LCs. Figure 1 presents LeCo-EAD architecture and its components are detailed below. 3.1 Domain Module The domain module includes the contents that are presented to the student on the web pages and is structured as conceptual maps (see Figure 2 for an example). The objective of a conceptual map is to contextualize the subject matter attributing meaning to the topics studied [2, 4]. In LeCo-EAD the concepts included in the map should be studied one by one. A set of exercises is associated to each concept and the students are supposed to work on them. Depending on the student score, other concepts of the map are liberated, according to a pre-requisites structure. A degree of importance (given by a percentage) is attributed to each concept in this structure. The exercises and respective answers also possess relative weights: the knowledge factors (KFs). The KF is based on the certainty factor model [12] and represent the degree of importance of each exercise to a particular concept. That is, the higher the value of an exercise KF, the more representative this exercise is for mastering the related concept.
3.2 Tutor Model LeCo-EAD tutor model handles the pedagogical decisions, which are made in accordance to the individual needs of each student. Thus, based on the pre-requisites structure, the tutor model manages the contents sequencing, deciding which topic should be presented to the student and when, and liberating the concepts represented in the conceptual map as the students reaches a satisfactory score to progress from one level to the next one.
Fig. 1. The LeCo-EAD architecture
3.3 Student Model Amongst the techniques used to represent the student knowledge, LeCo-EAD uses the perturbation or buggy student model [15]. This kind of model is an extension of the overlay model, which includes a subset of the expert’s knowledge as well as possible student misconceptions. These misconceptions are represented by the exercises alternatives that are relative to each problem but are incorrect. The closer the weight of the alternative is to -1.0, the more it indicates a student lack of knowledge if he or she chooses this particular alternative as the exercise response. Thus, for each concept included in the conceptual maps, the system represents (1) if the student knows or does not know it and (2) a quantitative measure of how much he or she knows about it. Hence, besides mapping the correct knowledge, the system handles the student incorrect knowledge through the KF negative values in the various incorrect answers situations. Each student model includes information about the student performance in solving the exercises, the tracing of the student’s accesses to the system, and the kind of LC attributed to the student based on the Likert scale.
3.4 Learning Companions LCs are artificial students that may interact with real students in different ways: guiding, behaving like a co-learner or, yet, provoking the student. In LeCo-EAD each kind of LC (collaborator, learner, or trouble maker) has a graphical representation related with its behaviour and the student actively participates in the process of choosing his or her LC through a Likert scale [16]. The Likert scale indicates the students’ predisposition in relation to objects, people, or events and was elaborated according to the following steps: (1) theory review, elaboration of the initial set of scale items, and reduction of items through a qualitative analysis; (2) collection of a pre-test sample, and pre-test; (3) statistics analysis; and (4) final scale elaboration [13]. The scale consists of an electronic form (questionnaire) that must be filled out by the student in the first time that he or she logs in the system. It aims to identify which kind of LC available in the system is appropriate to a particular student. 10%
20%
10%
is c omposed by Object
Class
send
Message
has allow
is a 10% Structures
10%
Instance 10%
Inheritanc e is a is a
is a
W hole-part Gen-Spec
10%
20%
Concept
Gen-Spec
0.8 Questions
0.1
0.6
0.4
Question Question Question Question 1
2
3
-1.0 Answers
20%
-0.8 A
4
1.0
1.0 0.6 -1.0 B
C
D
A
B
Fig. 2. A conceptual map in the domain of Object-Orientation
To each type of LC is associated a set of feedback messages that are presented to the student in two situations, namely: (1) when the system identifies that the student
needs support to solve a problem, recognized when he or she exceeds a preestablished time limit to present an answer to an exercise; or (2) when the system identifies a misunderstanding in the answer given by the student.
4 Updating the Student Model In LeCo-EAD, the student model is an overlay of the concepts, questions, and answers structure represented in the domain model. The process of updating the student model consists of updating the KFs based on the scores of the exercises presented in association with each concept and solved by the student. Updating the knowledge hierarchy is made by an algorithm that starts with the leaf-nodes (answer situation) and progress to the root (course) of the tree. The manipulation of the KFs is based on rules of the kind If evidence Then hypothesis. There is a KF for the evidence (KF(evidence)) and for the hypothesis to be accepted (KF(hypothesis)). The degree of certainty [12] for rules of the kind evidence/hypothesis is given by equation (1):
KF (evidence / hypothesis) = KF (evidence) * KF ( hypopthesis ) . (1) The rules of the kind evidence/hypothesis with respective KFs are applied to each pair answer/question situation of the contents hierarchy. The rules of the hierarchy presented on Figure 2 are detailed in Table 1. Table 1. Rules of the hierarchy presented on Figure 2
Quest.1
Quest.2
Rule1.1: If the student chooses option “A” => (KF(A) = 1) Then the student almost does not know question => (KF(almost does not know) = - 0.8) KF(Rule1.1) = KF(A)*KF(almost does not know) = -0.8 Rule1.2: If the student chooses option “B” => (KF(B) = 1) Then the student definitely knows question => (KF(definitely knows)= 1) KF(Rule1.2) = KF(B)*KF(definitively knows) = 1 Rule1.3: If the student chooses options “C” => (KF(C) = 1) Then the student probably knows question => (KF(probably knows) = 0.6) KF(Rule1.3) = KF(C)*KF(probably knows) = 0.6 Rule1.4: If the student chooses option “D” => (KF(D) = 1) Then the student definitely does not know question => (KF(definitely does not know) = -1) KF(Rule1.4) = KF(D)*KF(definitively does not know) = -1 Rule2.1: If the student chooses option “A” => (KF(A) = 1) Then the student definitely does not know question => (KF(definitely does not know) = -1) KF(Rule2.1) = KF(A)*KF(definitively does not know) = - 1 Rule 2.2: If the student chooses option “B” => (KF(B) = 1) Then the student definitely knows question => (KF(definitely knows) = 1) KF(Rule2.2) = KF(B)*KF(definitively knows) = 1
The KF values used in the concepts hierarchy follow a linguistic representation according to Table2. Table 2. Values attributed to linguistic meanings
Linguistic Meaning Definitely does not contribute/know Almost does not contribute/know Probably does not contribute Maybe does not contribute No information available Maybe contributes Probably contributes/knows Almost certainly contributes/knows Definitely contributes/knows
Value -1 -0.8 -0.6 -0.4 -0.2 to 0.2 0.4 0.6 0.8 1
As usually there are several questions related to a concept, the value of each rule referring to a particular concept should be incorporated as the student answers its related questions [12]. For handling multiple evidences referring to the same hypothesis, we consider: KF ( KF , KF ) = KF + KF x (1 − KF ) , If KF and KF are > 0 1 2 1 2 1 1 2 KF + KF 1 2 KF ( KF , KF ) = , If KF or KF are < 0 1 2 1 2 1 − min KF , KF 1 2
{
}
KF ( KF , KF ) = KF + KF x (1 + KF ) , If KF and KF are < 0 1 2 1 2 1 1 2 where: KF = Rule 1 knowledge factor; 1 KF = Rule 2 knowledge factor; and 2 KF ( KF , KF ) = Knowledge factor resulting from Rule 1 combined with Rule 2. 1 2 Intuitively, when the KF values are positive, there is an increase in the weight that results from the combination of the two rules. When they have different signs there is a decrease in the resulting weight, tending to zero. When the KF values are negative there is a decrease in the resulting weight, tending to -1.0. In any case, the resulting value of the KFs’ manipulation ranges from -1.0 to 1.0. Still considering the content hierarchy presented on Figure 2, updating the student model regarding the student knowledge about a concept (e.g., concept 1), is as follow: 1. The student model is initialized with zeros, indicating the student performance so far. That is, concerning LeCo-EAD, the student still does not know any concepts as he or she has not interacted with the system yet. 2. When the student chooses option “C” of question 1, Rule 1.3 is fired: KF(question1) = KF(C)*KF(probably knows) = 0.6 KF(question1/concept1) = 0.6*0.8 = 0.048
3. When the student chooses option “A” of question 2, Rule 2.1 is fired: KF(question2 ) = KF(A)*KF(definitely does not know) = -1 KF(question2/concept1) = -1 * 0.1 = -0.1 4. As there are two rules applied to a single concept: KF(question1/concept1) = 0.48 KF(question2/concept1) = -0.1 0 . 48 + ( − 0 . 1) KF ( concept 1) = 1 − min {0 . 48 , 0 . 1}
KF(concept1) = 0.38/0.9 = 0.42 5. The student model registers that the student masters concept 1 (Gen-Spec) with a value of 0.42.
5 Discussion The use of KFs in LeCo-EAD student model is based on the certainty factor model initially proposed for handling uncertainty in [3]. In spite of the fact that this model was criticised for not having a solid mathematical and probabilistic basis, it is computationally simple to implement, intuitive, and easy to manipulate [14]. In order to verify the adequacy of using KFs in LeCo-EAD, we carried out simulations considering the content hierarchies of the domain module with different number of levels. The first hierarchy considered was organized in four levels. The first level corresponded to the course and the levels below were units, concepts, and questions respectively. In the second case, we used a hierarchy that included only concepts and related questions. The certainty factor model did not present a satisfactory behaviour when the content hierarchy included many levels. The bigger the number of levels, the deeper the rules chaining mechanism has to operate. As a result, the final value of the knowledge accumulated for the student was distorted in deeper levels. In order to avoid this problem in LeCo-EAD inference mechanism, we limited the content hierarchy to two levels (as in the second type of contents hierarchy that we tested): concepts and related questions. We believe that more robust techniques, such as Bayesian networks, could be used for modelling and handling the student knowledge. On the other hand, the computational complexity of this particular technique [10] might not be appropriate for a web-based learning system that has to provide real-time feedback to the user. In addition, Bayesian techniques depend strongly on prior probabilities, which are often not available, to initialize the student model [11]. Besides the process of updating the KFs for representing the student performance, LeCo-EAD also keeps track of the students’ following actions within the system: the concepts that he or she studied, the questions answered, the time of access, the time of permanence at each web page, the LC initially indicated by the Likert scale, and if the student requests changing the type of LC. In summary, student modelling in LeCo-EAD generates two main outputs: the first one refers to the student learning represented by the KF values, which vary between
the range of -1.0 (the student did not master any of the concepts) and 1.0 (the student demonstrates that mastered all the concepts). A value next to zero indicates that the student answered the questions with alternatives that do not measure efficiently if he or she knows or does not know a particular concept. Leaving an exercise response blank also has this same effect. The second kind of output regards keeping track of the student’s actions, which prevents that LeCo-EAD repeats the concepts, exercises, and messages that were already presented to the student. Besides, all the interaction sessions of the student with the system are logged. Regarding system evaluation, we carried out an empirical study as a formative evaluation, similarly as [17] and [19]. LeCo-EAD prototype was tested by a group of 5 teachers and 20 students of a Computer Science undergraduate course on Object Oriented Programming in Java. The aim was to test the system functionalities and the student model operation regarding providing the input for the system adaptation. The formative evaluation brought to light some issues related with the system adaptivity. LeCo-EAD mechanism to choose the LC was considered sufficiently clear and direct by the empirical study participants, since the user participation in this process is explicit, either through the Likert scale or the feedback mechanism. This mechanism has a low demand upon the student in terms of the interaction: he or she just fills out the form of the attitude scale once in the beginning of the course, and afterwards controls the acceptance of the LC when closing the messages window. In addition, the conceptual map adaptation based on the student performance was considered appropriate and worked as expected.
6 Conclusions In this paper we presented the student modelling approach of LeCo-EAD. LeCo-EAD is a LCS for DL. It includes three types of LCs that adopt different pedagogical strategies and are always available to interact with the students via the web. The process of modelling the student in LeCo-EAD allows direct and quick updating, what is a desirable feature in web-based learning environments. It aims to monitor the student performance during the interaction with the didactic materials, keeping track of his or her actions. The former, i.e., monitoring the student performance, is carried out by updating the student model based on the inference mechanism of an expert system that uses the certainty factor model. The latter, i.e., keeping track of the student actions, is registered by the student model.
References [1]
[2]
Aimeur, E., Dufort, H., Leibu, D., Frasson, C.: Some Justifications for the Learning by Disturbing Paradigm. In: Du Boulay, B., Mizoguchi, R. (eds.): Proc. of 8th International Conference on AI in Education. IOS Press, Amsterdam (1997) 119-126 Ausubel, D.: Educational Psychology: A Cognitive View. Holt, Rinehart & Winston, New York (1968)
[3] [4]
[5] [6]
[7]
[8]
[9]
[10] [11]
[12] [13]
[14] [15]
[16] [17]
[18] [19] [20]
[21]
Buchanan, B.G., Shortliffe, E.H.: Rule-Based Expert Systems. Addison-Wesley, Reading, (1984) Cañas, A.J., Ford, K.M., Brennan, J., Reichherzer, T., Hayes, P.: Knowledge Construction and Sharing in Quorum. In: J. Greer (ed.): Proc. of 7th World Conference on AI in Education. AACE, Charlottesville (1995) 228-225 Chan, T.W.: Learning Companion Systems, Social Learning Systems, and Global Social Learning Club. International Journal of AI in Education 7(2) (1996) 125-159 Chan, T.W., Baskin, A.B.: Learning Companion Systems. In: Frasson, C., Gauthier G. (eds.): Intelligent Tutoring Systems: At the Crossroads of AI and Education. Ablex Publishing Corporation, New Jersey (1990) Cap. 1 Chan, T.W., Chou, C.Y.: Simulating a Learning Companion in Reciprocal Tutoring Systems. In: Proc. of Computer Support for Collaborative Learning’95 (1995). Available at: http://www-cscl95.indiana.edu/cscl95/chan.html. Chan, T.W., Chou, C.Y., Lin, C.J.: User Modeling in Simulating Learning Companions. In: Vivet, M., Lajoie, S. (eds.): Proc. of 9th International Conference on AI in Education. IOS Press, Amsterdam (1999) 277-284 Chou, C.Y., Chan, T.W., Lin, C.W.: Redefining the Learning Companion: The Past, Present, and Future of Educational Agents. Computers & Education 40(3) (2003) 255269 Coper, G.F.: The Computational Complexity of Probabilistic Inference Using Bayesian Belief Networks. AI Journal 42 (1997) 393-405 Devedzic, V., Harrer, A.: Architectural Patterns in Pedagogical Agents. In: Cerri, S. Gouarderes, G., Paraguaçu, F. (eds.): Intelligent Tutoring Systems. LNCS 2363. Springer-Verlag, Berlin (2002) 91-90 Durkin, J.: Expert Systems: Design and Development. Prentice-Hall, New York (1994) Faraco, R.A., Rosatelli, M.C., Gauthier, F.A.: Adaptivity in a Learning Companion System. In: Proc. of the 4th IEEE International Conference on Advanced Learning Technologies. IEEE Press (2004) in press van der Gaag, L.C.: A Pragmatical View on the Certainty Factor Model. The International Journal of Expert Systems: Research and Applications 7 (3) (1994) 289-300 Holt, P., Dubs, S., Jones, M., Greer, J.: The State of Student Modelling. In: Greer, J., McCalla, G. (eds.): Student Modelling: The Key To Individualized Knowledge-Based Instruction. Springer-Verlag, Berlin (1994) 3-35 Likert, R.: A Technique for the Measurement of Attitudes. Archives of Psychology (1932) Vol. 140 Meizalo, V., Torvinen, C., Suhonen, J., Sutinen, E.: Formative Evaluation Scheme for a Web-Based Course Design. In: Proc. of 7th Annual Conference on Innovation and Technology in Computer Science Education. ACM Press, New York (2002) 130-134 Nichols, D.: Issues in Design Learning by Teaching Systems. AAI-AI-ED Technical Report n. 107. Computing Department, Lancaster University, Lancaster (1994) Tedesco, P.: MArCo: Building an Artificial Conflict Mediator to Support Group Planning Interactions. International Journal of AI in Education 13 (2003) 117-155 Uresti, J.A.R.: Should I Teach My Computer Peer? Some Issues in Teaching a Learning Companion. In: Frasson, C., Gauthier, G., VanLenh, K. (eds.): Intelligent Tutoring Systems. LNCS 1839. Springer-Verlag, Berlin (2000) 103-112 VanLehn, K., Niu, Z., Siler, S. Gertner. A.S.: Student Modeling from Conventional Test Data: A Bayesian Approach without Priors. In: Goettl, B.P, Halff, H.M., Redfield, C.L., Shute, V.J. (eds.): Intelligent Tutoring Systems. LNCS 1452. Springer-Verlag, Berlin (1998) 434-443
