An Evaluation of Intelligent Learning Environment for Problem Posing
An Evaluation of Intelligent Learning Environment for Problem Posing Akira Nakano, Tsukasa Hirashima, and Akira Takeuchi Kyushu Institute of Technology Department of Artificial Intelligence 680-4 Kawazu, Iizuka 820-8502, Japan [email protected]
Abstract. In this paper, we describe an Intelligent Learning Environment which realizes the learning by problem posing. In the learning by problem posing, a learner poses problems through the interface provided by the ILE. The ILE has a function to diagnose the problems posed by the learner. By using the results of the diagnosis, the ILE helps the learner to correct wrong problems, or leads her/him in the next step of problem posing. We have developed the ILE. In this paper, we also report an experimental evaluation in an elementary school. By using the result of the experiment, we examined (1) usability of the ILE and (2) effectiveness of the ILE. In the ILE, the interface was implemented in Java, and the diagnosis module was implemented in Prolog. The current environment deals with simple arithmetical word problems solved by an addition or a subtraction.
1
Introduction
Several researchers about problem posing of arithmetical word problems suggested that problem posing is important to learn arithmetic. For example there are task analysis of problem posing [1], [2], examination of learning effects of problem posing [3], and case studies in classroom [4], [5]. Besides, “the Curriculum and Evaluation Standards for School Mathematics (in USA, 1989)”, and “Professional Standards for Teaching mathematics (in USA, 1991)” also indicated that it was important for learners to experience to pose problems. However, despite the importance of problem posing, it is not popular as a learning method in reality. In Learning by Problem Posing, in order to judge whether the problem is correct or not, a teacher has to examine each problem which learners posed. Therefore, it is difficult for teachers to use problem posing as a learning method. Based on this consideration, we are developing an Intelligent Learning Environment which realizes the learning by problem posing. The domain of the ILE is arithmetical word problems solved by an addition or a subtraction. In the ILE, when a learner poses a problem, the ILE diagnoses the problem and helps her/him to correct the wrong problem, or leads her/him in the next step of problem posing [6]. We have already developed a prototype of the ILE and used it in the lecture of the class experimentally [7]. However, in the experiment because we could not keep enough computers to provide each student, two or three students used one ILE. Therefore although we confirmed that learners could pose problems on the ILE, we could not evaluate the ILE in detail. In this paper, we report an evaluation of the ILE based on individual use by 4th grade elementary school students in class. In this experiment, we evaluate the ILE from the following two viewpoints, (1) whether the students can pose problems by using the ILE or not, and (2) whether the use of the ILE is effective to improve the student’s ability of problem posing or not. We call the former viewpoint “usability” and the latter viewpoint “effectiveness”. To evaluate the two points, we carried out pre-test, post-test, questionary for both students and teachers, and collected log data of problem
posing in the ILE. So, in order to consider about the usability, at first, we analyzed if subjects used the ILE enough. We then checked if the subjects whose results in the pre-test were low could pose solvable problems in the ILE. Moreover, we put several questions to several teachers who observed this class of problem posing on the ILE. We compared the results of the pre-test and the post-test in order to evaluate the effectiveness. If the result of the post-test is significantly higher than the result of the pre-test, we can judge the ILE is effective in problem posing, not only the quantity of posed problems but also the quality of them are important to evaluate the ability of problem posing. Because there is no standard method to evaluate the ability of problem posing, we also prepare a scoring method of posed problems in this paper.
2 2.1
ILE for problem posing Solution-Based Problem Posing
To develop an ILE for learning by problem posing, categorization of problem posing is important. We have categorized problem posing in the following three types: (1) Problem-Based Problem Posing, (2) Story-Based Problem Posing, and (3) SolutionBased Problem Posing [7]. The ability to pose problems is necessary to solve problems of which solution are not known. However, the ability often cannot be acquired only to solve problems. Therefore, the practice to pose problems is a promising way to acquire the ability. In this paper, we deal with “Solution-Based Problem Posing”. Solution-Based Problem Posing is the most popular type of problem posing, because it is effective to master the solution method directly [8], [9]. To deal with other types of problem posing on computer is our future work. Solution-Based Problem Posing is an effective practice to learn arithmetic. In Solution-Based Problem Posing, first, a solution method is given beforehand, then, learners pose problems which can be solved by the solution method. We call the solution method “stated solution”. The stated solution is defined by a learner, or given by a teacher or a system. Based on a stated solution, the learner has to pose problems which can be solved by the stated solution. Finally, the solution method of the problem posed by the learner has to be equal to the stated solution. We call the solution method of the posed problem “derived solution”. That is, when the derived solution is equal to the stated solution or when the derived solution is judged the stated solution, the result of the problem posing is correct in Solution-Based Problem Posing. So, we think that Solution-Based Problem Posing is learning method to learn relation between problem and solution.
2.2
Framework of ILE for Problem Posing
We developed an ILE for Solution-Based Problem Posing. We describe the ILE of problem posing based on the solutions in this section. The domain is arithmetical word problems which can be solved by an addition or a subtraction. Fig. 1 shows the framework of the ILE. In the ILE, when a learner poses a problem, the ILE diagnoses the problem in Problem Diagnosis Module. Moreover, by using result in Advice Generator, the ILE helps her/him to correct the wrong problem, or leads her/him in the next step of problem posing. The Problem Diagnosis Module and the Advice Generator and knowledge used in these modules are implemented by Prolog. Client and other module and information of Server are implemented by Java.
Fig. 1. The framework of the ILE.
Fig. 2. The first step to specify problem formula.
Fig. 3. The second step to pose problems.
Next, we explain graphical user interface in the ILE. The interface of the ILE provides learners with two steps to promote problem posing. In first step, a learner specifies a stated solution before s/he poses a problem, e.g., “5-X=3”. We call this solution method like “5-X=3” “problem formula”. So, in Fig. 2, the learner specifies a stated problem formula “5-X=3”. In second step, s/he poses problems which can be solved by the stated problem formula. In order that the ILE judges whether the result of the learner’s problem posing is correct or not, the ILE checks whether the derived problem formula is equal to the stated problem formula or not. Fig. 3 is the interface to pose a problem. This interface is explained in more details. Currently, the ILE can deal with only Change-Problem[10]. In Change-Problem, the quantity
in “the initial situation” is changed to the quantity in “the final situation” by the “change action”. The Change-Problem usually consists of three sentences: the first sentence describes the initial situation, the second sentence describes the change action, and the third sentence describes the final situation. Therefore, we prepare a “problem template” that composed of three “single sentence templates”. In Fig. 3, the problem template is shown in the left side panel. By filling in the blanks of three single sentence templates, the problem is completed. Moreover, in order to filling in the blanks of the templates, the ILE provide learners with several concepts in concept panel beforehand and ten-key. In Fig. 3, these are shown in the right side panel. In issue panel, the ILE provide learners with information generated based on the result of first step. In comment panel, the ILE provide learners with advice which is feedback about result of their operation in the interface and is generated in Advice Generator. Moreover, we explain the problem template in detail. The initial situation has the four information: “owner”, “object”, “number”, and “unit”. This means that “owner” has “object” and the number of “object” is “number”, then, the unit of the number is “unit”. The change action has the five information: “actor”, “object”, “number”, “unit”, and “action”. The ILE deals with the type of action where object moves between the object’s owner and other people, i.e., between two peoples. For example, the action where A receives B’s object from B is dealt with. However, the ILE does not deal with the type of action where object moves among three people. For example, the action where A receives B’s object from C is not dealt with. The final situation has the four information: “owner”, “object”, “number”, and “unit”. In learning by problem posing, the problem template and the sentence templates restrict the expression of problems. However, we think that using these templates don’t lose the main effect of learning by problem posing, because, at learning by arithmetical word problem, we think that it is most important to consider relation between concepts and numerical relations.
3
Experimental use of the ILE
A prototype of the ILE has been already developed. So two elementary school teachers permitted us to use the ILE in their arithmetic classes after they had used the ILE by themselves. Subjects were 55 students of elementary school in 4th grade. In this experiment, we could keep enough computers to provide each student. We evaluate the ILE from the following two viewpoints, (1) whether the students can pose problem by using the ILE or not, and (2) whether the use of the ILE is effective to improve the student’s ability of problem posing or not. We call the former viewpoint “usability” and the latter viewpoint “effectiveness”. The process of the experiment is as follows. (p1): We took 20 minutes to teach how to pose Change-Problems with several examples. (p2): In the pre-test, the subjects posed problems on paper in 10 minutes. We provided the subjects with problem formulas, and then, the subjects posed problems that can be solved by the problem formulas. In this pre-test, the following problems formulas were provided: “6+9=?”, “12-5=?”, “7+?=15”, “11-?=4”, “?+9=13” and “?-7=6”. (p3): The subjects posed problems with the ILE. 10 minutes were taken to explain the way to use the ILE, and 30 minutes were used to pose problems. Then, 4 teaching assistants were supporting how to use the ILE always. (p4): As the post-test, the subjects posed problems on paper again in the same method of (p2). (p5): We asked teachers and the subjects several questions. Through the experiment, we gathered the following data: (d1) logs of problem posing on the ILE, (d2) scores of problem posing on paper before using the ILE and after using the ILE, (d3) answers that teachers and students filled questionnaires. In this section, we explain the evaluation of the ILE based on these data.
3.1
Usability of the ILE
In lesson of problem posing on paper, it is difficult for a teacher to support each learner and it is difficult for the learner to write sentences on paper. Therefore, some students maybe fail to pose problems, because they are not good at write sentences. Therefore, the ILE provides learners with feedback to correct wrong problem and to lead them next problem posing. Moreover, the ILE provides learners with a tool to make sentences with combination between concepts. Therefore, learning by using the ILE is not practice to write sentences, but it is practice to make relations between concepts and numerical relation. From the results, we expected that the ILE would be usable for learners. We evaluate the usability of the ILE from the following four viewpoints. At first, (1) we analyzed the number of posed problems with the ILE. Then, (2) we checked if the subjects whose results in the pre-test were low could pose solvable problems in the ILE. Next, (3) we put several questions to teachers who participated at this class of problem posing on the ILE. By using the questions their impressions for the lesson of problem posing with the ILE are examined. Finally, (4) we put several questions to subjects. By using the question, their impressions for the learning of problem posing with the ILE are examined.
Fig. 4. The relation among amount of the posed problems with the ILE, a distribution of the subjects and the contents of the use of the ILE.
Fig. 4 shows two graphs at the same time. One graph is line graph which shows relation between amount of the posed problems with the ILE and number of the subjects. The other one is bar graph which shows relation between amount of the posed problems with the ILE and the contents of the posed problem with the ILE. At first, we explain each graph as follow. In the line graph of Fig. 4, the horizontal axis shows amount of the posed problems with the ILE, and the vertical axis shows number of subjects (the vertical axis is shown in the right side of Fig. 4). For example, when the value in the horizontal axis is 7, the value in the vertical axis is 9. It means that the number of the subjects who posed 7 problems with the ILE is 9. In the bar graph of Fig. 4, both the horizontal axis and the vertical axis shows amount of the posed problems with the ILE (the vertical axis is shown in the left side
of Fig. 4). That is, the value in the horizontal axis is the same value in the vertical axis. There are two cases in the contents of the posed problems with the ILE; one is that a subject posed a problem which is judged correct by the ILE, the other is that the subject posed the problem which is judged wrong by the ILE. For that reason, we divided one bar into two colors. Therefore, at each bar in the graph, the length of the part which is dark color shows “mean of the amount of the posed wrong problems” (The “wrong problem” means that the posed problems is judged wrong by the ILE) and the length of the part which is light color shows “mean of the amount of the posed correct problems” (The “correct problem” means that the posed problems is judged correct by the ILE). For example, when the value in the horizontal axis is 7, the value in the vertical axis is 7, and then, in the bar, the length of the part which is dark color is 2 and the rest, i.e.; the part which is light color is 5. It means that the subjects who posed 7 problems with the ILE posed wrong problems and correct problems at the rate of 2 to 5. That is, their correct answer rate is 71 %. By using both the line graph and the bar graph, for example, we can know at the same time that the number of the subjects who posed 7 problems with the ILE is 9 and their correct answer rate is 71 %. In the experiment, the mean of the amount of the problems posed by total subjects is 7.5. Forty-seven % subjects posed problems more than the mean of the amount of the problems posed by total subjects (i.e.; forty-seven % subjects posed more than 8 problems). We guess they posed problems actively with the ILE, although they posed many wrong problems (the correct answer rate of the subjects is 57 %). Fifty-three % subjects posed problems less than 7.5 with the ILE. However, they did not pose so many wrong problems (the correct answer rate of the subjects is 79 %). So, we guess that the subjects used the ILE carefully not to pose wrong problems. Four students posed less than two problems although the all problems were correct. Their log data of all operation of the ILE show that they did not operate the ILE at all in the second half of the class. So, we guess that the subjects did not use the ILE to pose problems.
Fig. 5. The relation between ability of problem posing and amount of the posed problems with the ILE.
Next, we check if the subjects whose results in the pre-test were low could use the ILE and if the subjects could pose solvable problems in the ILE. Fig. 5 shows two
graphs at the same time. One graph is line graph which shows relation between amount of the correct problems in the pre-test and number of the subjects. The other one is bar graph which shows relation between amount of the correct problems in the pre-test and amount of the posed problems with the ILE. At first, we explain each graph as follow. In the line graph of Fig. 5, the horizontal axis shows amount of the correct problems in the pre-test, and the vertical axis shows number of the subjects (the vertical axis is shown in the right side of Fig. 5). For example, when the value in the horizontal axis is 3, the value in the vertical axis is 9. It means that the number of the subjects who posed 3 correct problems in the pre-test (in the pre-test, we judged correct problem when the posed problem can be solved). In the bar graph of Fig. 5, the horizontal axis shows amount of the correct problems in the pre-test, and the vertical axis shows mean of the amount of the posed problems with the ILE (the vertical axis is shown in the left side of Fig. 5). In the same way as the bar of Fig. 4, we divided one bar into two colors. Therefore, at each bar in the graph, the length of the part which is dark color shows “mean of the amount of the posed wrong problems” and the length of the part which is light color shows “mean of the amount of the posed correct problems”. For example, when the value in the horizontal axis is 3, the value in the vertical axis is 6.9, and then, in the bar, the length of the part which is dark color is 2.7 and the rest, i.e.; the part which is light color is 4.2. It means that the subjects who posed 3 correct problems in the pre-test posed wrong problems and correct problems at the rate of 2.7 to 4.2. That is, their correct answer rate is 61 %. By using both graphs, for example, we can know at the same time that the number of the subject who posed 3 correct problems in the pre-test is 9 and their correct answer rate is 61 %. By using Fig. 5, at first, we classify the subjects into several types based on the result of the pre-test. Subjects who could pose six correct problems or five correct problems in the pre-test are 22. We consider that they have the high level ability of problem posing. So, we call them “high level subjects”. Subjects who could pose four correct problems or three correct problems in the pre-test are 22. We call them “middle level subjects”. Subjects who could pose two correct problems or a correct problem in the pre-test are 11. We call them “low level subjects”. Next, we examine how each type subject used the ILE. The correct answer rate of the low level subjects is 48 %, i.e., the worst in these level subjects. The correct answer rate of the high level subjects is 78 %, i.e., the best in these level subjects. From this result, we consider that it reflects the difference of ability of problem posing. The mean of the amount of the correct problems with the ILE of the low level subjects is 4.4, i.e., the worst in these level subjects. The mean of the amount of the correct problems with the ILE of the high level subjects is 5.7, i.e., the best in these level subjects. From this result, we consider that it reflects the difference of ability of problem posing, too. However, the low level subjects could pose several correct problems on the ILE. Therefore, we judge that the ILE was not hard for the low level subjects to use. Moreover, we consider that the result shows the function in order to correct wrong problem is a factor of the result. We consider that the reason in which the high and middle level subjects used the ILE carefully in order not to pose wrong problem. Moreover, in order to judge whether amount of the use of the ILE is enough or not, we asked teachers and the subjects several questions. Table 1 shows the result in which we asked teachers who used the ILE and presented 4 questions. These are as follows: (t-1) Did you judge that subjects using the ILE were earnest at the class?, (t-2) Did you judge that the subjects can use the ILE smoothly?, (t-3) Were feedback of the ILE helpful for subjects when they posed wrong problems? and (t-4) Were feedback of the ILE helpful for subjects when they posed correct problems? And, in order to investigate usability, we asked user of the ILE (i.e., they are this subject). Table 2 shows the result in which we asked subjects 3 question. These are as follows: (s-1) Did you enjoy problem posing on the ILE?, (s-2) Were feedback of the ILE helpful for you
when you posed wrong problems? and (s-3) Were feedback of the ILE helpful for you when you posed correct problems?
Table 1. An evaluation of the teachers. Question Yes No So-so (t-1) 6 0 1 (t-2) 7 0 0 (t-3) 6 0 1 (t-4) 5 0 2
Table 2. An evaluation of the 4th grade students. Question Yes No So-so (s-1) 47 8 0 (s-2) 48 6 1 (s-3) 41 10 4
Subjects’ behavior at class (t-1, t-2) and subjects’ interest (s-1) and the function of the ILE (t-3, t-4, s-2, s-3) are evidence of the above our interpretation which is that the subjects posed enough correct problems on the ILE. In the result of the Table 1 and the Table 2, these are evaluated high. That is, we think that the teacher thought this class using the ILE succeeded as class of learning by problem posing and subjects are interested in this class. In order to judge whether the ILE is learning material for learning by problem posing that elementary school students can use easily or not, we showed two Figures and two Tables and our interpretation of these data. In the result based on two Figures (Fig. 4 and Fig. 5), almost subjects, even the subjects who posed few problems, could pose correct problems with the ILE. Moreover, in the result of two Tables (Table 1 and Table 2), the teacher answered that the subjects could use the ILE and the function of the ILE was good, and the subjects answered that the class of the ILE was interesting. Therefore, in these results, we judge that the usability of the ILE is high.
3.2
Effectiveness of the ILE
In problem posing on the ILE, a learner has to find the proper combination among concepts and numerical relations. Because to find the proper combination is an important task of problem posing, the practice should be effective to improve problem posing ability. Therefore, we evaluate the effectiveness of the ILE from the comparison of the scores of the pre-test and the post-test. First, the method to give a score to each posed problem, and then the result of the comparison of the scores of the pre-test and the post-test is explained.
Scoring method of posed problems. Difficulty in problem formula. Problem formula has three types: (α) A ± B=X, (β) A ± X=C, (γ) X ± B=C (A,B,C are numerical values. X is a variable). “A” is the number in the initial situation, “B” is the number of the change action, and “C” is the number in the final situation. “X” is the number that is derived by the solution. In the α type, the answer is in the final situation. So this type of problem is the easiest one. In the β type, the answer is in the change action. In the γ type, the answer is in the initial situation. In learning by problem solving, generally, problems which can be solved by α type equation are the problems of the easiest type in these three types. And, problems which can be solved by γ type equation are the problems of the most difficult type. We think that there is the same difficulty of arithmetically thinking in problem solving and problem posing. So, we use the difference of formula type to score the posed problems.
Difficulty of operation. We think that each verb lets us remind each operation. For
example, “give” usually reminds us of “subtraction”, and “receive” usually reminds us of “addition”. This is the simplest pattern of the combination of a verb and an operation. For example, if we posed a problem by using “give” simply, the problem is like this: “Tom has 5 pieces of apple pies. Tom gives Nancy some pieces of his apple pies. Tom has 3 pieces of apple pies. How many pies of apple pies does Tom give Nancy?” We call the operation reminds by the verb “simple reminded operation”. Meanwhile, in the problem posing, not only the verb in the change action but also the combination of other concepts in the change action should be considered in order to decide the operation. So, all problems are classified in two cases. One is that an operation of derived problem formula is the same with the simple reminded operation. And then, the other is that an operation of derived problem formula is different from the simple reminded operation. We call this case “operation with the gap”. The gap means that there is difference between operation of derived problem formula and the simple reminded operation. For example, the case in which there is the gap is “Tom has 5 pieces of apple pies. Nancy gives Tom some pieces of her apple pies. Tom has 8 pieces of apple pies. How many pies of apple pies does Nancy give Tom?” The operator of derived problem formula is “addition”, but the simple reminded operation of “give” is “subtraction”. Therefore, this problem has the gap. So, we judge that a problem which has the gap is more difficult than a problem which does not has it.
Result of comparison between pre-test and post-test. In evaluation of the effectiveness of the ILE, we investigate whether the ability of problem posing of all subjects is improved or not, by using the ILE. We explained the method to evaluate each posed problem. The scoring method is as follows: if the problem type is the α type problem, the score of the problem is one. If the problem type is the β type problem, the score of the problem is two. If the problem type is the γ type problem, the score of the problem is three. Moreover, only if the problem has gap between an operation of derived problem formula and a simple reminded operation, the score of the problem is incremented one. So, by using the scoring method of the posed problems, we can get each learner’s scores of the pre-test and the post-test. We evaluate the effectiveness by using the difference of the scores between the pre-test and the post-test. In the pre-test and the post-test, the highest score is 18. Table 3. A comparison between the pre-test and the post-test. test subjects mean standard deviation pre-test 55 7.4 3.7 post-test 55 9.6 4.0 So, we tried t-test in order to investigate a change between the mean of scores of the pre-test and the post-test. Table 3 shows the relation of both tests. In the result, the difference of the change is significant. (Mean of the pre-test=7.4, Mean of the posttest=9.6; t-test, t(55)=2.00, p
Abstract. In this paper, we describe an Intelligent Learning Environment which realizes the learning by problem posing. In the learning by problem posing, a learner poses problems through the interface provided by the ILE. The ILE has a function to diagnose the problems posed by the learner. By using the results of the diagnosis, the ILE helps the learner to correct wrong problems, or leads her/him in the next step of problem posing. We have developed the ILE. In this paper, we also report an experimental evaluation in an elementary school. By using the result of the experiment, we examined (1) usability of the ILE and (2) effectiveness of the ILE. In the ILE, the interface was implemented in Java, and the diagnosis module was implemented in Prolog. The current environment deals with simple arithmetical word problems solved by an addition or a subtraction.
1
Introduction
Several researchers about problem posing of arithmetical word problems suggested that problem posing is important to learn arithmetic. For example there are task analysis of problem posing [1], [2], examination of learning effects of problem posing [3], and case studies in classroom [4], [5]. Besides, “the Curriculum and Evaluation Standards for School Mathematics (in USA, 1989)”, and “Professional Standards for Teaching mathematics (in USA, 1991)” also indicated that it was important for learners to experience to pose problems. However, despite the importance of problem posing, it is not popular as a learning method in reality. In Learning by Problem Posing, in order to judge whether the problem is correct or not, a teacher has to examine each problem which learners posed. Therefore, it is difficult for teachers to use problem posing as a learning method. Based on this consideration, we are developing an Intelligent Learning Environment which realizes the learning by problem posing. The domain of the ILE is arithmetical word problems solved by an addition or a subtraction. In the ILE, when a learner poses a problem, the ILE diagnoses the problem and helps her/him to correct the wrong problem, or leads her/him in the next step of problem posing [6]. We have already developed a prototype of the ILE and used it in the lecture of the class experimentally [7]. However, in the experiment because we could not keep enough computers to provide each student, two or three students used one ILE. Therefore although we confirmed that learners could pose problems on the ILE, we could not evaluate the ILE in detail. In this paper, we report an evaluation of the ILE based on individual use by 4th grade elementary school students in class. In this experiment, we evaluate the ILE from the following two viewpoints, (1) whether the students can pose problems by using the ILE or not, and (2) whether the use of the ILE is effective to improve the student’s ability of problem posing or not. We call the former viewpoint “usability” and the latter viewpoint “effectiveness”. To evaluate the two points, we carried out pre-test, post-test, questionary for both students and teachers, and collected log data of problem
posing in the ILE. So, in order to consider about the usability, at first, we analyzed if subjects used the ILE enough. We then checked if the subjects whose results in the pre-test were low could pose solvable problems in the ILE. Moreover, we put several questions to several teachers who observed this class of problem posing on the ILE. We compared the results of the pre-test and the post-test in order to evaluate the effectiveness. If the result of the post-test is significantly higher than the result of the pre-test, we can judge the ILE is effective in problem posing, not only the quantity of posed problems but also the quality of them are important to evaluate the ability of problem posing. Because there is no standard method to evaluate the ability of problem posing, we also prepare a scoring method of posed problems in this paper.
2 2.1
ILE for problem posing Solution-Based Problem Posing
To develop an ILE for learning by problem posing, categorization of problem posing is important. We have categorized problem posing in the following three types: (1) Problem-Based Problem Posing, (2) Story-Based Problem Posing, and (3) SolutionBased Problem Posing [7]. The ability to pose problems is necessary to solve problems of which solution are not known. However, the ability often cannot be acquired only to solve problems. Therefore, the practice to pose problems is a promising way to acquire the ability. In this paper, we deal with “Solution-Based Problem Posing”. Solution-Based Problem Posing is the most popular type of problem posing, because it is effective to master the solution method directly [8], [9]. To deal with other types of problem posing on computer is our future work. Solution-Based Problem Posing is an effective practice to learn arithmetic. In Solution-Based Problem Posing, first, a solution method is given beforehand, then, learners pose problems which can be solved by the solution method. We call the solution method “stated solution”. The stated solution is defined by a learner, or given by a teacher or a system. Based on a stated solution, the learner has to pose problems which can be solved by the stated solution. Finally, the solution method of the problem posed by the learner has to be equal to the stated solution. We call the solution method of the posed problem “derived solution”. That is, when the derived solution is equal to the stated solution or when the derived solution is judged the stated solution, the result of the problem posing is correct in Solution-Based Problem Posing. So, we think that Solution-Based Problem Posing is learning method to learn relation between problem and solution.
2.2
Framework of ILE for Problem Posing
We developed an ILE for Solution-Based Problem Posing. We describe the ILE of problem posing based on the solutions in this section. The domain is arithmetical word problems which can be solved by an addition or a subtraction. Fig. 1 shows the framework of the ILE. In the ILE, when a learner poses a problem, the ILE diagnoses the problem in Problem Diagnosis Module. Moreover, by using result in Advice Generator, the ILE helps her/him to correct the wrong problem, or leads her/him in the next step of problem posing. The Problem Diagnosis Module and the Advice Generator and knowledge used in these modules are implemented by Prolog. Client and other module and information of Server are implemented by Java.
Fig. 1. The framework of the ILE.
Fig. 2. The first step to specify problem formula.
Fig. 3. The second step to pose problems.
Next, we explain graphical user interface in the ILE. The interface of the ILE provides learners with two steps to promote problem posing. In first step, a learner specifies a stated solution before s/he poses a problem, e.g., “5-X=3”. We call this solution method like “5-X=3” “problem formula”. So, in Fig. 2, the learner specifies a stated problem formula “5-X=3”. In second step, s/he poses problems which can be solved by the stated problem formula. In order that the ILE judges whether the result of the learner’s problem posing is correct or not, the ILE checks whether the derived problem formula is equal to the stated problem formula or not. Fig. 3 is the interface to pose a problem. This interface is explained in more details. Currently, the ILE can deal with only Change-Problem[10]. In Change-Problem, the quantity
in “the initial situation” is changed to the quantity in “the final situation” by the “change action”. The Change-Problem usually consists of three sentences: the first sentence describes the initial situation, the second sentence describes the change action, and the third sentence describes the final situation. Therefore, we prepare a “problem template” that composed of three “single sentence templates”. In Fig. 3, the problem template is shown in the left side panel. By filling in the blanks of three single sentence templates, the problem is completed. Moreover, in order to filling in the blanks of the templates, the ILE provide learners with several concepts in concept panel beforehand and ten-key. In Fig. 3, these are shown in the right side panel. In issue panel, the ILE provide learners with information generated based on the result of first step. In comment panel, the ILE provide learners with advice which is feedback about result of their operation in the interface and is generated in Advice Generator. Moreover, we explain the problem template in detail. The initial situation has the four information: “owner”, “object”, “number”, and “unit”. This means that “owner” has “object” and the number of “object” is “number”, then, the unit of the number is “unit”. The change action has the five information: “actor”, “object”, “number”, “unit”, and “action”. The ILE deals with the type of action where object moves between the object’s owner and other people, i.e., between two peoples. For example, the action where A receives B’s object from B is dealt with. However, the ILE does not deal with the type of action where object moves among three people. For example, the action where A receives B’s object from C is not dealt with. The final situation has the four information: “owner”, “object”, “number”, and “unit”. In learning by problem posing, the problem template and the sentence templates restrict the expression of problems. However, we think that using these templates don’t lose the main effect of learning by problem posing, because, at learning by arithmetical word problem, we think that it is most important to consider relation between concepts and numerical relations.
3
Experimental use of the ILE
A prototype of the ILE has been already developed. So two elementary school teachers permitted us to use the ILE in their arithmetic classes after they had used the ILE by themselves. Subjects were 55 students of elementary school in 4th grade. In this experiment, we could keep enough computers to provide each student. We evaluate the ILE from the following two viewpoints, (1) whether the students can pose problem by using the ILE or not, and (2) whether the use of the ILE is effective to improve the student’s ability of problem posing or not. We call the former viewpoint “usability” and the latter viewpoint “effectiveness”. The process of the experiment is as follows. (p1): We took 20 minutes to teach how to pose Change-Problems with several examples. (p2): In the pre-test, the subjects posed problems on paper in 10 minutes. We provided the subjects with problem formulas, and then, the subjects posed problems that can be solved by the problem formulas. In this pre-test, the following problems formulas were provided: “6+9=?”, “12-5=?”, “7+?=15”, “11-?=4”, “?+9=13” and “?-7=6”. (p3): The subjects posed problems with the ILE. 10 minutes were taken to explain the way to use the ILE, and 30 minutes were used to pose problems. Then, 4 teaching assistants were supporting how to use the ILE always. (p4): As the post-test, the subjects posed problems on paper again in the same method of (p2). (p5): We asked teachers and the subjects several questions. Through the experiment, we gathered the following data: (d1) logs of problem posing on the ILE, (d2) scores of problem posing on paper before using the ILE and after using the ILE, (d3) answers that teachers and students filled questionnaires. In this section, we explain the evaluation of the ILE based on these data.
3.1
Usability of the ILE
In lesson of problem posing on paper, it is difficult for a teacher to support each learner and it is difficult for the learner to write sentences on paper. Therefore, some students maybe fail to pose problems, because they are not good at write sentences. Therefore, the ILE provides learners with feedback to correct wrong problem and to lead them next problem posing. Moreover, the ILE provides learners with a tool to make sentences with combination between concepts. Therefore, learning by using the ILE is not practice to write sentences, but it is practice to make relations between concepts and numerical relation. From the results, we expected that the ILE would be usable for learners. We evaluate the usability of the ILE from the following four viewpoints. At first, (1) we analyzed the number of posed problems with the ILE. Then, (2) we checked if the subjects whose results in the pre-test were low could pose solvable problems in the ILE. Next, (3) we put several questions to teachers who participated at this class of problem posing on the ILE. By using the questions their impressions for the lesson of problem posing with the ILE are examined. Finally, (4) we put several questions to subjects. By using the question, their impressions for the learning of problem posing with the ILE are examined.
Fig. 4. The relation among amount of the posed problems with the ILE, a distribution of the subjects and the contents of the use of the ILE.
Fig. 4 shows two graphs at the same time. One graph is line graph which shows relation between amount of the posed problems with the ILE and number of the subjects. The other one is bar graph which shows relation between amount of the posed problems with the ILE and the contents of the posed problem with the ILE. At first, we explain each graph as follow. In the line graph of Fig. 4, the horizontal axis shows amount of the posed problems with the ILE, and the vertical axis shows number of subjects (the vertical axis is shown in the right side of Fig. 4). For example, when the value in the horizontal axis is 7, the value in the vertical axis is 9. It means that the number of the subjects who posed 7 problems with the ILE is 9. In the bar graph of Fig. 4, both the horizontal axis and the vertical axis shows amount of the posed problems with the ILE (the vertical axis is shown in the left side
of Fig. 4). That is, the value in the horizontal axis is the same value in the vertical axis. There are two cases in the contents of the posed problems with the ILE; one is that a subject posed a problem which is judged correct by the ILE, the other is that the subject posed the problem which is judged wrong by the ILE. For that reason, we divided one bar into two colors. Therefore, at each bar in the graph, the length of the part which is dark color shows “mean of the amount of the posed wrong problems” (The “wrong problem” means that the posed problems is judged wrong by the ILE) and the length of the part which is light color shows “mean of the amount of the posed correct problems” (The “correct problem” means that the posed problems is judged correct by the ILE). For example, when the value in the horizontal axis is 7, the value in the vertical axis is 7, and then, in the bar, the length of the part which is dark color is 2 and the rest, i.e.; the part which is light color is 5. It means that the subjects who posed 7 problems with the ILE posed wrong problems and correct problems at the rate of 2 to 5. That is, their correct answer rate is 71 %. By using both the line graph and the bar graph, for example, we can know at the same time that the number of the subjects who posed 7 problems with the ILE is 9 and their correct answer rate is 71 %. In the experiment, the mean of the amount of the problems posed by total subjects is 7.5. Forty-seven % subjects posed problems more than the mean of the amount of the problems posed by total subjects (i.e.; forty-seven % subjects posed more than 8 problems). We guess they posed problems actively with the ILE, although they posed many wrong problems (the correct answer rate of the subjects is 57 %). Fifty-three % subjects posed problems less than 7.5 with the ILE. However, they did not pose so many wrong problems (the correct answer rate of the subjects is 79 %). So, we guess that the subjects used the ILE carefully not to pose wrong problems. Four students posed less than two problems although the all problems were correct. Their log data of all operation of the ILE show that they did not operate the ILE at all in the second half of the class. So, we guess that the subjects did not use the ILE to pose problems.
Fig. 5. The relation between ability of problem posing and amount of the posed problems with the ILE.
Next, we check if the subjects whose results in the pre-test were low could use the ILE and if the subjects could pose solvable problems in the ILE. Fig. 5 shows two
graphs at the same time. One graph is line graph which shows relation between amount of the correct problems in the pre-test and number of the subjects. The other one is bar graph which shows relation between amount of the correct problems in the pre-test and amount of the posed problems with the ILE. At first, we explain each graph as follow. In the line graph of Fig. 5, the horizontal axis shows amount of the correct problems in the pre-test, and the vertical axis shows number of the subjects (the vertical axis is shown in the right side of Fig. 5). For example, when the value in the horizontal axis is 3, the value in the vertical axis is 9. It means that the number of the subjects who posed 3 correct problems in the pre-test (in the pre-test, we judged correct problem when the posed problem can be solved). In the bar graph of Fig. 5, the horizontal axis shows amount of the correct problems in the pre-test, and the vertical axis shows mean of the amount of the posed problems with the ILE (the vertical axis is shown in the left side of Fig. 5). In the same way as the bar of Fig. 4, we divided one bar into two colors. Therefore, at each bar in the graph, the length of the part which is dark color shows “mean of the amount of the posed wrong problems” and the length of the part which is light color shows “mean of the amount of the posed correct problems”. For example, when the value in the horizontal axis is 3, the value in the vertical axis is 6.9, and then, in the bar, the length of the part which is dark color is 2.7 and the rest, i.e.; the part which is light color is 4.2. It means that the subjects who posed 3 correct problems in the pre-test posed wrong problems and correct problems at the rate of 2.7 to 4.2. That is, their correct answer rate is 61 %. By using both graphs, for example, we can know at the same time that the number of the subject who posed 3 correct problems in the pre-test is 9 and their correct answer rate is 61 %. By using Fig. 5, at first, we classify the subjects into several types based on the result of the pre-test. Subjects who could pose six correct problems or five correct problems in the pre-test are 22. We consider that they have the high level ability of problem posing. So, we call them “high level subjects”. Subjects who could pose four correct problems or three correct problems in the pre-test are 22. We call them “middle level subjects”. Subjects who could pose two correct problems or a correct problem in the pre-test are 11. We call them “low level subjects”. Next, we examine how each type subject used the ILE. The correct answer rate of the low level subjects is 48 %, i.e., the worst in these level subjects. The correct answer rate of the high level subjects is 78 %, i.e., the best in these level subjects. From this result, we consider that it reflects the difference of ability of problem posing. The mean of the amount of the correct problems with the ILE of the low level subjects is 4.4, i.e., the worst in these level subjects. The mean of the amount of the correct problems with the ILE of the high level subjects is 5.7, i.e., the best in these level subjects. From this result, we consider that it reflects the difference of ability of problem posing, too. However, the low level subjects could pose several correct problems on the ILE. Therefore, we judge that the ILE was not hard for the low level subjects to use. Moreover, we consider that the result shows the function in order to correct wrong problem is a factor of the result. We consider that the reason in which the high and middle level subjects used the ILE carefully in order not to pose wrong problem. Moreover, in order to judge whether amount of the use of the ILE is enough or not, we asked teachers and the subjects several questions. Table 1 shows the result in which we asked teachers who used the ILE and presented 4 questions. These are as follows: (t-1) Did you judge that subjects using the ILE were earnest at the class?, (t-2) Did you judge that the subjects can use the ILE smoothly?, (t-3) Were feedback of the ILE helpful for subjects when they posed wrong problems? and (t-4) Were feedback of the ILE helpful for subjects when they posed correct problems? And, in order to investigate usability, we asked user of the ILE (i.e., they are this subject). Table 2 shows the result in which we asked subjects 3 question. These are as follows: (s-1) Did you enjoy problem posing on the ILE?, (s-2) Were feedback of the ILE helpful for you
when you posed wrong problems? and (s-3) Were feedback of the ILE helpful for you when you posed correct problems?
Table 1. An evaluation of the teachers. Question Yes No So-so (t-1) 6 0 1 (t-2) 7 0 0 (t-3) 6 0 1 (t-4) 5 0 2
Table 2. An evaluation of the 4th grade students. Question Yes No So-so (s-1) 47 8 0 (s-2) 48 6 1 (s-3) 41 10 4
Subjects’ behavior at class (t-1, t-2) and subjects’ interest (s-1) and the function of the ILE (t-3, t-4, s-2, s-3) are evidence of the above our interpretation which is that the subjects posed enough correct problems on the ILE. In the result of the Table 1 and the Table 2, these are evaluated high. That is, we think that the teacher thought this class using the ILE succeeded as class of learning by problem posing and subjects are interested in this class. In order to judge whether the ILE is learning material for learning by problem posing that elementary school students can use easily or not, we showed two Figures and two Tables and our interpretation of these data. In the result based on two Figures (Fig. 4 and Fig. 5), almost subjects, even the subjects who posed few problems, could pose correct problems with the ILE. Moreover, in the result of two Tables (Table 1 and Table 2), the teacher answered that the subjects could use the ILE and the function of the ILE was good, and the subjects answered that the class of the ILE was interesting. Therefore, in these results, we judge that the usability of the ILE is high.
3.2
Effectiveness of the ILE
In problem posing on the ILE, a learner has to find the proper combination among concepts and numerical relations. Because to find the proper combination is an important task of problem posing, the practice should be effective to improve problem posing ability. Therefore, we evaluate the effectiveness of the ILE from the comparison of the scores of the pre-test and the post-test. First, the method to give a score to each posed problem, and then the result of the comparison of the scores of the pre-test and the post-test is explained.
Scoring method of posed problems. Difficulty in problem formula. Problem formula has three types: (α) A ± B=X, (β) A ± X=C, (γ) X ± B=C (A,B,C are numerical values. X is a variable). “A” is the number in the initial situation, “B” is the number of the change action, and “C” is the number in the final situation. “X” is the number that is derived by the solution. In the α type, the answer is in the final situation. So this type of problem is the easiest one. In the β type, the answer is in the change action. In the γ type, the answer is in the initial situation. In learning by problem solving, generally, problems which can be solved by α type equation are the problems of the easiest type in these three types. And, problems which can be solved by γ type equation are the problems of the most difficult type. We think that there is the same difficulty of arithmetically thinking in problem solving and problem posing. So, we use the difference of formula type to score the posed problems.
Difficulty of operation. We think that each verb lets us remind each operation. For
example, “give” usually reminds us of “subtraction”, and “receive” usually reminds us of “addition”. This is the simplest pattern of the combination of a verb and an operation. For example, if we posed a problem by using “give” simply, the problem is like this: “Tom has 5 pieces of apple pies. Tom gives Nancy some pieces of his apple pies. Tom has 3 pieces of apple pies. How many pies of apple pies does Tom give Nancy?” We call the operation reminds by the verb “simple reminded operation”. Meanwhile, in the problem posing, not only the verb in the change action but also the combination of other concepts in the change action should be considered in order to decide the operation. So, all problems are classified in two cases. One is that an operation of derived problem formula is the same with the simple reminded operation. And then, the other is that an operation of derived problem formula is different from the simple reminded operation. We call this case “operation with the gap”. The gap means that there is difference between operation of derived problem formula and the simple reminded operation. For example, the case in which there is the gap is “Tom has 5 pieces of apple pies. Nancy gives Tom some pieces of her apple pies. Tom has 8 pieces of apple pies. How many pies of apple pies does Nancy give Tom?” The operator of derived problem formula is “addition”, but the simple reminded operation of “give” is “subtraction”. Therefore, this problem has the gap. So, we judge that a problem which has the gap is more difficult than a problem which does not has it.
Result of comparison between pre-test and post-test. In evaluation of the effectiveness of the ILE, we investigate whether the ability of problem posing of all subjects is improved or not, by using the ILE. We explained the method to evaluate each posed problem. The scoring method is as follows: if the problem type is the α type problem, the score of the problem is one. If the problem type is the β type problem, the score of the problem is two. If the problem type is the γ type problem, the score of the problem is three. Moreover, only if the problem has gap between an operation of derived problem formula and a simple reminded operation, the score of the problem is incremented one. So, by using the scoring method of the posed problems, we can get each learner’s scores of the pre-test and the post-test. We evaluate the effectiveness by using the difference of the scores between the pre-test and the post-test. In the pre-test and the post-test, the highest score is 18. Table 3. A comparison between the pre-test and the post-test. test subjects mean standard deviation pre-test 55 7.4 3.7 post-test 55 9.6 4.0 So, we tried t-test in order to investigate a change between the mean of scores of the pre-test and the post-test. Table 3 shows the relation of both tests. In the result, the difference of the change is significant. (Mean of the pre-test=7.4, Mean of the posttest=9.6; t-test, t(55)=2.00, p
