The Adaptive Intelligent Personalised Learning Environment
2009 Ninth IEEE International Conference on Advanced Learning Technologies
The Adaptive Intelligent Personalised Learning Environment R. Costello and D.P. Mundy School of Arts and New Media University of Hull [email protected], [email protected] using information when learning. Felder et al. [1] suggest that learning styles are characteristics of cognitive, psychological, and affective behaviours that serve as indicators as to how learners perceive, interact, and respond to the learning environment. Whilst each individual may have a different learning style, it is possible, through generic learning styles to group and categorise learner approaches [2]. As evidenced in research [3] sometimes mismatches occur between the learning styles of the majority of students in a class and the teaching style of the professor; in these circumstances the students may become bored and inattentive in class, do poorly in tests, and get discouraged about the course and the lecturer too, so it is important to aim to make learning materials appropriate to the learner. To overcome these problems, domain experts strive for a balance of instructional methods. If a balance is achieved, then students will be taught partly in a manner they prefer, leading to increased comfort and motivation. This problem of providing diverse content for equally diverse learner groups is extremely difficult and unlikely to meet with success all of the time. However, advanced intelligent systems supporting the learning process have been making in-roads towards overcoming this issue. The concept of personalised learning environments is now starting to come to fruition. According to the UK Department for Education and Skills (now the Department for Children, Schools and Families) [4], personalised learning environments strive for an adaptive educational approach that is individual, interesting and tailored to learners’ needs and requirements. Personalised learning means high quality teaching that is responsive to the different ways students achieve their best results. A learning environment that responds to individual pupils, with materials chosen to take account of their needs, interests and aspirations, can lead to an enhanced learner experience. This paper focuses on the presentation of research results from the implementation of a personalised learning environment model, the Personalised Adaptive Filtering System (PAFS), and its associated supportive intelligent
Abstract As individuals the ideal learning scenario would be a learning environment tailored just for how we like to learn, personalized to our requirements. This has previously been almost inconceivable given the complexities of learning, the constraints within the environments in which we teach and the need for global repositories of knowledge to facilitate this process. Whilst it still is not necessarily achievable in its full sense this project represents a path towards the presented ideal. Findings from our research into the development of a model and intelligent algorithms for personalised learning are presented. Our model is built on the premise of an ideal system being one which does not just consider the individual but also considers groupings of likeminded individuals and their power to influence learner choice.
1. Introduction The process of learning can be recognised as individualistic, complex and sometimes chaotic. This learning process begins within the womb and does not stop until the day we die. Throughout our lives our learning approach continually evolves as our experiences in learning mould and shape our future experiences. In addition, the context of a particular learning experience may have an influence on the way we approach developing our understanding. Learners preferentially take in and process information in different ways to enable them to learn a given domain topic area; this can be achieved through application of the brain, senses and physical movement. An individual will apply these items in different ways dependent on circumstance and their own individual approach to learning. These approaches to learning can be determined to be different learning styles. Learning styles can be referred to as an individual’s preference of processing material; in other words, each of us may have different styles, with different characteristics, of acquiring and
978-0-7695-3711-5/09 $25.00 © 2009 IEEE DOI 10.1109/ICALT.2009.38
606
algorithms. PAFS is based on the combination of learning strategy theory and knowledge rich learning materials as the pedagogic foundation for adaptively matching student’s requirements to personalised learning environments. Section 2 will describe the related literature and current limitations. Section 3 briefly describes our model and underlying algorithms and section 4 presents our research findings. Finally we will summarise limitations and our results achieved.
•
2.2 Approaches to Matching The main component of a personalised learning environment is the algorithm used to perform matching between the learner requirements and learning resources. A number of different approaches have been used in the development of such an algorithm. Multiple researchers have used Artificial Neural Networks [16][10] to match learner profiles with learning objects. [17][18] have used the concept of swarm intelligence to provide personalised pathways to users through learning materials. Intelligent Algorithms [19][20] and Cluster Filtering [21] concepts have also been used most fundamentally around the ideas of keyword matching in knowledge bases to facilitate more personalized learner experiences. Only a small number of these intelligent systems [22] [23] have focused on taking into consideration the role of the group in guiding individual learning paths. Other limitations include: • Poor instructional design/implementation [24]
2. Related Literature In this section we the development of personalised learning environments and approaches to matching learner requirements with learning materials.
2.1 Personalised Learning Environments A number of educational institutions have focused on producing systems in which environments can adapt to the needs of the user, creating personalised learning experiences. There are a number of different approaches for achieving this, the most pertinent of which to this research are outlined below. In 1999, Dietinger et al. [5] proposed the Dynamic Background Library which uses an intelligent algorithm built around the concepts of keyword relevance and user profiles to filter large knowledge bases for data relevant to what the user wishes to learn about. Harun [6] described a Personalised Continuing Medical Education solution which focused on providing individually relevant learning materials based on expertise, interest and need. The algorithm at the heart of this solution needed to match personal attributes to the knowledge base to retrieve just in time materials. The Web Intelligent Trainer [7] provides a protected environment for novice users who can work on a module, as they would normally do, while the system silently reasons about their actions and offers adaptive tutoring. Most recently De Meo et al. [8] proposed the X-Learn system which is a multi-agent system for adaptive elearning based on user preferences, history, expertise and requirements. Limitations to existing models for personalised learning environments are stated below: • The degree of matching and mismatching between learners and learner materials [9][10] •
Changing learning processes [11]
•
Lack of consideration of learner technology skills [12]
•
Virtual Learning Environment (VLE) integration [14]
Flexibility [13][15]
•
Use of the wrong type of learning styles [25][26]
3. The Adaptive Environment In this paper we will briefly describe and provide some initial results for our Adaptive Intelligent Personalised Learning environment (AIPL) as outlined in Figure 1. The AIPL model was developed to create a personalised map between the individual learner and learning materials.
Figure 1 AIPL Model
3.1 The Learning Profile The learning profile plays an important part in providing information about the learner in the form of an assessment of learning needs, contextual information in 607
object markup allowing knowledge to be searched and retrieved with great efficiency and accuracy. In AIPL, a dynamic library resource can be used to facilitate learning on-line by enabling web-site addresses and ratings to be stored in conjunction with already written static materials.
relation to what the learner is learning about and individual repositories of preferred learning materials. All of this information can then be used by the intelligent algorithm within the system to assist in the filtering of relevant learning materials for the learner.
3.2 The Personalised Adaptive Filtering System (PAFS)
4. Research Findings Looking at previous research in the personalised learning field [27][28] the following areas for testing were identified: sensitivity of the algorithms, emotional response to the learning environment, analysis of interaction with the learning environment and the overall effectiveness of the solution. The test suite involved the development of a web system based on the AIPL model and the design and implementation of associated algorithms. The approach of the testing was built around the concept of a simple learning experience that of learning how to wire a plug. The author used twenty candidates to undertake the testing of the model. The candidates varied in age, computer literacy skills, education, and background to enable a diverse cross section to be analysed. Testing included analysis of the impact of each stage of the algorithm in personalising the learning experience for the user. Results are outlined in the sections below.
PAFS is the intelligent module which is at the heart of this personalisation system. PAFS at its most basic can simply provide a system for retrieving materials from the knowledge repository based on keyword relevance. At its most sophisticated PAFS takes into account user profiles, contextual requirements, semantic data and the role of the group in guiding individual learning paths. PAFS consists of a three-stage evolutionary algorithm. The first stage the Non-Semantic Response Algorithm (NSRA) is a keyword search that searches for defined keywords in specified learning materials and repositories. The NSRA uses a spider or web-crawler to search through Learning Objects to retrieve materials, which only have the specific words that the user choose or the learning profile provides. The NSRA would then identify the defined keywords within learning objects in the repository. Once keywords have been identified in documents a list of documents containing the keywords is retrieved. The user retrieves the on-line educational document and the algorithm reads the contents and provides a summary of keywords present within the selected document. The second stage, the Semantic Bridging Algorithm (SBA) was designed to overcome the problems associated with the NSRA. Due to the dynamics and variations of the particular keyword search, an automated process was introduced to accurately extract personalised educational metadata from the learning objects and compare it to the learner’s specific needs. To achieve this learning objects within the repository are semantically readable through the LOM and SCORM standards. The final stage, the Collaborative Categorization and Semantic Bridging Algorithm was designed to build upon the semantic data and provide the user with mechanisms to retrieve categorisations of learning objects based on the ratings of users with similar learning profiles (ie. group categorisation). This would assist the learner in choosing from lists of resources the most appropriate learning resource. The collaboration and categorisation system uses the Pearson Correlation Algorithm to enable the weight based rating of learning objects.
4.1 Sensitivity of the Algorithm Sensitivity can be related to the complexity of and the performance of the algorithms used. Sensitivity for this research has been measured using the retrieval rate of learning materials in the conduct of a basic and advanced search and a complex set of words searches. The performance has been measured using two types of key word searches enabling a comparisons to be made. In relation to the sensitivity of the three stage algorithm approach within AIPL we found that in general when judged against a human baseline the algorithm retrieved relevant learning objects to the particular scenario and at the final stage recommended specific materials designed for the needs of the learners. We found the more specificity involved in the algorithms parameters, the longer the algorithm took to complete however this reduced the number of relevant learning objects delivered to the users.
4.2 Emotional Response According to Lu et al [29] emotional response can be measured using a state set that is associated with the learning experience. The emotional state set consists of the following states, interest, confusion, frustration, and hopefulness. The emotional states of the candidates were important, because it would enable the authors to analysis
3.3 The Dynamic Background Library The Dynamic Background Library provides a centralised knowledge repository for AIPL. Standard taxonomies i.e. LOM, SCORM can be used for learning 608
the design specification against the theoretical concept of the AIPL model. The states were measured through direct questioning whilst the candidates were using the environment. The findings in relation to the emotional response to the system were quite positive. We found that of the test group selected 90% of users found the system to be none confusing and 80% found the system to be beneficial to their overall learning experience.
materials to allow test candidates to gain an idea of the effectiveness of this algorithm. However, it is noted from responses to this algorithm that the test group did not find this algorithm to be as effective as the initial two algorithms. The initial stage of the algorithm is not complex enough to provide a sufficiently personalized learning experience. Later stages depend on semantic metadata being present obviously this has serious time constraints and implications for the course developer.
4.3 Analysis of Interaction
5. Conclusions and Further Work
The aim of this group of tests was to measure individual responses to how the environment handled given problems. This gives an impression of candidate’s satisfaction with the pedagogical approach, the interface and functionalities therein. Candidates responded well to the example used to test the AIPL environment and in general felt that it was important for the course context and development to be shaped for online learning. However, coming out of this set of tests was the response that 1 in 10 of those sampled had no interest in working online. This fits in with issues around technology use identified by [22][24] in Section 2.1. In relation to the interface itself, in general, the test group had no real concerns. However there were clear indications of where improvements could be made.
Overall, the research detailed within this paper suggests that a model for the development of personalised learning environments (APIL) can be established and with work, complex algorithms can be used to most appropriately match learning resources with learner requirements. In addition, to this a model built on collaboration and feedback mechanisms can best support individual learning paths. Further work from this research is the application of the AIPL environment in more complex learning scenarios such as its implementation within a full time University module. In addition given the findings of the test suite more work should be carried out on the effectiveness of the algorithms particularly in relation to collaborative grouping and changes to the interface itself. An ideal would be to integrate the personalized learning environment within a modular VLE such as Sakai.
4.4 Overall Effectiveness The overall effectiveness of the model was measured through multiple factors. These factors included responses to the handling of student queries, fault identification, degrees of match to conceptual understanding and the recording of personal views. Overall, the response to the testing of the algorithms in relation to their effectiveness in retrieving relevant learning materials was very good. 70% of those tested found the functionality of the algorithm when dealing with their queries was between very good and excellent. A number of faults were identified within the system through this test suite but most were fairly minimal. When asked about the concept of AIPL and the degree in which AIPL’s algorithms met their conceptual ideals, for all algorithms the majority of candidates deemed them to be successful. Finally, candidates found that AIPL matched what it set out to achieve, which was to create a learner centric approach that would bridge together the individual learner to the learning materials.
6. Acknowledgements The authors wish to thank the School of Arts and New Media within the University of Hull for support in enabling the delivery of this paper.
7. References [1] R.M. Felder and R. Brent, "Understanding Student Differences." J. Engr. Education, 94(1), 57-72 (2005) [2] A. Gomes, A. Santos, L. Carmo, and A.J. Mendes, “Learning styles in an e-learning tool”, Proceedings of International Conference on Engineering Education – ICEE 2007, Coimbra, Portugal, September 3 – 7, 2007 [3] R.M. Felder and L.K. Silverman, "Learning and Teaching Styles in Engineering Education," Engr. Education, 78(7), 674681 (1988) [4] Department for Education and Skills, “A National Conversation about Personalised Learning”, DfES Publications, Crown Copyright 2004 [5] T. Dietinger, C. Guetl, B. Knögler, D. Neussl, and K. Schmaranz, “Dynamic Background Libraries - New Developments In Distance Education Using HIKS (Hierarchical Interactive Knowledge System)”, Journal of Universal Computer Science, vol. 5, no. 1 (1999), pp. 2-10
4.5 Limitations of model The final stage of the algorithm depends on the collaboration of users and a feedback response mechanism which can only be firmly developed over a long period of time. The authors pre-rated a sample set of learning 609
conjunction with the 16th ACM Conference on Hypertext and Hypermedia (HT’05) Salzburg, Austria, pp. 27-32 [21] T.Y. Tang and G. McCalla, “Smart Recommendation for an Evolving E-Learning System”, Architecture and Experiment . International Journal on E-Learning. 4 (1), pp. 105-129 [22] L. Capus, F. Curvat, O. Leclair, and N. Tourigny, “A Web environment to encourage students to do exercises outside the classroom: A case study”, Educational Technology & Society, 9 (3), pp.173-181 [23] C.J. Bonk, R.A. Wisher, "Applying collaborative and elearning tools to military distance learning: a research framework", US Army Research Institute for the Behavioral and Social Sciences, Washington, DC, Technical Report [24] K.A. Papanikolaou and M. Grigoriadou, “Accommodating learning style characteristics in Adaptive Educational Hypermedia”. Proceedings of the Workshop on Individual Differences in Adaptive Hypermedia in AH2004, Part I, Eidhoven, Nederlands, August 2004, p. 77-86 [25] S. King, E. Taylor, F. Satzinger, M. Carbonaro, and E. Greidanus, “Developing Interdisciplinary Team Competencies in a Blended Learning Course: Impact on Student Learning”, The Internet Journal of Allied Health Sciences and Practice. Jan 2008, Volume 6 Number 1 [26] S.A. White and I. Liccardi, “Harnessing Insight into Disciplinary Differences to Refine e-learning Design”, Frontiers in Education Conference, 36th Annual (2006), pp. 5-10 [27] J. Shepherd, J. Clendinning and L. Schaverien, “Rethinking e-learning design on generative learning principles”, Proceedings of Ascilite 2002, Auckland, NZ, December 2002 [28] Kindley, “Scenario-Based E-Learning: A Step Beyond Traditional E-Learning”, Learning Circuits, 2002 [29] C. Lu, J. Zhou, L. Shen and R. Shen, Techniques for Enhancing Pervasive Learning in Standard Natural Classroom. In Proceedings of ICHL'2008. pp.202-212
[6] M.H. Harun, “Integrating e-Learning into the workplace”, The Internet and Higher Education, Volume 4, Number 3, 2001 , pp. 301-310(10) [7] K. Kabassi and M. Virvou, “Personalised adult e-training on computer use based on multiple attribute decision making”, Interacting with Computers, Volume 16, Issue 1, February 2004, pp. 115-132 [8] P. De Meo, A. Garro, G. Terracina and D.Ursino, “Personalizing learning programs with X-Learn, an XML-based, ‘user-device’’ adaptive multi-agent system”, Information Sciences: an International Journal, Volume 177 , Issue 8 (April 2007), pp. 1729-1770 [9] S. Visser, S. McChlery, N. Vreken, “Teaching styles versus learning styles in the accounting sciences in the United Kingdom and South Africa: a comparative analysis”, Meditari Accounting Research, Vol. 14, No.2 (2006), pp.97-112 [10] M.T. Tomlinson and B.C. Love, “From Pigeons to Humans: Grounding Relational Learning in Concrete Examples”, Proceedings of the Twenty-First National Conference on Artificial Intelligence, July 16–20, 2006, Boston, Massachusetts, pp.199-204 [11] J. Juhary, “Cadets’ Expectations of Their Learning Environment at the National Defence University of Malaysia”, MERLOT Journal of Online Learning and Teaching, Vol. 4, No. 4, December 2008, pp. 549 – 559 [12] P. Van Rosmalen, H. Vogten, R. Van Es, H. Passier, P. Poelmans, and R. Koper (2006), “Authoring a full life cycle model in standards-based, adaptive e-learning”, Educational Technology & Society, 9 (1), 72-83 [13] V.M. Chieu, "COFALE: An Authoring System for Supporting Cognitive Flexibility", Sixth IEEE International Conference on Advanced Learning Technologies (ICALT'06), Kerkrade, The Netherlands, 5-7 July 2006, pp.335-339 [14] O. Conlan, D. Dagger, and V. Wade, "Towards a Standardsbased Approach to e-Learning Personalization using Reusable Learning Objects", E-Learn 2002, World Conference on ELearning in Corporate, Government, Healthcare and Higher Education, Montreal, September 2002 [15] M. Meccawy, P. Brusilovsky, H. Ashman, M. Yudelson and O. Scherbinina, “Integrating Interactive Learning Content into an Adaptive E-Learning System: Lessons Learned”, Proceedings of World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education 2007, Chesapeake, VA, pp. 6314-6319 [16] K. Scalise, “Differentiated e-Learning: Five Approaches through Instructional Technology”, International Journal of Learning Technology 2007 - Vol. 3, No.2 pp. 169 – 182 [17] J.F. Peters, C. Henry and S. Ramanna, “Reinforcement learning in Swarms that Learn”, Proceedings of the 2005 IEEE/WIC/ACM International Conference on Intelligent Agent Technology, pp. 400 - 406 [18] S. Gutierrez, A. Pardo and C.D. Kloos – “Finding A Learning Path: Towards A Swarm Intelligence Approach”, Proceedings of IASTED Conference on Web Based Education 2006, Puerto Vallarta, Mexico, pp. 94-99 [19] D. Gaševic and M. Hatala, “Ontology mappings to improve learning resource search”, British Journal of Educational Technology, Volume 37, Number 3, May 2006 , pp. 375389(15) [20] A. Carbonaro, R. Ferrini, and M.A. Zamboni, “Considering semantic abilities to improve a web based distance learning system”, Proceedings of the International Workshop Held in
610
The Adaptive Intelligent Personalised Learning Environment R. Costello and D.P. Mundy School of Arts and New Media University of Hull [email protected], [email protected] using information when learning. Felder et al. [1] suggest that learning styles are characteristics of cognitive, psychological, and affective behaviours that serve as indicators as to how learners perceive, interact, and respond to the learning environment. Whilst each individual may have a different learning style, it is possible, through generic learning styles to group and categorise learner approaches [2]. As evidenced in research [3] sometimes mismatches occur between the learning styles of the majority of students in a class and the teaching style of the professor; in these circumstances the students may become bored and inattentive in class, do poorly in tests, and get discouraged about the course and the lecturer too, so it is important to aim to make learning materials appropriate to the learner. To overcome these problems, domain experts strive for a balance of instructional methods. If a balance is achieved, then students will be taught partly in a manner they prefer, leading to increased comfort and motivation. This problem of providing diverse content for equally diverse learner groups is extremely difficult and unlikely to meet with success all of the time. However, advanced intelligent systems supporting the learning process have been making in-roads towards overcoming this issue. The concept of personalised learning environments is now starting to come to fruition. According to the UK Department for Education and Skills (now the Department for Children, Schools and Families) [4], personalised learning environments strive for an adaptive educational approach that is individual, interesting and tailored to learners’ needs and requirements. Personalised learning means high quality teaching that is responsive to the different ways students achieve their best results. A learning environment that responds to individual pupils, with materials chosen to take account of their needs, interests and aspirations, can lead to an enhanced learner experience. This paper focuses on the presentation of research results from the implementation of a personalised learning environment model, the Personalised Adaptive Filtering System (PAFS), and its associated supportive intelligent
Abstract As individuals the ideal learning scenario would be a learning environment tailored just for how we like to learn, personalized to our requirements. This has previously been almost inconceivable given the complexities of learning, the constraints within the environments in which we teach and the need for global repositories of knowledge to facilitate this process. Whilst it still is not necessarily achievable in its full sense this project represents a path towards the presented ideal. Findings from our research into the development of a model and intelligent algorithms for personalised learning are presented. Our model is built on the premise of an ideal system being one which does not just consider the individual but also considers groupings of likeminded individuals and their power to influence learner choice.
1. Introduction The process of learning can be recognised as individualistic, complex and sometimes chaotic. This learning process begins within the womb and does not stop until the day we die. Throughout our lives our learning approach continually evolves as our experiences in learning mould and shape our future experiences. In addition, the context of a particular learning experience may have an influence on the way we approach developing our understanding. Learners preferentially take in and process information in different ways to enable them to learn a given domain topic area; this can be achieved through application of the brain, senses and physical movement. An individual will apply these items in different ways dependent on circumstance and their own individual approach to learning. These approaches to learning can be determined to be different learning styles. Learning styles can be referred to as an individual’s preference of processing material; in other words, each of us may have different styles, with different characteristics, of acquiring and
978-0-7695-3711-5/09 $25.00 © 2009 IEEE DOI 10.1109/ICALT.2009.38
606
algorithms. PAFS is based on the combination of learning strategy theory and knowledge rich learning materials as the pedagogic foundation for adaptively matching student’s requirements to personalised learning environments. Section 2 will describe the related literature and current limitations. Section 3 briefly describes our model and underlying algorithms and section 4 presents our research findings. Finally we will summarise limitations and our results achieved.
•
2.2 Approaches to Matching The main component of a personalised learning environment is the algorithm used to perform matching between the learner requirements and learning resources. A number of different approaches have been used in the development of such an algorithm. Multiple researchers have used Artificial Neural Networks [16][10] to match learner profiles with learning objects. [17][18] have used the concept of swarm intelligence to provide personalised pathways to users through learning materials. Intelligent Algorithms [19][20] and Cluster Filtering [21] concepts have also been used most fundamentally around the ideas of keyword matching in knowledge bases to facilitate more personalized learner experiences. Only a small number of these intelligent systems [22] [23] have focused on taking into consideration the role of the group in guiding individual learning paths. Other limitations include: • Poor instructional design/implementation [24]
2. Related Literature In this section we the development of personalised learning environments and approaches to matching learner requirements with learning materials.
2.1 Personalised Learning Environments A number of educational institutions have focused on producing systems in which environments can adapt to the needs of the user, creating personalised learning experiences. There are a number of different approaches for achieving this, the most pertinent of which to this research are outlined below. In 1999, Dietinger et al. [5] proposed the Dynamic Background Library which uses an intelligent algorithm built around the concepts of keyword relevance and user profiles to filter large knowledge bases for data relevant to what the user wishes to learn about. Harun [6] described a Personalised Continuing Medical Education solution which focused on providing individually relevant learning materials based on expertise, interest and need. The algorithm at the heart of this solution needed to match personal attributes to the knowledge base to retrieve just in time materials. The Web Intelligent Trainer [7] provides a protected environment for novice users who can work on a module, as they would normally do, while the system silently reasons about their actions and offers adaptive tutoring. Most recently De Meo et al. [8] proposed the X-Learn system which is a multi-agent system for adaptive elearning based on user preferences, history, expertise and requirements. Limitations to existing models for personalised learning environments are stated below: • The degree of matching and mismatching between learners and learner materials [9][10] •
Changing learning processes [11]
•
Lack of consideration of learner technology skills [12]
•
Virtual Learning Environment (VLE) integration [14]
Flexibility [13][15]
•
Use of the wrong type of learning styles [25][26]
3. The Adaptive Environment In this paper we will briefly describe and provide some initial results for our Adaptive Intelligent Personalised Learning environment (AIPL) as outlined in Figure 1. The AIPL model was developed to create a personalised map between the individual learner and learning materials.
Figure 1 AIPL Model
3.1 The Learning Profile The learning profile plays an important part in providing information about the learner in the form of an assessment of learning needs, contextual information in 607
object markup allowing knowledge to be searched and retrieved with great efficiency and accuracy. In AIPL, a dynamic library resource can be used to facilitate learning on-line by enabling web-site addresses and ratings to be stored in conjunction with already written static materials.
relation to what the learner is learning about and individual repositories of preferred learning materials. All of this information can then be used by the intelligent algorithm within the system to assist in the filtering of relevant learning materials for the learner.
3.2 The Personalised Adaptive Filtering System (PAFS)
4. Research Findings Looking at previous research in the personalised learning field [27][28] the following areas for testing were identified: sensitivity of the algorithms, emotional response to the learning environment, analysis of interaction with the learning environment and the overall effectiveness of the solution. The test suite involved the development of a web system based on the AIPL model and the design and implementation of associated algorithms. The approach of the testing was built around the concept of a simple learning experience that of learning how to wire a plug. The author used twenty candidates to undertake the testing of the model. The candidates varied in age, computer literacy skills, education, and background to enable a diverse cross section to be analysed. Testing included analysis of the impact of each stage of the algorithm in personalising the learning experience for the user. Results are outlined in the sections below.
PAFS is the intelligent module which is at the heart of this personalisation system. PAFS at its most basic can simply provide a system for retrieving materials from the knowledge repository based on keyword relevance. At its most sophisticated PAFS takes into account user profiles, contextual requirements, semantic data and the role of the group in guiding individual learning paths. PAFS consists of a three-stage evolutionary algorithm. The first stage the Non-Semantic Response Algorithm (NSRA) is a keyword search that searches for defined keywords in specified learning materials and repositories. The NSRA uses a spider or web-crawler to search through Learning Objects to retrieve materials, which only have the specific words that the user choose or the learning profile provides. The NSRA would then identify the defined keywords within learning objects in the repository. Once keywords have been identified in documents a list of documents containing the keywords is retrieved. The user retrieves the on-line educational document and the algorithm reads the contents and provides a summary of keywords present within the selected document. The second stage, the Semantic Bridging Algorithm (SBA) was designed to overcome the problems associated with the NSRA. Due to the dynamics and variations of the particular keyword search, an automated process was introduced to accurately extract personalised educational metadata from the learning objects and compare it to the learner’s specific needs. To achieve this learning objects within the repository are semantically readable through the LOM and SCORM standards. The final stage, the Collaborative Categorization and Semantic Bridging Algorithm was designed to build upon the semantic data and provide the user with mechanisms to retrieve categorisations of learning objects based on the ratings of users with similar learning profiles (ie. group categorisation). This would assist the learner in choosing from lists of resources the most appropriate learning resource. The collaboration and categorisation system uses the Pearson Correlation Algorithm to enable the weight based rating of learning objects.
4.1 Sensitivity of the Algorithm Sensitivity can be related to the complexity of and the performance of the algorithms used. Sensitivity for this research has been measured using the retrieval rate of learning materials in the conduct of a basic and advanced search and a complex set of words searches. The performance has been measured using two types of key word searches enabling a comparisons to be made. In relation to the sensitivity of the three stage algorithm approach within AIPL we found that in general when judged against a human baseline the algorithm retrieved relevant learning objects to the particular scenario and at the final stage recommended specific materials designed for the needs of the learners. We found the more specificity involved in the algorithms parameters, the longer the algorithm took to complete however this reduced the number of relevant learning objects delivered to the users.
4.2 Emotional Response According to Lu et al [29] emotional response can be measured using a state set that is associated with the learning experience. The emotional state set consists of the following states, interest, confusion, frustration, and hopefulness. The emotional states of the candidates were important, because it would enable the authors to analysis
3.3 The Dynamic Background Library The Dynamic Background Library provides a centralised knowledge repository for AIPL. Standard taxonomies i.e. LOM, SCORM can be used for learning 608
the design specification against the theoretical concept of the AIPL model. The states were measured through direct questioning whilst the candidates were using the environment. The findings in relation to the emotional response to the system were quite positive. We found that of the test group selected 90% of users found the system to be none confusing and 80% found the system to be beneficial to their overall learning experience.
materials to allow test candidates to gain an idea of the effectiveness of this algorithm. However, it is noted from responses to this algorithm that the test group did not find this algorithm to be as effective as the initial two algorithms. The initial stage of the algorithm is not complex enough to provide a sufficiently personalized learning experience. Later stages depend on semantic metadata being present obviously this has serious time constraints and implications for the course developer.
4.3 Analysis of Interaction
5. Conclusions and Further Work
The aim of this group of tests was to measure individual responses to how the environment handled given problems. This gives an impression of candidate’s satisfaction with the pedagogical approach, the interface and functionalities therein. Candidates responded well to the example used to test the AIPL environment and in general felt that it was important for the course context and development to be shaped for online learning. However, coming out of this set of tests was the response that 1 in 10 of those sampled had no interest in working online. This fits in with issues around technology use identified by [22][24] in Section 2.1. In relation to the interface itself, in general, the test group had no real concerns. However there were clear indications of where improvements could be made.
Overall, the research detailed within this paper suggests that a model for the development of personalised learning environments (APIL) can be established and with work, complex algorithms can be used to most appropriately match learning resources with learner requirements. In addition, to this a model built on collaboration and feedback mechanisms can best support individual learning paths. Further work from this research is the application of the AIPL environment in more complex learning scenarios such as its implementation within a full time University module. In addition given the findings of the test suite more work should be carried out on the effectiveness of the algorithms particularly in relation to collaborative grouping and changes to the interface itself. An ideal would be to integrate the personalized learning environment within a modular VLE such as Sakai.
4.4 Overall Effectiveness The overall effectiveness of the model was measured through multiple factors. These factors included responses to the handling of student queries, fault identification, degrees of match to conceptual understanding and the recording of personal views. Overall, the response to the testing of the algorithms in relation to their effectiveness in retrieving relevant learning materials was very good. 70% of those tested found the functionality of the algorithm when dealing with their queries was between very good and excellent. A number of faults were identified within the system through this test suite but most were fairly minimal. When asked about the concept of AIPL and the degree in which AIPL’s algorithms met their conceptual ideals, for all algorithms the majority of candidates deemed them to be successful. Finally, candidates found that AIPL matched what it set out to achieve, which was to create a learner centric approach that would bridge together the individual learner to the learning materials.
6. Acknowledgements The authors wish to thank the School of Arts and New Media within the University of Hull for support in enabling the delivery of this paper.
7. References [1] R.M. Felder and R. Brent, "Understanding Student Differences." J. Engr. Education, 94(1), 57-72 (2005) [2] A. Gomes, A. Santos, L. Carmo, and A.J. Mendes, “Learning styles in an e-learning tool”, Proceedings of International Conference on Engineering Education – ICEE 2007, Coimbra, Portugal, September 3 – 7, 2007 [3] R.M. Felder and L.K. Silverman, "Learning and Teaching Styles in Engineering Education," Engr. Education, 78(7), 674681 (1988) [4] Department for Education and Skills, “A National Conversation about Personalised Learning”, DfES Publications, Crown Copyright 2004 [5] T. Dietinger, C. Guetl, B. Knögler, D. Neussl, and K. Schmaranz, “Dynamic Background Libraries - New Developments In Distance Education Using HIKS (Hierarchical Interactive Knowledge System)”, Journal of Universal Computer Science, vol. 5, no. 1 (1999), pp. 2-10
4.5 Limitations of model The final stage of the algorithm depends on the collaboration of users and a feedback response mechanism which can only be firmly developed over a long period of time. The authors pre-rated a sample set of learning 609
conjunction with the 16th ACM Conference on Hypertext and Hypermedia (HT’05) Salzburg, Austria, pp. 27-32 [21] T.Y. Tang and G. McCalla, “Smart Recommendation for an Evolving E-Learning System”, Architecture and Experiment . International Journal on E-Learning. 4 (1), pp. 105-129 [22] L. Capus, F. Curvat, O. Leclair, and N. Tourigny, “A Web environment to encourage students to do exercises outside the classroom: A case study”, Educational Technology & Society, 9 (3), pp.173-181 [23] C.J. Bonk, R.A. Wisher, "Applying collaborative and elearning tools to military distance learning: a research framework", US Army Research Institute for the Behavioral and Social Sciences, Washington, DC, Technical Report [24] K.A. Papanikolaou and M. Grigoriadou, “Accommodating learning style characteristics in Adaptive Educational Hypermedia”. Proceedings of the Workshop on Individual Differences in Adaptive Hypermedia in AH2004, Part I, Eidhoven, Nederlands, August 2004, p. 77-86 [25] S. King, E. Taylor, F. Satzinger, M. Carbonaro, and E. Greidanus, “Developing Interdisciplinary Team Competencies in a Blended Learning Course: Impact on Student Learning”, The Internet Journal of Allied Health Sciences and Practice. Jan 2008, Volume 6 Number 1 [26] S.A. White and I. Liccardi, “Harnessing Insight into Disciplinary Differences to Refine e-learning Design”, Frontiers in Education Conference, 36th Annual (2006), pp. 5-10 [27] J. Shepherd, J. Clendinning and L. Schaverien, “Rethinking e-learning design on generative learning principles”, Proceedings of Ascilite 2002, Auckland, NZ, December 2002 [28] Kindley, “Scenario-Based E-Learning: A Step Beyond Traditional E-Learning”, Learning Circuits, 2002 [29] C. Lu, J. Zhou, L. Shen and R. Shen, Techniques for Enhancing Pervasive Learning in Standard Natural Classroom. In Proceedings of ICHL'2008. pp.202-212
[6] M.H. Harun, “Integrating e-Learning into the workplace”, The Internet and Higher Education, Volume 4, Number 3, 2001 , pp. 301-310(10) [7] K. Kabassi and M. Virvou, “Personalised adult e-training on computer use based on multiple attribute decision making”, Interacting with Computers, Volume 16, Issue 1, February 2004, pp. 115-132 [8] P. De Meo, A. Garro, G. Terracina and D.Ursino, “Personalizing learning programs with X-Learn, an XML-based, ‘user-device’’ adaptive multi-agent system”, Information Sciences: an International Journal, Volume 177 , Issue 8 (April 2007), pp. 1729-1770 [9] S. Visser, S. McChlery, N. Vreken, “Teaching styles versus learning styles in the accounting sciences in the United Kingdom and South Africa: a comparative analysis”, Meditari Accounting Research, Vol. 14, No.2 (2006), pp.97-112 [10] M.T. Tomlinson and B.C. Love, “From Pigeons to Humans: Grounding Relational Learning in Concrete Examples”, Proceedings of the Twenty-First National Conference on Artificial Intelligence, July 16–20, 2006, Boston, Massachusetts, pp.199-204 [11] J. Juhary, “Cadets’ Expectations of Their Learning Environment at the National Defence University of Malaysia”, MERLOT Journal of Online Learning and Teaching, Vol. 4, No. 4, December 2008, pp. 549 – 559 [12] P. Van Rosmalen, H. Vogten, R. Van Es, H. Passier, P. Poelmans, and R. Koper (2006), “Authoring a full life cycle model in standards-based, adaptive e-learning”, Educational Technology & Society, 9 (1), 72-83 [13] V.M. Chieu, "COFALE: An Authoring System for Supporting Cognitive Flexibility", Sixth IEEE International Conference on Advanced Learning Technologies (ICALT'06), Kerkrade, The Netherlands, 5-7 July 2006, pp.335-339 [14] O. Conlan, D. Dagger, and V. Wade, "Towards a Standardsbased Approach to e-Learning Personalization using Reusable Learning Objects", E-Learn 2002, World Conference on ELearning in Corporate, Government, Healthcare and Higher Education, Montreal, September 2002 [15] M. Meccawy, P. Brusilovsky, H. Ashman, M. Yudelson and O. Scherbinina, “Integrating Interactive Learning Content into an Adaptive E-Learning System: Lessons Learned”, Proceedings of World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education 2007, Chesapeake, VA, pp. 6314-6319 [16] K. Scalise, “Differentiated e-Learning: Five Approaches through Instructional Technology”, International Journal of Learning Technology 2007 - Vol. 3, No.2 pp. 169 – 182 [17] J.F. Peters, C. Henry and S. Ramanna, “Reinforcement learning in Swarms that Learn”, Proceedings of the 2005 IEEE/WIC/ACM International Conference on Intelligent Agent Technology, pp. 400 - 406 [18] S. Gutierrez, A. Pardo and C.D. Kloos – “Finding A Learning Path: Towards A Swarm Intelligence Approach”, Proceedings of IASTED Conference on Web Based Education 2006, Puerto Vallarta, Mexico, pp. 94-99 [19] D. Gaševic and M. Hatala, “Ontology mappings to improve learning resource search”, British Journal of Educational Technology, Volume 37, Number 3, May 2006 , pp. 375389(15) [20] A. Carbonaro, R. Ferrini, and M.A. Zamboni, “Considering semantic abilities to improve a web based distance learning system”, Proceedings of the International Workshop Held in
610
