LNCS 4061 - A User-Orientation Evaluation ... - Springer Link
A User-Orientation Evaluation Framework: Assessing Accessibility Throughout the User Experience Lifecycle Alexandros Mourouzis1, Margherita Antona1, Evagelos Boutsakis1, and Constantine Stephanidis1, 2 1 Foundation
for Research and Technology – Hellas (FORTH), Institute of Computer Science, 71300 Heraklion, Greece {mourouzi, antona, boutsak,cs}@ics.forth,gr 2 University of Crete, Department of Computer Science, 71 409 Heraklion, Greece
Abstract. Today, as the users and contexts of use of Information and Communication Technology (ICT) become more diverse, there is a significant need to understand all the factors that may affect the quality of the user-experience, and to measure them systematically. This paper proposes an evaluation framework for specifying and measuring the user-orientation of interactive products. The term “user-orientation” refers to the extent to which target users (will) find the product acceptable at all phases of the user experience lifecycle. The proposed framework incorporates accessibility as a basic determinant of acceptability and long-term adoption. It can be employed effectively in the evaluation of systems that are aimed to be accessible and usable by diverse users (e.g., public systems) or by people with disability.
1 Introduction In order to develop better methods for designing new computer technology, testing designs and predicting how users will respond to new products, it is important to understand why a person may fail or resist using an interactive product. Since the (target) user population of computer technology became more diverse, researchers and practitioners realised that utility and usability alone, although salient acceptance factors, do not necessarily imply that take-up rates of a system will reach their full potential. For instance, in designing and delivering information technology for the public, accessibility needs to be guaranteed, and interfaces are required to cope with a variety of users, including people with different cultural, educational, training, and employment background, novice or experienced users, the very young and the elderly, and people with different types of disability. In fact, accessibility is considered as a prerequisite for usability, since there cannot be optimal interaction if there is no possibility of interaction in the first place [1]. As a result, traditional usability-oriented design and evaluation approaches are not sufficient for delivering high-quality systems that are able to accommodate the needs of diverse user populations with different abilities, skills, requirements and preferences, and to be used in dynamic contexts of use (tasks, equipment and environmental conditions). An analysis of the involved sources of variability reveals the necessity of developing and adopting more comprehensive approaches, capable of specifying and K. Miesenberger et al. (Eds.): ICCHP 2006, LNCS 4061, pp. 421 – 428, 2006. © Springer-Verlag Berlin Heidelberg 2006
422
A. Mourouzis et al.
measuring the user-orientation of products and their interfaces. In this paper, the term “user-orientation” refers to the extent to which target users (will) find the product acceptable at any phase of their ‘interaction’ with it, i.e., throughout the entire user experience lifecycle [2]. This paper defines a user-orientation evaluation framework, which incorporates accessibility as a basic determinant of system acceptability, and describes how such a framework can be employed effectively in the evaluation of systems and their interfaces, including systems that are aimed to be accessible and usable by diverse users (e.g., public systems) or systems developed for people with disability.
2 Research and Methodological Approach Common models of technology acceptance have their roots in a number of diverse theoretical perspectives. One of the most popular is that of Innovation Diffusion Theory [3], which seeks to identify significant perceived characteristics of technology which may impede its adoption by users. On the other side, in social psychological research, theorists seek to identify determinants of behaviour within the individual rather than the technology. For instance, the Theory of Reasoned Action (TRA) [4] has been used to investigate how user beliefs and attitudes are related to individual intentions to perform. TRA provides a complete rationale for the flow of causality from external stimuli (such as system design features) through user perceptions to attitudes about the technology, and finally to actual usage behaviour. Therefore, for the definition of a comprehensive evaluation framework ,one needs to determine, on the one hand, whether a user can use a product - subject to the characteristics of the user and the context of use, and on the other hand, whether a user will use the product - subject to the user’s behavioural situation (see Fig. 1). In terms of user characteristics, key personal differences may be characterized as physiological, psychological or socio-cultural [5] and may involve variances in gender, physical and cognitive abilities, language, culture, experience, background, etc. The context of use involves diversity in terms of user tasks, equipment (at the user site), and in social and environmental conditions [6]. Finally, in terms of the user’s behavioural situation, the Technology Acceptance Model [7] derived from TRA identifies two salient factors: perceived usefulness and perceived ease of use. Perceived usefulness can be considered as the degree to which a potential user believes that using a particular system would match individual goals, and perceived ease of use as the degree to which a potential user believes that accessing and using a particular system would be free of effort. More recent approaches have identified additional constructs, such as, for example, perceived risk (PR), i.e., the degree of uncertainty regarding possible negative consequences of using a product. In [8] the authors propose that PR comprises the facets of performance, financial, time, psychological, social, privacy and overall risk. Furthermore, when measuring user satisfaction as a key determinant of user acceptance, the fact that satisfaction is heavily influenced by expectations should be taken into account. In other words, perceptions of product quality stem from a comparison of what customers feel a product should offer (i.e., their expectations) with the way the functions are actually delivered. A number of factors influence user
A User-Orientation Evaluation Framework
423
expectations, e.g., previous experience, personal needs, implicit service communication, values and beliefs, views about the provider, explicit service communication, and word-of-mouth communication [9].
Fig. 1. Conditions of use: user’s characteristics, context of use and behavioural situation encompass determinant acceptance factors towards and during the use of a product
3 A Comprehensive Evaluation Framework 3.1 A Systematic Perspective In the development of the proposed approach, a number of theoretical and empirical models of technology acceptance were considered. Emphasis though has been put on the user’s decision making process towards the use and adoption of a system. The main generic perspective looks at the process of discovering, identifying the need or interest to use, reaching (i.e., accessing), using, and re-using a system (see Fig.2).
Fig. 2. An abstract model of the user experience lifecycle
Regarding the phase of product usage, and towards long term adoption, previous studies based on empirical data [10] have identified a sequence of three patterns of use: (a) exploration in width, which refers to the phase of the preliminary familiarisation of the user with the interactive environment of a system; (b) occasional (long
424
A. Mourouzis et al.
term) use, which refers to the phase of common use of a system; and (c) exploration in depth, which refers to specialised usage. The proposed approach implies careful consideration of all the factors which may influence an individual at each phase of the lifecycle and concluding upon the likelihood that the individual will proceed or not to a subsequent phase. Ultimately, this allows predicting or assessing the likelihood that a non-user will eventually become a faithful product user (i.e., the evaluating acceptability of the system). 3.2 Framework Definition User-orientation (and thus system acceptability) is measured by the extent to which: − the product1 is made visible to non-users (visibility), − non-users are motivated to gain a personal experience of the system (perceived usefulness & ease of use), − actual users find it easy and acceptable to reach the product (availability/approachability), − actual users find it useful, easy and acceptable to interact with the product (quality of interaction experience), − previous users are motivated to become long term users (relationship maintainability and subjective usefulness & ease of use), − product users are not offered more promising and satisfying alternatives (competitiveness).
Fig. 3. Overview of the user-orientation evaluation framework
1
Note that the product can be of various types including service, software, hardware , function or any combination of these.
A User-Orientation Evaluation Framework
425
Emphasis is given to the Conditions of Use (see Fig.3) and to the fact that the level of user-orientation depends on the specific circumstances in which the product is used, including specific user goals, user characteristics (physiological, psychological, socio-cultural), contexts of use (user’s tasks, equipment and surrounding social and physical environment) and user’s behavioural situations (perceived usefulness, ease of access / use and risks, and expectations). In cases of products that are aimed to be used repeatedly and to be adopted for long term use, emphasis is also given to the fact that the level of user-orientation depends on the current phase of usage lifecycle, including first time and novice usage (seeking product exploration in width), moderate usage (seeking occasional product exploitation), and expert usage (seeking exploration and exploitation in depth). 3.3 User-Orientation Measurements As it becomes apparent from the above definition of the framework, user-orientation reflects the overall product quality perceived by users as a total of the following product qualities (revised from [2]). Visibility. Visibility refers to the degree to which a system can become known to individual2 non-users. Obviously, the actual location of the system is a major visibility factor. Furthermore, visibility can be increased by providers through publicity strategies. Products can, however, be visible to a certain degree even if no promotion takes place, for example through easy location by means of popular web search engines. Awareness might result from unintentionally coming across a product, e.g., while surfing on the Internet. Naturally, the accessibility of the location of the system as well as of the publicity media used is a major factor of a system’s visibility. Perceived Usefulness and Ease of Use. These qualities refer to the usefulness and ease of (access and) use of the system from the viewpoint of individual non-users. These are related to the available information regarding the product and to the extent to which the product appears to be suitable with respect to the user’s particular goals and needs. This also comprises a variety of tangible aspects, such as time and cost savings resulting from the product itself (rather than the way it is delivered). Finally, it can also incorporate less tangible aspects, such as personal intrinsic gratification that can be derived from the fun of, for example, participating in an attractive learning experience/training course. Other social aspects may also play a role, such as prestige and social desirability. Availability / Approachability3. Availability refers to the degree to which all types of potential individual users can reach the entry point (s) of the system. Certainly, accessibility (e.g., for anyone, at any time, from anywhere) of the carrier / storage medium of a system is a major factor for its availability / approachability. At this stage, particular needs and requirements of diverse user populations, such as people with disability, are considered with regards to available ‘routes’ for reaching the product. 2
3
The term “individual” user refers to individual Conditions of Use (user characteristics, context of use, and behavioural situations). The use of the term accessibility (in its literal sense) is avoided here in order to ensure that this is not confused with as ease of access to people with disability.
426
A. Mourouzis et al.
Quality of Interaction Experience. This characteristic encompasses the quality of interaction perceived by actual individual users and refers to the degree to which a system can be used to achieve useful and quality results (i.e., lead to subjective satisfaction). It can also be perceived as the total of the user-orientation of the system’s functions of subjective importance to the individual user. Relationship Maintainability (Subjective Usefulness and Ease of Use). This refers to the degree to which a good relationship with individual system users is effectively cultivated and maintained while the user is not working on the system (e.g., by means of informing the user for new functionality, content updates, changes of status, etc.). The provider may need to adopt specific strategies to ensure the sustainability of this relation. For example, the system provider may offer a richer service package in order to maximally fit the goals and needs of individual users. Competitiveness. Competitiveness is the degree to which the system is conceived by individual users to be more appropriate for them than other available alternatives. For instance, this may be improved by informing the user about reviews of the system and market rankings. 3.4 Accessibility: A Ubiquitous Issue in the User Experience Lifecycle Accessibility can be defined as the extent to which the sequences of input actions of a product, and the associated feedback that lead to successful product use, are possible to be performed by the user, with respect to the individual’s limitations emerging from the particular conditions of use (adapted from [1]). In other words, accessibility ensures that an individual can use a product, whereas usability ensures that the individual finds it easy and satisfying to use it. Thus, further to the framework, accessibility, for instance for people with disability, is an issue penetrating all phases of the user experience lifecycle, raising questions such as “is the product (service, software, hardware, function) visible to people with disability?” or “can the entry point of the system be reached by people with disability?”.
4 Employing the Framework for User Interface Evaluations The proposed framework can also be employed to specify and measure the userorientation of any system part (e.g., interaction devices and peripherals), including individual functions of a system and their corresponding user interfaces (UIs). In other words, when moving deeper into the evaluation of subsystems and system functions, the framework can be iteratively applied to each corresponding UI component (both physical and virtual). Indicatively, assume a function “view incoming emails” provided through an icon in a Web mail application. The function’s visibility is decreased, for example, if the icon is placed at an inappropriate location in design, or increased if information regarding the existence of this function is provided through a help module. Perceived usefulness and ease of use (i.e., prior actually using it) is decreased if the icon does not reflect the utility of the function, or increased if a help module offers a narrative
A User-Orientation Evaluation Framework
427
description of the function’s utility and guidance on using it. Reachability / approachability is decreased if the icon is not accessible (e.g., for blind users) or if it is hard to reach for motor-impaired users who navigate through tabbing. Quality of interaction starts from assessing the behaviour of the icon itself (e.g., is appropriate feedback provided while the icon is being pressed) and continues with assessing each component of the triggered dialogue. Relationship maintainability (i.e., while the user is not logged in the system) is increased, for example, through sms notifications of new incoming mails. Finally, competitiveness can be improved by offering the user information about the performance (success rates etc.) of this function. In general, in assessing a user interface and in order to claim high levels of overall user-orientation, (a) each function needs to be highly user-oriented individually, and at the same time (b) an analogy needs to be achieved between the importance of each function to the user and the corresponding levels of user-orientation of each individual function (see Table 1). Table 1. User-orientation evaluation of a user interface consisting of three functions
Function Function 1 Function 2 Function 3
Importance to the user High Moderate Low
Function’s user orientation X (must be higher or equal to Y) Y (must be higher or equal to Z) Z (must be at least acceptable)
5 Conclusions and Future Work The presented framework4 provides a holistic approach for expert-based or user-based evaluations of interactive systems, specifying at an abstract level all factors that may influence the quality of the user experience with a given system. In the context of user studies, the framework can be employed for setting goals, measurements and data collection instruments for a test, and for preparing test materials such user questionnaires, task scenarios, etc. The framework may also be employed in conducting walkthrough evaluations similarly to traditional cognitive walkthroughs, which “use an explicitly detailed process to simulate a user’s problem-solving process at each step in the human-computer dialogue, checking to see if the simulated user’s goals and memory for actions can be assumed to lead to the next correct action” [11]. The main difference is that, further to the framework, the aim is to simulate a user’s reasoned action process at each step in the human-computer dialogue, checking to see if the simulated user’s beliefs, external stimuli (such as system design features) and intentions to perform can be assumed as a prerequisite to lead to the next interaction step. To this end, this method can be considered as a behavioural walkthrough method, rather than a cognitive one, for inspections of user-orientation, rather than of usability. A paper-based tool implementing this method has been developed and validated in the
4
Part of this work has been carried out in the framework of the European Commission funded project eUSER (“Evidence-based support for the design and delivery of user-centred on-line public services”, Contract no. 507180).
428
A. Mourouzis et al.
context of the eUSER project5, and an online interactive version is also planned for later on. Future plans also include formal validation of the proposed framework throughout evaluation experiments of various types of systems, including mainstream systems developed following a Design for All approach, and systems dedicated to people with disability.
References 1. Savidis, A., & Stephanidis, C.: Unified User Interface Design: Designing Universally Accessible Interactions. In: International Journal of Interacting with Computers, 16 (2) (2004). 243-270. 2. eUSER: eUSER Conceptual and Analytical Framework (first version). In: Kevin Cullen (Ed), eUSER Deliverable D1.1, Part A (2004) 3. Rogers, E.: Diffusion of Innovations. New York: The Free Press (1993) 4. Fishbein, M., & Ajzen, I.: Belief, Attitude, Intention and Behavior: An Introduction to Theory and Research, Addison-Wesley, Reading, MA (1975) 5. Benyon, D., Crerar, A., Wilkinson, S.: Individual Differences and Inclusive Design. In: Stephanidis, C. (Ed.) User Interfaces for All - Concepts, Methods, and Tools. Mahwah, NJ: Lawrence Erlbaum Associates (ISBN 0-8058-2967-9) (2001) 21-46 6. ISO 9241 ISO DIS 9241 – Part 11: Guidance on usability (1994) 7. Davis, F.: Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13, 3 (1989) 319-340 8. Featherman, M.S. & Pavlou, P.A.: Predicting e-Services Adoption: A Perceived Risk Facets Perspective. In: International Journal of Human-Computer Studies, 59, 4 (2003) 451-474 9. MORI: Measuring & Understanding Customer Satisfaction. A MORI Review for the Office of Public Services Reform. London: The Prime Minister’s Office of Public Services Reform (2002) 10. Antona, M., Mourouzis, A., Kartakis, G., & Stephanidis, C.: User requirements and usage life-cycle for digital libraries. In: Proc. of the HCII2005 Conference (CD-ROM) (2005) 11. Mack, R. L., Nielsen, J.: Executive Summary. In Nielsen, J., and Mack, R.L. (Eds.), Usability Inspection Methods. John Wiley & Sons, New York, NY (1994) 1-23
5
See http://www.euser-eu.org/Document.asp?MenuID=123
for Research and Technology – Hellas (FORTH), Institute of Computer Science, 71300 Heraklion, Greece {mourouzi, antona, boutsak,cs}@ics.forth,gr 2 University of Crete, Department of Computer Science, 71 409 Heraklion, Greece
Abstract. Today, as the users and contexts of use of Information and Communication Technology (ICT) become more diverse, there is a significant need to understand all the factors that may affect the quality of the user-experience, and to measure them systematically. This paper proposes an evaluation framework for specifying and measuring the user-orientation of interactive products. The term “user-orientation” refers to the extent to which target users (will) find the product acceptable at all phases of the user experience lifecycle. The proposed framework incorporates accessibility as a basic determinant of acceptability and long-term adoption. It can be employed effectively in the evaluation of systems that are aimed to be accessible and usable by diverse users (e.g., public systems) or by people with disability.
1 Introduction In order to develop better methods for designing new computer technology, testing designs and predicting how users will respond to new products, it is important to understand why a person may fail or resist using an interactive product. Since the (target) user population of computer technology became more diverse, researchers and practitioners realised that utility and usability alone, although salient acceptance factors, do not necessarily imply that take-up rates of a system will reach their full potential. For instance, in designing and delivering information technology for the public, accessibility needs to be guaranteed, and interfaces are required to cope with a variety of users, including people with different cultural, educational, training, and employment background, novice or experienced users, the very young and the elderly, and people with different types of disability. In fact, accessibility is considered as a prerequisite for usability, since there cannot be optimal interaction if there is no possibility of interaction in the first place [1]. As a result, traditional usability-oriented design and evaluation approaches are not sufficient for delivering high-quality systems that are able to accommodate the needs of diverse user populations with different abilities, skills, requirements and preferences, and to be used in dynamic contexts of use (tasks, equipment and environmental conditions). An analysis of the involved sources of variability reveals the necessity of developing and adopting more comprehensive approaches, capable of specifying and K. Miesenberger et al. (Eds.): ICCHP 2006, LNCS 4061, pp. 421 – 428, 2006. © Springer-Verlag Berlin Heidelberg 2006
422
A. Mourouzis et al.
measuring the user-orientation of products and their interfaces. In this paper, the term “user-orientation” refers to the extent to which target users (will) find the product acceptable at any phase of their ‘interaction’ with it, i.e., throughout the entire user experience lifecycle [2]. This paper defines a user-orientation evaluation framework, which incorporates accessibility as a basic determinant of system acceptability, and describes how such a framework can be employed effectively in the evaluation of systems and their interfaces, including systems that are aimed to be accessible and usable by diverse users (e.g., public systems) or systems developed for people with disability.
2 Research and Methodological Approach Common models of technology acceptance have their roots in a number of diverse theoretical perspectives. One of the most popular is that of Innovation Diffusion Theory [3], which seeks to identify significant perceived characteristics of technology which may impede its adoption by users. On the other side, in social psychological research, theorists seek to identify determinants of behaviour within the individual rather than the technology. For instance, the Theory of Reasoned Action (TRA) [4] has been used to investigate how user beliefs and attitudes are related to individual intentions to perform. TRA provides a complete rationale for the flow of causality from external stimuli (such as system design features) through user perceptions to attitudes about the technology, and finally to actual usage behaviour. Therefore, for the definition of a comprehensive evaluation framework ,one needs to determine, on the one hand, whether a user can use a product - subject to the characteristics of the user and the context of use, and on the other hand, whether a user will use the product - subject to the user’s behavioural situation (see Fig. 1). In terms of user characteristics, key personal differences may be characterized as physiological, psychological or socio-cultural [5] and may involve variances in gender, physical and cognitive abilities, language, culture, experience, background, etc. The context of use involves diversity in terms of user tasks, equipment (at the user site), and in social and environmental conditions [6]. Finally, in terms of the user’s behavioural situation, the Technology Acceptance Model [7] derived from TRA identifies two salient factors: perceived usefulness and perceived ease of use. Perceived usefulness can be considered as the degree to which a potential user believes that using a particular system would match individual goals, and perceived ease of use as the degree to which a potential user believes that accessing and using a particular system would be free of effort. More recent approaches have identified additional constructs, such as, for example, perceived risk (PR), i.e., the degree of uncertainty regarding possible negative consequences of using a product. In [8] the authors propose that PR comprises the facets of performance, financial, time, psychological, social, privacy and overall risk. Furthermore, when measuring user satisfaction as a key determinant of user acceptance, the fact that satisfaction is heavily influenced by expectations should be taken into account. In other words, perceptions of product quality stem from a comparison of what customers feel a product should offer (i.e., their expectations) with the way the functions are actually delivered. A number of factors influence user
A User-Orientation Evaluation Framework
423
expectations, e.g., previous experience, personal needs, implicit service communication, values and beliefs, views about the provider, explicit service communication, and word-of-mouth communication [9].
Fig. 1. Conditions of use: user’s characteristics, context of use and behavioural situation encompass determinant acceptance factors towards and during the use of a product
3 A Comprehensive Evaluation Framework 3.1 A Systematic Perspective In the development of the proposed approach, a number of theoretical and empirical models of technology acceptance were considered. Emphasis though has been put on the user’s decision making process towards the use and adoption of a system. The main generic perspective looks at the process of discovering, identifying the need or interest to use, reaching (i.e., accessing), using, and re-using a system (see Fig.2).
Fig. 2. An abstract model of the user experience lifecycle
Regarding the phase of product usage, and towards long term adoption, previous studies based on empirical data [10] have identified a sequence of three patterns of use: (a) exploration in width, which refers to the phase of the preliminary familiarisation of the user with the interactive environment of a system; (b) occasional (long
424
A. Mourouzis et al.
term) use, which refers to the phase of common use of a system; and (c) exploration in depth, which refers to specialised usage. The proposed approach implies careful consideration of all the factors which may influence an individual at each phase of the lifecycle and concluding upon the likelihood that the individual will proceed or not to a subsequent phase. Ultimately, this allows predicting or assessing the likelihood that a non-user will eventually become a faithful product user (i.e., the evaluating acceptability of the system). 3.2 Framework Definition User-orientation (and thus system acceptability) is measured by the extent to which: − the product1 is made visible to non-users (visibility), − non-users are motivated to gain a personal experience of the system (perceived usefulness & ease of use), − actual users find it easy and acceptable to reach the product (availability/approachability), − actual users find it useful, easy and acceptable to interact with the product (quality of interaction experience), − previous users are motivated to become long term users (relationship maintainability and subjective usefulness & ease of use), − product users are not offered more promising and satisfying alternatives (competitiveness).
Fig. 3. Overview of the user-orientation evaluation framework
1
Note that the product can be of various types including service, software, hardware , function or any combination of these.
A User-Orientation Evaluation Framework
425
Emphasis is given to the Conditions of Use (see Fig.3) and to the fact that the level of user-orientation depends on the specific circumstances in which the product is used, including specific user goals, user characteristics (physiological, psychological, socio-cultural), contexts of use (user’s tasks, equipment and surrounding social and physical environment) and user’s behavioural situations (perceived usefulness, ease of access / use and risks, and expectations). In cases of products that are aimed to be used repeatedly and to be adopted for long term use, emphasis is also given to the fact that the level of user-orientation depends on the current phase of usage lifecycle, including first time and novice usage (seeking product exploration in width), moderate usage (seeking occasional product exploitation), and expert usage (seeking exploration and exploitation in depth). 3.3 User-Orientation Measurements As it becomes apparent from the above definition of the framework, user-orientation reflects the overall product quality perceived by users as a total of the following product qualities (revised from [2]). Visibility. Visibility refers to the degree to which a system can become known to individual2 non-users. Obviously, the actual location of the system is a major visibility factor. Furthermore, visibility can be increased by providers through publicity strategies. Products can, however, be visible to a certain degree even if no promotion takes place, for example through easy location by means of popular web search engines. Awareness might result from unintentionally coming across a product, e.g., while surfing on the Internet. Naturally, the accessibility of the location of the system as well as of the publicity media used is a major factor of a system’s visibility. Perceived Usefulness and Ease of Use. These qualities refer to the usefulness and ease of (access and) use of the system from the viewpoint of individual non-users. These are related to the available information regarding the product and to the extent to which the product appears to be suitable with respect to the user’s particular goals and needs. This also comprises a variety of tangible aspects, such as time and cost savings resulting from the product itself (rather than the way it is delivered). Finally, it can also incorporate less tangible aspects, such as personal intrinsic gratification that can be derived from the fun of, for example, participating in an attractive learning experience/training course. Other social aspects may also play a role, such as prestige and social desirability. Availability / Approachability3. Availability refers to the degree to which all types of potential individual users can reach the entry point (s) of the system. Certainly, accessibility (e.g., for anyone, at any time, from anywhere) of the carrier / storage medium of a system is a major factor for its availability / approachability. At this stage, particular needs and requirements of diverse user populations, such as people with disability, are considered with regards to available ‘routes’ for reaching the product. 2
3
The term “individual” user refers to individual Conditions of Use (user characteristics, context of use, and behavioural situations). The use of the term accessibility (in its literal sense) is avoided here in order to ensure that this is not confused with as ease of access to people with disability.
426
A. Mourouzis et al.
Quality of Interaction Experience. This characteristic encompasses the quality of interaction perceived by actual individual users and refers to the degree to which a system can be used to achieve useful and quality results (i.e., lead to subjective satisfaction). It can also be perceived as the total of the user-orientation of the system’s functions of subjective importance to the individual user. Relationship Maintainability (Subjective Usefulness and Ease of Use). This refers to the degree to which a good relationship with individual system users is effectively cultivated and maintained while the user is not working on the system (e.g., by means of informing the user for new functionality, content updates, changes of status, etc.). The provider may need to adopt specific strategies to ensure the sustainability of this relation. For example, the system provider may offer a richer service package in order to maximally fit the goals and needs of individual users. Competitiveness. Competitiveness is the degree to which the system is conceived by individual users to be more appropriate for them than other available alternatives. For instance, this may be improved by informing the user about reviews of the system and market rankings. 3.4 Accessibility: A Ubiquitous Issue in the User Experience Lifecycle Accessibility can be defined as the extent to which the sequences of input actions of a product, and the associated feedback that lead to successful product use, are possible to be performed by the user, with respect to the individual’s limitations emerging from the particular conditions of use (adapted from [1]). In other words, accessibility ensures that an individual can use a product, whereas usability ensures that the individual finds it easy and satisfying to use it. Thus, further to the framework, accessibility, for instance for people with disability, is an issue penetrating all phases of the user experience lifecycle, raising questions such as “is the product (service, software, hardware, function) visible to people with disability?” or “can the entry point of the system be reached by people with disability?”.
4 Employing the Framework for User Interface Evaluations The proposed framework can also be employed to specify and measure the userorientation of any system part (e.g., interaction devices and peripherals), including individual functions of a system and their corresponding user interfaces (UIs). In other words, when moving deeper into the evaluation of subsystems and system functions, the framework can be iteratively applied to each corresponding UI component (both physical and virtual). Indicatively, assume a function “view incoming emails” provided through an icon in a Web mail application. The function’s visibility is decreased, for example, if the icon is placed at an inappropriate location in design, or increased if information regarding the existence of this function is provided through a help module. Perceived usefulness and ease of use (i.e., prior actually using it) is decreased if the icon does not reflect the utility of the function, or increased if a help module offers a narrative
A User-Orientation Evaluation Framework
427
description of the function’s utility and guidance on using it. Reachability / approachability is decreased if the icon is not accessible (e.g., for blind users) or if it is hard to reach for motor-impaired users who navigate through tabbing. Quality of interaction starts from assessing the behaviour of the icon itself (e.g., is appropriate feedback provided while the icon is being pressed) and continues with assessing each component of the triggered dialogue. Relationship maintainability (i.e., while the user is not logged in the system) is increased, for example, through sms notifications of new incoming mails. Finally, competitiveness can be improved by offering the user information about the performance (success rates etc.) of this function. In general, in assessing a user interface and in order to claim high levels of overall user-orientation, (a) each function needs to be highly user-oriented individually, and at the same time (b) an analogy needs to be achieved between the importance of each function to the user and the corresponding levels of user-orientation of each individual function (see Table 1). Table 1. User-orientation evaluation of a user interface consisting of three functions
Function Function 1 Function 2 Function 3
Importance to the user High Moderate Low
Function’s user orientation X (must be higher or equal to Y) Y (must be higher or equal to Z) Z (must be at least acceptable)
5 Conclusions and Future Work The presented framework4 provides a holistic approach for expert-based or user-based evaluations of interactive systems, specifying at an abstract level all factors that may influence the quality of the user experience with a given system. In the context of user studies, the framework can be employed for setting goals, measurements and data collection instruments for a test, and for preparing test materials such user questionnaires, task scenarios, etc. The framework may also be employed in conducting walkthrough evaluations similarly to traditional cognitive walkthroughs, which “use an explicitly detailed process to simulate a user’s problem-solving process at each step in the human-computer dialogue, checking to see if the simulated user’s goals and memory for actions can be assumed to lead to the next correct action” [11]. The main difference is that, further to the framework, the aim is to simulate a user’s reasoned action process at each step in the human-computer dialogue, checking to see if the simulated user’s beliefs, external stimuli (such as system design features) and intentions to perform can be assumed as a prerequisite to lead to the next interaction step. To this end, this method can be considered as a behavioural walkthrough method, rather than a cognitive one, for inspections of user-orientation, rather than of usability. A paper-based tool implementing this method has been developed and validated in the
4
Part of this work has been carried out in the framework of the European Commission funded project eUSER (“Evidence-based support for the design and delivery of user-centred on-line public services”, Contract no. 507180).
428
A. Mourouzis et al.
context of the eUSER project5, and an online interactive version is also planned for later on. Future plans also include formal validation of the proposed framework throughout evaluation experiments of various types of systems, including mainstream systems developed following a Design for All approach, and systems dedicated to people with disability.
References 1. Savidis, A., & Stephanidis, C.: Unified User Interface Design: Designing Universally Accessible Interactions. In: International Journal of Interacting with Computers, 16 (2) (2004). 243-270. 2. eUSER: eUSER Conceptual and Analytical Framework (first version). In: Kevin Cullen (Ed), eUSER Deliverable D1.1, Part A (2004) 3. Rogers, E.: Diffusion of Innovations. New York: The Free Press (1993) 4. Fishbein, M., & Ajzen, I.: Belief, Attitude, Intention and Behavior: An Introduction to Theory and Research, Addison-Wesley, Reading, MA (1975) 5. Benyon, D., Crerar, A., Wilkinson, S.: Individual Differences and Inclusive Design. In: Stephanidis, C. (Ed.) User Interfaces for All - Concepts, Methods, and Tools. Mahwah, NJ: Lawrence Erlbaum Associates (ISBN 0-8058-2967-9) (2001) 21-46 6. ISO 9241 ISO DIS 9241 – Part 11: Guidance on usability (1994) 7. Davis, F.: Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13, 3 (1989) 319-340 8. Featherman, M.S. & Pavlou, P.A.: Predicting e-Services Adoption: A Perceived Risk Facets Perspective. In: International Journal of Human-Computer Studies, 59, 4 (2003) 451-474 9. MORI: Measuring & Understanding Customer Satisfaction. A MORI Review for the Office of Public Services Reform. London: The Prime Minister’s Office of Public Services Reform (2002) 10. Antona, M., Mourouzis, A., Kartakis, G., & Stephanidis, C.: User requirements and usage life-cycle for digital libraries. In: Proc. of the HCII2005 Conference (CD-ROM) (2005) 11. Mack, R. L., Nielsen, J.: Executive Summary. In Nielsen, J., and Mack, R.L. (Eds.), Usability Inspection Methods. John Wiley & Sons, New York, NY (1994) 1-23
5
See http://www.euser-eu.org/Document.asp?MenuID=123
