Publikationen

Auf dieser Seite findest Du eine Übersicht meiner Veröffentlichungen, in denen ich mich intensiv mit der Messung und Optimierung der User Experience auseinandersetze. Die Publikationen spiegeln dabei meine langjährige Auseinandersetzung mit Methoden, Werkzeugen und Studien wider, die darauf abzielen, die Qualität interaktiver Systeme besser zu verstehen und zu steigern.

Ein zentrales Instrument meiner Forschung war und ist der User Experience Questionnaire (UEQ), der unter www.ueq-online.org verfügbar ist. Mit diesem Fragebogen lässt sich die Wahrnehmung der Nutzer*innen erfassen und quantifizieren, um damit wertvolle Impulse für die Weiterentwicklung von Produkten, Dienstleistungen oder Prozessen abzuleiten.

Darüber hinaus findest Du hier auch Arbeiten, die sich allgemein mit der Messbarkeit von UX beschäftigen, sowie Veröffentlichungen, in denen ich verschiedene Perspektiven, Methoden und Herausforderungen der praktischen Anwendung diskutiere. Mein Ziel ist es, Forschung und Praxis noch enger miteinander zu verknüpfen und so zu einer besseren Gestaltung interaktiver Angebote beizutragen.

2024

A Benchmark for the UEQ+ Framework: Construction of a Simple Tool to Quickly Interpret UEQ+ KPIs
Meiners, Anna-Lena; Schrepp, Martin; Hinderks, Andreas; Thomaschewski, Jörg
Link zum vollständigen Artikel: https://doi.org/10.9781/ijimai.2023.05.003
Questionnaires are a highly efficient method to compare the user experience (UX) of different interactive products or versions of a single product. Concretely, they allow us to evaluate the UX easily and to compare different products with a numeric UX score. However, often only one UX score from a single evaluated product is available. Without a comparison to other measurements, it is difficult to interpret an individual score, e.g. to decide whether a product’s UX is good enough to compete in the market. Many questionnaires offer benchmarks to support researchers in these cases. A benchmark is the result of a larger set of product evaluations performed with the same questionnaire. The score obtained from a single product evaluation can be compared to the scores from this benchmark data set to quickly interpret the results. In this paper, the first benchmark for the UEQ+ (User Experience Questionnaire +) is presented, which was created using 3.290 UEQ+ responses for 26 successful software products. The UEQ+ is a modular framework that contains a high number of validated user experience scales that can be combined to form a UX questionnaire. Currently, no benchmark is available for this framework, making the benchmark constructed in this paper a valuable interpretation tool for UEQ+ questionnaires.

Selecting the Appropriate User Experience Questionnaire and Guidance for Interpretation: the UEQ Family
Kollmorgen, Jessica; Hinderks, Andreas; Thomaschewski, Jörg
Link zum vollständigen Artikel: https://doi.org/10.9781/ijimai.2024.08.005
Measuring the user experience (UX) of products, systems and services is individual depending on the research question. On the one hand, the user’s goals and environment play a role in the subjective evaluation. On the other hand, different UX factors are relevant depending on the product. In this case, it is practical to have a questionnaire family as an aid, whose questionnaires are geared towards these different use cases. The User Experience Questionnaire (UEQ) family allows researchers and practitioners to choose the right tool for efficient UX measurement from three questionnaire versions. This article summarizes the UEQ, its short version (UEQ-S) and a modular framework (UEQ+) with overall 27 UX factors and purposes in over 30 different languages. In addition, specific instructions and assistance are provided for the statistical evaluation and interpretation of the questionnaire results. With the help of a key performance indicator (KPI), benchmarks and an importance-performance analysis (IPA), the realization of UX measurements is made easier for researchers and practitioners. To make it even more convenient to choose the right questionnaire from the UEQ family, influencing factors on the UX measurement and recommendations for action are given.

Requirements for User Experience Management – A Tertiary Study
Hinderks, Andreas; Dominguez Mayo, Francisco José; Escalona, Maria José; Thomaschewski, Jörg
Link zum vollständigen Artikel: https://doi.org/10.9781/ijimai.2024.01.004
Today’s users expect to be able to interact with the products they own without much effort and also want to be excited about them. The development of a positive user experience must therefore be managed. We understand management in general as a combination of a goal, a strategy, and resources. When applied to UX, user experience management consists of a UX goal, a UX strategy, and UX resources. We conducted a tertiary study and examined the current state of existing literature regarding possible requirements. We want to figure out, what requirements can be derived from the literature reviews with the focus on UX and agile development. In total, we were able to identify and analyse 16 studies. After analysing the studies in detail, we identified different requirements for UX management. In summary, we identified 13 requirements. The most frequently mentioned requirements were prototypes and UX/usability evaluation. Communication between UX professionals and developers was identified as a major improvement in the software development process. In summary, we were able to identify requirements for UX management of People/Social, Technology/Artifacts, and Process/Practice. However, we could not identify requirements for UX management that enabled the development and achievement of a UX goal.

2023

A Benchmark for the UEQ+ Framework: Construction of a Simple Tool to Quickly Interpret UEQ+ KPIs
Meiners, Anna-Lena; Schrepp, Martin; Hinderks, Andreas; Thomaschewski, Jörg
Link zum vollständigen Artikel: https://doi.org/10.9781/ijimai.2023.05.003
Questionnaires are a highly efficient method to compare the user experience (UX) of different interactive products or versions of a single product. Concretely, they allow us to evaluate the UX easily and to compare different products with a numeric UX score. However, often only one UX score from a single evaluated product is available. Without a comparison to other measurements, it is difficult to interpret an individual score, e.g. to decide whether a product’s UX is good enough to compete in the market. Many questionnaires offer benchmarks to support researchers in these cases. A benchmark is the result of a larger set of product evaluations performed with the same questionnaire. The score obtained from a single product evaluation can be compared to the scores from this benchmark data set to quickly interpret the results. In this paper, the first benchmark for the UEQ+ (User Experience Questionnaire +) is presented, which was created using 3.290 UEQ+ responses for 26 successful software products. The UEQ+ is a modular framework that contains a high number of validated user experience scales that can be combined to form a UX questionnaire. Currently, no benchmark is available for this framework, making the benchmark constructed in this paper a valuable interpretation tool for UEQ+ questionnaires.

Trust, Perspicuity, Efficiency: Important UX Aspects to Consider for the Successful Adoption of Collaboration Tools in Organisations
Meiners, Anna-Lena; Hinderks, Andreas; Thomaschewski, Jörg
Link zum vollständigen Artikel: https://doi.org/10.1007/978-3-031-49425-3\textunderscore 9
Collaboration tools are heavily used in work, education, and leisure. Yet, what makes a good collaboration tool is not well researched. This study focuses on what users expect of collaboration tools by investigating how they are used and which UX aspects are important to users when using them. In a survey, 184 participants described their use of collaboration tools and then rated the importance of 19 given UX aspects in their specific scenario. Results show that seven UX aspects are almost universally seen as most important. Additionally, five aspects seem to be especially relevant in specific usage domains. It is indicated that the context of use, especially the usage domain, influences which UX aspects are important to users. These results can be used by organisations as a guideline when selecting a collaboration tool suitable for their members in order to successfully adopt a tool.

Introduction to special issue on Agile UX: challenges, successes and barriers to improvement
Schön, Eva-Maria; Da Silva, Tiago Silva; Hinderks, Andreas; Sharp, Helen; Thomaschewski, Jörg
Link zum vollständigen Artikel: https://doi.org/10.1016/j.infsof.2023.107193
The integration of Agile software development and User Experience (UX) has become a growing field of research, as both approaches play critical roles in building digital products and services. In this special issue on Agile UX, the current state of the field is explored through a combination of systematic literature reviews and qualitative and quantitative studies. The special issue provide an overview of the key trends, challenges, and successes in combining Agile and UX, and highlight the importance of involving stakeholders throughout development. The shift from plan-driven approaches to Agile UX approaches has brought a focus on human values and a better understanding of the importance of considering users‘ needs. We present recent advances in research and practice, showing that Agile UX is a continuous journey towards changing user behavior by delivering value.

On the Importance of UX Quality Aspects for Different Product Categories
Schrepp, Martin; Kollmorgen, Jessica; Meiners, Anna-Lena; Hinderks, Andreas; Winter, Dominique; Santoso, Harry B.; Thomaschewski, Jörg
Link zum vollständigen Artikel: https://doi.org/10.9781/ijimai.2023.03.001
User experience (UX) is a holistic concept. We conceptualize UX as a set of semantically distinct quality aspects. These quality aspects relate subjectively perceived properties of the user interaction with a product to the psychological needs of users. Not all possible UX quality aspects are equally important for all products. The main use case of a product can determine the relative importance of UX aspects for the overall impression of the UX. In this paper, the authors present several studies that investigate this dependency between the product category and the importance of several well-known UX aspects. A method to measure the importance of such UX aspects is presented. In addition, the authors show that the observed importance ratings are stable, i.e., reproducible, and hardly influenced by demographic factors or cultural background. Thus, the ratings reported in our studies can be reused by UX professionals to find out which aspects of UX they should concentrate on in product design and evaluation.

Development of a Shared UX Vision Based on UX Factors Ascertained Through Attribution
Winter, Dominique; Hausmann, Carolin; Hinderks, Andreas; Thomaschewski, Jörg
Link zum vollständigen Artikel: https://doi.org/10.9781/ijimai.2023.04.001
User experience (UX) is an important quality in differentiating products. For a product team, it is a challenge to develop a good positive user experience. A common UX vision for the product team supports the team in making goal-oriented decisions regarding the user experience. This paper presents an approach to developing a shared UX vision. This UX vision is developed by the product team while a collaborative session. To validate our approach, we conducted a first validation study. In this study, we conducted a collaborative session with two groups and a total of 37 participants. The group of participants comprised product managers, UX designers and comparable professional profiles. At the end of the collaborative session, participants had to fill out a questionnaire. Through questions and observations, we identified ten good practices and four bad practices in the application of our approach to developing a UX vision. The top 3 good practices mentioned by the participants include the definition of decision-making procedures (G1), determining the UX vision with the team (G2), and using general factors of the UX as a basis (G3). The top 3 bad practices are: providing too little time for the development of the UX vision (B1), not providing clear cluster designations (B2) and working without user data (B3). The results show that the present approach for developing a UX vision helps to promote a shared understanding of the intended UX in a quickly and simply way.

2022

Approaches to manage the user experience process in Agile software development: A systematic literature review
Hinderks, Andreas; Dominguez Mayo, Francisco José; Thomaschewski, Jörg; Escalona, Maria José
Link zum vollständigen Artikel: https://doi.org/10.1016/j.infsof.2022.106957
Context: Software development companies use Agile methods to develop their products or services efficiently and in a goal-oriented way. But this alone is not enough to satisfy user demands today. It is much more important nowadays that a product or service should offer a great user experience — the user wants to have some positive user experience while interacting with the product or service.Objective: An essential requirement is the integration of user experience methods in Agile software development. Based on this, the development of positive user experience must be managed. We understand management in general as a combination of a goal, a strategy, and resources. When applied to UX, user experience management consists of a UX goal, a UX strategy, and UX resources.Method: We have conducted a systematic literature review (SLR) to analyse suitable approaches for managing user experience in the context of Agile software development.Results: We have identified 49 relevant studies in this regard. After analysing the studies in detail, we have identified different primary approaches that can be deemed suitable for UX management. Additionally, we have identified several UX methods that are used in combination with the primary approaches.Conclusions: However, we could not identify any approaches that directly address UX management. There is also no general definition or common understanding of UX management. To successfully implement UX management, it is important to know what UX management actually is and how to measure or determine successful UX management.

UX Poker: Estimating the Influence of User Stories on User Experience in Early Stage of Agile Development
Hinderks, Andreas; Winter, Dominique; Dominguez Mayo, Francisco José; Escalona, Maria José; Thomaschewski, Jörg
Link zum vollständigen Artikel: https://doi.org/10.9781/ijimai.2022.11.007
Agile methods are used more and more frequently to develop products by reducing development time. Requirements are typically written in user stories or epics. In this paper, a new method called UX Poker is presented. This is a method to estimate the impact of a user story on user experience before development. Thus, there is the opportunity that the product backlog can also be sorted according to the expected UX. To evaluate UX Poker, a case study was conducted with four agile teams. Besides, a workshop followed by a questionnaire was conducted with all four agile teams. The goal of being able to estimate the UX even before development was achieved. Using UX Poker to create another way to sort the product backlog can be considered achieved in this first evaluation. The results show that UX Poker can be implemented in a real- life application. Additionally, during the use of UX Poker, it was found that a shared understanding of UX began. The participants clarified in the team discussion about UX Poker what related to influence the user stories had on UX and what UX meant for their product.

A Benchmark for the UEQ+ Framework: Construction of a Simple Tool to Quickly Interpret UEQ+ KPIs
Meiners, Anna-Lena; Schrepp, Martin; Hinderks, Andreas; Thomaschewski, Jörg
Link zum vollständigen Artikel: https://doi.org/10.9781/ijimai.2023.05.003
Questionnaires are a highly efficient method to compare the user experience (UX) of different interactive products or versions of a single product. Concretely, they allow us to evaluate the UX easily and to compare different products with a numeric UX score. However, often only one UX score from a single evaluated product is available. Without a comparison to other measurements, it is difficult to interpret an individual score, e.g. to decide whether a product’s UX is good enough to compete in the market. Many questionnaires offer benchmarks to support researchers in these cases. A benchmark is the result of a larger set of product evaluations performed with the same questionnaire. The score obtained from a single product evaluation can be compared to the scores from this benchmark data set to quickly interpret the results. In this paper, the first benchmark for the UEQ+ (User Experience Questionnaire +) is presented, which was created using 3.290 UEQ+ responses for 26 successful software products. The UEQ+ is a modular framework that contains a high number of validated user experience scales that can be combined to form a UX questionnaire. Currently, no benchmark is available for this framework, making the benchmark constructed in this paper a valuable interpretation tool for UEQ+ questionnaires.

2020

Korrelationen zwischen UX-Fragebögen
Meiners, Anna-Lena; Hinderks, Andreas; Thomaschewski, Jörg
Link zum vollständigen Artikel: https://doi.org/10.18420/muc2020-ws105-375
Zur Messung der User Experience sind zahlreiche Fragebögen entwickelt worden. Einige von ihnen sind so konstruiert worden, dass sie einen Key Performance Indicator (KPI) berechnen. In diesem Beitrag sind wir der Forschungsfrage nachgegangen, ob die KPIs von etablierten Fragebögen miteinander vergleichbar sind. Zu Beantwortung dieser Frage haben wir eine Studie mit 199 Teilnehmern mit dem Testobjekt YouTube durchgeführt. Ziel war es, die Korrelation der KPIs zwischen dem User Experience Questionnaire+ (UEQ+), der System Usability Scale (SUS) und dem Net Promotor Score (NPS) zu berechnen. Im Ergebnis zeigte sich, dass der UEQ+ Faktor \glqqIntuitive Bedienung\grqq hoch mit der SUS KPI korreliert. Alle anderen Korrelationen weisen moderate bis geringeWerte auf. Abschießend kann zusammengefasst werden, dass die KPIs der einzelnen Fragebögen nicht generell und allgemein vergleichbar sind. In Detail kann aber eine Vergleichbarkeit auf Skalenebene durchgeführt werden.

Measuring User Experience Quality of Voice Assistants
Klein, Andreas M.; Hinderks, Andreas; Schrepp, Martin; Thomaschewski, Jörg
Link zum vollständigen Artikel: https://doi.org/10.23919/CISTI49556.2020.9140966
Virtual digital assistants are predicted to experience enormous growth in enterprise applications worldwide. Popular voice assistants are integrated into many devices and systems. Measuring the quality of user experience (UX) must, therefore, also be of great interest to evaluate and improve voice assistants. In this article, three scales for measuring the UX aspects of voice assistance systems are presented, which, in combination with a new questionnaire concept, allow these measurements. It is shown with which method the scales have been constructed and analysed, especially for systems with voice user interfaces. The results are three new scales for voice communication that can be combined with the modular questionnaire concept of the UEQ+.

Construction of UEQ+ Scales for Voice Quality: Measuring User Experience Quality of Voice Interaction
Klein, Andreas M.; Hinderks, Andreas; Schrepp, Martin; Thomaschewski, Jörg
Link zum vollständigen Artikel: https://doi.org/10.1145/3404983.3410003
The UEQ+ is a modular framework for the construction of UX questionnaires. The researcher can pick those scales that fit his or her research question from a list of 16 available UX scales. Currently, no UEQ+ scales are available to allow measuring the quality of voice interactions. Given that this type of interaction is increasingly essential for the usage of digital products, this is a severe limitation of the possible products and usage scenarios that can be evaluated using the UEQ+. We describe in this paper the construction of three specific scales to measure the UX of voice interactions. Besides, we discuss how these new scales can be combined with existing UEQ+ scales in evaluation projects.

Measuring User Experience Quality of Voice Assistants
Klein, Andreas M.; Hinderks, Andreas; Schrepp, Martin; Thomaschewski, Jorg
Link zum vollständigen Artikel: https://doi.org/10.23919/CISTI49556.2020.9140966
Virtual digital assistants are predicted to experience enormous growth in enterprise applications worldwide. Popular voice assistants are integrated into many devices and systems. Measuring the quality of user experience (UX) must, therefore, also be of great interest to evaluate and improve voice assistants. In this article, three scales for measuring the UX aspects of voice assistance systems are presented, which, in combination with a new questionnaire concept, allow these measurements. It is shown with which method the scales have been constructed and analysed, especially for systems with voice user interfaces. The results are three new scales for voice communication that can be combined with the modular questionnaire concept of the UEQ+.

Construction of Voice Communication Scales for the UEQ+ Framework: Protocol for Measuring User Experience Quality of Voice Assistants
Klein, Andreas M.; Hinderks, Andreas; Schrepp, Martin; Thomaschewski, Jörg

An SLR-tool: search process in practice
Hinderks, Andreas; Mayo, Francisco José Dominguez; Thomaschewski, Jörg; Escalona, Maria José
Link zum vollständigen Artikel: https://doi.org/10.1145/3377812.3382137
Systematic Literature Reviews (SLRs) have established themselves as a method in the field of software engineering. The aim of an SLR is to systematically analyze existing literature in order to answer a research question. In this paper, we present a tool to support an SLR process. The main focus of the SLR tool (https://www.slr-tool.com/) is to create and manage an SLR project, to import search results from search engines, and to manage search results by including or excluding each paper. A demo video of our SLR tool is available at https://youtu.be/Jan8JbwiE4k.

Applying Importance-Performance Analysis (IPA) to Interpret the Results of the User Experience Questionnaire (UEQ)
Hinderks, Andreas; Dominguez Mayo, Francisco José; Meiners, Anna-Lena; Thomaschewski, Jörg
Link zum vollständigen Artikel: https://doi.org/10.13052/jwe1540-9589.1926
In recent years, user experience questionnaires have established themselves to measure various aspects of User Experience (UX). In addition to these questionnaires, an evaluation tool is usually offered so that the results of a study can be evaluated in the light of the questionnaire. As a rule, the evaluation consists of preparing the data and comparing it with a benchmark. Often this interpretation of the data is not sufficient as it only evaluates the current User Experience. However, it is desirable to determine exactly where there is a need for action. The User Experience Questionnaire (UEQ) is a common and valid questionnaire with an evaluation tool to measure and analyse the User Experience for a product or service. In our article we present an approach that evaluates the results from the User Experience Questionnaire using the importance-performance analysis (IPA). The aim is to create another possibility to interpret the results of the UEQ and to derive recommendations for action from them. In a study with 467 participants, we validated the approach presented with YouTube, WhatsApp, and Facebook. The results show that the IPA provides additional insights from which further recommendations for action can be derived.

Exploring Voice Assistant Risks and Potential with Technology-based Users
Klein, Andreas M.; Hinderks, Andreas; Rauschenberger, Maria; Thomaschewski, Jörg
Link zum vollständigen Artikel: https://doi.org/10.5220/0010150101470154
Voice user interfaces (VUIs) or voice assistants (VAs) such as Google Home or Google Assistant (Google), Cortana (Mircosoft), Siri (Apple) or Alexa (Amazon) are highly available in the consumer sector and present a smart home trend. Still, the acceptance seems to be culture-dependent, while the syntax of communication poses a challenge. So, there are some basic questions: `Why do people buy VAs?‘ `What do they use them for?‘ `What could be improved in the future?‘. We explore the opinion of a German technology-based user group to identify the challenges and opportunities of VAs. We focus on the interaction behaviour, frequency of use, concerns, and opinions of this target group as they show a higher variety of interaction as well as privacy concerns in representative population studies. Our preliminary findings confirm previous results (missing accuracy of commands and serious concerns about privacy issues) and show that technology-based users from Germany are intensive users, althou gh with particular concerns about data collection. Probably, there is a correlation between privacy concerns and speech intelligibility as queries relating to VAs are problematic due to repetitions and refinement.

2019

Developing a UX KPI based on the user experience questionnaire
Hinderks, Andreas; Schrepp, Martin; Dominguez Mayo, Francisco José; Escalona, Maria José; Thomaschewski, Jörg
Link zum vollständigen Artikel: https://doi.org/10.1016/j.csi.2019.01.007
Decisions in Companies are made typically by using a number of entirely different key figures. A user experience key figure is one of many important key figures that represents one aspect of the success of the company or its products. What we aim in this article is to present to those responsible for a product a method of how a user experience key performance indicator (UX KPI) can be developed using a UX questionnaire. We have developed a UX KPI for use in organizations based on the User Experience Questionnaire (UEQ). To achieve this, we added six questions to the UEQ to measure the importance of the UEQ scales. Based on the UEQ scales and the scores given for importance, we then developed a User Experience Questionnaire KPI (UEQ KPI). In a first study with 882 participants, we calculated and discussed the UEQ KPI using Amazon and Skype. The results show that the six supplementary questions could be answered independently of the UEQ itself. In our opinion, the extension can be implemented without any problems. The resulting UEQ KPI can be used for communication within an organization as a key performance indicator.

UX-Methoden richtig auswählen
Raddy, Katinka; Winter, Dominique; Hinderks, Andreas; Thomaschewski, Jörg
Link zum vollständigen Artikel: https://doi.org/10.18420/muc2019-up-0347
In unterschiedlichen Situationen müssen UX-Professionals geeignete Usability- und UX-Methoden für ein Projekt auswählen. Der Human-Centered-Design-Prozess gibt jedoch keine Methoden vor, so dass je nach Produkt und Projektgröße vom Produktentwicklungsteam Auswahlentscheidungen aufgrund von Erfahrungen und Vorlieben getroffen werden. Dabei greift das Team oft auf Methoden zurück, bei denen sich die UX-Professionals in der Durchführung sicher fühlen. Ob die Methoden aber ideal zur Aufgabenstellung passen oder die Aufwände angemessen sind, kann durch diese Vorgehensweise leicht übersehen werden. In unserem Workshop lernen die Teilnehmer*innen zunächst verschiedene Methoden-sammlungen und -klassifikationen kennen. Sie werden in die Lage versetzt projektabhängige Lern- und Durchführungsaufwände von Methoden zu bestimmen und Maßnahmen zur Erlernbarkeit eben dieser Methoden vorzunehmen. Darüber hinaus werden Sie befähigt die Auswahl geeigneter Methoden nachvollziehbar zu begründen.

Interpreting the Results from the User Experience Questionnaire (UEQ) using Importance-Performance Analysis (IPA)
Hinderks, Andreas; Meiners, Anna-Lena; Mayo, Francisco; Thomaschewski, Jörg
Link zum vollständigen Artikel: https://doi.org/10.5220/0008366503880395
User Experience Questionnaire is a common and valid method to measure the User Experience (UX) for a product or service. In recent years, these questionnaires have established themselves to measure various aspects of UX. In addition to the questionnaire, an evaluation tool is usually offered so that the results of a study can be evaluated in the light of the questionnaire. As a rule, the evaluation consists of preparing the data and comparing it with a benchmark. Often this interpretation of the data is not sufficient as it only evaluates the current User Experience. However, it is desirable to determine exactly where there is a need for action. In our article we present an approach that evaluates the results from the User Experience Questionnaire (UEQ) using the importance-performance analysis (IPA). The aim is to create another possibility to interpret the results of the UEQ and to derive recommendations for action from them. In a first study with 219 participants, we validated the approach presented with YouTube and WhatsApp. The results show that the IPA provides additional insights from which further recommendations for action can be derived.

Applicability of User Experience and Usability Questionnaires
Hinderks, Andreas; Winter, Dominique; Schrepp, Martin; Thomaschewski, Jörg

To be successful, interactive products need to fulfil user expectations and create a positive user experience (UX). An established method to measure UX involves questionnaires. What we aim in this paper is to present a list of user experience and usability questionnaires and its applicability for different digital products. A total of 13 questionnaires on usability and UX were analysed for this paper, and 25 factors were extracted from those questionnaires. A study was conducted based on this collection of factors with N=61 students. The study investigated the perceived importance of usability and UX factors for seven digital products. The goal was to have a collection of usability and UX factors that could be combined for suitable products evaluation. The results of the study revealed that no questionnaire covered all the factors perceived important by the participants.

A Methodology for Integrating User Experience Methods and Techniques into Agile Software Development
Hinderks, Andreas

Agile methods are used more and more frequently to develop products to reduce development time. However, for developing desirable products with good user experience, Agile methods alone are not enough. This Ph.D. thesis explores how an iterative and incremental process model in the context of Agile methods must be designed in order to manage and improve product quality in terms of user experience. For this purpose, the design science research methodology is used. Using existing and new methods, an iterative and incremental process model is developed, which is validated in an international study. This Ph.D. thesis contributes to the knowledge base of software development by providing a process model that offers an iterative and incremental improvement of product quality in terms of user experience and process quality.

Interpreting the results from the User Experience Questionnaire (UEQ) using Importance-Performance Analysis (IPA)
Hinderks, Andreas; Meiners, Anna-Lena; Dominguez Mayo, Francisco José; Thomaschewski, Jörg
Link zum vollständigen Artikel: https://doi.org/10.5220/0008366503880395
User Experience Questionnaire is a common and valid method to measure the User Experience (UX) for a product or service. In recent years, these questionnaires have established themselves to measure various aspects of UX. In addition to the questionnaire, an evaluation tool is usually offered so that the results of a study can be evaluated in the light of the questionnaire. As a rule, the evaluation consists of preparing the data and comparing it with a benchmark. Often this interpretation of the data is not sufficient as it only evaluates the current User Experience. However, it is desirable to determine exactly where there is a need for action. In our article we present an approach that evaluates the results from the User Experience Questionnaire (UEQ) using the importance-performance analysis (IPA). The aim is to create another possibility to interpret the results of the UEQ and to derive recommendations for action from them. In a first study with 219 participants, we validated the approach presented with YouTube and WhatsApp. The results show that the IPA provides additional insights from which further recommendations for action can be derived.

2018

A Benchmark for the Short Version of the User Experience Questionnaire
Hinderks, Andreas; Schrepp, Martin; Thomaschewski, Jörg
Link zum vollständigen Artikel: https://doi.org/10.5220/0007188303730377
o enable an interactive product to provide adequate user experience (UX), it is important to ensure the quantitative measurability of this parameter. The User Experience Questionnaire (UEQ) is a well-known and popular method for such a UX measurement. One of the key features of this questionnaire is a benchmark that helps to interpret measurement results by a comparison with a large dataset of results obtained for other products. For situations where filling out the entire UEQ is too time-consuming, there is a short version (UEQ-S). However, there is currently no sufficient data available to construct an independent and interpretable benchmark for this short version. This paper examines the efficiency of using a modified version of the existing benchmark of the full UEQ for this purpose. The paper also presents some additional evaluation results concerning the UEQ-S.

Vergleich von UX Fragebögen: Evaluationsbericht von Whatsapp, Netflix und Moodle mit meCUE, Visawi und UEQ
Hinderks, Andreas; Schrepp, Martin; Thomaschewski, Jörg
Link zum vollständigen Artikel: https://doi.org/10.13140/RG.2.2.36124.18560
In diesem Evaluationsbericht sind die detaillierten Ergebnisse einer Studie mit drei verschiedenen UX Fragebögen und drei Testobjekten dargestellt. In dieser Studie sollten UX Fragebögen miteinander verglichen werden, die sowohl in einer deutschen Version frei zur Verfügung stehen als auch über die notwendigen Auswertetools verfügen. Wir haben den meCUE, VisAWI und UEQ für die Studie ausgewählt. Die Studie ist motiviert durch die Notwendigkeit, den Zusammenhang zwischen der bewerteten Qualitäten der drei UX Fragebögen zu verstehen. Da alle drei UX Fragebögen Teilaspekte der User Experience messen, ist davon auszugehen, dass eine Korellation zwischen den einzelnen Dimensionen der UX Fragebögen vorhanden ist.

A Benchmark for the Short Version of the User Experience Questionnaire
Hinderks, Andreas; Schrepp, Martin; Thomaschewski, Jörg
Link zum vollständigen Artikel: https://doi.org/10.5220/0007188303730377
To enable an interactive product to provide adequate user experience (UX), it is important to ensure the quantitative measurability of this parameter. The User Experience Questionnaire (UEQ) is a well-known and popular method for such a UX measurement. One of the key features of this questionnaire is a benchmark that helps to interpret measurement results by a comparison with a large dataset of results obtained for other products. For situations where filling out the entire UEQ is too time-consuming, there is a short version (UEQ-S). However, there is currently no sufficient data available to construct an independent and interpretable benchmark for this short version. This paper examines the efficiency of using a modified version of the existing benchmark of the full UEQ for this purpose. The paper also presents some additional evaluation results concerning the UEQ-S.

Steigerung der organisationalen UX-Kompetenz
Winter, Dominique; Hinderks, Andreas; Thomaschewski, Jörg
Link zum vollständigen Artikel: https://doi.org/10.18420/muc2018-up-0135
Zur Verbesserung der User Experience aller Produkte einer Organisation geht, brauchen Organisationen geeignete Maßnahmen zur Steigerung ihrer UX-Kompetenz. Gemeinsam mit 30 UX-Professionals wurden in einem Workshop 24 Maßnahmen zusammengetragen, die zur Verbesserung verwendet werden können. Diese Maßnahmen wurden anschließend von weiteren UX-Professionals auf ihre Wirkung bewertet. Der Einfluss der Verbesserungsmaßnahmen auf Individuen, Gruppen und Organisation ist entscheidend für die Auswahl der empfohlenen Maßnahmen. Es wurden 11 Maßnahmen her-ausgearbeitet, die zur Förderung der UX-Kompetenz von Organisationen empfohlen werden können.

Herausforderungen beim Einsatz der Faktorenanalyse bei der Konstruktion von UX Fragebögen
Hinderks, Andreas; Thomaschewski, Jörg
Link zum vollständigen Artikel: https://doi.org/10.18420/muc2018-ws16-0504
Bei der Konstruktion von User Experience Fragebögen werden verschiedene statistische Methoden angewendet. Eine weit verbreitete Methode ist die Faktorenanalyse. Diese kann sowohl bei der Konstruktion als auch als Bestätigung des Faktorenmodells eines Fragebogens dienen.In diesem Beitrag werden wir einen Überblick über den Einsatz der Faktorenanalyse geben und mögliche Probleme bezüglich der Ermittlung des Faktorenmodells diskutieren. Dabei zeigen wir anhand von iterativ ermittelten Faktorenmodellen, dass es kein eindeutiges Faktorenmodell gibt. Diese Erkenntnis hat unweigerlich Auswirkungen auf den Einsatz der Faktorenanalyse, die wir im abschließenden Kapitel dieses Beitrags diskutieren werden.

Welcher UX-Fragebogen passt zu meinem Produkt?
Thomaschewski, Jörg; Hinderks, Andreas; Schrepp, Martin
Link zum vollständigen Artikel: https://doi.org/10.18420/muc2018-up-0150
UX-Fragebögen werden oft eingesetzt, um die User Experience eines Produktes zu ermitteln. Etabliert haben sich im deutschsprachigen Raum einige Fragebögen, die einen unterschiedlichen Bekanntheitsgrad haben und in der Praxis oftmals aufgrund der Bekanntheit verwendet werden.In diesem Beitrag werden die Dimensionen der etablierten Fragebögen (AttrakDiff2, meCUE, SUS, SUMI, UEQ und VisAWI) verglichen und es wird gezeigt, dass die Fragebogenauswahl anhand der Produkteigenschaften und deren UX-Faktoren gezielt stattfinden kann.

Vergleich von UX Fragebögen
Hinderks, Andreas; Schrepp, Martin; Thomaschewski, Jörg
Link zum vollständigen Artikel: https://doi.org/10.18420/muc2018-mci-0363
Wir untersuchen drei verbreitete Anwendungen (Whatsapp, Netflix, Moodle) mit drei im deutschspra-chigen Raum häufig verwendeten UX Fragebögen (VISAWI, meCUE, UEQ). Ziel ist die Beziehung der Skalen dieser Fragebögen näher zu untersuchen. Überraschenderweise zeigte sich, dass die Korre-lationen der Skalen innerhalb eines Fragebogens stets deutlich höher sind, als zwischen verschiedenen Fragebögen. Zusätzlich hat die Art des untersuchten Produkts offenbar einen Einfluss auf die Bezie-hungen zwischen den Skalen verschiedener Fragebögen.

2017

Konstruktion einer Kurzversion des User Experience Questionnaire
Schrepp, Martin; Hinderks, Andreas; Thomaschewski, Jörg
Link zum vollständigen Artikel: https://doi.org/10.18420/muc2017-mci-0006
Der User Experience Questionnaire (UEQ) ist ein weit verbreiteter Fragebogen zur Messung der subjektiv wahrgenommenen User Experience. Der Fragebogen ist mit 3-5 Minuten Zeitbedarf schon recht effizient, aber für einige spezielle Anwendungsszenarien noch zu aufwändig. Wir beschreiben in diesem Beitrag die Erstellung einer Kurzversion des UEQ mit nur 8 Items. Es werden erste Evaluationsergebnisse für diese Kurzversion vorgestellt.

Die UX KPI: Wunsch und Wirklichkeit
Schrepp, Martin; Hinderks, Andreas; Thomaschewski, Jörg
Link zum vollständigen Artikel: https://doi.org/10.18420/muc2017-up-0100
Manager sind daran gewöhnt, Informationen zu wichtigen Aspekten der aktuellen Unternehmenssituation in einer einzigen Kennzahl (z.B. Umsatz, Gewinn, NPS) zusammenzufassen. Das führt naturgemäß zum Wunsch, auch bezüglich UX eine solche Kennzahl (KPI) zu nutzen. Damit eine gute UX erreicht wird, soll ein Produkt leicht zu erlernen, effizient zu bedienen, ästhetisch gestaltet sein, etc. Eine gute UX erfordert also eine Reihe unterschiedlicher UX Qualitäten, deren Wichtigkeit dabei noch zwischen Produkten und Nutzergruppen variieren kann. Eine UX KPI muss dies natürlich berücksichtigen, um einen vernünftig interpretierbaren Wert zu liefern. Wir werden zeigen, wie man auf Basis existierender UX Fragebögen zu einer interpretierbaren UX KPI kommen kann. Zusätzlich werden die Vorteile, aber auch die Gefahren einer solchen Reduktion der Messung von UX auf eine einzige Kennzahl besprochen.

Cultural Differences in the Perception of User Experience
Santoso, Harry; Schrepp, Martin; Hinderks, Andreas; Thomaschewski, Jörg
Link zum vollständigen Artikel: https://doi.org/10.18420/muc2017-mci-0272
Many products are developed for an international audience. But product design is typically done by a small group of designers, which is often homogeneous concerning their cultural background. In addition, the adaption of products to different cultural contexts is an expensive exercise and is thus in most cases not even considered in the product planning. Currently not much is known on the influence of cultural aspects on the interaction design of products, since doing empirical research in this area is quite difficult. We investigate how Indonesian and German students judge common products concerning their user experience. We present some first results and possible explanations of the detected differences.

Construction of a Benchmark for the User Experience Questionnaire (UEQ)
Schrepp, Martin; Hinderks, Andreas; Thomaschewski, Jörg
Link zum vollständigen Artikel: https://doi.org/10.9781/ijimai.2017.445
Questionnaires are a cheap and highly efficient tool for achieving a quantitative measure of a product’s user experience (UX). However, it is not always easy to decide, if a questionnaire result can really show whether a product satisfies this quality aspect. So a benchmark is useful. It allows comparing the results of one product to a large set of other products. In this paper we describe a benchmark for the User Experience Questionnaire (UEQ), a widely used evaluation tool for interactive products. We also describe how the benchmark can be applied to the quality assurance process for concrete projects.

UX-Kompetenz von Organisationen: Wie kann die UX-Kompetenz einer Organisation gesteigert werden?
Winter, Dominique; Bittenbinder, Sven; Hinderks, Andreas; Thomaschewski, Jörg
Link zum vollständigen Artikel: https://doi.org/10.18420/muc2017-up-0003
Um die User Experience einzelner Produkte zu verbessern stehen bereits einige Methoden zur Verfügung. Möchten wir aber alle Produkte verbessern, müssen wir unseren Blick auf die herstellende Organisation richten. Dabei stellt sich die Frage, wie wir die UX-Kompetenz einer ganzen Organisation verbessern können. Gemeinsam mit den Teilnehmern suchen wir Antworten auf diese Frage. Wir erarbeiten die häufigsten Problemstellen und finden Ansatzpunkte zur Verbesserung. Dabei fokussieren wir uns auf zwei entscheidene Bereiche zur Kompetenzentwicklung: Erzeugen neuer Kompetenz und Verbesserung der Anwendbarkeit der bestehenden Kompetenz.

Welche UX Faktoren sind für mein Produkt wichtig?
Winter, Dominique; Hinderks, Andreas; Schrepp, Martin; Thomaschewski, Jörg
Link zum vollständigen Artikel: https://doi.org/10.18420/muc2017-up-0002
Langwierige Diskussionen, bei denen sich die beteiligten Personen über einzelne Aspekte eines Designs nicht einigen können, sind für erfahrene UX-Professional nichts Neues. Solche Konflikte sind in der Regel kraft- und zeitraubend. Diese Konflikte resultieren oft aus unterschiedlichen Einschätzungen bzgl. der Wichtigkeit bestimmter Qualitätseigenschaften eines Produkts. Allerdings ist den meisten Produktteams, und damit den Beteiligten am Design-Prozess, nicht bewusst welche Aspekte wichtig sind und welche vernachlässigt werden können. Das macht die Diskussion in solchen Konfliktsituationen umso schwieriger, da die Beteiligten aus ihrer jeweiligen Perspektive Recht haben.Wir stellen einen Prozess vor, der die Reduzierung von Abstimmungskonflikten unterstützt und die Produktentwicklung dadurch beschleunigt. Als Grundlage verwenden wir 16 klar beschriebene UX-Faktoren, welche in empirischen Studien ermittelt und bzgl. ihrer Wichtigkeit für bestimmte Produktgruppen bewertet wurden.

Design and Evaluation of a Short Version of the User Experience Questionnaire (UEQ-S)
Schrepp, Martin; Hinderks, Andreas; Thomaschewski, Jörg
Link zum vollständigen Artikel: https://doi.org/10.9781/ijimai.2017.09.001
The user experience questionnaire (UEQ) is a widely used questionnaire to measure the subjective impression of users towards the user experience of products. The UEQ is a semantic differential with 26 items. Filling out the UEQ takes approximately 3-5 minutes, i.e. the UEQ is already reasonably efficient concerning the time required to answer all items. However, there exist several valid application scenarios, where filling out the entire UEQ appears impractical. This paper deals with the creation of an 8 item short version of the UEQ, which is optimized for these specific application scenarios. First validations of this short version are also described.

Construction of a Benchmark for the User Experience Questionnaire (UEQ)
Schrepp, Martin; Hinderks, Andreas; Thomaschewski, Jörg
Link zum vollständigen Artikel: https://doi.org/10.9781/ijimai.2017.445
Questionnaires are a cheap and highly efficient tool for achieving a quantitative measure of a product’s user experience (UX). However, it is not always easy to decide, if a questionnaire result can really show whether a product satisfies this quality aspect. So a benchmark is useful. It allows comparing the results of one product to a large set of other products. In this paper we describe a benchmark for the User Experience Questionnaire (UEQ), a widely used evaluation tool for interactive products. We also describe how the benchmark can be applied to the quality assurance process for concrete projects.

2016

Adaption of User Experience Questionnaires for Different User Groups
Schrepp, Martin; Pérez Cota, Manuel; Gonçalves, Ramiro; Hinderks, Andreas; Thomaschewski, Jörg
Link zum vollständigen Artikel: https://doi.org/10.1007/s10209-016-0485-9
Products should guarantee a sufficiently high user experience for all intended user groups. A good user experience means not only that it is sufficient to allow users to work effectively and efficiently with a product, but also that non-task-related quality aspects need to be considered. To attain various levels of user experience thus requires an efficient method to access the subjective impressions of a larger number of users toward the product. The authors show that this can be done with little effort by using a user experience questionnaire. However, such questionnaires must be adapted to the language and even to the level of language understanding of the intended target groups. Using the example of the creation of a Portuguese language version and a special version for children, the paper illustrates how the quality of the resulting adapted questionnaire can be evaluated.

User Experience mit Fragebögen evaluieren – Tipps und Tricks für Datenerhebung, Auswertung und Präsentation der Ergebnisse
Schrepp, Martin; Hinderks, Andreas; Thomaschewski, Jörg
Link zum vollständigen Artikel: https://doi.org/10.18420/muc2016-up-0016
Fragebögen erlauben es, die User Experience eines Produkts sehr effizient zu messen. Speziell bei Online-Fragebögen ist der notwendige Zeitaufwand für die Datenerhebung minimal, so dass sehr viele Nutzer befragt werden können. Bei der Anwendung eines Fragebogens lauern aber auch einige Fallstricke, die den Erfolg einer Evaluation gefährden können. Wir zeigen anhand konkreter Fragestellungen und praktischer Projekte, was man bei Datenerhebung, Auswertung und Kommunikation der Ergebnisse beachten sollte.

Iterative Ermittlung der Stichprobengröße bei der Faktorenanalyse
Hinderks, Andreas; Schrepp, Martin; Thomaschewski, Jörg
Link zum vollständigen Artikel: https://doi.org/10.18420/muc2016-mci-0241
Fast alle aktuell verwendeten Fragebögen zur Messung von Usability bzw. User Experience basieren auf faktorenanalytischen Verfahren. Bei der Durchführung der Faktorenanalyse ist unter anderem die Größe der Stichproben von Bedeutung. Zur besseren Interpretation der gängigen Empfehlungen über die Stichprobengröße wurde ein Algorithmus entwickelt, der eine iterative Faktorenanalyse durchführt. Die so gewonnenen Ergebnisse der einzelnen Faktorenanalysen wurden untereinander verglichen. Die Ergebnisse zeigen, dass die existierenden Empfehlungen bzgl. der Stichprobengröße praktisch nur sehr begrenzt weiterhelfen.

2014

Applying the User Experience Questionnaire (UEQ) in Different Evaluation Scenarios
Schrepp, Martin; Hinderks, Andreas; Thomaschewski, Jörg

A good user experience is central for the success of interactive products. To improve products concerning these quality aspects it is thus also important to be able to measure user experience in an efficient and reliable way. But measuring user experience is not an end in itself. Several different questions can be the reason behind the wish to measure the user experience of a product quantitatively. We discuss several typical questions associated with the measurement of user experience and we show how these questions can be answered with a questionnaire with relatively low effort. In this paper the user experience questionnaire UEQ is used, but the general approach may be transferred to other questionnaires as well.

Evaluation des UEQ in einfacher Sprache mit Schülern
Hinderks, Andreas; Schrepp, Martin; Thomaschewski, Jörg

Kinder und Jugendliche sind eine relevante Zielgruppe für viele Produkte. Etablierte Fragebögen zur Messung der User Experience werden in der Regel mit Erwachsenen entwickelt und validiert und setzen daher ein gewisses Ausmaß an Sprachverständnis und Abstraktionsvermögen voraus. Bei der Anwendung mit Kindern und Jugendlichen können sich hier Probleme ergeben. Für den User Experience Questionnaire (UEQ) wurde deshalb eine spezielle Version für Kinder und Jugendliche erstellt. Diese wurde bisher aber nur mit studentischen Teilnehmern validiert. Wir beschreiben die Evaluation dieser Variante des UEQ mit Schülern. Die Ergebnisse zeigen, dass die verwendeten Items ab einer gewissen Altersstufe ausreichend gut verstanden werden. Allerdings konnten auch einige potentiell problematische Items identifiziert werden.

Evaluation des UEQ in einfacher Sprache mit Schülern
Hinderks, Andreas; Schrepp, Martin; Thomaschewski, Jörg
Link zum vollständigen Artikel: https://doi.org/10.1524/9783110344486.323
Kinder und Jugendliche sind eine relevante Zielgruppe für viele Produkte. Etablierte Fragebögen zur Messung der User Experience werden in der Regel mit Erwachsenen entwickelt und validiert und setzen daher ein gewisses Ausmaß an Sprachverständnis und Abstraktionsvermögen voraus. Bei der Anwendung mit Kindern und Jugendlichen können sich hier Probleme ergeben. Für den User Experience Questionnaire (UEQ) wurde deshalb eine spezielle Version für Kinder und Jugendliche erstellt. Diese wurde bisher aber nur mit studentischen Teilnehmern validiert. Wir beschreiben die Evaluation dieser Variante des UEQ mit Schülern. Die Ergebnisse zeigen, dass die verwendeten Items ab einer gewissen Altersstufe ausreichend gut verstanden werden. Allerdings konnten auch einige potentiell problematische Items identifiziert werden.

2012

User-eXperience (UX) meets Return-on-Investment (RoI)
Boris Oliver Kneisel; Katharina Göring; Hinderks, Andreas; Christine Krämer; Oliver Siegmund; Elena Gocheva
Link zum vollständigen Artikel: https://doi.org/10.13140/RG.2.1.2458.3924
Die UX-Branche befindet sich aktuell in einem Reifeprozess. User eXperience (UX) Services verändern sich hin zu einer industrialisierbaren Standarddienstleistung. Dies hat Auswirkungen auf das Berufsbild des UX-Professionals, da sich die Branchenteilnehmer zunehmendem Kommoditisierungsdruck ausgesetzt sehen. Um dem herrschenden Wandel zu begegnen hat die GUPA 2011 den Arbeitskreis \glqqBusiness Case Usability / RoI-UX\grqq gegründet. Der Arbeitskreis stellt sich und den aktuellen Arbeitsstand auf der UP12 vor.

Konstruktion eines Fragebogens für jugendliche Personen zur Messung der User Experience
Hinderks, Andreas; Schrepp, Martin; Rauschenberger, Maria; Olschner, Siegfried; Thomaschewski, Jörg

Der User Experience Questionnaire (UEQ) ist ein etablierter Fragebogen zur Messung des Nutzungserlebens, der als semantisches Differential mit 26 Gegensatzpaaren realisiert ist. Fragebögen dieses Typs setzen ein genaues Verständnis der verwendeten Items durch die befragten Personen voraus. Beim Einsatz des UEQ zur Befragung von jugendlichen Nutzern einer Web-Seite wurden Probleme in Bezug auf die Verständlichkeit einiger Begriffe durch diese Zielgruppe beobachtet. Hauptprobleme waren Fremdwörter und Begriffe, deren konkrete Interpretation im gegebenen Befragungskontext ein höheres Maß an Abstraktionsfähigkeit voraussetzt. Um diese Probleme zu beheben, wurde eine spezielle Version des UEQ in einfacher Sprache konstruiert. Wir beschreiben in diesem Beitrag den Konstruktionsprozess und erste Evaluationsstudien dieser Version.

2011

Benutzererlebnis bei Unternehmenssoftware: Ein Praxisbericht über die Umsetzung attraktiver Unternehmenssoftware
Rauschenberger, Maria; Hinderks, Andreas; Thomaschewski, Jörg

Unternehmen können heute nicht mehr ohne speziell entwickelte Unternehmenssoftware arbeiten. Obwohl eine sehr hohe Anzahl derartiger Softwareprodukte am Markt vorhanden ist, sind die Erfahrungen der Benutzer meist nicht positiv, da zum Beispiel die Gebrauchstauglichkeit nicht optimal erfüllt wird. Der Benutzer kann das Softwareprodukt nicht optimal nutzen und die Gesamtattraktivität der Software sinkt. Aber wie lässt sich die Attraktivität von Unternehmenssoftware steigern, ohne hohe Kosten zu produzieren? In diesem Artikel wird gezeigt, wie das Unternehmen MSP Medien Systempartner die Attraktivität von Tempestiva, eines seiner Softwareprodukte, mittels Zukunftswerkstatt, Prototyping und Benutzertest kosteneffizient in einem kleinen Softwareentwicklungsteam gesteigert hat.