The System Usability Scale – How to measure usability and UX
Whereas until a few years ago the system itself was the focus of development, it is now the customer or user. The system is developed according to the needs of the customer to increase acceptance. But how can you measure usability and UX?
What is UX and why is it hard to measure?
User experience (UX) plays a central role in the development of systems, but is often mistakenly equated with terms such as user interface or usability. To create a common understanding of the term user experience, we are guided by the associated DIN standard, which states in summary:
“UX is a person’s perception and reaction before, during or after using a product, system or service.” (according to DIN EN ISO 9241)
This includes the emotions, psychological and physiological reactions, expectations and behavior of the user. User experience is understood as a consequence, influenced by the design, functionality and performance characteristics of a product. Likewise, prior knowledge and characteristics of the respective user:in can play a role as well as brand perception or the context of use.
The definition already makes it clear that the UX is far more difficult to measure than parameters such as unit numbers or throughput times. Is it possible to make such an abstract concept measurable? The answer to that is yes. With the help of methods from software evaluation (UX evaluation), the usability and application logic of a system can be evaluated and measured.
That is why it is important important to measure UX
The development of systems has changed in recent years. Whereas until a few years ago the system itself was the focus of development, it is now the customer or user. The system is developed according to the customer’s needs in order to increase the customer’s willingness to buy or use it, to support him optimally in his processes and to develop him further in his work. In this context, the question arises whether usability and relevance, as well as the effectiveness of measures to increase them, can be measured. With the help of a software evaluation, opportunities for improvement in the system are identified. This reduces the need for support and increases data quality. This in turn leads to increased customer and employee satisfaction.
A brief overview of survey methods
There are various methods for collecting user feedback. A primary distinction is made between two types of methods: measurements and surveys.
As part of the measurements, key figures on UX are collected directly by the system. Contact with the respondent is not mandatory for the survey. The Task Success Rate is a key figure collected in this way: It measures how often a task was successfully completed and puts it in relation to all attempts. The average time required for a task, the Task Completion Time, can also be collected by the system. Another well-known example of a metric collected through measurement is the conversion rate. In other words, the proportion of users of the system who perform a certain action, for example, the proportion of visitors to a website who create a user account.
In the surveys, standardized questions or questionnaires are presented to the users in order to determine key figures from the results. A simple example of this is the Single Ease Question: immediately after completing a task, the respondent is asked to rate on a Likert scale from 1 to 7 the question “How easy was that?”. Other examples include the Net Promoter Score (“How likely are you to recommend this system/application to friends or acquaintances?”) and the Customer Effort Score, which asks how much effort the user puts into using a system.
Surveys are somewhat more laborious to collect, as they depend heavily on the commitment of the user:in. However, qualitative follow-up questions on system analysis can also be integrated in their framework in order to be able to put the results obtained into the right context and interpret them. This measure is also necessary for measurements in order to be able to improve the system in a user-centered way.
Our tip to get started with measuring UX: The SUS.
Quantification based on scientific findings
A classic method to survey the location of a system in terms of user experience is the System Usability Scale (SUS). Even though the SUS does not consider the entire UX, but is limited to the partial aspect of usability, this metric is characterized primarily by two aspects for evaluating a system: It is simple and has been tested in practice many times.
This standardized questionnaire from 1986 by John Brooke is technology-independent and allows the subjectively perceived “usability” of a system to be assessed. Contrary to what one might think, despite its age, SUS is still one of the most widely used metrics for measuring usability. The simplicity of SUS is characterized by the scientifically selected and defined questions that allow for an unbiased result, as well as the quick applicability regardless of the sample size.
The SUS includes ten questions that alternate between positive and negative wording:
The evaluation of the SUS is done with the help of the Likert scale and a point evaluation (see figure). The values of the scale are added and multiplied by 2.5. In our example, this results in the following calculation:
(4+0+0+3+2+2+2+2+2+2) * 2,5 = 47,5.
A maximum of 100 points can be achieved. A score of 68 or more is considered a good result. This classification is the result of the evaluation of over 500 studies in which the SUS was carried out with a total of over 5000 users. Depending on where your system scores, the SUS shows how satisfied users are with the system or whether measures should be taken to improve its “usability”. It should be noted that with the help of SUS it is possible to quickly determine whether there are problems with the use of a system. However, no direct statement can be made as to which problems these are. At first, this is only possible either via additional and voluntary user feedback or via further subsequent measures.
Those who don't measure, make crap
In the modern world, where systems are becoming more and more complex, users more demanding and requirements generally increasing, it would be almost irresponsible to develop software and systems purely on gut feeling. Reliable figures, facts and assessments are needed to make informed decisions. In this article we have presented you some methods to be able to build such a decision base yourself.
In general, we do not recommend conducting specially created surveys. We recommend using existing and proven metrics to avoid unknowingly (possibly negatively) influencing respondents’ opinions. The order of the questions “How satisfied are you with the overall system?” and “How satisfied are you with feature X from the overall system?” alone has a significant influence on the result: For example, those who first praise the overall system in the highest terms will then be more reticent about criticizing sub-aspects. Here, the complexity of applying methods of how to measure UX becomes clear.
We are happy to support you from the definition of the task, through the preparation, implementation and evaluation of an SBS survey, to the identification of improvement potential and the targeted derivation of measures. Our goal is the best user experience for your users!
