User Experience Analysis of Duolingo Using User Experience Questionnaire

— The internet is one of the vital means for everyone to get various information easily and exact like they’re looking for. The use of internet-based learning that is applied in modern times is very influential in the field of education compared to the past, because it can develop language skills in a country, besides that increasingly sophisticated technology can help students learn in a structured manner. One of the impacts we can see or feel is on the learning process. With the internet, it is so much easier either for the students or the teachers. One of the well-known applications in the world is Duolingo. Duolingo is one of many applications that give so much influence to language learning applications. More than 300 million people already use Duolingo for their learning. The purpose of this experiment is to analyze the User Experience of the Duolingo application. The experimental method was applied using surveys distributed via social media. There are 103 Duolingo users who were willing to take the surveys and answer all of the questions given. The result of the survey showed Novelty’s scale has the lowest mean, and Perspicuity’s scale has the highest. That means some of Duolingo’s users found that the application is less interesting. Hence, that could affect the effectiveness of the application.


I. INTRODUCTION
The use of internet-based learning that is applied in modern times is very influential in the field of education compared to the past, because it can develop language skills in a country, and the increasingly sophisticated technology can help students learn in a structured manner.Student learning that can be applied is E-learning which is specifically designed to make it easy for everyone to understand the language they want to learn (Suartama et al, 2020).One example of an application that can be used is Duolingo.This application has a simple User interface (UI) so it is easy for users to understand when they want to learn the desired language because many languages and interesting features have been provided in this application (Ajisoko, 2020).
The problem that often occurs in the learning process in general, is the decline in student interest because there are no tools that can support learning at that time, so students become more passive and find it difficult to apply the correct pronunciation or writing structure vocabulary in their learning (Abdulrahaman, 2020).As for other problems, most countries are more focused on their main language without little to no motivation to learn another country's language, which can pose a problem for foreign students as that means it is difficult to communicate because others do not understand the language of their country.That means they have to learn a new language on their own.There are so many options to learn foreign languages such as watching some tutorials on YouTube, buying a dictionary, watching a lot of movies with the language you want to learn, or listening to music and learning the lyrics.But as time goes on, the internet keeps evolving and creating new inventions to make our lives so much easier.One of the most convenient ways to learn foreign languages is through a mobile application.There's a mobile app and web called Duolingo which provides more than 30 languages to learn (Nugroho & Surjono, 2019).
The research conducted is directed at evaluating the User Experience of the Duolingo application as measured using the User Experience Questionnaire with the features provided to determine the quality and functionality of the application in stages with a survey conducted to compare the feasibility of features from the appearance of the application (Laksono et al, 2020).There are so many advantages of using Duolingo, if the app is already installed on a device, the user just has to choose which language they want to learn.Moreover, Duolingo gives users choices of which level of user's proficiency in the language.It can be intermediate or advanced and so on.Therefore, users don't have to learn from the beginning if they already know the basics.Duolingo is very convenient, especially for students who are interested in learning other languages and want to communicate with others around the world (Nugroho & Surjono, 2019).
The Duolingo application is well-known and has many users.That makes it important for people to know its quality so they can be aware of whether others use it because of its effectiveness in language learning or just its popularity.One of the many ways to measure the quality of an app is through User Experience as it represents the user's perception and results after using an app.Therefore, this research is the basis for the analysis of the User Experience Duolingo application using the User Experience Questionnaire that has been implemented (Ajisoko, 2020).
The Internet has become one of the vital ways to gain various information (Arkorful & Abaidoo, 2017).Start from teachers, students, workers, and even homemakers.We are already in an era where technology-based in everywhere.One of the impacts we can see is on the learning process.As we know, back then learning was only in a classroom and there is a teacher who explains the material in front of the class using a whiteboard and sourced from books (Ekayana, 2021).And there is technology-based learning called E-learning that uses the use of electronic technologies.Having the same goal in conveying knowledge by sharing information in a more flexible learning process can help teachers and students have a good relationship (Maham Sheikh et al, 2021).E-learning can also be more interesting for students who don't get bored and end up failing some courses.There are so many applications that have fascinating designs that can help students in the learning process (Handayani et al, 2020).E-learning refers to a learning system that is carried out online to provide knowledge that can be accessed through technology.E-learning has proved to be the best means in the corporate sector.The schools which use E-learning technologies are a step ahead of those which still have the traditional approach to learning.So, it's time to finally step forward with the concept of non-electronic teaching with the added help of books and lectures but the importance and effectiveness of technologybased learning cannot be taken lightly or ignored.
The growing availability of mobile technologies has contributed to an increase in mobile-assisted language learning in which learners can autonomously study a second language anytime or anywhere (Loewen et al, 2019).Duolingo is one of many applications that give so much influence to language learning applications.More than 300 million people already use Duolingo for their learning (Shortt, 2021).As time goes on, Duolingo is already seen as a representation of Mobile-Assisted Language Learning (MALL) time by time.MALL has been reported to increase either teacher or students' speaking, listening, and communication skills.Duolingo provides approximately 38 languages to learn.As soon as users already downloaded the app or success on the sign-up page, users can be started to choose which language they want to learn.And after they finally succeed, they can choose their level of proficiency in the language.If the user completes one lesson every day, it will take them on a streak or unlock new achievements.Duolingo sometimes provides some optional challenges to the users for maintaining their streak or gives a reward after completing the challenge.
User Experience is an important aspect of technology that has changed and developed over the years.It extends the concept of usability beyond just effectiveness, efficiency, and satisfaction (Quiñones et al, 2018).Many say it is quite difficult to get a general definition of User Experience as it is connected with a vast variety of dynamic concepts and is too malleable.However, some studies have their own interpretation of the definition of UX (Maslov et al, 2021).According to ISO, UX means "person`s perceptions and responses resulting from the use and/or anticipated use of a product, system or service" (ISO, 2010).Additionally, it represents all factors of user interaction regarding the company, its service, and its products (Zardari et al, 2021).The concept of User Experience is also mentioned to be the user`s subjective, situated, complex, and dynamic experience.It is a concoction of experience, feelings, and impulse which goes past human-computer interaction's traditional characteristics of applications (Mansson et al, 2020).
In another study, User Experience is said to have similarities to usability, although there are some differences in meaning and objectives.Usability objectives are interaction attributes that are associated with a system's function in order to achieve its purpose, while the objectives of User Experience are the features of an interaction with a product linked to emotions, beliefs, and other aspects (Sabukunze & Arakaza, 2021).Another distinction is User Experience is affected by the expectation of the user even before the use of the product (Mochammad et al, 2021).
We can say that User Experience is directly involved with the acceptance of a product (Nurhudatiana & Caesarion, 2020).It is important because frankly, a user is the one that uses the application or system which means their satisfaction will affect the purchase of a product (Badran & Al-Haddad, 2018).Therefore it is necessary to find measurements for a successful and effective User Experience.Knight proposed that to further understand UX, we need to focus on the technical product's two categories: pragmatic characteristics and hedonic characteristics (Knight, 2019).Pragmatic characteristics involve features that help a user achieve their goal and hedonic characteristics mean making the experience exciting and stimulating.Pragmatic characteristic is related to efficiency, output quality, perspicuity, and dependability while hedonic characteristic is associated with stimulation and novelty.Kim proposed the UX Honeycomb Model, which has seven dimensions of usability.It includes usefulness, usability, desirability, findability, accessibility, credibility, and value (Kim, 2020).
UEQ is a widely used questionnaire to measure the User Experience of a product.The german version of UEQ was first made in 2005 based on a data analytical approach.It is used to maintain a practical significance of the distinct qualities of each scale.At first, 229 items related to User Experience were made.It was then reduced to 80 items by the evaluations of experts (Laksono, 2020).And then, 6 scales and the items representing each scale were extracted from the previous 80 items.Each item in the scale is represented by two terms with opposite meanings (Schrepp, 2017).
There are three aspects represented by the scales, pure affective aspect, pragmatic quality aspect, and hedonic quality aspect.The pure affective quality aspect is represented by Attractiveness, the pragmatic quality aspect (goal-oriented) is represented by Perspicuity, Efficiency, and Dependability, and lastly, the hedonic quality aspect (nongoal-oriented) is represented by Stimulation and Novelty (Laugwitz, 2008).

II. METHODS
This study used an experimental methodology.We have made a survey to collect the data.The survey was distributed through various social media platforms.It consists of 26 questions regarding the User Experience of Duolingo as well as questions about the participants' name, age, and period of usage.In addition, this survey is shared with Duolingo Users and uses convenience sampling.With that, participants have the freedom to fill out the survey or not.Based on the information obtained from the Play Store, there are at least 300+ million learners that chose to use Duolingo, this is our population.To determine the sample, we use the Slovin Formula with a 10% margin of error. (1) Based on the Slovin Formula in (1), assume that the sample size (n), the population size (N), and the 10% margin of error (E) are applied in the calculation.
From the calculation in (2), we can conclude that we need approximately 100 people's feedback about their experience after using Duolingo.
The survey consists of 26 questions which were taken from the UEQ website.The questions include a 7-point Likert scale with opposing terms on each side.

III. RESULTS AND DISCUSSION
The survey reached 103 respondents.From the 103 responses, we found that 15 participants are below 18 years old, 80 participants are between 18 and 23 years old, and 8 participants are above 23 years old.For the respondent's usage period of Duolingo Application (Shown in Figure 2), 49.5% of users used Duolingo for less than 1 year, 40.8% of users used Duolingo for around 1 until 5 years, and 9.7% of users used it for more than 5 years.From 103 respondents, we only took 101 respondents because 2 of the respondents didn't take the surveys seriously and filled the answer with the same scale.We analyzed each scale's values with the given Excel Data Analysis Tool.The mean of each scale is displayed in Table I and Figure 3, where the Attractiveness mean is 1,523, Perspicuity 1.681, Efficiency 1.564, Dependability 1.418, Stimulation 1.334, and Novelty 0.955.In order to measure the quality of Duolingo based on User Experience, we compared it to the benchmark given by the Data Analysis Tool, which can be seen in Table II and the comparison in Figure 4. Based on the benchmark, most of the scales fall into the Above Average category.Furthermore, according to the handbook (Schrepp, 2017), a score above 0,80 is considered to be in a good category.Therefore, we can conclude that the quality of the Duolingo application is pretty high.Although, none of them can be put into the Excellent category.Perspicuity holds the highest mean which means Duolingo is easily comprehensible, this can be beneficial for new users as they didn't have an understanding of how to use the application at first.Meanwhile, Novelty holds the lowest score so we can conclude that Duolingo lacks uniqueness.This can be detrimental to the quality of the application as over time, users will get bored of the content and stop using the application.That behavior is reflected through the period of usage.Based on the survey, most participants used Duolingo for less than 1 year.Therefore, Duolingo needs to find some brand new ideas to make their application more interesting.

IV. CONCLUSION
The intention of this experiment is to analyze the User Experience of Duolingo.The experiment was conducted by using a survey that was distributed through social media.
The result of the survey shows that Perspicuity holds the highest score (1.681) which means the application is really easy to learn and use.While Novelty holds the lowest score which is 0.955.Overall, it can be concluded that all of the scale means of Duolingo fall in a good category based on the benchmark values.According to the result, it is recommended to improve the Novelty of the application with new ideas.
As with most studies, this study is subject to limitations.This study implements convenience sampling to collect data, which means there may be some bias in the data.For future studies, we hope to be able to implement other sampling methods in order to gain more authentic data and also to analyze the User Experience of other languagelearning applications to compare with Duolingo.

Figure 1 .
Figure 1.UEQ questions for the survey using the Indonesian version

Table I .
UEQ Scale Mean of Duolingo

Table II .
UEQ Scales Benchmark