A Case Study on the use of Gamification in the Flipped Classroom

The present article corresponds to the result of an internship at the Polytechnic Institute of Bragança. After accomplishing a systematic literature review looking for the clarification of the main concepts of gamification and flipped classroom, it was performed a case study which main purpose was to identify and analyze how students react to the use of educational games as diagnostic or formative evaluation. Many tools were evaluated, but only four were chosen, which were: Kahoot!, Socrative, Quizizz and Nearpods. Subsequently, these tools were distributed to the third year of the course of management degree through Google Classroom and shared with other professors of the European community through the platform AduLeT (Advanced use of Learning Technologies in Higher Education). That contributed to share and reuse strategies to face the world’s current scenario due to the COVID-19 pandemic. We can affirm henceforth that all the use experiences, which related a learning method with one of the tools referred, showed satisfactory results involving a greater dynamic and interest during the classes.


INTRODUCTION
A certain event which happens in the society changes it. The spectrum of this change is the one that can be different. The worldwide event caused by the pandemic dispersion of the corona virus SARS-CoV-2, responsible for the COVID-19, has influenced drastically the way of teaching and learning around the world in the past few months. All the changes have their consequences, and to face this worldwide pandemic, some educational institutions had to adapt their teaching and learning methods to continue the academic year. Based on that, this research was considered even more relevant since it verified the applicability of the creation of interactive educational resources using authorship tools of objects of learning namely platforms of learning based in games which are: Kahoot!, Quizizz, Nearpod and Socrative which would be available in a course and shared in the platform Google Classroom. Some of these platforms are more than formative assessment tools, but mainly we can consider them as audience response systems and the use of these applications, without integration with other platforms, are small examples about the true gamification process. In fact, we emphasize the use of this tools to implement simple gamebased strategies in classroom.
This study aimed to analyze how students react to the application of the platforms mentioned above in their educational content, as a way of diagnostically or formatively evaluating their knowledge in synchronous classes through videoconference held by the teacher of the course. Therefore, it is important to measure the impacts generated during the learning processes and the students' perception of these tools as means of evaluation.
This research was based in a systematic literature review to understand the main concepts of gamification, flipped classroom and the mostly used tools game-based, to then carry out a case study with a class of thirty-five students from the degree in administration at the Polytechnic Institute of Bragança. To evaluate all the four tools previously referred there were performed four sessions using a different platform and its interactive resource in each of the sessions. To map the class profile and collect the student's perceptions it was elaborated a form in Google Forms.
The structure of the current article is i) show the methodologies referring to the main results of the systematic literature review; ii) clarify the concepts of flipped classroom and the gamification emphasizing its benefits in motivation, commitment, involvement or engagement; iii) show the results of the case study emphasizing the presentation and data analysis collected through the semi structured questionnaire and the platforms statistic tools as well as observing each one of the synchronic sessions of video conferencing; iv) refer that all use experiences in which every learning method is related to one of the technologies referred were shared with other professors of the European community through the platform AduLeT (Advanced Use of Learning Technologies in Higher Education); v) finally, the main conclusions of this case study are showed.

II.
METHODOLOGY Initially, we started the systematic literature review of scientific articles, dissertations and theses stored in the main bibliometric basis such as Web of Science and Scopus. This 2021 16th Iberian Conference on Information Systems and Technologies (CISTI) 23 -26 June 2021, Chaves, Portugal ISBN: 978-989-54659-1-0 methodology can be resumed in the following steps: (i) question formulation; (ii) studies location of the bibliometric basis; (iii) evaluation and selection of the studies; (iv) analysis and synthesis; (v) results report.
The systematic literature review allowed us to reunite some of the most relevant studies about themes such as the flipped classroom and the gamification considering the creation of interactive educational resources and using the most relevant data basis which aim was to conduct a critical and extensive literature review. The review of the systematization process aimed to reinforce the study avoiding the disadvantages which would probably occur in an ordinary or not systematic review (figure 1). This systematic review does not include the metaanalysis as the primary studies which were integrated in a summarized way. However, its integration was not combined statistically, it corresponded to a qualitative systematic review [1].  [2] For the evaluation and selection of studies, in the second stage, the first parameter was used by the title and year of publication, 2005 -2020. For the year publication, 63 studies and, after, 83 studies were discarded due to lack of relationship for the purpose of this research. As a result of reading the abstract and introduction, 17 articles were also eliminated. The criteria for verify eligibility was precisely to do application of these concepts in the context of classroom or a case study either in the application of the flipped classroom methodology as in the application of any tool that applies the idea of gamification as kahoot!, socrative, nearpod or others.
Subsequently, four interactive educational resources were elaborated. They were applied to the third-year class of a curricular unit of the course of management degree of the Polytechnical Institute of Bragança. Therefore, it was applied the case study methodology bearing in mind that it was a specific phenomenon and that many analysis tools were used to interpret the data. Both the external and internal factors were analyzed. [3]. YIN (2010) [4] affirms that it is important to comprehend how and why the problematics involved occur and also adds that "the case study method is not just a qualitative research, even though it can be recognized in the variety of quantitative research options. The case study research goes beyond one type of qualitative research, it uses a mixture of qualitative and quantitative evidences".
There were 35 students in the class which majority are young people who are between 18 and 22 years old. 60% of the class is represented by girls and 90% of it claim that they have never had any kind of gamification experience during the classes. Based on that, this survey is characterized as a descriptive and qualitative research. It is also important to mention that referring to the data treatment, three chronological poles were proposed by Bardin (2006) [5]: i) pre-analysis; ii) material exploration and iii) results treatment, inference, and interpretation.
In the first part, it was shown to the students the main purposes of the project as well as the tools which were going to be used to set the class profile and find out some criteria such as age, gender, if the students already knew the tools and if they preferred any kind of teaching methodology. In the second part, the educational interactive resources were applied and distributed into four sessions. In the final part, the reports available in the platforms Kahoot!, Quizizz, Nearpod and Socrative as well as the semi structured questionnaire sent to the students by Google Forms and an interview with the professor through email were used to interpret and understand the results. Based on the information collected, the following terminal categories were defined to study: i) motivation and engagement ii) students' and professor's educational perception iii) technical and functional factors.

III. CONTRIBUTION TO THE STATE OF THE ART
Due to the globalization, the pedagogical methodologies can be implemented with the educational technologies resources which can support and most of the time can also promote disruptive processes. According to Hiltz and Turoff (2005) [6], the conventional model of in person courses focused on the teacher loses space for a pedagogy supported by digital tools and focused on the students. It is important to emphasize that one of the biggest challenges in every single educational process is to keep the student motivated, which is a major factor in the learning process and in their engagement during the activities proposed [7].

A. Motivation and engagement
One of the problems faced during the classes is the level of the students' participation and how well the teachers are able to make them motivated and engaged during the proposal activities, which is an ancient question when analyzing the teaching and learning process [8]. Referring to this main question, this study approaches two strategies, which are the gamification and the flipped classroom, to verify their applicability as important factors which can increase the students' participation, motivation, and engagement. Bishop and Verleger (2013) [9], while studying the flipped classroom, recognize the importance of the interactive learning activities in 2021 16th Iberian Conference on Information Systems and Technologies (CISTI) 23 -26 June 2021, Chaves, Portugal ISBN: 978-989-54659-1-0 group and the use of computers in extra activities outside the classroom. This is related to the gamification and the use of online platforms to create interactive educational resources once the strategy based in educational games can embrace many motivational factors. Monteiro et al. (2012) [10] defends that one of the determining factors would be the fact that the professor combines different languages resources and provides more interaction and discussion during the classes where the students can feel free to express their opinions and the professor can interfere in a more effective way.
First, it is important to highlight that motivation can be intrinsic and originated inside the individual or extrinsic and related to external factors. Busarello (2016) [11] in his book "gamification: principles and strategies", in the fourth chapter writes about that in the games context and recognize that to make an individual interested, intrinsic motivation, it is fundamental that the activity makes him curious, entertained and satisfied. On the other hand, referring to the motivation generated by external factors, the author affirms that it is necessary an external reward and, in the games, it is represented by the scores and ranking.
Besides that, the author emphasizes that the strategy used during the games approaches cognitive, emotional, and social areas and, because of that, it tends to be assertive. The first one is related to the individual autonomy provided by the stage's functionalities inside the game. The emotional area is related to the success and the failure and the satisfaction after completing a task. The last cognitive area, which is the social one, is just the competition promoted between the classmates.

B. Flipped Classroom
The search for new methods, which can provide the students participation, motivation and can, even, reduce their absences, is constant in literature. Valente (2014) [12] reinforce the educational professionals' engagement and their attempt to innovate their teaching methods and he mentions the flipped classroom as a great solution which is used by some famous universities worldwide. The flipped classroom consists in inverting the logic inside the class. In the traditional method, the students learn the subjects at school and do the homework at home. In the flipped classroom the students are supposed to study the school subjects at home and do their homework inside the class [13].
In his study, Amstelveen (2019) [14] indicates this approach as an alternative to the professors who want to use more technological resources, change the classes schedule, and focus on the student. Yang (2015) [15] alredy points it out as a significant method to innovate and to renovate the educational model. In general, when we compare the flipped classroom with the more traditional methods, we find out that it shows more promising results (Valente, 2014) [12], but Amstelveen (2019) [14] mentions that there is no guarantee to increase the apprenticeship when comparing it to a more traditional method once it depends on the professors' capacity to maximize the apprenticeship.
Some other studies and reflections such as Hamdan and McKnight (2013) [16] could be highlighted once they demonstrate the convenience that this method could provide to the students. They would be able to do the activities anytime, anywhere and they would be able to do it as many times as they need. Rolo (2016) [17] also quotes that the young people can adapt better in this new scenario because of their abilities to deal with technological tools. It also influences the exploitation during the flipped classroom once the multimedia tools are one of the main bases.
Certainly, there are also some negative aspects as in any other teaching learning approach. Soldevilla (2015) [18] in his study share the negative results that he obtained due to the lack of resources, time and the lack of the students' consciousness face to the importance of their participation in the method success. Referring to the students that are not very autonomous, it can increase significantly in the flipped classroom. Bergmann & Sams (2012) [13] point out that the fact that the students have already studied the subjects at home may not solve their doubts immediately, on the other hand they can study in their own rhythm and because of that they can make questions more precisely in the classroom. Therefore, the beginning of the class is used to clarify the doubts and to solve and correct the exercises which prevent the student to apply incorrect concepts on his lessons. Referring to that, it is clearly that the time logic during the classes is also flipped. In the traditional method, the beginning of the class is focused to solve the homework and to comprehend the students' problems. Moran & Milsom (2014) [19] used this method in a fifteen-student class in a University of the United States whose problem had been the lack of the students' content applicability and their miss understanding during the class reading. Due to that fact, they use the flipped classroom in attempt to make the class environment more interactive and collaborative. In that class, the results were satisfactory. The students were more satisfied with the increase of discussions during the classes and with the opportunity to learn with each other.
In short, it can be said that the benefits when using this method surpasses the most traditional methods. However, it is important to mention that changing a learning teaching process which has been used for decades is very complex for both the students and the professors. Therefore, it is required dedication and conscious about the change's necessity [20].

C. Gamification
Searching for the innovation and the students' engagement, another strategy mentioned in the literature is the gamification. It is defined by Kapp (2012) [21] as the use of the applied principles in the game development and other contexts which are not applied to games with the potential to intrigue and motivate people. Referring to that, it is essential the applicability of this approach to the educational software and many methods can be used besides the gamification such as videoconferencing, quizzes, mental maps, and simulation.
Signori and Guimarães (2016) [20] point out that the difference present in the gamification is "the use of the game elements as rewards systems and competitive social mechanisms". That results in a positive and directed connection between the motivation and the engagement. This statement befits with the conclusion Kapp (2016) [22] made in his research in which the gamification allows the students to dedicate more and to assume bigger responsibilities and consequently individualizing their learning. It is also considered a method 2021 16th Iberian Conference on Information Systems and Technologies (CISTI) 23 -26 June 2021, Chaves, Portugal ISBN: 978-989-54659-1-0 which allows to involve the student, reinforce the content, expose the progress and the following direction which are important motivation agents. Another important aspect is the competition promoted by the game model. França and Reategui (2013) [23] point out that the use of gamification can promote both a collaborative and competitive environment and although there is not an agreement in the literature about the positive and directed relationship between competition and motivation in their study, the students showed a very satisfactory result.
In the literature, there are many interactive educational platform applications in which one of them is the use of the Kahoot! instead of a traditional test in the Pharmacy College of the Federal University of Minas Gerais. The quiz had 25 questions and the timing was between 60 to 120 seconds. In the students' perception, it was possible to take a fair review. 80% of the students considered this method as something more attractive than the traditional tests and the competitiveness was able to surpass the limitations that this tool had [24].

IV. STUDY CASE RESULTS
The use of educational games has great potential to engage the student in the learning process [22]. However, the way the sessions will be guided can influence the results considerably and because of that it is necessary that the professors pay attention during the creation of the project in three steps: the game presentation, the game execution, and its discussion according to Kapp (2016) [22], mentioning Hays (2005) and Sitzmann (2011). In the first session, the class dynamic and how to use the tool were explained to the students. The first three session were accomplished in a synchronized way with the platforms Kahoot!, Quizizz, and Nearpod. Only in the last one it was used the platform Socrative in which was applied the flipped classroom, mainly because fit well with the strategy of the teacher for those type of contents. In all these platforms the questionnaires had ten questions and were used in the end of the classes with the duration of approximately 30 to 40 minutes. In the first session, we opted to start with the platform Kahoot!. During the game, it was guided by the professor who was responsible to read the questions and the alternatives with the students in attempt to encourage them mentioning some comments about the subject.
In the second session, we used the platform Quizizz and as the students had already known the platform, it was slightly more intuitive and because of that the timing to enter the game classroom was lightly smaller. The same happened to the Nearpod. However, with the Socrative platform we chose the asynchronous mode to understand better how the flipped classroom worked. We also used this mode because this platform does not show an immediate feedback, so the questions could be discussed during the class.

A. Motivation and engagement
Motivation and engagement are the factors responsible for these tools applicability and responsible to modify the class traditional methodology. In the students comments the words funny and interesting were very used. That is one of the main strategies of the gamification applicability which motivates and engages the students during the classes [23]. One student affirmed: "I thought quite interesting the fact that we can learn in a different and funnier way" another one said: "I didn't know the games, but I found them good tools to attract the students' attention during the classes". They both reinforced both innovation and satisfaction during the activity.
The professor also confirmed the platforms benefit. Previously, she had experienced only in person methodologies which were mainly expositive and traditional. She also affirmed that she wants to use the platforms, especially in the synchronous mode, but during the in-person classes.

B. Students' and professor's educational perception
During the classes, the students were satisfied face to the application of the tools. Besides their verbal comments, the questionnaire also showed that they considered the tools as a good feature to learning and evaluation. Before the activities, 77.4% of the class preferred the traditional method and despite this number falling to 50% it is yet an important data. Referring to this, another important aspect to point out is the students' lack of interest face to the flipped classroom which was known by only 25% of the class beforehand. Initially, only one student preferred this methodology and that number reduced to zero after the activities were done. Therefore, it is concluded that this method is still not very implemented in the academic community both by the students and professors. That was their first experience and as Amstelveen (2019) [14] said before it changes the experience directly. Despite that, the students considered the Socrative as the most potential platform to achieve the pedagogical aspects and also the flipped classroom.
In attempt to evaluate the experience, it was proposed to the students a seven-level scale in which (1) very bad, (2) bad, (3) not satisfactory, (4) satisfactory, (5) good, (6) very good, (7) excellent. The criteria were the use as evaluate method, contribution to the learning process, means of learning the content and class strategy and its dynamic. In all of the criteria the most used term was the very good and the terrible one was not indicated in none of the criteria. These data were reinforced in the students' comments in the end of the questionnaire. One student defined the learning as funny in the comment: "I think that's a funnier way of evaluating our knowledge and to give more joy to the classes". Another one defined as innovative in the statement: "It is a very innovative method and despite its small use, it has such a great potential".
Comparing the criteria in the different platforms by students, figure 2, the criteria were: (i) interactivity , (ii) easy handling, (iii) visual resources, (iv) pedagogical potentiality, (v) learning intuitiveness, (vi) dynamism, (vii) funniest learning. These referenced items were mentioned in the literature as important gamification means which may be applied in the educational context [19]. Considered the most interactive tool, the Kahoot! obtained 16 votes, followed by Socrative which had 7. It is also interesting to point out that the interaction created by the Socrative was in the following class, during an available moment for discussion once the simulation happened in an asynchronous way. The Kahoot! and the Quizizz were considered the easiest tools, but the Kahoot! won with a three points difference. The class also preferred the Kahoot! when compared to the other virtual resources.

C. Platforms selection: technical and functional factors
Place for the selection of the platforms for the application in the curriculum unit it was considered the (i) research volume already existent and basing on the systematic literature review; (ii) the experiences report collected through the platform AduLeT and (iii) technical factors of each platform available in a free version which was measured empirically.
The platform AduLeT is an online community of practices (CoP) to share technologies enhanced learning (TEL) appropriated to the recommended method for a specific learning situation. This CoP shares the TEL methodological practices in the students' formation and the use of educational technologies in their learning process. Thereby, the following framework shows the relation between the platform and the TEL method as well as where the selected tools fit in, their definition and their benefits. The last analysis criterion was the platform functionalities test. During the scientific initiation internship, many platforms were evaluated and then the tools functionalities were highlighted. In a certain way, all of the four platforms chosen have their similarities, but it is undeniable that each one of them presents its own particularities and benefits. The platforms were evaluated in the following criteria: the quiz construction, the execution, and the reports. The chart below shows in a summarized way the functionalities used in each platform: Following the order of the parameters exposed in the chart, the first one is the type of questions included in each platform. The Kahoot! is the one that cares more about a more dynamic presentation. That was observed because the questions have a characters limit, and they can only be true or false questions or multiple choices. That fact is even more restrictive in the Nearpod because it allows only multiple choices questions (ignoring other type of learning objects and activities), the time is reduced and there is also characters limit. Due to that parameter, the Quizizz and the Socrative are less rigid because they allow the professors to create open-ended questions in which the Socrative answers are short sentences and the Quizizz's are discursive. In both cases the answer is more subjective and personal, so they are not evaluated, and the professor is responsible to correct them.
The Kahoot! and the Quizizz are the most versatile when we consider the multimedia elements available. The Nearpod is limited considering only this specific type of exercise, but its free version platform allows the professor to add some other resources such as pdf, video, a BBC link, QTVR, etc. The Socrative allow us to add only images but allow us to include one picture to each alternative while in, for example, Nearpod it can be done just once in the statement.
The Kahoot! Is the only platform which allows people to indicate a different level of difficulty in each question throughout three types of evaluation. It is also possible to cancel the points in some question which can be very interesting in the beginning of the game as if the students were in a certain type of training test. All the platforms allow us to show the answers in the end, but only in the Socrative that is an optional feature. That makes the correction faster and it also minimize the frustration during the learning process.
Finally, the last comparative parameter are the reports available. Kahoot!, Quizizz and Socrative stand out when compared to the Nearpod because they show the results in two versions in a more friendly way. Every time the students answer an interactive assessment question in a Nearpod presentation, results are saved on a report that teachers can later access via the Content Tool or app. A PDF or CSV version of this Report can be sent to your email.

V. CONCLUSIONS
The main purpose of this study was to identify and analyze how students react to the platforms application and corresponding games referred in the educational content as means of diagnostic or formative evaluation. It was concluded that the results were satisfied and that the classes became more interesting and dynamic. This can enhance the academic community, once one of the biggest challenges is centered in the students' engagement difficulty during the classes. Analyzing especially the current classes scene in which they are forced to be online it is really inevitable the reinvention of the conventional teaching-learning methods.
As a suggestion of possible works in the future is interesting to explore other functionalities of each tool and go deep on the application of the flipped classroom. The Nearpod has many functions such as interconnected platform with the BBC news for the inclusion of news, the combination of PowerPoint and games, videos, but only one of its functionalities was really used which was "the climb timing". Though, testing its functionalities is extremely relevant. The Kahoot!, Socrative and Quizizz also allow the students to play in teams which develop some other motivational factors. Referring to the flipped classroom it was noticed that it is necessary a better preparation to involve more the students and the teacher, so the information sharing can get bigger and the learning more interpersonal.
We also intend to apply this type of study to non-higher education institutions, involving classes from the three school groups in the city of Bragança, in order to understand how they take advantage of the integration of these tools in the e-Learning platforms used: Microsoft Teams or Google Classroom.