Use of Learning Management System (LMS): A Study in a Brazilian and Portuguese Universities

Learning Management Systems (LMS) are mainly used as a didactic support for the presential classes. However, because of the tools they provide, they are also essential to support b-learning and e-learning process. Because of this, most universities rely on some LMS through many or all of the courses. This paper describes a study performed in two different universities regarding the students’ perception about the usefulness and the usability of two different LMS: Moodle and Sakai. Through a focus group, students shared their opinion and discussed about their experience. In addition, data from the log of the platforms was also analysed, to assess how teachers use the LMS tools. In general, it can be stated that when the teacher knows how to use the LMS, students feel more comfortable using it and have a better learning experience. Regarding the tools, teachers mostly rely on the repository of documents.


INTRODUCTION
The technological development has effects in many areas of human development. Although usually associated to Innovation and to the Information and Communication, the impact on Education is both significant and transversal, starting to be ubiquitous both inside and outside of classes. The most prevalent expression of technology is the digital mediation between teachers, students and learning-experiences. This mediation is performed through special-purpose platforms, known as Virtual Learning Environments (VLE), providing support to individual management of learning experiences and activities (Georgouli, 2011). VLE are also known by different terms, such as elearning, because it combines Information and Communication Technologies (ICT) with learning process, Course Management Systems (CMS), or Learning Management Systems (LMS), the term we will be using in this paper. In addition to supporting file hosting in different formats, such as text, image, sound, and others, they provide a broad set of tools, a https://orcid.org/0000-0002-3444-1408 b https://orcid.org/0000-0002-4992-8614 c https://orcid.org/0000-0002-9170-5078 that can be used to facilitate interaction between colleagues and lecturers. These can be synchronous, asynchronous, individual or group, such as message boards, forums, chat rooms, videoconferencing, etc. (Sánchez & Hueros, 2010).
LMS provide a web-based user interface, accessible anytime, anywhere through a conventional web browser. Usually, each course will have a specific area, with several tools at the teachers' and students' disposal for managing interactions and learning-experiences. These includes integrated support for the creation, organization, delivery, communication, collaboration, and evaluation of activities and content.
Learning Management Systems are not exclusive to e-learning environments. They have gained space in traditional classroom, increasing flexibility by providing individual areas for students and permanent access to content and other learning experiences (Kunnath, 2017). Moreover, they stand out among today's media for enabling collaborative learning, interactivity and different forms of learning by diversifying representations of the same content (Lacerda & Silva, 2015).
LMSs are widely used as a "repository of files" and few teachers explore their potential and didacticpedagogical possibilities (Tenório et al., 2019). In turn, students reveal some common concerns such as: when they should enroll in the course; if the enrollment of the traditional class already guarantees access to the LMS; which credentials to use to join the LMS; about the activities and the deadline for delivery; about the locations and times available for access to the LMS; how to communicate with teachers and how to deliver activities. These questions reveal that there is a need for technical guidance to start the process of using LMS. In addition there is also evidence showing a demand for specialized services that require permanent staff that can provide technical support in production and follow up of the e-learning courses offered (Seixas et al., 2012).
By encouraging the use of these systems, it is possible to use the LMS in another pedagogical perspective, expanding the use of its tools for the construction of knowledge. (Lacerda & Silva, 2015). While pedagogical and discipline expertise are considered the foundation of the teaching process, interaction in online learning systems requires additional communication and technological competencies to achieve learning outcomes (Farhan et al., 2019).
Particularly with e-learning, teachers can act as course constructors and facilitators, guiding the learner through their educational experiences. For learners the opportunity to take advantage of continuous learning through the use of free didactic materials and interactions with peers, using a suitable technological environment that enhances their growth. (De Medio et al., 2020;Lopes & Mesquita, 2015).
The aim of this paper is to understand how Learning Management Systems have been used in higher education, through the analysis of two different platforms (Moodle and Sakai) in two different universities (on in Portugal and another in Brazil). For that, a quantitative and qualitative methodology was followed, supported by a focus group and the historical log of the platforms' usability.

METHODOLOGY
The methodology uses three instruments in a quantiqualitative approach. Data was thus collected with an online, open-questions, questionnaire for teachers, focus group for students and the numerical logs of the LMS platforms was statistically analyzed. Two focus group were organized, with students from Portugal and the other with students from Brazil. The focus group was recorded and transcribed and later analyzed for category identification. A focus groups was selected because students tend to discuss their worries in group and provide a more natural perception.
An open questions questionnaire was conducted with teachers, to assess their opinion regarding the use and importance of an LMS. This choice was also intentional, because teachers tend to express better their opinion without concerns or restrictions in this way.
Finally, information from the usage record of the platforms (logs) was also used, providing quantitative information about user access and tools used. These provide a measurement of the importance and popularity of each tool in the learning process.

Focus Group with Students
For the collection of the perceptions of students regarding the use of LMS, a Focus Group was organized and structured, in order to develop more indepth description of students' experiences. Students were organized in two groups: one with 8 Brazilian students from computer science, electronic engineering, food engineering, environmental engineering, chemistry engineering and bioprocess engineering, and another with 11 Portuguese students from business management, pre-school and elementary education, computer engineering and animal science technology. Students were randomly selected and their participation optional. The students that agreed to participate, were welcome and participated in the Focus Group session.
The meetings were conducted by a moderator who ensured that all students expressed their opinion. The session was held in compliance with the Declaration of Helsinki and had their audio recorded with the permission of the participants for later transcription of the comments (Bardin, 2015).
Eight primary questions were asked during the focus groups (Table 1), with follow-up questions to probe for additional details as necessary.

Questionnaire with Teachers
An open-questions questionnaire was conducted with Brazilian teachers, users of the LMS. The questionnaire was composed of eight open questions, available through Google Forms (Table 2). Six teachers responded in writing and their answers was subjected to content analysis, according to a rigorous qualitative methodology.

Historical Usage Data (Logs)
The use of LMS is ongoing, in the Portuguese higher education institution for 6 years, now, and all the records are kept since 2013. This historical data registered in the LMSs' log files was statistically processed using analysis of variance and Tukey test at 5% significance level.

ANALYSIS AND DISCUSSION
The analysis process started with the definition of virtual systems tools and analysis historical data. Next, the results of focus group meetings are describe. Finally, the perception of teachers are presented.

Definition of LMS Tools
A Learning Management System provides a set of tools with special purpose to support communication, assessment, time-management and access to content. Among the many tools, it is possible to identify the most meaningful and available to the Brazilian and Portuguese Higher Education Institution with their respective functions (Table 3).

Historical Data (logs)
The log files collected from the LMS include information about seven tools: Calendar, Announcements, Resources, Assignments, Online testes, Messages and Forums. Figure 1 represents the sum of the use of the tools from 2013 to 2019, in all the curricular units offered in the period. The Resources were used 633,099 times during the evaluated period, being the most used tool by teachers and students. This tool allows the teacher to upload handouts for students to download. It is possible to store files, books, articles, etc.
Messages, Assignments and Announcements tools were used 141,909, 111,043 and 90,258 times, respectively (Figure 1). Table 4 presents the average usage of each tool per year and per curricular unit from 2013 to 2019. The use of Resource tool increased over time, as shown in Table 4. The average use of Resources in 2018/2019 was 38.30 times per course offered in the period.
Messages and Announcements had their use decreased over time, while the use of the Assignments tool had its peak in the year 2016/2017 with average use of 14.40 times per course offered (Table 4).  Online tests have been used as often since the system was launched, in 2013. The use of the Calendar tool has decreased over time and the Forums tool has been less used over the period but has increased in use over time.
The number of curricular units per year where the LMS was minimally used is shown in Figure 2. Minimum usage was considered when the sum of the use of all tools ranged from zero to 20 times per curricular unit. The percentage of minimum use of the LMS, in the curricular units offered between 2013 and 2019, ranged from 26% to 37%. The tools used in these course units were Messages, Resources and Announcements.

Focus Group with Students
When asked which tools were known to them, Portuguese students answered Resources, which they use to obtain content material; Online tests; Assignments, to submit all tasks; Calendar, as some teachers post course information; and Messages, where they can exchange messages with the teacher and classmates. Brazilian students also identified the Archive tool to download support material posted by the teacher; Questionnaires and Assignments, used for student evaluation; Message Box and Topics, which refers to how the course content is organized by the teacher.
Portuguese students considered the Resources, Assignments and Messages tools to be the most important, as they provide access to course content, assign assignments for teacher evaluation, and communicate with the teacher as needed. Brazilian students listed the Online Testing tool as very interesting. They mentioned that being able to receive email alerts each time the teacher posts material is very important. They do not have the habit of login in periodically, preferring being notified instead.
Regarding initial guidance on using the systems, Portuguese students said they attended a lecture or received a tutorial on how to use the LMS and found the use of the system to be quite intuitive. Brazilian students reported that they did not receive any initial instructions for using the LMS and found the system unintuitive. Some students believe that it would not be important to receive initial usage guidance if the system were more intuitive. Others would like to have received a tutorial. Asked what could be improved to make it more intuitive, students responded that its screen is very polluted, and that important information is hidden. Besides that, the student needs to register in each course, through a password given by the subject teacher, because the LMS is not linked to the academic system of the Brazilian university.
Portuguese students appreciate the Resources tool and have exam results as soon as they finish their Online tests. Brazilian students appreciate the organization of subjects by content and the ease of submitting an assignment to the teacher.
The good experiences of Portuguese students are the ability to access the system from wherever you are, just needing internet access, to be able to exchange messages with colleagues and the teacher and to be able to participate in exams if in the impossibility of being physically present. For Brazilian students, what provided good experiences was the centralization of content and the organized availability of support material for the discipline. One student commented that the task submission system works well and another student had a great experience with online assessments because he knew about his performance right after the quiz completion, realizing how much he needed to achieve the average grade. Everyone agreed that if the teacher knows how to use the tool well, the student's experience in the system will be better.
Portuguese students reported that they do not like to respond to mandatory teacher assessment surveys, an extension added by the university under the mandatory law for teacher evaluation. If the student does not respond, he loses access to other features of the LMS, while the questionnaire is not finished. Brazilian students reported that the system often goes down. One student did not like the experience of attending the non-classroom calculus discipline, because the teacher put only text material in the files and when there was a video class, it was of poor quality. Other reports were misused by the teacher, such as opening more than one send box for the same assignment and not finding the file the student sent.
Both Portuguese and Brazilian students reported that they do not use virtual systems to communicate with teachers and classmates. When they want to talk to the teacher, they use email, and when they want to talk to classmates, they use social networks like WhatsApp or Facebook. Portuguese students understand the communication tools for use by the teacher when they need to advise they have changed the date or place of the class. Brazilian students suggested that the teachers could open a chat, during student hours, to answer the questions of those students who are not present at the university.
Suggestions for improving the use of the Portuguese university LMS were creating a specific place for students to consult their grades, preparing and standardizing the way teachers organize the material available in Resources. Brazilian students suggested that the academic system should be integrated with the LMS.
Finally, when asked what would be needed in the system, if there were no face-to-face classes, Portuguese students said they would like to have access to live or recorded video classes, with the possibility of interaction with the teacher. They see it as necessary to receive a tutorial on the main points of the class to learn what to focus on, as well as a guide on how to operate the system to access content. Another is the implement of templates and guidelines for teachers to use the system. When the material is organized, the student can better follow the course, as it will not be in the classroom in the presence of the teacher.
Brazilian students do not see the LMS as a learning platform. Basically, they see it as a content repository. If the course is structured like this, there is no need for the teacher to put material in the system, just indicate the book that students should read and mark the day of assessment. They commented that the platform is not prepared for this because it does not have an online class tool (unaware of the BigBlueBottonBM tool). They suggested hosting video lessons on YouTube and sharing the link via the LMS. One student suggested that the teacher use games to motivate students to continue the course and another student concluded that when the teacher used the system tools well, he could learn more within the virtual platform.

Questionnaire with the Teachers
Six teachers from the field of electronic engineering, food engineering, environmental engineering, mechanical engineering and computer science answered a questionnaire with questions related to the use of the LMS in their pedagogical practices. Only one teacher mentioned that he does not use the LMS, did not receive any training and considers it important to have an initial training. When he tried to use the LMS by himself, he was unsuccessful and found the system unintuitive.
Of the teachers who use the system, two took a basic training course and the rest stated that they learned to use the system alone. The computer science teacher thinks that it would not be important to take a course to start using the system. Another teacher stated that he would only be interested in taking a course if the classes were divided into basic, intermediate and advanced level, because he already had basic knowledge of the tools.
The best-known tools for the teacher group are Files, Database, Folder, Assignments and Forum. The most commonly used tools are Files, Assignments and Folder. Teachers would like to learn to use the Lesson, BigBlueBottonBM and Workshop tools. The tools teachers find necessary for a fully online course are File (100 %); Forum and Folder (83 %); Workshop, Lesson, Quiz and Assignments (67 %); BigBlueBottonBM, Chat and Database (50 %).
When asked what they like most about using Moodle, they mentioned using a quiz from a question bank because students can get instant feedback. Teachers appreciate the fact that the system allows control over reception of Assignments so that they are organized.
Regarding the good experiences in using Moodle, the ease of making material available to students in a private environment, the creation of a question bank, setting dates for assignment delivery, content archiving, asynchronous activity possibilities, immediate feedback of test results and system flexibility, were mentioned.
The difficulties encountered by teachers in using the LMS were trying to upload a large ebook and fail and unable to fix tasks directly, because system does not allow to increase image size. All teachers mentioned that the system interface is not intuitive.
If the course were entirely e-learning, teachers believe that the material should be made available in the form of recorded video lessons and some would be delivered in real time using the BigBlueBottonBM tool; hold Forums, Assignments, Quiz and Chats. But more important than technology would be learning how to build teaching practices using system tools.

CONCLUSIONS
Both the Portuguese and Brazilian universities LMS have similar tools and are mostly used for teacher material repository and student task upload. When the teacher knows how to use the system, the student's learning experience is better. How easy it is to follow the course through the system depends on how organized the materials are. The better organized, the easier it is for students. Portuguese students are more comfortable using the system than the Brazilian students. In the Portuguese university, the use of LMS is encouraged, as a mean of sharing of content and information between students and teachers. In the Brazilian university there are other tools that are used for this purpose, besides the LMS. Both systems have tools to deliver online courses, but the teacher must have knowledge of how to apply teaching practices using the tools to facilitate and motivate students to adhere to learning.