Design of An Intelligent Tutoring System – Student Model: Predicting Learning Style

- Education is very important for everyone, not only for acquiring knowledge but also for improving quality of life and well-being. An Intelligent Tutoring System (ITS) is a computer system that can provide personalized and adaptive learning assistance and support to students. This system is designed to offer effective guidance to students based on their individual abilities and learning styles. ITS utilizes artificial intelligence (AI) technology to understand students’ abilities and provide guidance tailored to their needs. Recently, there have been methods to predict learning styles, such as through questionnaires on the EducationPlanner website, but these determinations are often too general. This study aimed to predict the learning styles used by specific students for specific subjects. Researchers conducted this study at XYZ University to determine the learning styles of certain students or groups. With this information, instructional materials and methods can be uniquely designed to cater to the needs of these groups. Based on the evaluation results, the study found that the Logistic Regression model was the best, with a precision of 0.5653 and a hamming loss value of 0.3468. This research demonstrates that information from six selected subjects (English, Religion, Civics, Arts, Physics, and Geography) can be used to determine students’ learning styles.


I. INTRODUCTION
The Change is a necessity, the inevitability that is needed and desired is a change accompanied by progress.In the industrial era 4.0, industry players are connected and communicate with each other which will make decisions without human involvement.A number of industrial sectors are ready to enter the industrial era 4.0, including the education sector at universities.Discussing the education sector will be closely related to digitalization.Many learning and teaching activities have also been carried out through video conferencing platforms and learning management systems that can help communicate and discuss material provided by instructors interactively without being limited by distance.
Teaching and learning process in tertiary institutions cannot be separated from the role of the lecturer who will provide lecture material.In Indonesia, a person can teach if he meets the academic qualifications determined by the Indonesian government, one of which is in Ministry of Education decree 44 2015 concerning National Higher Education Standart Article 26, as follows (Kementerian Riset, Teknologi, 2015),shown in Table I: From the data provided above, it can be seen that university lecturers in Indonesia have academic standards and workloads regulated by the government.However, in reality, many students still encounter difficulties in understanding the course materials, and one of the contributing factors is the less interactive or passive teaching methods (Gunawan Tambunsaribu & Yusniaty Galingging, 2021).
On the other hand, each student is unique and diverse.The objective of this research is to explore alternative methods by creating a predictive model in machine learning that can predict students' learning styles.This allows educators to understand the needs of students in determining learning strategies for each specific type of learner.This component is included in the student model which is one of the components in a larger system known as the Intelligent Tutoring System (ITS).
Several studies suggest the use of resources offered by artificial intelligence (AI) to provide adaptative learning environment based on student characteristics and enable high-level interactions (Sein Minn, 2022).ITS allows for the modeling of a student's knowledge level and utilizes teaching strategies to enhance or correct a student's knowledge.They are based on the development and implementation of artificial intelligence (AI) methods and techniques, and as a result, the content and presentation methods of topics can be tailored to individual student abilities, shown in Figure 1.psychology and education (Woolf, 2009) Figure 1 illustrates how ITS applies the theory of the teaching and learning process while incorporating various disciplines and educational domains to select effective teaching tactics for students.Psychology studies can be used to examine how students behave, understand how they learn, and determine the best ways to motivate them.Computer science is utilized to develop applications and determine the technology needed to assist students (Woolf, 2009).ITS is built based on four interconnected software modules, shown in Figure 2. The primary goal of the Student Module is to ensure that the system possesses fundamental knowledge about each student so that it can respond appropriately, engage the student's interest, and enhance learning.The Student Module will generate teaching strategies, which can be seen in the Figure 3   In the real classroom, teachers need to know understand the learning styles of their students to fit their teaching strategies (Cantina, 2021).ITS can adapt to learning styles use a formal questionnaire to collect information about learners (Yotta, 2023).
To achieve the highest level of understanding in learning outcomes, every learner possesses a unique learning style.Some textbooks described learning style as an individual style, the other half defined it as a choice or strategy.Texts for beginners studying education tended to take a more positive view of the use of learning styles, whereas texts for beginners studying educational psychology took a more neutral position.Matching instructional strategies to student learning styles was advised in 25% of the textbooks (Wininger et al., 2019).There are three main types of learning, namely visual, auditory, and tactile/ kinesthetic, according to (Yaseen et al., 2021).
Machine Learning (ML) techniques in Intelligent Tutoring Systems (ITS) are used to achieve various goals, including: Building Predictive Models for Student Learning Styles: This assists educational institutions in understanding and integrating learning styles to maximize student learning outcomes (Marouf & Abu-naser, 2019).
Identify Effective Teaching Strategies: ML targets individual skills and assesses student knowledge (through practices or teaching actions -hints or explanations) to provide better teaching interventions (Woolf, 2009).Multivariate Imputation refers to techniques that use information from multiple variables (features) to predict and fill in missing values in the variable being imputed.One of the techniques of multivariate imputation is Iterative Imputation, which involves an iterative approach where missing values are filled based on model predictions that take into account other variables (Fouad et al., 2021).
According to (Berrar, 2018), K-Fold Cross-validation is one of the most widely used methods of data sampling for assessing the generalization ability of predictive models.Its utility includes aiding in reducing bias from relying solely on a single data split and assisting in optimizing model parameters.In this method, the training data is divided into k subsets of the same size or nearly the same size (k-fold), where k is a positive integer, shown in Figure 4.The data will be partitioned into k sub-sections or folds.For example, in Figure 6 using 3-fold crossvalidation with 12 data points, there will be 3 iterations.In the first iteration, the first subset will use the first fold for testing, while the other folds will be used for training.In the second iteration, the second subset will use the second fold for testing, and the other folds for training.Similarly, in the third iteration, the third subset will use the third fold for testing, and the other folds for training.Although, lack of a formal framework for selecting the number of the neighborhood k is problematic (Melek et al., 2017).
A well-established and independent industrial process model for data mining tasks is CRISP-DM.It consists of six iterative phases, spanning from business understanding to the deployment process, shown in Figure 5. Logistic Regression is one of the most popular machine learning algorithms in supervised learning techniques.This algorithm is used to analyze the relationship between one or more independent variables (predictor factors) and a dependent variable that is binary (two categories) or nominal that can be treated as binary (for example, yes/no, success/failure, 1/0).Logistic Regression can be employed to classify observations using various types of data and can easily determine the most effective variables for classification.The Figure 6 below depicts the logistic function.

II. METHODS
The research is conducted by creating a machine learning model using a supervised learning algorithm.The method to design the machine learning model will follow the CRISP-DM approach, with the following Figure 7 stages: The stages of the research are carried out by analyzing the business needs that have been explained previously, analyzing the data requirements needed for making machine learning models, tidying up the data that has been collected, processing and preparing data, making machine learning models, evaluating the results of making machine models learning, and system design to implement machine learning models as student models in predicting student learning styles at XYZ University.

Business Understanding
The exploration process is conducted through interviews with several undergraduate lecturers at XYZ University.Currently does not have a mapping of students based on their learning styles.Referring to the Education Planner website, which provides self-assessment for determining learning styles, the complexity of implementation would be higher if the university wants to integrate this self-assessment application with the existing Learning Management System (LMS).Additionally, the input process would take longer as it requires answering 20 questions in the assessment form.
Therefore, predictive modeling using logistic regression in machine learning is needed to predict learning styles more efficiently.This is because the input data is derived from students' actual high school (SMA) grades and is more time-efficient since predictions can be made with only a few student grades inputted in a relatively short time.

Data Understanding
Data was obtained from the research subjects by administering a questionnaire using an online survey tool.The data sample consists of active student data for the year 2023 from XYZ University.The questionnaire data is divided into two parts: • The first part consists of 20 questions sourced from the EducationPlanner website questionnaire at: http:// www.educationplanner.org/students/self-assessments/learning-styles-quiz.shtml# to obtain the student labels, shown in Figure 8.If a student receives the label "Visual" from the selfassessment learning style, the researcher assigns 1 point to the Visual column, 0 points to the Auditory and Tactile columns.A student can receive one or two labels with different combinations.If a student receives both "Visual" and "Tactile" labels, the researcher assigns 1 point to both the Visual and Tactile columns and assigns 0 points to the Auditory column.In Figure 11, there are 3 columns for the label variable and 14 columns for the predictor variables.

Data Description
The results of the data collected have the

Data Preparation
In this phase, data from non-active students will be excluded from the dataset as an exclusion criterion.Non-standard data quality can be addressed through data cleansing techniques such as removing rows of data that have a value of 0 for all subjects.

Modelling
The creation of the machine learning model is carried out using two methods: Logistic Regression and Random Forest, shown in Figure 10.• Split the data into training and testing sets using k-fold cross-validation; the average evaluation scores will be taken from 5-fold.• Next, conduct a final evaluation to determine the best algorithm, whether it is logistic regression or random forest.• The best algorithm will be used as the final model for deployment.
• The model selection involves 5-fold cross-validation, with each fold having its training and test sets.The training set is utilized for binary relevance modeling, and the resulting model will then be used to evaluate its corresponding test set.

Model Evaluation
After obtaining the final model from binary relevance, an evaluation will be conducted using the precision score and Hamming loss metrics.

Deployment
Once the machine learning model has been established, the model can be implemented to predict students' learning styles.The output of each prediction will be stored in a database and used for retraining the model so that it can continue to update if there are changes in the prediction patterns.This approach ensures that the model remains accurate and relevant as new data becomes available.

III. RESULTS AND DISCUSSION
One student can have more than one learning style, so the classification problem addressed in this research is a multi-label classification with three response variables, each having a cardinality of 2 (0 and 1).Two machine learning algorithms will be employed: Logistic Regression representing a linear model, and Random Forest representing an ensemble model.Hyperparameter tuning using Bayesian Search and 10-Fold cross-validation will also be utilized to select the best hyperparameters for each model.Furthermore, model selection will be carried out using 5-Fold cross-validation.

Logistic Regression
One of the folds from the outer cross-validation (CV) will be used for further exploration of the model.Here is a list of coefficients from the L2 Logistic Regression model built on Fold-5, shown in Table III.Each variable is already on the same scale, so the absolute values of the coefficients above can measure the impact of each variable on the prediction outcome.Even small changes in the predictor variable values with large absolute coefficients can have relatively significant effects on the response variable's outcome.Therefore, in this case, variables with large absolute coefficient values will be used as conditions for feature selection in the logistic regression model.Based on the logistic regression model built on Fold-5, the best features obtained are as follows: Therefore, English, Religion, Civics, Arts, Physics, and Geography scores are the values that have the most significant influence on determining the learning style with the Logistic Regression model.Here are the best hyperparameters for the Logistic Regression model, shown in Table IV.The best hyperparameters for the optimization algorithm 'solver' in the logistic regression model are all among 'liblinear,' 'sag,' or 'saga' with varying regularization strength 'C.'The L2 logistic regression model tends to prefer larger values of 'C,' while the L1 logistic regression model tends to prefer smaller values of 'C.'.

Random Forest
One of the folds from the outer cross-validation (CV) will be used for further exploration of the model.Here is the list of feature importance from the Random Forest model built on Fold-5, shown in Figure V. Based on the feature importance of the trained random forest model, it can be observed that English and Arts scores are the most crucial features in distinguishing learning styles.When each feature is sorted based on its feature importance from high to low, a significant difference in values can be seen between the most important variables and others.This indicates the importance of English and Arts scores for this random forest model, shown Figure 11.In the binary Visual model using random forest, the feature importance score for English is more than twice the feature importance score for Geography, which is the second most important variable, shown Figure 12.In the binary Auditory model using random forest, the feature importance score for English is more than three times the feature importance score for Civic Education (PPKN), which is the second most important variable, shown Figure 13.In the binary Tactile model using random forest, the feature importance score for English is only around 1.4 times the feature importance score for Biology, which is the second most important variable.Overall, it is indicated that English scores play the most crucial role in distinguishing Visual from Non-Visual learning styles and Auditory from Non-Auditory learning styles.Additionally, Arts scores play the most crucial role in distinguishing Tactile from Non-Tactile learning styles.The results obtained from these feature importance scores align well with the findings from the data exploration.
Therefore, in this case, variables with high feature importance values will be used as conditions for feature selection in the random forest model.Based on the random forest model constructed in Fold-5, the following features are identified as the best: PPKN, Kesenian, Fisika, Biologi, Geografi, Ekonomi} Thus, the scores for English, Religious Studies (Agama), Civic Education (PPKN), Arts, Physics, Biology, Geography, and Economics are the values that have the most significant impact in determining learning styles using the Random Forest model.
Here are the optimal hyperparameters for the Random Forest model, shown Table VI.The selected optimal hyperparameter values for each binary Random Forest model vary significantly.The Auditory model tends to build a relatively complex model, as seen from the relatively large 'max_depth' values both with and without feature selection.

Evaluation
The evaluation was performed using 5-fold crossvalidation.In each fold, the multilabel model was evaluated using commonly used metrics for multilabel cases, namely hamming loss.In addition to hamming loss, precision score was also used because it is quite suitable for the case being studied.Therefore, precision score is more recommended than specificity for this case.Here are the evaluation results of the multilabel model with 5-fold CV using hamming loss metric.From the perspective of hamming loss, Logistic Regression with L2 penalty outperforms Logistic Regression with L1 penalty and Random Forest, although the difference in loss between Logistic Regression L1 and L2 is not very significant.
However, when each model is trained with only the best features, the hamming loss values become lower, indicating that feature selection plays a role in improving performance.When all models are compared, both models built with feature selection and those without it, it is shown that the Logistic Regression L2 model with feature selection outperforms the others in terms of hamming loss, shown Table VIII.Here are the evaluation results of the multilabel model with 5-fold cross-validation using the precision score metric, shown Table IX.From the perspective of precision score, Logistic Regression with L2 penalty outperforms Logistic Regression with L1 penalty and Random Forest.In addition to having the highest average precision score, Logistic Regression L2 also has the lowest standard deviation of precision score.
However, when each model is trained with only the best features, the precision score values become higher, indicating that feature selection plays a role in improving performance.When all models are compared, both models built with feature selection and those without it, it is shown that Logistic Regression L1 model with feature selection outperforms the others in terms of precision score.However, the difference in precision score between the models built with feature selection is relatively small, shown in Table X.Based on the cross-validation results with precision scores, Random Forest manages to outperform 3 out of 5 experiments, with the highest score reaching nearly 70% achieved on fold-5.Although Random Forest has a fairly impressive performance, its score variance is also the highest among the models.On fold-3, its precision score is the lowest compared to the other models.On the other hand, Logistic Regression has lower variance in performance with a higher average precision score compared to the other models.

IV. CONCLUSION
Machine learning modeling with feature selection can predict students' learning styles by inputting 6 subject grades (English, Religion, Civics, Arts, Physics, and Geography).This can facilitate educational institutions, allowing educators to design teaching strategies tailored to each student.Therefore, it can be concluded that the binary relevance model based on logistic regression with feature selection is the best model according to the results of K-Fold Cross Validation, with an average precision score of 0.5653 and an average hamming loss of 0.3468, outperforms the binary relevance model based on random forest, which only has an average precision score of 0.5643 and an average hamming loss of 0.3613 Researcher hopes that this study can be extended by obtaining primary data directly from educational institutions (high schools) and by employing other prediction models such as XGBoost and Multi-layer Neural Network.By utilizing such primary data and different prediction models, the author anticipates that the performance of the constructed models could be further optimized.

Figure 1 .
Figure 1.The ITS field is based on three disciplines: computer science,psychology and education(Woolf, 2009)

Figure 2 .
Figure 2. Intelligent Tutoring System Architecture Component Features below.

Figure 3 .
Figure 3. Basic architecture of an ITS (Morales-rodríguez et al., n.d.) SMOTE works by performing oversampling on the minority class.The technique involves the following steps: Selecting a sample from the minority class.Choosing several nearest neighbors of the selected sample (k-nearest neighbors).Creating new synthetic examples by combining the selected sample with its nearest neighbors through line segments connecting them.By following these steps, SMOTE increases the number of samples in the minority class by generating synthetic examples that represent the variations within that class.Imputation in machine learning refers to the process of replacing or filling in missing values in a dataset with estimated values.Missing values can occur in a dataset for various reasons, such as data collection errors, faulty sensors, or the inherent unavailability of certain data.One type of imputation in the context of filling missing values in a dataset is Multivariate Imputation.

Figure 8 .
Figure 8. Selft assesment questionnare • Second part consists of 14 questions regarding the high school subject scores, including the grades for Indonesian Language, English, Mathematics, Physical Education, History, Civic Education, Arts, Religion, Physics, Chemistry, Biology, Sociology, Geography, and Economics, shown in Figure 9.

Figure 9 .
Figure 9. Data with label

Figure 10 .
Figure 10.Model Process Creation of a machine learning model involves various stages, namely: • Import dataset.• Perform global preprocessing.•Split the data into training and testing sets using k-fold cross-validation; the average evaluation scores will be taken from 5-fold.• Next, conduct a final evaluation to determine the best algorithm, whether it is logistic regression or random forest.• The best algorithm will be used as the final model for

Table I .
Lecturer academic qualifications

Table II .
Table II following attributes: Data Attribute

Table IV .
LR -Best Hyper Parameter

Table V .
Random Forest Coefficient

Table VI .
RF -Best Hyper Parameter

Table VII .
Hamming Loss Metric

Table VIII .
Hamming Loss Metric (feature selection)

Table IX .
Precission Score Metric