Leveraging deep learning and graph analysis for enhanced course recommendations in online education

Abstract

Online education growth requires the development of effective customized recommendation systems to improve student involvement and educational performance. This research, suggests new hybrid model based on Convolutional Neural Networks (CNNs) with graph analysis to improve online course recommendations by delivering more tailored suggestions to students. Our suggested process starts with extracting raw student and course data from a database which is preprocessed for training a CNN model. The purpose of this CNN model is to identify essential characteristics from student records and educational performance data and course information for predicting student course selection probabilities. The proposed model then defines the students as the fundamental entities of a graph network while behavioral and educational relationships create the edges between them. The application of graph analysis serves to detect behavioral patterns among students while predicting new possible relationships through solutions to cold start problems. This goal is achieved by applying the link prediction technique on the constructed graph. The generated course recommendations result from combining information derived from CNN models and graph analysis techniques. Experimental results based on 12,898 students from the Islamic Azad University E-Campus Tehran, Iran validated this hybrid approach through better performance than traditional methods which yielded precision at 0.8336 and F1-Score at 0.3347 thus confirming its ability to provide precise relevant course recommendations.

Introduction

The explosion in online learning platforms revolutionizes the educational world because it provides access to resources that were previously out of reach worldwide. However, such an immense number of courses faces learners with a very important problem: the selection of courses best fitted to their skills, interests, and learning objectives. Moreover, courses have varying quality and content; this demands the need for developing sophisticated recommendation systems in order to effectively guide learners^1,2. Traditional recommendation techniques such as collaborative filtering, content-based filtering, and matrix factorization have been instrumental in addressing this challenge; they are not without limitations. Issues such as data sparsity, cold start problems, and the need for expensive feature extraction limits their effectiveness and scalability^3,4,5.

Recommender systems are widely applied in Internet applications to assist users in finding their favorite items or services when they face too many options or too much information⁶. In preparation for the post-COVID era, the global education market recently accelerated its pace of transitioning from offline education to online education^7,8. The integration of recommender systems with e-learning systems will enable the filtering and selecting of useful and relevant resources for each learner, thus reducing the time needed to choose among them. The role of recommender systems is very important when developing an e-learning system to guide learners, hence ensuring a personalized learning environment, called adaptive learning⁹. The main recommendation systems can be considered to be one of the information-filtering tools, which predicts user preference for an item¹⁰, reducing information overload caused by the large volume of information present on the web¹¹.

This research examines the challenges and issues in recommender systems in the field of online education and attempts to fill important research gaps. The first research gap is related to the limitations of recommender systems in personalizing recommendations. Current methods cannot effectively consider individual characteristics and behavioral histories of candidates in providing educational recommendations, and this research attempts to address this shortcoming. The second research gap is related to the inefficiency of representational methods in modeling the relationships between individuals. Previous methods have not provided us with a detailed view of modeling the relationships between individuals, courses, and other conceptual factors. Therefore, our proposed method will seek to solve the first challenge using deep learning techniques and the second challenge using graph analysis and user network clustering. In general, solving these challenges will respond to the shortcomings felt in previous works, which is why this research was formed.

Our research includes three main innovations that can help improve recommender systems in online education. First, the use of convolutional neural networks to extract complex patterns related to course selection and modeling people’s behavior in choosing courses can reduce the complexity of the problem and provide more accurate recommendations. Second, the use of graph analysis, graph clustering, and link prediction techniques can effectively solve the cold start challenge in course recommendation systems and thereby connect new users with users who already have behavioral records in the system model. And the third innovation is the result of combining the first and second innovations, which, by integrating the CNN model and the graph analysis model, allows for the simultaneous combination of individual and relational characteristics of people’s information and helps provide more accurate and personalized recommendations.

This study contributes to research on recommendations in online education by demonstrating the following:

• The use of CNN helps to extract complex patterns related to course selection and model people’s behavior in course selection.

• The use of graph analysis, graph clustering, and link prediction can effectively solve the cold start challenge in course recommendation systems, thereby connecting new users with users who have a behavior in the modeled system.

• The combination of CNN and graph analysis is the result of the synergy of the two previous initiatives, which, by integrating these two models, allows the simultaneous combination of individual characteristics and information relationships of users, ultimately helping to provide more accurate and personalized recommendations.

The paper is organized as follows: Sect. "Literature review" reviews the similar research in the literature review, Sect. "Research method" outlines the proposed methodology, while Sect. "Implementation results" discusses the findings of the study and conclusions are made in Sect. "Conclusion".

Literature review

Research efforts have increased rapidly due to the growing number of online learning platforms to build efficient course recommendation systems. These systems function as guidance tools for learners to access suitable educational resources which improves their learning process and produces better educational results. This segment delivers an analytical review of different recommender system strategies which operate within the online education sector.

Gulzar et al.¹² introduced Personalized Course Recommender System (PCRS) which uses both hybrid methodology and ontology-based methods as its foundation. The researchers developed a framework which increased learners’ learning both efficiently and satisfyingly through customized curriculum selections. This combined recommendation methodology successfully addresses traditional recommendation system flaws while resolving the individual approach weaknesses according to the authors. Online education demand has intensified because of the COVID-19 pandemic thus demonstrating the necessity for reliable course recommender systems. In response to this, Li and Kim¹³ developed DECOR, a deep learning-based course recommender system. The researchers conducted investigations to solve data sparsity and scalability issues and proved that DECOR delivers better recommendations than conventional systems.

Deep learning methods have recently become popular tools for designing modern e-learning recommendation platforms. Liu et al.¹⁴ delivered an extensive review about deep learning-based recommendation systems which operate within e-learning environments. The authors examined multiple deep learning architectures in their research which included multilayer perceptron machines, recurrent neural networks, convolutional neural networks, neural attention mechanisms and recommendation systems based on deep reinforcement learning for this domain. The research by Salau et al.¹⁵ investigated deep learning methods for user opinion prediction in e-learning together with other domains. The authors demonstrated through their research how quantitative assessment enables better prediction of future progress in recommender systems development.

Kulkarni et al.¹⁶ extended deep learning capabilities to build an adaptive e-learning system which applies hybrid deep learning for individualized recommendation engines. The research developers extracted text features then optimized their weighted features through term frequency-inverse document frequency (TF-IDF) and GloVe embeddings techniques. The experimental results demonstrated that recommendation efficiency improved according to their findings. The combination of deep collaborative filtering and wide linear models forms a hybrid recommendation model for K12 online education according to Gong et al.¹⁷. The joint application of different modeling approaches led to higher Area Under Curve (AUC) results by 12.7% compared to standalone collaborative filtering systems according to their research findings. Ren et al.¹⁸ addressed the problems from the COVID-19 pandemic in online education through the development of an LSTM-based deep course recommendation system with attention integration. The recommendation system used multiple data formats consisting of course video, audio, title, introduction, demographics and user feedback to boost recommendation accuracy.

The authors from Safarov et al.¹⁹ developed a deep neural network solution to help e-learning platforms handle resource overload and discover appropriate educational content for teachers. The system used synchronous sequences together with heterogeneous features to achieve precise recommendations and minimize cold-start issues. ALBERT (A Lite BERT) serves as the foundation for Nanda et al.‘s²⁰ personalized educational recommender system which enhances student education through contextualized word representation and semantic meaning understanding in learning materials and user accounts and interactions. Li et al.²¹ studied all wide-ranging uses of deep learning methods within recommendation systems through their extensive review. The review demonstrated that graph, convolutional and recurrent networks play a crucial role in different domains which include social networks, e-commerce and e-learning.

The research of Ahmad et al.²² developed Attributed Bipartite Network Embedding for Course Recommender System (ABiNE-CRS) to boost course recommender systems through combined implicit feedback with high-order collaborative relations and content similarity which produced superior results than current approaches. The author Wang²³ examined a neural network-based intelligent recommendation system which tackles information overload problems in online education. The proposed model produced reduced absolute error together with faster prediction speeds and demonstrated stronger educational resource recommendation capabilities than different recommendation algorithms. EBSAAS stands for Edge-Based Student Attentiveness Analysis System which uses facial detection algorithms together with deep learning technology according to Abdulkader et al.²⁴. The developed system demonstrated better performance than current industry standards as well as providing instructors with evaluation opportunities to enhance their teaching practices in edge-based online education methods. Song et al.²⁵ developed DFDLOF as a machine learning framework which applies individualized learning pathways to enhance student success through immediate feedback systems and specifically tailored educational resources. The researchers at Chinnadurai et al.²⁶ established a deep learning system which connected clustering methods to supply precise individualized learning recommendations to minimize information sparsity and startup problems. Mekala et al.²⁷ developed an improved deep hybrid model for online course recommendation which achieved better results than other recommendation systems and improved learner knowledge when dealing with limited data while optimizing training efficiency and classification accuracy.

The research by Cha et al.²⁸ evaluated AI-based course recommender systems because they show promise to assist students in achieving their educational goals and professional development and intellectual advancement. The researchers pointed out that recommendation systems create an uncertain environment for decision-making while being adaptive in nature. The researchers from Xu et al.²⁹ presented a new recommendation method that combined collaborative filtering with knowledge graph elements to improve the recommendation precision through item relationship analysis. Student learning outcome prediction along with matrix factorization from Nguyen et al.³⁰ provides course recommendations through performance data analysis while depending on the quality of performance data. Yang and Cai³¹ established a bilateral knowledge graph-enhanced online course recommendation framework to combine user characteristics with item semantics for improved precision and cold start solution yet its success relies on the quality of the knowledge graph and model complexity. Specifications of each reviewed paper are listed in Table 1.

Table 1 Summary of the reviewed works.

Research initiatives from the previous years have progressed the practice of course suggestion for online educational settings remarkably. Some research gaps still exist despite the valuable findings obtained from previous studies. The majority of current recommendation approaches face problems pertaining to sparse data while dealing with the cold start issue particularly when they depend only on collaborative filtering systems or content-based systems. The proposed approach resolves these restrictions through CNNs from deep learning technology since these networks efficiently extract essential details from mixed data types. Graph analysis techniques incorporated into this system makes it capable of modeling student-course relationships for creating accurate individualized recommendations. The proposed model integrates strong approaches to solve previous method limitations and deliver better customized course recommendations to students.

Research method

This section presents a hybrid approach, based on CNN and graph analysis, to course recommendations for students. This approach leverages the capability of CNN in extracting complex features from both structured and unstructured data, along with graph analysis techniques that identify behavioral patterns in students for the improvement of recommendation accuracy and the effectiveness of educational recommendation systems. After that, this section will explain the data collection process and its structure, followed by the details of the proposed method.

Data collection

For this study, we make use of the enhanced database that includes knowledge on 12,898 students of the Islamic Azad University E-Campus, Tehran, Iran. The MySQL 5.5 database contains knowledge on various details such as student choices amongst multiple semesters, course specifications, and personal characteristics of the students.

The data in this dataset is organized into three key tables:

• Student Table (S): This table carries personal and academic information about each student, including their student ID, first name, and last name, age, gender, the study field, specialization, year of entry in the university, overall GPA, number of completed credits, failures and withdrawals, previous academic history, academic interests, extracurricular activities, among other information regarding individual characteristics. Note that additional information has been excluded.

• Course Table (C): This table includes information about each course, such as course code, course title, prerequisites, corequisites, the instructor associated with the course (identified by a unique identifier), course level (introductory, intermediate, advanced), subject area, and other related course specifications, with additional information having been ignored.

• Student Course Selection Table (SC): This table illustrates the relationship between students and courses, including student ID, course code, semester, grade achieved, and course completion status.

Before using the data, a data cleaning process was conducted by removing incomplete, duplicate, and inconsistent records, as well as extracting the information needed for analysis. In addition, students with only one term of course selection were excluded from the analysis. It is important to note that during this data cleaning, any identifying information, including the names of students, was removed and each student’s ID in the overall database was replaced with a new, unique identifier.

Figure 1 shows the overall structure of the database used in the present study, showing schematically the relations between several tables and what information each table includes.

Fig. 1

The structure of the tables comprising the dataset of this research.

In Fig. 1, table S contains the student’s personal information and table C shows the course specifications. Table SC also stores the course selection information of students in different semesters. Each of these data sets provides the necessary input for the course recommendation model, which will be described below in detail. The data collected in this study plays a very important role in the development of the course recommendation model. Using this data, it is possible to identify the behavioral patterns of students and provide them with accurate and personalized recommendations.

Proposed method

This section provides a description of the proposed hybrid approach, which includes course recommendations for students by CNN models and graph analysis. Figure 2 illustrates the proposed hybrid strategy for providing recommendations based on its constituent phases.

Fig. 2

Steps of providing recommendations in the suggested approach.

The suggested recommender system’s steps can be summarized as follows:

• 1. Feature Extraction using CNN: This step involves the extraction and cleaning of raw data from the database. It then converts the data into an appropriate format that can serve as an input for the convolutional neural network. Later, the convolutional neural network extracts key features from the input data. These key features contain information about students and courses that may help the model in finding the concealed patterns within data. The CNN model applied at this step is trained on the probability that a student will choose to take some course depending on his attributes, performance, and academic history.

• 2. Graph Analysis and Link Prediction: During this step, students are regarded as nodes of a graph and the connections between them are modeled with regard to educational and behavioral relationships. Graph analysis techniques are used to identify the behavioral pattern of students; students with similar characteristics are then organized into clusters. Moreover, link prediction methods are used to overcome the cold start problem of the proposed recommender system, by forming a set of potential relationships not covered by the data. This information will help the model to give better recommendations.

• 3. Information Integration and Recommendation: Finally, by combining the information extracted from the convolutional neural network and the graph analysis, the proposed model will be able to deliver precise and personalized recommendations to students.

Given the independence of data in the first two phases, these processes are executed offline and in parallel. After the CNN models and the graph connections are formed through the first and second phases, this information is integrated in the third phase to enable the recommendation process to be executed online. Each of these steps will be explained in detail below.

Feature extraction using CNN

The first phase of the proposed approach is the extraction of features related to students’ course selection, done by a CNN model. The prerequisite for this model’s correct operation is to prepare the data format for this CNN, which includes the data preprocessing and transformation.

• A) Data preprocessing and preparation.

Data preprocessing is a very important step in the machine learning model development. During this process, raw data is extracted from the database and transformed into a format suitable for input to the CNN. This process begins with the extraction of raw data from the database, which includes information about students, courses, and students’ course selections. Student information contains features such as age, gender, field of study, overall GPA, number of completed credits, and so on. Course information includes features such as course title, prerequisites, corequisites, course level, and subject area. Course selection information consists of student ID, course code, semester, grade achieved, and course completion status. The course selection data for each student is organized as a sequence based on their academic semesters. Thus, for each student, it is possible to create N data records, each of which specifies the characteristics of the student in one of their N academic semesters.

After data extraction, cleaning of data is done. In this step, incomplete, duplicate, and inconsistent data is removed. Once the data is cleaned, it is transformed into a suitable format that can be used as an input to the CNN. It includes the following:

• Converting Student Information to Dense Vectors: In this step, information of all students is explained in the form of equal length vectors. Quantitative features including age, overall GPA, and completed credits are normalized by using the max-min strategy to bring them to the same scale. On the other side, nominal features such as the field of study and specialization are converted into numeric vectors by substituting each unique nominal value with a unique number.

• Course Information to Dense Vectors: Like in the previous case, the features of each course are described as numerical vectors of fixed length.

• Creating (Student, Course) Pairs: For each record created for a student, pairs are generated against all courses.

• Creating Labels: For each (student, course) pair, a label is assigned to indicate whether the student (based on the academic semester and the conditions of their descriptive record) selected that course or not. These labels serve as the target for the machine learning model.

After completing the preprocessing steps, the data is in a suitable format for training the convolutional neural network. In the next step, the CNN will be trained on this data to predict the likelihood of a student selecting a course. This structure will be described in further detail below.

• B) Proposed CNN model for predicting course selection.

The data samples formed in the format of pairs (student, course) and with the target labels assigned for each sample, based on the selection history of a student, are used to train the deep learning model. Deep learning models have emerged as a transformative force across numerous domains, demonstrating remarkable potential to solve a wide array of complex problems that were previously intractable or required significant manual effort³². Modeling the individuals’ behaviors is one of these domains which can benefit from the potentials of deep learning models such as CNNs. The CNN model used in the proposed approach is a one-dimensional network that contains input layers, convolutional blocks for feature extraction, and the necessary layers to predict the target variable. The architecture of the overall CNN model is shown in Fig. 3.

Fig. 3

Architecture of the CNN employed in the proposed recommender system.

According to Fig. 3, the CNN model used in the proposed method consists of three convolutional blocks. In this CNN model, one-dimensional convolutional and pooling layers are utilized to process the selected features. Consequently, it is natural that the input layer of this 1D CNN is defined as one-dimensional and accepts the extracted data from the previous step unchanged. In addition, the 1D CNN uses the sigmoid function as the activation function in its first convolutional block, while the ReLU activation function is applied in the second and third convolutional blocks. The model concludes with the necessary layers for predicting the target variable. In this regard, two fully connected layers, with dimensions of 100 and 2 respectively, reduce the dimensions of the extracted feature maps. Finally, the likelihood of a student selecting a course for the input pair (student, course) is determined using a sigmoid layer with a single neuron.

In order to obtain the best performance model for feature extraction, an extensive search strategy was carried out to tune the hyperparameters of the CNN. The hyperparameters involved in tuning the configuration of the CNN during the search process were filter size of each convolutional layer, numbers of filters in the convolutional layer, types of activation function, and types of pooling function. Also, the model’s performance was evaluated across various training configurations, including the training algorithm and the mini-batch size. Table 2 lists the hyperparameters considered for configuring the CNN model.

Table 2 Hyperparameters considered for configuring the CNN model.

The configuration search and its suitability evaluation were conducted based on the validation error criterion. As a result, the configuration described in this section for the CNN model (Fig. 3) yielded the lowest validation error following the comprehensive search. Furthermore, training the CNN model with the Adam optimizer and a mini-batch size of 32 achieved the best performance on the dataset used. The resulting trained model can predict the likelihood of a student selecting a course in the form of a numeric value.

Graph analysis and link prediction

In the second phase of the proposed method, graph analysis techniques are employed to analyze student features, allowing for a more precise scope of recommendations. To suggest appropriate resources to a student, dominant behavioral patterns of similar students can be utilized. Link prediction can thus be employed as an effective strategy in this regard. However, high complexity associated with this strategy renders it difficult to apply this strategy on large datasets. The use of clustering techniques can reduce the problem’s complexity. To accomplish this, the data structure describing the students is initially transformed into a graph, which is then clustered into groups based on similarities among students. Finally, link prediction techniques are applied within the cluster where the target student is located, enabling the extraction of a set of candidate students for recommendation modeling based on the results of this algorithm. Given the data independence between this phase and the first, parallel processing techniques are utilized in the proposed method to expedite the execution of this phase.

• A) Constructing the student graph and clustering.

The second phase of the proposed method begins with describing the student data structure in the form of a graph. In this step, each student is considered a vertex of the graph, and each edge is used to represent a high similarity between the characteristics of students within the same field of study. To create the aforementioned graph structure, an edge is constructed for each pair of students, such as i and j, who are enrolled in the same academic program. This approach results in the database being represented as a collection of full subgraphs, with each subgraph representing a student from a given field of study. These entire subgraphs only describe the similarity of the students’ fields of study through their links, not the similarity of the students’ characteristics. Subsequently, for each subgraph, the pairwise similarity among its members is calculated using the Minkowski distance metric, which is then used to weight the connections within each subgraph³³. The Minkowski distance is a distance metric in normed vector spaces and is recognized as a generalization of both the Euclidean and Manhattan distances. The general formula for the Minkowski distance is as follows:

$$\:D\left(x,\:y\right)=\:{\left(\sum\:_{i=1}^{n}{\left|xi\:-\:yi\right|}^{p}\right)}^{\frac{1}{p}}$$

(1)

In this equation, $\:D\left(x,\:y\right)$ describes the Minkowski distance between the n-dimensional vectors x and y, with$\:\:{x}_{i}$ and $\:{y}_{i}$ representing the corresponding elements of the i^th entry of vectors x and y, respectively. Additionally, p is the Minkowski parameter (where p ≥ 1). In the proposed method, the similarity between two records is defined based on the inverse of their Minkowski distance. The next step in the proposed method involves decomposing each subgraph into a set of communities based on the calculated weight values. To achieve this, the tree structure used in³⁴ can be employed to describe a graph. This tree structure ensures that each vertex of the graph is examined only once, thereby reducing the computational complexity of the proposed method. To identify communities through the formed weighted graph, the maximum spanning tree corresponding to the weighted graph is first created using Prim’s algorithm. This results in the removal of some low-weight connections in the graph. After establishing the maximum spanning tree, the next step is to remove the edges with the greatest weight from the obtained tree. If the graph in question contains N nodes, it is possible to remove $\:\left\lfloor {\frac{{N - 1}}{2}} \right\rfloor$ edges with the highest weight from the maximum spanning tree to form $\:\left\lfloor\frac{N+1}{2}\right\rfloor\:$local communities. Students within each of these local communities exhibit the highest similarity in terms of features and behavior, as their internal connections have the greatest weight. These communities are used as input for the second step of the proposed method, allowing the application of link prediction techniques on the members of each of the derived local communities to identify candidate nodes for providing recommendations.

• B) Filtering similar students based on link prediction.

After describing the set of students in the form of a graph and subsequently clustering it, the link prediction strategy is employed to identify suitable nodes for recommendations. This step (and the subsequent steps in the proposed method) comprises the online phases of the recommendation system. This means that these processes are used to provide recommendations for new samples.

The basic link prediction algorithm considers communication paths of varying lengths between the graph vertices and performs link prediction based on this. The link prediction algorithm consists of two phases: communication processing and similarity calculation. In the communication processing phase, all connections between vertices are described in the form of a matrix containing one-step connections (paths of length one). Thus, if there is a connection from vi to vj in the graph, the identifier corresponding to the destination node of the connection (i.e., vj will be stored in the cell of row vi and column vj. This matrix can represent all paths resulting from one-step connections between nodes. In the similarity calculation phase, the created one-step path matrix is used to generate paths of greater lengths. For example, by combining two one-step paths, a two-step path can be created.

To combine two paths and produce longer paths, all connections in the graph must be examined through three nested loops, and this complexity will increase as the path length l increases. The computational complexity of this algorithm is cubic, specifically O(n³), which makes it inefficient for large-scale graphs. To address this issue, the proposed method divides each subgraph into clusters of similar students, and the link prediction algorithm performs recommendations based on these subgraph clusters.

Accordingly, the link prediction algorithm is executed on the cluster to which the target student belongs, and a similarity matrix for the vertices is formed based on their connections. In the final step, a set of students is recommended as an output of the algorithm that have the highest similarity to the target student. To achieve this, after generating paths of length l, it’s necessary to calculate the similarity between the nodes.

The proposed method is based on the utilization of two metrics: centrality and path similarity characteristics, to develop a similarity matrix for node connections. For computing the similarity between students using the link prediction algorithm, the similarity matrix is updated according to the number of paths of different lengths actually linking the nodes. If v_x and v_y are two vertices in the graph, one can construct path matrices of lengths two and three, under the condition that no vertex is repeated. The greater the number of these paths, the higher the likelihood of similarity. Assuming a matrix with the lengths of all paths containing one and two edges for all pairs of vertices in the graph, then the similarity between the two vertices v_x and v_y can be calculated as follows:

$$\:sim\left({v}_{x},{v}_{y}\right)=\frac{{C}_{{v}_{y}}}{{\sum\:}_{\forall\:z\in\:L\left({v}_{x}\right)}{C}_{{v}_{z}}}+\sum\:_{i=2}^{l}\frac{1}{i-1}.\:\frac{\left|pat{h}_{{v}_{x},{v}_{y}}^{i}\right|}{\prod\:_{j=2}^{i}\left(n-j\right)}$$

(2)

In the above equation, n represents the number of nodes in the graph, and l is the maximum length considered for a path between two nodes, v_x and v_y. Additionally, $\:z\in\:L\left({v}_{x}\right)$ specifies the set of nodes that have a connection of at least length 2 with node v_x. In these paths, no cycles can exist. $\:\frac{1}{i-1}$ is a decay factor that weights paths of different lengths. For instance, paths of length two are considered with a coefficient of $\:\frac{1}{2-1}=1$, while paths of length three are weighted with a coefficient of $\:\frac{1}{3-1}=0.5$ in the similarity calculation. In the above equation, $\:\left|pat{h}_{{v}_{x},{v}_{y}}^{i}\right|$ indicates the total number of acyclic paths of length i between the two nodes v_x and v_y. Finally, $\:{C}_{{v}_{y}}\:$represents the semi-local centrality of node v_y. To compute the semi-local centrality, the nearest neighbors of the node and their neighbors are calculated and used in the centrality computation. The calculation of semi-local centrality for a node like i is performed using the following equation³⁵:

$$\:Q\left(u\right)=\sum\:_{w\in\:{\Gamma\:}\left(u\right)}N\left(w\right)$$

(3)

$$\:LC\left(i\right)=\sum\:_{u\in\:{\Gamma\:}\left(i\right)}Q\left(u\right)$$

(4)

where in the above equations, $\:{\Gamma\:}\left(\text{u}\right)$ is the set of neighbors of node u, N(w) is the number of nearest neighbors and next nearest neighbors of node w. This method has lower computational complexity compared to global methods and retrieves neighbors of w in two steps to compute N(w). By applying the link prediction algorithm to students within a cluster, a similarity matrix is generated that predicts the similarity among students in that cluster. Subsequently, the process of filtering similar students is conducted based on the highest likelihood in this matrix. Thus, students are considered similar to student i if they have the highest values in the i^th row of the similarity matrix. At the end of this phase, the set of nodes with the highest similarity based on link prediction is merged with the set of nodes that have direct connections to the target student. This combined set serves as input for the third phase of the proposed method, where the filtering of rules and the provision of recommendations based on this data take place.

Providing recommendations

The third phase of the proposed method takes the “trained CNN model N” and “similar students S” as inputs to perform the recommendation process by combining the information extracted based on them. For this purpose, each recommended course for student x is ranked using the following equation:

$$\:F=\sum\:_{i=1}^{\left|S\right|}\sum\:_{j=1}^{\left|C\right|}{\delta\:}_{i,j}\times\:sim\left({v}_{x},{s}_{i}\right)\times\:N({s}_{i},{c}_{j})$$

(5)

In the above equation $\:sim\left({v}_{x},{s}_{i}\right)$ represents the similarity between the target student x and the i^th candidate student from set S, which is calculated through the link prediction algorithm in the second phase of the proposed method. It is important to note that set S also includes student x itself. In addition, $\:\left|C\right|$ denotes the number of courses being evaluated for recommendation, and $\:N({s}_{i},{c}_{j})$ is the output value obtained from the CNN model based on the data pair student I, course j, predicting the likelihood that student I will select course j. Finally, $\:{\delta\:}_{i,j}\:$is a binary function that equals one only if the course selection history of student s_i does not include course j. After ranking the set of recommended options using the above equation, the members of this set are presented to student x based on the resulting rankings.

Implementation results

Our experiments were performed using MATLAB 2020a software and the evaluations were based on three key metrics: Precision, Recall, and F1-Score. During the experiments, the hyper-parameters of the CNN were implemented according to the setting introduced in Sect. "Feature extraction using CNN". The CNN was implemented using the MATLAB’s Deep Learning Toolbox. This model was trained using Adam optimizer and a mini-batch size of 32 within 350 epochs. During implementing the graph analysis phase of the proposed method, the Minkowski parameter in Eq. (1) was set as $\:p=2$. Additionally, the maximum path length parameter for the link prediction step in Eq. (2) was set as $\:l=3$. The experiments were designed in two separate research phases, each of which specifically examined and analyzed different aspects of the proposed method. This approach allows us to more accurately evaluate the performance of the algorithms and their effects on the results of the recommender system and to gain a deeper understanding of the analyses. Furthermore, we compared the proposed method’s performance with the approaches PCRS¹² and DECOR¹³. In the following, we will examine these two phases and the results obtained in more detail.

Evolution metrics

This research estimates the performance of the proposed model using three important parameters: True Positives, False Positives, and False Negatives, and then, these parameters are employed to describe the efficiency of the recommender system using precision, recall, and f-measure. The main focus of these metrics will help us gain a wide view regarding the performance of the model in identifying positives and negatives and explain the strengths and weaknesses of the model in detail.

• True Positive: The model correctly identifies Positive cases as Positive.

• False Positive: The model incorrectly classifies Negative cases as Positive.

• False Negative: The model inaccurately predicts Positive cases as Negative.

Precision: It is the measure of the model correctly identifying positive samples out of the actual positive instances³⁶.

$$\:Precision=\frac{TruePositive}{TruePositive+\text{F}\text{a}\text{l}\text{s}\text{e}\text{P}\text{o}\text{s}\text{i}\text{t}\text{i}\text{v}\text{e}}$$

(6)

Recall: It is a measure for the model’s performance in terms of correctly detecting the positive samples within an actual positive pool. This measure is also known as sensitivity or true positive rate³⁶.

$$\:Recall=\frac{TruePositive}{\text{F}\text{a}\text{l}\text{s}\text{e}\text{N}\text{e}\text{g}\text{a}\text{t}\text{i}\text{v}\text{e}+TruePositive}$$

(7)

F1-Score: The harmonic mean of precision and recall is called the F1-Score³⁶.

$$\:\text{F}1-\text{S}\text{c}\text{o}\text{r}\text{e}\:=\frac{2\times\:Precision\times\:recall}{Precision+recall}$$

(8)

Phase 1: evaluation of model generalizability

In order to evaluate the generalizability of the model, we analyzed different iterations of the proposed method to understand whether or not this model could preserve its good performance under various conditions and with data of different natures. In this experiment, we considered the number of recommendations to be one; in other words, the number of recommendations in this experiment is equal to one. To perform this evaluation, we repeated the cross-validation steps 10 times and looked at the relevant criteria. By doing so, we can estimate the stability and generalizability of the model more precisely, thus allowing us to draw more valid inferences from the results. In this way, we can ensure good performance of the model under various conditions and with various data.

Figure 4 shows the precision for different folds. This figure shows that the proposed method’s precision outperforms the comparative methods in each case. Higher precision by the proposed method than the APRIORI algorithm shows that this algorithm has been able to provide better performance using its model compared to methods that used to be based on common rules or data patterns. This significant difference indicates that we have been able to significantly increase the efficiency of conventional recommender systems by using deep learning techniques. In addition, the superiority of the proposed method in the precision criterion compared to the PCRS and DECOR methods also indicates the fact that this approach has performed better in identifying and recommending correct examples. In other words, when the recommender system recommends an item, the probability of the recommendation being correct in the proposed method is higher than in the compared methods. These results indicate that our method can be used as a more efficient tool in recommender systems.

Fig. 4

Evaluation of the precision metric for model generalizability.

Figure 5 shows the recall value with respect to the number of different iterations, and it is clear that in 9 out of 10 iterations, the proposed method has a significant advantage over other methods in terms of recall criterion. However, there is a point about the recall criterion that the recall value is very low compared to the precision value. The sharp decrease in recall can be justified by the fact that we have considered the number of recommendations equal to one. While in each data record, we have, each person has selected at least four items on average. Therefore, when we provide only one recommendation out of these four recommendations, it is natural that with this one choice alone we cannot cover all the previous choices of the person and as a result, the recall value will decrease. One possible solution to increase the recall value is to increase the number of recommendations, which this approach will be examined in the next experiment. Overall, the comparison of the proposed method against the APRIORI algorithm also replicated the same results, showing that the use of deep neural network architecture has led to a significant increase in efficiency compared to conventional or traditional models for recommender systems. Our proposed method has also managed to achieve a higher average recall overall compared to the PCRS and DECOR methods. This therefore reflects the high success of our approach in this area.

Fig. 5

Evaluation of the recall metric for model generalizability.

Figure 6 shows the F1-Score value, which examines the overall performance of the proposed method compared to other methods. As can be seen, the F1-Score value for the proposed method is still higher than other comparative methods. This result well reflects the interaction between the Precision and Recall metrics and shows that the proposed method has been able to provide better performance overall than the comparative methods. In other words, this F1-Score value indicates the success of our approach in achieving the desired balance between Precision and Recall, thus improving the overall quality of the algorithm’s performance.

Fig. 6

Evaluation of the recall F1-Score for model generalizability.

Phase 2: evaluating model performance for changes in the number of recommendations

During this phase, we tested the model’s performance based on different numbers of recommendations. To accomplish this, we changed the number of recommendations for the ranges of 1 to 7 and analyzed the median of the Precision, Recall, and F1-Score based on these changes. Also, during this experiment, cross-validation for each of these recommendation values was used and repeated 10 times. The average results received by these evaluations are represented in the corresponding graph. This method helps us to better understand the impact of the number of recommendations on the model’s performance and to be able to analyze its efficiency in different conditions more accurately.

Figure 7 shows that the Precision value decreases as the number of recommendations increases. More precisely, it decreases from 0.84 to 0.41 for the proposed method. This decrease in precision can be justified because when the number of recommendations increases, the probability that additional recommendations, unlike the existing data, will be less constrained increases. For example, when we provide 7 recommendations, in the real training data record, each person may have selected only four to five items. Therefore, the additional items that we provide as recommendations are naturally not present in the real data and are therefore recorded as errors. Therefore, it is natural that with the increased number of recommendations, the Precision value decreases. However, in general, the proposed method has still kept its distance from the comparative methods and generally performs better than them. All these results clearly confirm our claim in Fig. 4 and show that the outputs generated by the proposed method are more likely to be selected and accepted by users. The result is that this method has been successful in enhancing the user experience and improving recommender system performance.

Fig. 7

Evaluation of the precision metric for number of recommendations.

Figure 8 shows the recall value in response to changes in the number of recommendations, clearly confirming the discussion we had in Fig. 5. As expected, the recall value also increases with the increase in the number of recommendations. This increase is because when we increase the number of recommendations, the probability that these additional recommendations are among the individual’s previous choices and are somehow consistent with his contextual choices increases clearly. Meanwhile, our proposed method shows higher recall values ​​than other comparative methods and has been able to maintain its distance in all iterations. This clearly indicates the fact that the items recalled or recalled by the proposed method are more likely to be included in the users’ actual choices. In other words, these results confirm that our method is able to show greater accuracy in the identification and suggestion of options that are actually relevant to users. This clearly indicates the positive impact due to the proposed method on the performance of recommender systems and strengthens its reliability in providing appropriate recommendations.

Fig. 8

Evaluation of the recall metric for number of recommendations.

Figure 9 shows the F1-Score value and generally indicates that as the number of recommendations increases, the F1-Score value also increases. However, after six items, i.e. when we provide six or more recommendations, it is observed that the F1-Score value does not increase any more. The reason that justifies this situation is that by reaching this number of recommendations, we can no longer provide useful suggestions because valuable suggestions are already among the initial recommendations and we only provide additional options in the future. This clearly shows that our proposed method has been able to perform accurate ranking of recommendable items and provide the most appropriate options to the user at the very initial step. In other words, the efficiency of this method is that it can make the best and most relevant suggestions available at the very beginning and does not require searching for additional options. Finally, the superiority of the F1-Score value of the proposed method over other comparative methods indicates that this approach has been able to demonstrate better performance and provide more accurate recommendations to users. This emphasizes the high efficiency and reliability of the proposed method in providing personalized recommendations and shows that this algorithm is able to identify users’ needs well.

Fig. 9

Evaluation of the F1-Score metric for number of recommendations.

Discussion

The experimental data in this section demonstrates that our proposed methodology achieves superior effectiveness through its usage of CNN and graph analysis techniques for online course recommendation systems. Our approach outperforms baseline models PCRS¹² and DECOR¹³ because it delivers superior precision and recall performance at the same time as higher F1-Scores obtained while providing recommendations to students.

During the first phase of experiments (Sect. "Phase 1: evaluation of model generalizability"), our hybrid model confirmed its capability to deliver highly precise recommendations with one offered suggestion (Fig. 4). The CNN component demonstrates its ability to discover student-course meaningful representations which leads to precise predictions of course selection probability. Our method surpasses the performance of PCRS¹² which signifies that data-driven feature extraction from CNNs is able to understand intricate student course interrelations that may be difficult to express in ontology-based systems. Our approach demonstrates better precision than DECOR¹³ because integrating graph analysis to model student relationships provides additional valuable information for recommendation refinement.

Our approach shows promise in finding suitable courses among previously selected courses despite the lower recall values observed in the first phase of experiments (Fig. 5) due to the single recommendation limitation. The model demonstrated improved recall performance in the second phase of experiments (Sect. "Phase 2: evaluating model performance for changes in the number of recommendations") through additional recommendations which validated its capacity to identify diverse relevant courses (Fig. 8). Recommender systems face a common challenge between achieving maximum precision and recall as shown in Figs. 7 and 8. Our research indicates increasing recommendation quantities enhances the discovery of relevant items yet reduces precisions in the process which underscores a need to strike ideal user experience through enhancing both metrics. Figures 6 and 9 show that our hybrid method delivers optimal performance according to F1-Score measurements due to its balanced precision and recall achievements. The F1-Score reaches equilibrium when the recommendation list exceeds six items which mirrors research findings about how users handle recommendation lists in the field of recommender systems.

• A) Theoretical implications.

Theoretical research about hybrid approaches in recommender systems receives advancement through our study. The framework successfully merges CNNs which extract complex data features (based on Liu et al.¹⁴ and Salau et al.¹⁵) with graph analysis techniques (explored in Ahmad et al.²² and Xu et al.²⁹) for addressing the cold start problem to deliver new recommendations in online education. The combination of different modeling paradigms produces superior recommendation systems than using a single method according to our research results. The research adds to our comprehension about applying deep learning and graph-based approaches to educational data as well as user conduct within online educational domains.

• B) Potential managerial implications.

The findings from our research deliver multiple management opportunities to both online education platform providers and course designers. The proven enhancement of recommendation precision indicates that organizations using this hybrid system will achieve greater student participation thus resulting in better course completion statistics and improved educational results. The system detects behavioral patterns of students thereby enabling data collection which educators can use to develop curricula and improve courses. Educators achieve better understanding of their learning material effectiveness through exploring instances where similar students enroll in particular paths or when multiple subjects tend to be studied together. Graph analysis methods enable the solution of the cold start problem which proves advantageous for platform retention of new students who lack previous interactions on the system. The knowledge about the best recommendation amount will help designers create interfaces that enhance user experience on the platform.

Our research demonstrates convincing experimental evidence which validates the combined CNN and graph analysis framework for making course recommendations on online platforms. These research outcomes generate important theoretical understandings for recommender system research while providing applicable recommendations for improving the management of online education platforms.

Practical implications

The combination of CNNs and graph analysis for online course recommendations provides practical benefits to enhance online education platforms. Experimental data shows our recommendation system delivers better performance in precision and F1-Score than PCRS and DECOR which supports student satisfaction improvement. The precision levels of our approach remained consistently high throughout all tests which demonstrates that students would find the recommended courses relevant to their actual selection choices. The recommendation process of PCRS¹² aims to match learner needs but our solution achieves enhanced accuracy using deep learning applied to feature extraction.

The student behavioral pattern analysis capabilities of our system through graph analysis described in our methodology enable academic advisors to obtain useful insights. Our approach functions like the adaptive e-learning system of Kulkarni et al.¹⁶ by detecting students in early stages of falling behind during learning. Educational and behavioral relationships studied by advisors lead to improved guidance that produces better retention outcomes together with enhanced student learning performance. The system meets an extensive requirement within online education according to Li and Kim¹³ who identify the need for enhanced learning experiences.

The system utilizes its capability to review student choices along with course dependencies to schedule courses effectively which results in better course meeting student expectations and lower dependency conflicts. The system provides useful outcomes that both simplify administrative procedures and create better learning circumstances for students. The frequency analysis of selected courses enables better resource distribution by providing data to support popular programs which is a practical issue that Safarov et al.¹⁹ address in their discussion about resource management challenges.

Phase 2 evaluation of model performance with different recommendation quantities (Figs. 7 and 8, and 9 in Sect. "Phase 2: evaluating model performance for changes in the number of recommendations") provides useful information about the best number of recommendations to present. The number of recommendations directly impacts recall but negatively affects precision. The F1-Score reaches a steady point after six recommendations in our experiments which indicates a practical boundary for additional recommendations to enhance quality since extra suggestions could reduce precision. The understanding of the recommendation window will guide interface design by enabling appropriate option selection which addresses the balance between student needs and excessive irrelevant suggestions with deep learning approaches like DECOR¹³.

The practical results generated by our combination strategy prove to be highly significant according to our findings. Virtual education platforms that integrate deep learning features with graph-based behavioral patterns will deliver better course recommendations to users thus generating increased learning satisfaction together with better academic plans and efficient resource management. The experimental results against methods PCRS¹² and DECOR¹³ demonstrate practical advantages which support the claims made in Sect. "Phase 1: evaluation of model generalizability" and "Phase 2: evaluating model performance for changes in the number of recommendations".

Limitations and future directions

Although, the proposed approach yields encouraging results, there are several limitations which leads to directions for further research. It can be pointed out that one of the first limitations is scalability in the model. Since it is relatively higher in computational complexity when compared to the basic methods, this may face scalability challenges in the case of large datasets. This increase in computational complexity is influential in the execution process and response time of the model, especially in conditions with large data. In order to improve the performance of the model and solve this limitation, the use of optimal processing techniques and the use of hardware accelerators can be effective. These techniques can reduce the computation time and increase the efficiency of the model, and thus help to improve scalability.

Another limitation is related to the interpretability of the model. Although our proposed method using deep learning techniques has been effective in improving the performance of the model, one of the fundamental problems of these models is the difficulty in interpreting the connections and decisions that are made based on them. Unlike methods like APRIORI that explicitly state all the rules and relationships used to make recommendations, deep learning models tend to be perceived as “black boxes.” This can be considered a disadvantage compared to traditional models, even if their performance is weaker. To mitigate this problem and increase the interpretability of the model in future work, we can use strategies such as Explainable AI (XAI) methods or techniques such as Attention Mechanisms. These methods can allow us to make the model’s decision-making process more transparent and understandable, thereby improving the model’s ability to interpret data. These actions can help us not only improve the model’s performance, but also achieve greater user trust in the results.

Additional real-time data sources, including logs of student’s activity and forum discussions, can also be used to improve the quality of the recommendations. Taking into account things like current load that a student has or the available amount of time the student has to spend on the material, or the speed at which a student browses through the material, can make recommendations more locally specific. Implementing ideas of collaborative filtering can use the collective intelligence to enhance the recommendation quality. Generating methods to enhance the understanding of the recommendation logic can help to improve the acceptance of recommendations. In this way, we can extend the practical applicability of the proposed approach and promote the advancement of more effective and individualized online learning.

Conclusion

This paper presented a new framework for improving online course recommendations through a hybrid approach that leverages CNN models and graph analysis. In this process, raw data was extracted from the database and prepared for input into a neural network. The network was able to predict the probability of course selection based on the profile and academic performance by analyzing the features associated with students and courses. Students were also modeled as nodes in a graph and their educational and behavioral relationships were analyzed. Using graph analysis techniques, behavioral patterns were identified and new relationships were predicted, which helped solve challenges such as the cold start problem. The model ultimately succeeded in providing accurate and personalized recommendations to students, which increased their satisfaction and possibly improved learning outcomes. The results of simulation experiments showed that the model achieved a precision of 0.8336 and an F1-Score of 0.3347 compared to other similar methods, and these findings confirmed the effectiveness of the hybrid approach in providing accurate and relevant recommendations.