Tutor intervention during early stages of the course results in higher pass ratesTertiary Education Industry
They wanted to make the most of their tutor’s time by helping the students to whom it matters the most. FOYI was involved in solving the problem of identifying the students who have the most likelihood to pass if the tutor would help them during the course work.
Challenge
The tertiary education customer proposed that a preliminary analysis be conducted to understand the nature of the problem and the scope of the project. As a result of this initial analysis, the following challenges were uncovered.
1. The course structure and grading changed over the years.
The nature of scoring and the EFTS have changed for more than 10% of the courses over the years. This meant that historical data cannot be used as-is for training machine learning models.
2. Some key attributes of the students were not permitted to be used on account of the privacy policy.
From the existing reports, it was well known that some student attributes such as pass rate for previous courses are good indicators of current course pass rates. However, such information was not permitted to be used in this project.
3. The board of the institution wanted to have the knowledge of this model retained within their team.
The team had a couple of recent graduate hires for data science and they needed to be taken along the journey of this project i.e. on the job training.
Approach
Given the nature of the challenges, FOYI was able to call out at the very beginning that the project duration would run into a few months and not just a few weeks. In order to ensure that the project was within the budget, FOYI proposed a more advisory approach to this project. This meant that FOYI would provide the detailed set of steps to be taken at each stage of the project and then facilitate 2 day workshops with their data science teams to analyse the outcome and propose next steps.
The documentation of the proposed approach would be shared by FOYI while the heavy lifting in terms of the deployment and the final as-built documentation will be handled by the internal data scientists.
Outcome
The 3 key deliverables of this project and their benefits are as follows.
1. A heuristic engagement model.
Benefit: A simple and logic based formulation was crafted to ensure there is an explainable score right from the first week of the course. This helped the tutors to engage via email with the students having a low score.
2. A probabilistic pass likelihood model.
Benefit: A predictive model based on historical data of comparable courses and predicting the pass probability each week. This helped the tutors to call the students for following up on assignments on a weekly basis.
3. Academic style documentation with literature reviews and empirical methodology.
Benefit: This documentation that followed the academic rigor helped the data team showcase their scientific approach to the executive leadership and sustain their buy-in during the course of the project.