This article was first published in University World News.
Like most sectors, higher education has experienced a considerable degree of AI hype over the past couple of years. But despite the bold promises, we now appear to be somewhere within what the Gartner hype cycle refers to as the “trough of disillusionment”, as many higher education institutions fail to realise as much value from AI as they had initially hoped.
AI applications within the student recruitment journey face particular challenges, making it difficult for institutions to understand what’s credible. As a sector, higher education is unique. Journeys to enrolment can look drastically different and are not transactional, so machine learning algorithms are harder to build.
Some of the misunderstanding lies in the misapplication of the term ‘AI’. We see this in the actual branding of services, with a lot of automation and workflow tools incorrectly badged as AI. More importantly though, we see this misunderstanding manifest in institutions applying AI in the wrong places – in student-facing tools that are expected to revolutionise the student experience but inevitably fall short.
The problem is that it’s nearly impossible for individual universities, on their own, to achieve a data foundation that is reliable enough to enable the development of AI tools.
Universities can more meaningfully look to apply AI on repetitive and high-volume administrative tasks, in turn affording staff more time to focus on valuable, positive human experiences with prospective students.
Accuracy of content
Time and again, we see institutions drawn towards investing in AI to deliver student-facing chatbots and personal assistants, or to help write email templates and other outbound marketing campaigns.
Imagine the scenario a director of student recruitment might be facing, with budget pressures, a need to deliver international student numbers, and possibly with other senior voices in the institution talking about a desire to implement AI. Because they are not technologists, they will look towards the solutions they can understand. On the surface, chatbots might seem logical – a solution to high website traffic and volumes of enquiries.
But this introduces one of the key issues around training machine learning models: accuracy of content. Chatbots are typically trained by scraping a university’s website, so they will only be as good as the content that already exists. We know that universities find it notoriously difficult to keep websites up to date. That means the end result might be a chatbot that can only answer about 10% of student queries.
This is not just a higher education problem. The difficulty of getting customer-facing AI applications right is a universal issue that even huge global brands struggle with. The failed ‘Just Walk Out’ technology in Amazon Fresh stores springs to mind!
Back-end processes
Focusing on back-end applications means that institutions can redirect staff to higher value student-facing activities, and don’t need to increase their professional services headcount to keep up with demand.
A good example is applying AI in quality assurance practices. Machine learning models exist that are able to assess phone calls with students and provide feedback and student sentiment analysis to staff. This might not sound earth-shattering, but it creates huge efficiencies, can be delivered at scale and it provides in-cycle student decision-making insights faster than would otherwise be possible.
Another hugely exciting emerging area for AI in back-end processes lies in its ability to make predictions and help universities optimise their activity. Which students are most likely to enrol? Where should universities prioritise resources? Which interventions would increase conversions?
While this type of machine learning propensity modelling is starting to be introduced to the market, it is incredibly difficult to deliver. And it brings us to the second fundamental issue with using higher education data to train intelligent models: consistency.
Higher education data sets are subject to fluctuations that make the year-on-year comparisons required to build machine learning models difficult. Changes to immigration policies, internal decisions to speed up or slow down application processing for certain markets and changes to deadlines or courses all affect this.
Without the existence of consistent data sets, propensity modelling is impossible to deliver at scale. For it to be credible, it needs to be based on aggregated data from a much bigger pool of institutions that is able to standardise outside of normal operating data, with the machine learning element overlaid on top of that.
This breadth and depth of student journey and decision-making data is not available at an individual institution level, and very few organisations out there can claim to own this type of data set either. Without this, it makes most claims of ‘we’re going to be the next big AI company in student recruitment’ frankly uncredible.
Questions to ask
To try and understand the value of what they’re buying into, universities might want to consider having a standard set of questions to ask providers.
Of course, this depends on what the proposed AI solution is, but broadly it should cover: what data is the model trained on? How do they quality assure for accuracy? Is the data set proprietary? What are their data responsibilities and where is the data being stored? And – last but not least – what is the actual machine learning element or is this just automation?
The current economic backdrop for universities means that it’s critical they get maximum value out of any technology investment.
We know that from an applicant perspective, there is an increased demand for that all-important human connection. Student-facing AI applications are simply not good enough to get anywhere close to being able to deliver on that yet, so institutions should instead focus on the behind-the-scenes applications that might be less ‘showy’ but are infinitely more credible. In doing so, perhaps we can gradually move away from the ‘trough of disillusionment’ towards the ‘slope of enlightenment’.
Rachel Fletcher is CEO and co-founder of UniQuest, experts in engaging, enrolling and retaining students for universities.
