In just a few short years, generative AI has gone from curiosity to imperative in higher education. Today, it’s reshaping everything from curriculum design to campus IT strategy and accelerating the need for robust AI infrastructure in higher education. Institutions that don’t modernize their digital foundations risk falling behind. Once seen as a disruptive force to manage, AI is now recognized as a transformative tool to embrace. The University of South Florida, for example, is launching the Bellini College of Artificial Intelligence, Cybersecurity, and Computing, a move that shows AI’s enduring place in academia, provided institutions prioritize building the digital backbone to support it.
A Broader Shift in Attitude
The evolution in higher education towards AI integration mirrors a broader trend across SLED organizations. As Scottie Ray, Head of Solutions Engineering, Public Sector at Cloudflare, noted in a recent talk: “We’re already at 75 percent of global knowledge workers that are using [generative AI] in some context, and three-fourths of all developers are using this in some sort of code.”
But preparing students for the AI-driven workforce is only part of the equation. Institutions must also reimagine their digital foundations. “You have to get students to understand AI and hopefully embrace it and leverage it,” said Dan Kent, Field CTO at Cloudflare. “How students use AI will be quite different depending on the field that they’re in, but I think we have to move faster than we’ve ever moved before in terms of any workforce in its need to be aware, trained, and possibly building new AI tools so that students are better prepared when they graduate.”
The stakes are high. Colleges and universities must prove their relevance to demonstrate that the time and money invested in a degree is worth it in an AI-transformed economy. As the value proposition of traditional educations faces scrutiny, institutions that fail to act risk falling behind and even becoming obsolete.
From Training to Inference: A New Technical Reality
Traditionally, AI development focused on training large language models, a resource-intensive but predictable process. But the real architectural challenge began with the rise of inference, or real-time AI execution. “When we think about training and the impact to architecture, it’s predictable,” Ray explained. “Inference sort of changed the game. Everything gets really spiky, everything gets really lumpy, and it gets really unpredictable.” This shift matters for higher education because inference is where AI becomes embedded in daily campus life from tutoring bots to advising systems. Once a model is live, it must serve users in real time.
The Need for Responsive Infrastructure
Unpredictability demands agility. “You need a modern compute substrate,” Ray said. This is especially true in higher ed, where resource constraints and seasonal traffic spikes – think registration, admissions, and advising – are common.
Ray highlighted modern compute environments that scales automatically to meet demand without the overhead or complexity of traditional infrastructure. “They scale up without any DevOps tuning, and they scale to zero easily,” he said. This means universities can rapidly deploy AI services when needed and shut them down just as quickly, combining performance reliability with cost efficiency.
Why Edge Architecture Matters Now
The physical location of compute also matters more than ever. Universities aren’t just running models, they’re coordinating data across cloud environments, APIs, internal systems, and increasingly, open-source AI tools. Performance and latency are mission-critical, especially for real-time applications.
“This is also the orchestration engine,” Ray explained, referring to the edge, “where the part of the app that interacts with other LLMs, maybe a constellation of LLMs or other APIs… has to be close to those sources as well.” A programmable global edge enables AI tools to run near users while maintaining integration with backend systems, essential for remote learners and globally distributed research teams.
AI as Infrastructure, Not Just an App
As AI matures, universities need to stop treating it as a one-off application. Generative AI and agentic AI will fundamentally reshape the student journey, the workforce, and the value proposition of higher education. Instead, they must integrate AI into everything from learning management systems to cybersecurity and compliance frameworks.
Imagine an AI-powered academic advisor that helps students select the right major and courses based on their strengths, interests, and future job prospects. Or an intelligent career coach that connects graduating students with real-time labor market data and hiring managers, aligning their skill set with emerging opportunities. Or a virtual financial aid assistant that helps students navigate complex loan, grant, and scholarship options, with contextual guidance, 24/7.
To make this a reality, institutions must start treating AI as a foundational layer of their digital infrastructure, as opposed to siloed applications.
“We need to stop thinking about AI agents, LLMs, these types of things as just an app,” Ray argued. “We need to think about it as a technology primitive… and tie those into a holistic approach to solving the problem that we want to solve.” This perspective shift is key to futureproofing campus IT. Not only to support AI, but to empower faculty and students with intelligent systems that evolve alongside them.
The Real Opportunity for Higher Ed
The stakes go beyond efficiency or IT modernization; AI presents a chance to reimagine how institutions serve students, empower faculty, and accelerate research. But that promise depends on whether the infrastructure can keep up.
“The network is the computer,” Ray said, echoing John Gage’s famous words. “And in the context of cloud-based services, it’s that platform that’s huge.” By investing in flexible, distributed, AI-ready infrastructure, colleges and universities can move beyond experimentation toward real-world applications that enhance learning, automate tasks, and personalize experiences at scale.
In a moment when higher ed must do more with less, AI-ready infrastructure becomes a competitive differentiator as well as a strategic imperative for earning trust, proving value, and staying relevant in the years ahead.