Unavailable for most of day 1 and 2, UVic says Wi-Fi is fixed
“The Help Desk is receiving reports of intermittent connectivity for UVic wireless network. System administrators are aware of the issue and are working to fix the issue.”
Sent from UVic’s computer help desk at 10:44 a.m., September 8, this brief two sentence description of the wide-ranging Wi-Fi problem doesn’t exactly encompass the severity of UVic’s paralyzed network.
As students logged into UVic Wi-Fi from their phones and computers for the first day of school, Wi-Fi access points began failing across campus. A network bug ended up taking down entire chunks of the network for two days, disrupting wi-fi access for thousands of students, instructors, and staff.
Too many devices were accessing the network from the same areas. The bug was a result of too many IP addresses, or connections, accessing the network at the same time. IP addresses are unique numbers given to each device connected to a WiFi network.
Users were trying to connect to access points. These access points exist within the UVic Wi-Fi system and are spaced out geographically around campus. Each group of access points is known as a DHCP pool. When one of those pools can’t handle the number of IP addresses trying to connect to it, the system is vulnerable to collapse.
This is what happened to UVic’s Wi-Fi. Too many connections were made at different access points, causing a shut down in the system.
First-year engineering student Sam Morrow recalls seeing his professor struggle to introduce Brightspace in an in-person class of roughly three hundred on the clogged network on his first day of classes.
“I’m not surprised at all. It’s the university dealing with more students that they’ve had in more than two years,” said Morrow. “I don’t think anyone else was surprised.”
For students who were able to connect to the internet and access UVic’s website, a bright yellow banner at the very top announced the outage.
Speaking with the Martlet, Morrow paints a vivid scene.
“You’ve just arrived in your first-ever class,” said Morrow. “You’re scrambling to open your laptop to take notes and you can’t access the syllabus for the course.”
Morrow says he is reliant on the network to access note-taking software and that all his classes were affected on the first day.
“It was certainly a little stressful for the first little bit [and] as an introduction to my first-ever classes,” he said.
Behind the scenes, UVic technicians were working long hours trying to get the network up and running. They were well aware of the anxiety and frustration the crash has caused students and professors.
Ron Kozsan, UVic’s director of infrastructure services, told the Martlet that the server crash happened due to increased connectivity exacerbating a network bug.
According to Kozsan, UVic’s Wi-Fi system runs off over 1500 access points, each divided into groups known as DHCP pools. A bug in the system caused DHCP pools to go offline once the number of available connections in each pool had been exceeded.
“The bug was, once you exceeded [the limit], people couldn’t log on anymore,” he said. “That’s kind of what was biting people last week.”
Kozsan said that UVic’s network services technicians worked with the network vendor Cisco to find a work-around. He says he is hopeful they have found a solution.
“Often in our world, we’re doing things like trying to figure out what combination of tricks you can do to get through this,” said Kozsan. “So the workaround was a whole bunch more groups with a lot fewer of the access points in each group.”
What this means is that the number of DHCP pools on campus has been expanded and each pool now hosts fewer IPs. Kozsan says that this change will make it easier for the system to handle the number of people on campus.
In 2020, UVic undertook a “complete overhaul” of their network. Kozsan says that while the lack of activity was great because it allowed technicians to switch out the hardware unimpeded.
But this also prevented them from catching any load-related issues.
“One of the funny things about a pandemic and with nobody being on campus, the guys could just fly at it and upgrade everything without disrupting anyone,” he said. “Because there were not very many people around, we had to go about a year before we knew if there were going to be any load related issues, and guess what, that was [what happened in the first week].”
According to Kozsan, UVic isn’t the only school to have encountered these problems. Several schools in the U.S. had load related problems last week.
He says that network services have also worked this summer to expand access to Wi-Fi in certain spots across campus. This includes tables set up around campus, and the front of the library.
With COVID-19 distancing measures in place, he says it was important to expand service outside the main class buildings.
“Prior to this summer, if you were outside you were lucky if you got Wi-Fi,” said Kozsan.
For now, UVic’s network is holding up. Kozsan hopes that students will have access to stable WiFi on campus while Cisco finds a permanent software fix.
