CompSci teacher sets lab task: Accidentally breaking the university
- Reference: 1738569491
- News link: https://www.theregister.co.uk/2025/02/03/who_me/
- Source link:
This week, meet a reader we’ll Regomize as “Don” who is now a college professor, but early in his teaching career found himself running a networking course in a dedicated lab that had a Windows PC for every student. Don had his students run Linux instead.
“The lab had its own switch in an open rackmount at the back of the room. That connected to a router which served the building floor that we were on,” Don told Who, Me?
[1]
One of Don’s lessons aimed to educate about Dynamic Host Configuration Protocol (DHCP), the protocol that automatically assigns IP addresses to devices on a network. DHCP works its magic with a server that detects client devices and hooks them up.
[2]
[3]
Don demonstrated DHCP by having all his students disconnect from the campus network. He then fired up a DHCP server in the lab and assigned it a 192.168/16 subnet – that’s the range of IPv4 addresses that aren’t routable on the open internet. Choosing that address range should have meant that his DHCP server didn’t bother anyone outside the lab.
About 15 minutes into the exercise, a member of the university’s tech team purposefully strode into the lab, found to the switch, and “energetically disconnected its backbone link.”
[4]
The university techie then glared at Don, who recognized malice and frustration in the look, before exiting the lab.
[5]Tired techie botched preventative maintenance he soon learned wasn't needed
[6]Developers feared large chaps carrying baseball bats could come to kneecap their ... test account?
[7]Life lesson: Don't delete millions of accounts on the same day you go to the dentist
[8]Brackets go there ? Oops. That’s not where I used them and now things are broken
Don was flustered for a moment but then remembered the university policy of turning off computers when they weren’t being used.
That mean PCs around the building were often turned on, and when they powered up they’d go looking for a DHCP server.
Don had inadvertently made his server the machine most likely to respond first. But as it was built to keep traffic in the lab, folks in the rest of the university who used it could not reach the internet as they desired.
Unbeknown to Don, his DHCP server had caused a torrent of calls to the campus helpdesk. Which was why they tech staffer had stared daggers in his direction.
[9]
Don doesn’t know how the uni tech team figured out he was the culprit within minutes of his server booting up, but he’s impressed they did.
And the next time he taught this exercise, he did it with virtual machines on student PCs.
“That we got the machines of the day to run four or more VMs at the same time is rather surprising,” he mused. “Maybe it was a testament to Linux's lighter memory footprint?”
Did you make a mess when studying tech? [10]Click here to send us an email about your experience. Your academic records are safe with us – we’ll find a way to give your story a high grade! ®
Get our [11]Tech Resources
[1] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/networks&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2Z6Ch2grroCZoV3csRxceDwAAAII&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/networks&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Z6Ch2grroCZoV3csRxceDwAAAII&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/networks&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33Z6Ch2grroCZoV3csRxceDwAAAII&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/networks&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Z6Ch2grroCZoV3csRxceDwAAAII&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[5] https://www.theregister.com/2025/01/27/who_me/
[6] https://www.theregister.com/2025/01/20/who_me/
[7] https://www.theregister.com/2025/01/13/who_me/
[8] https://www.theregister.com/2025/01/06/who_me/
[9] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/networks&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33Z6Ch2grroCZoV3csRxceDwAAAII&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[10] mailto:whome@theregister.com
[11] https://whitepapers.theregister.com/
Re: walked to the lab to disconnet the switch?
> why didn't the techies just remotely disconnect the uplink port from the router?
Maybe every previous year they did that. But this time there was a different techie (a techy techie?) who had had enough and finally wanted to make a point
Re: walked to the lab to disconnet the switch?
Maybe the switch was old enough -- TFA didn't give any dates -- that it didn't have the necessary features to practically allow remote administration.
Back in the late 1990s I saw at my workplace removed-from-service 3Com switches whose "remote" admin connection was an RS-232C port on the back. The cable-length limits of RS-232C wouldn't allow a remote admin box at the other end of our building, let alone across campus!
I crashed one of two graphics domains
When a mere spotty undergraduate in 1991, I was working on my final year project. The official title of the project was "The parallel computation of iterative fractals", but what it really was was "calculate the Mandelbrot set on Transputers". The Transputers were on a Meiko Computing Surface (MCS), and was the most powerful system the university owned.
The MCS sat in a little room next to the main machine room, and its processors were split into several domains of various processor counts, and two of them contained a graphics board. We didn't always have direct access to it (researchers sometimes wanted to use it) so we had to use text or X terminals dotted around campus. These didn't have the means to output graphics from the MCS so I implemented a text output option which still ran on the graphical domain for the odd occasion I wanted to test.
We were generally displaying graphical results in a 1280x1024 resolution, but this wasn't suitable for text so I normally lowered it to something like 50x50 when at a terminal. Note the word "normally" as this is important. One time, whilst sat at a DEC Station, I connected to the domain, wired up the processors, loaded my code, and hit "RUN". I should have taken the other meaning of "RUN" into consideration as a survival technique. I'd forgot to lower the resolution and was trying to output quite a lot of data to a terminal session which, oddly enough, did not have a terminal capable of displaying 1280 characters on a line. This also wasn't a gigabit network so it was going to take some time. I experienced a classic "Oh F***" second, hastily did the CTRL-C so beloved of failures, and hoped for the best.
The program running didn't quit, but the link to the MCS did. I tried to re-connect to the domain but it thought it already had an active connection (which it did - mine - but was now dead). Unfortunately, the process on the graphics domain carried on churning out text data to a session no longer connected, and then sat there waiting to find out what I wanted to do. A quick chat with one of the MCS admins confirmed my fears - the domain was locked to my session and could not be stopped without rebooting the whole MCS. A reboot wasn't scheduled for a couple of days. I'd locked one of two graphics domains on the most powerful system the university owned through a simple absence of a comment marker.
I got on with my project diary and kept my head down for a week.
Re: I crashed one of two graphics domains
Inspired by James Gleick's 'Chaos', I wrote a little programme to draw the classic Mandelbrot Set. It ran on my home, pre-Windows, Amstrad. I had to let it run overnight to get a grainy picture plotted on the screen which I was then reluctant to stop and lose the image. Wanting a better, more detailed picture, I knew our work computers had more memory......
Our server had a big (40 Mb!!) HDD which we used simply as file storage. In those days it was heavily under-utilised. I set the program to send its output to be stored there. Everything else on the system ground to a halt which although not uncommon, was unexpected and only I knew the probable culprit.....
It was good friend Peter Norton who brought his Toolbox to resolve the chaos I caused. How we laughed......
Re: I crashed one of two graphics domains
Reminds me of the tiny little postscript file that could keep a laser-jet printer occupied for hours, and then output one beautiful, A4, 300 DPI, colour Mandelbrot image. This made use of the fact that postscript is Turing complete, so of course you can write a Mandelbrot program in it. And sure enough, someone did. To the delight of some, and chagrin of other, I might add. In part this was motivated by the fact that the printer had more memory and a faster processor (RISC, as I recall) than even a power PC back in the day. In part it was motivated by the "let's try it" attitude that has lead to many great things, many more wasted hours, and occasionally assorted disasters.
There is also a postscript ray tracer, which has a similar effect.
Been there, done that
In a previous life a colleague and I were doing some scheduled maintenance in our main server room AKA tidying up the cabling mess that had manifested itself over the years.
The network featured many VLANs including at least 2 for voice.
Our main PBX servers were in this room and plugged into ports which had the correct voice VLAN configured. At the end of the maintenance we decided on a whim ("while we're here") to move the PBX servers to different ports in the rack because it resulted in a shorter, neater cable path. I configured the ports by untagging the appropriate voice VLAN, we moved the connections and went on our way.
Not long after, we started getting calls that PCs were not working properly. It took us some time to work out what had happened, but we eventually discovered:
The old ports were correctly configured with the untagged VLAN and nothing else.
The new ports had the correct untagged VLAN but all the other VLANs were tagged - (in my defence this was the default for a switch interconnect port)
Our PBX ran its own DHCP server
The PBX DCHP server didn't understand VLAN tags and was responding to ALL DHCP requests, not just the handsets.
The consequence was that some (but not all) DHCP requests on a tagged network was being responded to by the PBX, breaking connectivity to the client that received it.
I think we managed to convince management it was something else but it took way longer than it should have to work out what was going on...
Been there
> Don doesn’t know how the uni tech team figured out he was the culprit
Maybe the university's network bod was a graduate of "Don's" class and recalled this particular exercise.
I was asked to set up some Citrix testing many, many moons ago on a test network. I was a little surprised when someone ran up to my desk and told me I'd taken down a large part of the Production Citrix estate. Turns out the network I was told was isolated really wasn't. Luckily for me I was able to point to an email where I'd been given the work, confirmed the network was (allegedly) isolated and would not cause issues.
Fun times all round that day!
dhcping
Given demonstrator Don is now a Professor this could have happen 30 years ago or longer if he meant Unix rather than Linux. In any case campus networks pre 2000 were pretty rough and ready. Pay way too low to attract or retain networking experience or talent - it was the .com boom so just knowing the difference between 10Base2 and 10BaseT made you an expert.
This disaster movie was pretty common once consumer wifi routers were petty cash purchases.
Students would bring in their old routers to connect their phone or notebook to the local LAN for internet access or figured, logically I suppose, that the RJ45 LAN ports were as good as switch as any for offices chronically under provisioned with Ethernet ports.
These renegade DHCP servers were a nuisance interfering with departmental servers but given those two redundant servers were a lot faster than the consumer grade rubbish, clients normally got or renewed a legitimate lease (every 6 hours.) The departmental LAN was strictly firewalled from the campus network so little risk of a visitation from the glaring tech.
Usually running dhcping to flush out these blighters and arpwatch to detect new devices on the network. Being able to track the MAC address to the switch port and thence to the physical location lent one an air of omniscience which encouraged good behaviour.
Needless to say when it came to connecting unauthorised equipment to the network the staff were the worst offenders having lost their innate fear of heights and open windows.
The last rogue DHCP server I encountered a few years ago was an i-phone bridged through a desktop onto the local wired network. I never knew the details as it was quickly located by others, using the MAC address I provided, who never revealed the who, what and where.
walked to the lab to disconnet the switch?
why didn't the techies just remotely disconnect the uplink port from the router?
Surely the switches management interfaces where on a different vlan?
Surely the university used ip helpers on the vlans configs?
Surely the techies didn't use dhcp for the switch IP's
surely the techies could have setup a static IP on their machines if they also fell foul of the malicious dhcp server.
i guess these where in the times when these things where not well understood or implemented.
Still impressive the techies knew exactly what room the malicious system was in and could go directly to it.
Maybe the disconnect was a show of annoyance after they rectified the issue remotely.