Hmmmmm, how to cool that overheating CPU, if only there was a solution...
- Reference: 1624865407
- News link: https://www.theregister.co.uk/2021/06/28/who_me/
- Source link:
Regomised as "Peter", our hero was working in the IT department of what he described as a "medium-sized city" somewhere on the West Coast of the US. It was the early 1980s, and computing duties were being performed by a Univac mainframe.
It was a high-end (for the Series 90) 90/80; "Sperry Univac's answer to the IBM 370," Peter told us. This example featured a water-cooled central processor and was designed to shut down if things got too hot.
[2]
"I was startled," he told us, "when I got off the bus on my way to work one day to see a wide stream of water sluicing along a downtown street.
[3]
[4]
"It was coming from opened fire hydrants. I found that my office building was at the top of the stream. Inside, it sounded like a fountain; all of the taps in all of the restrooms had been opened. Fellow programmers (as coders used to be called) warned me "Don't drink the water.'"
The problem was the mainframe. It had been getting rather hot and bothered and the engineers had hit on a seemingly simple solution. More cooling needed? Add more water, of course!
[5]
A hose, having been liberated from a garden, had been run from a handy faucet to the radiator and attached. The valves were opened and the engineers prepared to congratulate themselves on their genius.
Except things did not go quite as expected. The pressure in the radiator was rather high. High enough, in fact, that the mains water, under a lower pressure, did not trouble the cooling system. Likely a lucky escape, because who knows what untreated water would have done to the internals.
Not so lucky, however, were those downstream since the chemicals in the coolant began to go places where they shouldn't as the flow went the wrong way. Don't drink the water indeed.
[6]
Ill-thought-out solution swiftly cut-off, the way to resolve things was to turn on taps and hydrants in an effort to flush out the contaminants.
The cock-up made the news. Peter recalled: "The evening paper featured a picture of flooded gutters with the headline 'Main Street.'"
But at least they were liquid cooled, eh?
[7]Updating in production, like a boss
[8]Do you come from a land Down Under? Where diesel's low and techies blunder
[9]Thanks, boss. The accidental creation of a lights-out data centre – what a fun surprise
[10]Congestion or a Christmas cock-up? A Register reader throws himself under the bus
[11]Big red buttons and very bad language: A primer for life in the IT world
The Register asked a number of people in the pumping and chemical industry if such a thing was possible and the answer, after lots of complicated discussions about "reverse flow" and "composition profile", was "story sounds viable." So, there you have it.
That said, the incompetence required to think that attaching a garden hose to a mainframe cooling system was a good idea is a thing to behold.
We like world-beating incompetence in the files of Who, Me? and On Call. So email your story to [12]Who, Me? in exchange for a spin on the Regomiser and confess all. ®
Get our [13]Tech Resources
[1] https://www.theregister.com/Tag/who-me
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2YNmdwsmjuwBQ1sd0daYv8AAAAMo&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YNmdwsmjuwBQ1sd0daYv8AAAAMo&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YNmdwsmjuwBQ1sd0daYv8AAAAMo&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YNmdwsmjuwBQ1sd0daYv8AAAAMo&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[6] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YNmdwsmjuwBQ1sd0daYv8AAAAMo&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[7] https://www.theregister.com/2021/06/21/who_me/
[8] https://www.theregister.com/2021/06/14/who_me/
[9] https://www.theregister.com/2021/06/07/who_me/
[10] https://www.theregister.com/2021/05/31/who_me/
[11] https://www.theregister.com/2021/05/24/who_me/
[12] mailto:whome@theregister.com
[13] https://whitepapers.theregister.com/
Re: They wanted to play W:ET!
Sounds like the Mainframe was hosed...
Should have used the sprinkler
Spraying cold water over the outside of the radiator would have worked!
Anybody who knows anything about plumbing ...
... knows you don't plumb two water sources together. Ever. For lots of reasons.
A cousin of mine decided it would be nice to build a manifold so he could close a valve or two, open a couple others, and thus easily prime either of the two jet-pumps on his irrigation systems ... with the city water supply as the priming water.
Several thousand dollars in fines, and quite the tongue lashing from the raised-rural Judge later, and now he understands why most of us have a pickle/olive barrel full of water handy to the pump for priming purposes. Once the pump is running, top up the barrel for next time.
Re: Anybody who knows anything about plumbing ...
"Anybody who knows anything about plumbing..."
That sentence is where your entire comment falls down. Plenty of sensible people know nothing useful about plumbing - myself included. Actually I find plumbing to be one of those topics that seems simple on the surface but has many ways to come back and bite you in the ass!!
And those sensible people will sensibly ask a plumber how to go about doing something concerning plumbing, otherwise they're not so sensible.
So I fail to see how Jake's comment falls down.
I swear I've heard this story before with a flooded parking garage as a bonus.
There was talk around the office that one of our customers would be returning their Cray-2 instead of buying it, which meant we'd have to test it to resell for the most money.
Ron was against testing. He didn't trust that fancy chlorofluronated crap Cray used, and he didn't like 'water' cooling in general.
"Last time I worked on something like this I had to replace my Mustang!"
Ron claimed to have worked at a company with a Univac, and the thing kept getting hot. One of the 'engineers', who had worked on nuclear subs in the past, thought it was air bubbles in the cooling system. "Starvation caused by insufficient reserve volume", he said.
Mister Bull Nuke decided they would 'burp' the cooling system by attaching a couple of valves and hoses. One to vent the air and steam from the hot side into a city drain in the garage, another to add water on the cold side from a tap.
Ron continued on. "There isn't any real reserve volume in a Sperry-Univac, and it wasn't starvation, it was cavitation."
By the time the 'engineer' gave up, they had eight inches of water in the small underground garage and his Mustang was a total loss.
A total loss from sitting in a mere 8 inches of water?
Story sounds all wet to me. I'm fairly certain I've driven Mustangs of most vintages through more than 8 inches of water before, with nary a problem.
8 inches of water barely gets to cover half the height of the wheels. I don't see how a car of any kind can be totalled by that.
You might need to replace the wheels, depending on the corrosiveness of the liquid, but worse than that I can't think of.
Oh, oh, D'oh
A little knowledge is a dangerous thing!
Who thought water and computers mixed?
Not a "who me?" but it reminds me of the time (also back in the early 1980's) when the firm I was working for had their own IBM mainframe to control company stock, manufacturing and finance. It sat in a separate building, in its own room with its own water cooling. Back up, BTW, was a duplicate sitting across the pond, with a nightly satellite link. Two design flaws in the building:
1) One of the above ground pipes was plastic, and
2) The floor drain wasn't at the lowest point.
Needless to say, one day maintenance needed a bit of height to fix something (probably something as innocuous as changing a light-bulb) and a water pipe provided a convenient step up. Being plastic. it sheared off and water sprayed into the room. Fortunately, the spray didn't target any vents but the bunded floor started to resemble a paddling pool. Panic ensued as the computer was hastily shut down. I wasn't working in that department but was one of the folk who had to start placating the several hundred other people who now found they had no access to the data they needed to input or withdraw stock, run any of the four manufacturing shops or any of the myriad other back office jobs.
Fortunately, the computer was unharmed and, following some urgent baling and re-plumbing, normality resumed by the following day.
Not IT, but involving many tonnes of copper and heat; I have seen organisations in the USA calling in local fire response services to point their pumps at substation transformers to increase their cooling capabilities over and above what the traditional oil radiator-and-fan approach can do.
It is, from the standpoint of cooling highly effective. One has to question whether asset management or planning practises are fit for purpose to "survive" on one transformer rather than installing two rated to take the load in question.
For the record, in the UK, it is can be a criminal act to operate HV equipment beyond it's rated capability.
A/C because not hard to identify what organisation I must be in to be able to write this.
Water main street
Should the headline not have been 'Water main street'?
The first/only water cooled ICT 1301?
It wasn't designed that way.
I was working as a computer operator during my sixth form hols at a well known engineering company in the Black Country. They were a go-ahead organistion housing the kit and a resident ICT engineer in a state-of-the-art glass fronted office block facing south. Is the penny starting to drop?
Anyway it being the swinging 60s the company had purchased a spanking new ICT 1904. It magically had
So - the false ceiling was torn down to install the AC revealing an extensive network of water pipes. Cold water pipes as it turned out. On a blistering August summer's day the un-airconditioned air produced an absolute deluge as it condensed on the pipes right on to the top of the 1301.
The ICT engineers horror - not to mention the DPM as meltdown looked inevitable. But equally magically a dozen or so fire buckets were rapidly emptied and arranged along the top of the 1301. But buckets are round so complete coverage was impossible and the waterfalls were fickily moving along the exposed pipes necessitating constant re-configuration of the the computer's new bucket collection. But it worked the 1301 never missed a beat. But then it wasn't running GEORGE.
Perhaps that was the hardest day's work I have ever done - except, possibly the night before payroll day when one of the punch card machines supplying the data went unaligned and the 1301 reader rejected and halted after every fifth card. This had to be re-punched by one hand while holding up the errant card to spot the holes with the other before the payroll program would continue. The prospect of the Friday pay packet being empty was not an option for a company that didn't want its management lynched.
They wanted to play W:ET!
They just ended up flooding their infrastructure when the servers retaliated with a buffer overflow.