There ain't no problem that can't be solved with the help of American horsepower – even yanking on a coax cable
- Reference: 1601622908
- News link: https://www.theregister.co.uk/2020/10/02/on_call/
- Source link:
Today's story takes us back to the mid-1970s, "the Triassic age of computers," as our reader "Chuck" put it.
"The mainframe ruled," he recalled, "and the Rex of the... species was IBM. The punch card was still prevalent but beginning to evolve into input by 8" floppy and the growing number of data-entry terminals."
Those with long memories might remember the IBM 3270 series of terminals, the descendants of which could be found frustrating users until the latter days of the last century. Chuck remembered the 3278 being the most prolific of the breed: "This was a text-only 'green screen' that displayed 80 x 24 characters with a reserved line at the very bottom to give a few status indications."
The thing was about the size and weight of an old 17" CRT and popped up in industries around the globe. Thirty-two could be connected to a controller ("usually a 3271 or 3272," said Chuck) which was then connected via a high-speed (ish – this was the 1970s) channel to a mainframe. The terminals themselves were connected via ordinary coax, which allowed the kit to be quite some distance away from the controller ("a thousand feet," reckoned Chuck).
Help! My printer won't print no matter how much I shout at it! [2]READ MORE
Chuck was working in IBM's service branch back in those days and had been well trained in the 3270 system. He had been placed on a three-month stint in a call centre supporting field technicians.
"For younger readers to whom the term 'Call Centre' or 'Help Desk' gives memory of rage-induced breakage of handset or keyboard, in those days they were staffed by people who were trained on the product or software, who spoke your native language, desired to help, and would stay on the line as long as needed, even into the night if the problem was severe. Really," he reminisced.
The call came in from an on-site operative faced with an unusual problem. A new terminal had been set up in a bank that would not "see" its controller, which was some distance away.
"The coax metered OK," said Chuck, but swapping out the terminal with a known good one made no difference, nor did replacing the circuit card in the controller. Perhaps the distance was causing a timing issue, but after the software team took a look, it was decided that it had to be the cable.
Another cable was pulled through and had the same problem, "with the contractor pointing the 'I told you so' finger at the service person."
Troubleshooting continued for the rest of the day, but still success eluded the team.
It was no good. A team for the factory would have to be dispatched, and the expensive trip was about to be made when the field operative ("Customer Engineer" or CE in IBM-speak) called again. The problem had been solved.
In desperation, the CE had walked the 500 feet of underground cable in the hope of finding some giant electromagnetic emitter that could be screwing with the signals in the coax below. He reached the end, "then noticed that the spool of wire that had been ordered to length still had a hundred feet left on it..."
Odd.
A few pointed questions and (we imagine) a shuffling of feet later, and the solution to the problem was found. The cable had been pulled through by what Chuck called a "gang of heavies". A tough job, and the team had eventually hit on the brilliant wheeze...
... of simply tying the pull rope for the coax to a handy pickup truck and letting Ford's finest take the strain.
Sorted.
"Of course," said Chuck, "the cable was now stretched to only half of its original diameter and the electrical characteristics were obviously far beyond anything that the equipment could recognize.
"The service call was terminated, and the problem never heard from again."
Ever had to deal with a "solution" that involved a pick-up truck and a leaden foot? Or done an empathetic 500 feet walk in someone else's shoes during support call? Share all with an email to [3]On Call . ®
Get our [4]Tech Resources
[1] https://www.theregister.com/Tag/on-call
[2] https://www.theregister.com/2020/09/25/on_call/
[3] mailto:oncall@theregister.com
[4] https://whitepapers.theregister.com/
the difference between an engineer and sn installer.
Quite.
One evening, some two decades back, we were upgrading the firmware on the FDDI hubs strewn around the site, a veterinarian pharmaceuticals research/production plant. Topo was a double ring, so one hub being rebooted should not have affected any of the other hubs. Still, it turned out it did as with a number of reboots a couple of the other hubs started complaining about the loss of one ring.
So, two of us grabbed the keys to the buildings and their data cabinets, a pair of walkie-talkies and flashlights, and set off into the night in the general direction of where the problem was. Now a break in a fiber could really be anywhere between hub A and hub B, although it probably wouldn't be somewhere halfway in a multicore run in an underground duct, as it would be very unlikely a disturbance would have taken out just a single strand. So we started with checking the patch panels and the leads to the hubs.
Nothing immediately obvious, so checking the connectors was next. And pulling out one of them the cause of the problem glaringly presented itself as the fiber just dropped out of the connector body.
It turned out there had been a couple of new CAT5 patches installed the day before, and apparently one of the installers had dropped a screwdriver or something on the hapless fiber connector below, somehow breaking the fiber in such a way that it came loose inside of the connector body after which that installer monkey just pushed it back in.
The next day facility services were instructed to put plexiglass covers over the fiber patches in all data cabinets.
Re: the difference between an engineer and sn installer.
"veterinarian"
About 30 years ago I was rewiring my Victorian terraced house. The problem was getting the cables the 10 metres under the ground floor floorboards. The crawl space was too small to get under there. So, I pulled up the floor in one corner, tied some string around the cat's collar and shoved him down the hole. Then went to the far corner where I'd also pulled up some floor and rattled the kitty crunchies. Perfect! He came right through. Unfortunately, when he was set up for doing the next wire, he wasn't hungry anymore and spent the next five hours exploring his new domain before emerging looking like Dick van Dyke out of Mary Poppins...
Never work with children or animals?
That said - I've often wished for a trained weasel or something strong and agile enough to pull a cable through a difficult run!
Penguins are probably only suitable for aquatic cable runs... >>
Re: Never work with children or animals?
I knew someone who had a Polecat that would do it for a whole Mouse. The trick was to send a pull string through with the Polecat, as it one only worked once a day!
Re: Never work with children or animals?
That must have been a rather smelly solution
Re: Never work with children or animals?
A friend uses his ferrets occasionally. Tinned sardines as bait (YMMV).
While we're on this topic, a good tool for lobbing cable over hanging ceilings is a crossbow.
Re: Never work with children or animals?
I recall the story of a clicker trained rat, also with a pull 'cable' (or rather a pull twine).
Re: Never work with children or animals?
I vaguely remember a story, possibly on here about a chap doing that with a crossbow and hitting the fire sprinker?
Was it you, Jake?
Re: the difference between an engineer and sn installer.
Did he have the Martian accent (or whatever that was) to go with the look?
Re: the difference between an engineer and sn installer.
Remember, you don't own the cat, the cat owns you.
Re: the difference between an engineer and sn installer.
The coil of stiff nylon or fibre glass used by electricians is called a mouse in the UK, so I guess a cat is a step up.
A friend of mine in the UK specialises in building extensions, he had s similar problem running a cable quite some distance under a floor.
He solved it by borrowing an RC 4x4 toy from the house owner's son, tied some cord to it and after two or three goes got it to the right spot.
Re: cats [and rats]
I sent my fancy rat to do the same. Knotted her a lovely little string harness and called her through with a bit of chocolate.
Another Pharma Story
So I'm there early 2000's, my names not Steven & its ages ago....
New Computers going into a custom suite & beyond that a glass wall & doors into a lab full of pharmaceutical testing equipment, all enviroment factors tightly controlled & the door if open longer than the time it takes to walk through sounds an alarm.
Then an installer breaks the new pristine unused & by extension expensive isolated area by drilling a hole to run a cable through the glass*. While I am installing PC's & one of the senior Lab guys notices that something suddenly isn't quite right i.e. why does that alarm keep going off!
*I don't recall if he decided to go via the drywall or had the wit to have (Or fetch) a drill bit for going through glass.
Closest I've had to that ....
.... would be a PC that dropped off the network. I finally traced the fault to a Cat 5 cable that had been severed by a workman (Electrical or HVAC, never actually identified them) where they'd decided to be helpful and fix the problem by reconnecting the wires using a 'chocblock'.
For some people BFI is the default option. You can normally identify this sort by the fact that their toolkit comprises of five different size of hammer, and little else.
Re: Closest I've had to that ....
An electrician working on my house chopped through the aerial cable and choc blocked it. My TV stopped getting a picture after that... I ended up replacing the entire cable as it was 40 years old...
Re: Closest I've had to that ....
Almost exactly the same. Having doors blocked off, I explicitly pointed out to the builder that a phone cable ran right by the corner of the doors and to avoid it. He said fine.
Later that day at work, connected back into home, my home connection suddenly becomes incredibly unreliable. I get home and find a screw going right into the cable. Perhaps more surprisingly when I removed the screw it started working again (I'm guessing it was shorting the two lines rather than actually having broken them).
I told the builder and he was furious with his guys as (he said!) he had shown it to them and been very clear!
Re: Closest I've had to that ....
Builders pffft! I left explicit instructions to be alerted should the power need to be shut off in my house. F$CK@rs powered down the whole house, with my two node ESXi powered up. They were fired after that phase of the job...
Re: Closest I've had to that ....
Vermin media installers....... told the idiots that I did'nt know where the power lines were in the walls
1 decides to drill through the wall from front room to cupboard under the stairs...
BANG out go the lights.
Stupid fekker went straight through the power feed to the lighting distribution box... was lucky his lights didnt go out
Re: Closest I've had to that ....
The guys replacing my barge boards, soffits and gutters were most assiduous in working around the television aerial, sadly less so about clouting the satellite dish with their ladder. It looked fine, but was sufficiently out of shape to drop a whole mux. Ah well, the new one is nice and shiny.
They were wonderfully creative swearers, these lads from Armagh, which brightened up more than a few online meetings as I worked from my office while they did they hammering and drilling.
Re: Closest I've had to that ....
Same thing here. Except the muppet who severed the cable was ME, and it was the phone cable into my house which runs up the wall behind a wysteria bush that needed trimming. I soldered it back together rather than using a choc block, and it's been fine ever since. :-)
Re: Closest I've had to that ....
How to bodge repair outdoor cables!
https://youtu.be/7Wh5gM8GM70
Re: Closest I've had to that ....
Why can't the Yanks pronounce the letter "L"? The word is "Solder" not "Sodder". GRRRRR!
Re: Closest I've had to that ....
Because the word is from the Middle English soudure would be my guess. Not our fault you lot corrupted your own word along the way.
Re: Closest I've had to that ....
I find the drift of this thread worrying. Right now the road is being dug up to replace our connection to the water main.
Re: Closest I've had to that ....
BFI? Sorry, tried looking up, but come up with British Film Institute or Bureau of Fearless Ideas
Re: Closest I've had to that ....
BFI - I'd never heard it either, but I bet it means Brute Force and Ignorace. I'm certainly going to use it in the future.
Re: Closest I've had to that ....
Aeons ago - I ran a building controller network - set up using an RS432 or something ring (it was a while ago) suddenly found I couldnt talk to one of the devices.
Tried everything remotely (1200 Baud modems FTW) that could be done and, sadly, had to attend site (an art college.... shame about the cable routing through the roof space over the studios... I was young!!)
Nothing wrong with the cable in the buildings, however whilst crossing the site where they were building new blocks, I noticed a footing trench across the cable run.
Unsrprisingly, upon peering into the trench, I saw the ends of the cable and, on closer inspection, mashed condiut on the spoil tip. 1-0 to the JCB I guess.
The digger driver (who was sat in his cab) even had the audacity to claim it wasn't him!
N.B. the cable run was clearly marked on the plans the groundworks guys were working with...but some architect decided he wanted a trench there anyway.
In a similar vein I had a call last week from an old employer. They were stripping out part of the building and someone sliced through the cable for the gps clock antenna so it stopped working (as this provides the time to the site it could be an issue).
Apparently the same people also sliced through some analogue phone cables as well. Now based on the building there are about 20 fibre multi cores going through the same area if they had sliced through them....... the entire site would have been dead for a week or two.
Mind you not on the cable stretching line but in a similar story. That building used to be wired with ibm type 1 cabling. We had a problem with a terminal not working once. The lead went from a panel to the controller wiring (about 2m) but wouldn’t work. We started on pulling the cable out from the bottom of the cabinet. A while later we had got to the building door (about 60m) and the cable was still connected at both ends. Someone had put a very long cable in as a patch and as the panel still went to the terminal near the door there must have been around 200m of cable involved. We pulled the lead out and swapped it for a shorter one, the terminal came to life immediately!
Staples
Cable routing along a skirting board/doorframe used to be a thing in small offices. Many times I saw cables skewered with a staple leg (there are 'correct' staples to do it) - but if mis-aligned, the staple goes through the cable.
interesting effects - I saw one which would glitch when someone walked accross an old stock room with squeaky floorboards.
Re: Staples
I've already written about my first foray as a SysAdmin back at the early 90s, at the Dean's office of my local Uni. The network was a small spool of coax cable stretched on a bus of 7-8 PCs and a Novell Netware server. No fancy tech trays near the ceiling ir dedicated wall mounts there, just a small metal tray (enough for one coax cable) running above the skirt board.
It all worked awesome until someone would move the coax from behind the computer desks and it would become jammed beneath the cart wheels in the office chairs. Easy peasy, replace a length of coax, crimp the terminals and call the day off.
At the start of the winter, the network went down and I couldn't for the mother of $DEITY find the faulty coax segment. Until I started crawling my way from the Netware server down to the first PC in the bus, only to find someone had pulled the cable from its tray and placed it over a gas heater. The minute the Dean's secretary came in from the chilly morning and set it on, the coax's core melted and the network went down.
Single points of failure always do.
I landed a contract to install two big, garage sized, Memorex tape backup robots at a large number-crunching outfit once. Before I bid on the job, the VP of operations gave me the grand tour. He was proud of all his redundancy. He had two power lines coming in to two separate rooms, with a motor-generator, a large battery consisting of dozens of telco-style lead-acid batteries, a generator, and monitoring systems for each room-full of gear. The 48 Volts was switched by a box at the corner where the two rooms met, brought into the main building via a 5" conduit, where it was switched to two separate computer rooms. Even the links between outlying offices were redundant T-1 and T-3 lines. There was a third "data center" that was dark, to be used for spares "just in case". It was designed to provide non-stop operations, and it did a pretty good job of it. Even the Halon had built-in redundancy.
Until a semi-truck carrying some of my Memorex kit backing into the receiving dock went off course & cut the 5" conduit. The security cameras caught the sparks quite nicely :-)
Two weeks after installing the tape robots, I had a proposal for a more geographically diverse version of the same thing on the VP's desk. I didn't land that contract, alas.
Was working as a telephone fitter in the early eighties, and went to a client's premises to fit a wall-mounted phone extension.
The client had decided where he wanted it, and helpfully marked the wall with a big cross in marker pen. It was in the middle of a bare external wall, with no electrical sockets or switches anywhere near it, so I was happy to fit it where indicated. (This was in the days before cable finders were a thing).
So I marked out the bracket, drilled the first hole, no problem. Started drilling the second hole, big flash and bang... Oops!
Turns out some electrician, in the dim and distant past, had run an electrical cable diagonally across the wall from top to bottom, and then plastered it in with no steel capping or anything over it.
45 degree electricians
I had the same thing in our flat - you could clearly see diagonal channels up the living room wall. God knows what the sparky was thinking (or not)...
Re: 45 degree electricians
What he’d do with the couple of quid he’d saved on cable; or the next job where he'd use the "saved" cable?
Yep, got that in our kitchen
When replacing it after we moved in, took the old wall units off, hmm, why is there an 8mm hole here in the middle with burn marks round it? Stud/wire detector out, wire (in metal conduit fortunately) runs diagonally from top corner to socket in middle of wall.
Wall now has appropriate areas shaded and labelled in big letters, just in case I forget in the future!
I am not a sparky but even I know wires should run vertically!
You must be English..
diagonal cable runs
I'll raise you this:
Visiting a friend's flat in a former Soviet republic, the cable came out of the plaster in the corner of the room, spanning about 4" to the adjoining wall, there it went back into the plaster to a socket about 3" further along that wall. "Plans say the socket goes there, on that wall, I've only got this much cable..."
Beware mystery cable runs
It is the late 1980's I am asked to visit a customer campus that is experiencing strange ethernet issues. I turn up at the campus that has two buildings that are separated by about 20 feet at their nearest points. I quickly confirm that they really do have an ethernet problem, to the point that is almost unusable. Do the normal troubleshooting questions, anything changed? / Is there any physical activity on site etc etc
Get a negative from the IT boys, so in the end it was up with the DC floor tiles and start to examine the transceivers (As the back bone was 10MB Thick Ethernet), found the transceivers and all looking good. But strangely the Ethernet cable seemed to go out of the DC wall. Ask the IT Manager where the cable is routed, he tells me that they dug a trench about 6 months ago to the other building to connect a small server room to the backbone. Had to convince them to get someone to dig up where the cable had been buried - Unbelievably they had just buried the cable in the ground and was looking very much the worse for wear !
Apart from the electrical issues (Electrical Potential issues) connecting two buildings with a cable which in itself could cause issues, the cable which was very much not intended to be buried in the ground outside and started to breakdown the outer coating which had caused the failure.
Advised them to get new cable, keep it in the DC and invest in proper connectivity solution to connect their buildings on the campus.
Customer complained that no-one told them that they could not use the cable outdoors and that we should take some of the responsibility and compensate them - But thankfully someone with a modicum of knowledge quickly shut that down on the customer side.
For me what was amazing is that it actually worked good enough for about 6 months !
Re: Beware mystery cable runs
Just to piggy-back on this one ... They do make what purports to be direct burial network cables. My advice? Always run them in properly installed conduit anyways. Trust me on this.
Re: Beware mystery cable runs
You can do this with some success, but you need to dig quite deep, use SWA cables, back fill with some pea gravel and add yellow a big plastic tape with 'Buried Cables' written on it, before you fill in the ditch. That said, you're still better off using ducting as well.
https://www.tlc-direct.co.uk/Main_Index/Cable_Index/Network_Cable_External/index.html
https://www.tlc-direct.co.uk/Main_Index/Necessities_Index/Electric_Cable_Below_Tape/index.html
Re: Beware mystery cable runs
Had to run 1Gbps Ethernet over 80m to my brothers office. Could not afford 'proper' conduit, but 100m of buried grade plastic water pipe worked perfectly. And I purposely put no warning tape over it :-)
Re: Beware mystery cable runs
+1 for using alkathene pipe for conduit! Easy to blow* cable through with a compressor, resilient, and cheap too.
*Or suck it through, apparently. See a few comments down ↓↓↓
Cable fights back!
Supposedly my firm had an installer suffer an eye injury because he was looking down a conduit while a buddy blew a fiber down the tube.
Didn't witness it myself, but the mandatory training did gain another entry.
Blowing fibre.
When OpenReach came to install our FTTP your man told me that it was a long run, a few hundred metres at least, and because we live on a hill, it's hard to blow the cable up the conduit. (They attach a carrier bag to the end of the fibre with an elastic band, and then use compressed air to blow it along the pipe) So, I suggested that he carried the reel of fibre up the hill to our property, and blew it back down the duct. It took a good 20 seconds for it to register, but he finally realised that this was a good idea™. Unsurprisingly, the fibre went through like a rat up (down?) a drainpipe, job done. We never saw that guy again, and I like to think that's because he got a promotion when he reported back his brilliant ruse...
Re: Blowing fibre.
Hint: Don't use a compressor. Use a decent shopvac. Really. It works better.
soo...
Ever had to deal with a "solution" that involved a pick-up truck and a leaden foot?
Yup. One evening, I was bimbling off to get some food. Route took me along a popular cable route to a large BT exchange. Noticed an outside plant crew had plonked a couple of cones in the road, lifted some duct covers and weren't wearing hi-viz or any of the usual stuff an OP crew would have. So called the police and BT, who turned up and nicked the cable thieves just as they were lashing the cables to their van ready to pull them. Sadly cables had already been cut.
So we were moving a large datacentre into new buildings. DC in one hall, people in an office next door. Cabling needed to be run between the two in the new ducts installed for the job. Contractors said 'done' and pointed to the neat coil of nylon cord left at one end. There's an art to using draw cords to pull cables so you still have a draw cord in the ducts.
Same site, some time later and the risers between the machine room on one floor and the ops room below were full. So contractors hired over a weekend to cut a hole in the concrete slab so we could run more cabling down to ops. There was quite a bit of cable under the floor, so contractors moved some out of the way. Coiling slack fibre into a neat skein comfortably exceeded said fibres bend radius.
Different employer, different sites, on different sides of a river. Contractors hired to do directional/mole drilling across the river to make a new duct. All going well until mole went through a gas main, then a power cable. Fault detection was aided by the jets of flame. Think that was a 6-figure repair job.
And many more. Drilling through building walls to make entry (not in the Hatton Garden sense) and hitting buiding's water mains. Finding unexploded bombs or long forgotten buried fuel tanks. There's a reason why excess construction charges can be, or seem to be excessive. Thanks to affordable digital cameras though, it helped me explain to clients why their service was going to be delayed.
Re: soo...
Who is it on here that said that any story is instantly funnier if it includes the phrase "flames shot out!"?
Re: soo...
Regarding draw cords, my attitude to this sort of work, is a simple one. Make sure system is left as you would expect it to be.
If you need a draw cord to fit your cable, replace it so the next time it is there for you or whoever comes after you.
Same as if working on a mechanical system ensure refitted bolts are copper greased so they come off in a few years, the next guy will thank you for it.
Re: soo...
the next guy will thank you for it.
More importantly, the "next" guy might be you again.
That is a nice example of the difference between an engineer and sn installer.