A paper clip, a spool of phone wire and a recalcitrant RS-232 line: Going MacGyver in the wonderful world of hotel IT
- Reference: 1587107704
- News link: https://www.theregister.co.uk/2020/04/17/on_call/
- Source link:
Today's tale comes from "Jeff", who spent a good portion of the decade that gave us The A-Team , Knight Rider and the IBM PC working in the hotel IT business.
" [2]RS-232 wiring ," he recalled, "was normal between machines." There was no email or messaging back then, just teletype and faxes, nor was there the remote access or hasty sending of a patch taken for granted today. "A small business computer was size of a washer and dryer and cost around $100,000."
RS-232 cables were also the bane of Jeff's existence.
A call came in from a hotel complaining that telephone charges were not making it from the phone switch (located at one end of the hotel) to the computer (helpfully positioned at the other.)
The cables were plugged in correctly and things had been working properly up until now. Naturally, the user was convinced that nothing had changed but still those charges were not getting through.
OK brainiacs, we've got an IT cold case for you: Fatal disk errors on an Amiga 4000 with 600MB external SCSI unless the clock app is... just so [3]READ MORE
Jeff began the troubleshooting process over the phone, kicking off by testing the Async BiSync Converter (ABC) board with the aid of a paperclip.
For those not familiar with the delights of ABC devices, the things were hardware protocol converters, usually with a pair of RS-232 ports (one async and one bisync.) They were often found converting data from one source into something another device would understand. In this case, helping the phone system and hotel computer communicate.
And the paperclip? "Disconnect the RS-232 plug," instructed Jeff over the phone, "bend a paperclip into a 'U', plug into pins 2 & 3, run the test program on the host machine…" A DIY loopback, if you will.
Pins 2 and 3 of an RS-232 plugs are used for receiving and transmitting data (with subtle differences between the original 25-pin incarnation and the "modern" 9-pin versions).
This hack has fond and not-so-fond memories of making cables for ancient daisy-wheel printers back in the day as well as the odd acoustic coupler. The various types of serial connection remain seared into the grey matter to this very day.
In the case of the hotel, all was OK. The card was good. Jeff delicately asked if "something" might be happening at the hotel. Maybe a new tower was being added?
"No... but they are remodelling the third floor."
Ah.
The construction crew had kicked off work that morning and cheerfully sliced through the RS-232 line (helpfully installed without conduit) while demolishing walls.
Jeff's solution was of the quick and dirty variety. A spool of phone wire was picked up from the local Radio Shack and ran along the floor from phone switch to computer. The ends were stripped, the cores bent over to form a makeshift pin and stuck into holes 2, 3 and 7 of the RS-232 port.
The data was flowing once more.
Naturally, Jeff explained the fix was very temporary, that the cable would need to be placed inside a proper conduit and the correct connectors wired up. "No need!" the user cheerfully responded. "I got the carpet guy tucking it under the rug. We are good!"
What could possibly go wrong?
Tune in next week for the second instalment of Jeff's adventures in RS-232 with The Mystery of the Million-Dollar Bar Bill.
Ever talked a user through a potentially computer destroying procedure? Or have a particular RS-232 confession that you need to get off your chest? Send an email to [4]On Call and tell all. Just don't ask us to make you an RS-232 cable as a reward, OK? ®
Sponsored: [5]Legacy Modernization: Finding Your Way With Low-Code
[1] https://www.theregister.co.uk/Tag/on-call
[2] https://www.lammertbies.nl/comm/cable/rs-232
[3] https://www.theregister.co.uk/2020/04/10/on_call/
[4] mailto:oncall@theregister.co.uk
[5] https://go.theregister.co.uk/tl/1936/-8553/legacy-modernization-finding-your-way-with-low-code?td=wptl1936
Re: Proper lash up
The "6 months - tops" lash-up code that I wrote in 1992 is still running I hear. I wonder what happened to the project that it was a stop-gap for?
Re: Proper lash up
You're probably using the lash-up that I put together in 4.1BSD (now called 4.1aBSD) for part of the TCP/IP stack to be included in 4.2BSD[0]. It was supposed to be one of those "just get us through the demo, dammit" hacks. I can't remember exactly when it was included, but I was working on it over Christmas/NewYears 1981.
[0] Just to cut the usual pack of idiots putting words into my mouth off at the socks, no, I didn't write the whole stack. That's why I said "part of". It was only about 120 lines of C in total.
Re: Proper lash up
That is the problem. You get a temporary, Heath Robinson solution in place to tide the user over until they can get the problem fixed.
At least, that is you interpretation.
On the other hand, the user just sees that the problem has been resolved, end of story.
Re: Proper lash up
Nothing is as permanent as a temporary solution.
Re: Proper lash up
Except for a temporary tax hike.
Re: Proper lash up
Where I live, a temporary viaduct over the railway was built in the early 70's to allow easy access to the dockyards. It's still in use and is now commonly referenced as the "Interim Overpass". Maybe one day it will be renamed to "Permanent Overpass" and everything will be fine.
Re: Proper lash up
The town I grew up in has a tower of which the literal translation of the name into English is "The New Tower". It is only a bit more than 7 centuries old and slightly younger than the nearby "The New Market".
Re: Proper lash up
Do a search for "TEMPORARY, I HOPE HOPE HOPE". Or see this excellent poetic commentary on it.
[1]http://www.thecodelesscode.com/case/234
[1] http://www.thecodelesscode.com/case/234
Re: Proper lash up
That deserves more than the one upvote I can give you, have a ===>
Truely
Swept under the rug!
Re: Truely
The bane of my existence: BT Red Care burglar alarm systems.
Around 1998 a very bright spark decided that my IT department should have a Redcare burglar alarm installed, just in case, you know, a very clever thief managed to get through the armed security at King Charles Street, into a government department that had more armed police in the reception area, got down into the basement (onboarding for new staff involved escorting them to reception until they knew how to find their way along the maze of basement corridors) and found their way to a security pass-blocked door.
BT duly arrived and installed the RedCare line and it worked. Just as the engineer was about to leave someone mentioned that the emergency external lines had all gone down.
Cue: "Before you go Mr BT Engineer what did you just do?" and the sheepish response that he could not find a spare line to use, so he opened the trunking, sliced though the bunch of external telephone lines, bar one he used for RedCare, and closed the trunking. I asked him if he would mind fixing the lines he had "accidentally" severed ... "since one of those lines is a direct link to a government security centre and we might need it if there was something petty .. like a national emergency".
The BT engineer apparently filed a report that he felt that he had been intimidated and not allowed to leave the building. The reality was that I refused to allow someone to escort him out of the building and he must have spent some time wandering the basement corridors before being tracked down by security and escorted out of the building.
Back in the very early 90's...
...a colleague and I used to wire all the RS232 connected terminals in, as well as cable any new phone extensions into the PBX.
I remember we had great fun using only green wires wherever we could get away with it, knowing that anyone following us in a few years would be fuming at the lack of additional colours.This was reserved mostly for where we had to put connectors through walls or make adapters of course - we had no choice in the colour of the cores in the main cable.
I do remember once wiring in about a dozen new phones and of course I had to test them so every time I connected on, I'd call it from another office.
Imagine my surprise as I walked into the office I'd been putting them in to see him pick up a ringing phone (not even one of the ones I'd put in) to hear him shouting down the phone "will you just fuck off?!? I'm busy!!!".
Apparently he thought it was me calling to test a line again and in his annoyance, didn't notice it was an external ring. Or an existing phone. He says it sounded like a very nice, very confused old lady on the other end of the line, just before he hung up in horror!
Happy days.
Re: Back in the very early 90's...
Ah colour wires in very early 90s ...
My first IT transformation project was re-doing all the network at a big warehouse ... The network part went completely OK, we even replaced this old 10base2 BNC cable that was only here for distance reason by a proper fiber ...
The phone part didn't go so well: the contractor, when re-cabling all phones, had assumed correct wire colours were used and nothing worked.
Explanation: the bozo from a dodgy company who was normally doing phone wiring would use whatever cable of the day he had all over the place, for many years. Everything was therefore messed up.
We had to re-cable every office one by one to make it work ...
Good times, yes :)
PS: forgot to mention, no-one knew an RS-232 line was going from the entrance of the warehouse up to the HR office for the badges. It was borrowing our LAN cabling panels which we had removed and replaced. Took me days to understand this and restore this darn badging system !
Thanks for reminding me...
... of the related Spitting Image tune https://m.youtube.com/watch?v=CDlj0jBtYmQ
Re: Thanks for reminding me...
Damn. I was just about to post that too
RS-232...
I have not-so-fond memories of RS-232 from those days, debugging both hardware and software could be a right pain in the proverbial.
I ended up making my own test lead set, two cables with 9 and 25 pin D-connectors at one end and a 25 pin at the other, plus a single long extension and a bought in test/breakout box. It served me well for a few years when serial printers and comms were common.
Hardware/software issues included twice developing suitable hardware and drivers to run a plotter over the same serial line as the terminal. For reasons lost in the mists of time we couldn't use the AUX (or was it PRT?) port on the DEC VT-220 for the first so had to construct a wire tap, for the second we could just set the VT to pass-through but had to tweak the drivers as one of the Calcomp plotter commands would cause the terminal to lock up.
In a later job we had a MicroVAX 2000 with 4 separate RS-485 connected serial port hubs. The hardware was second-hand (it had been the test/demo kit for the MRP system provider) so the hubs came with a single backplane. It made far more sense to run a single RS-485 connection to the more distant offices instead of large bundles of serial cables but DEC wanted a ridiculous sum for separate PSUs for each hub. I handed it over to our in-house electronics engineer who built me 3 wall boxes to act as RS-485 terminations and power supplies so the hubs could just plug in.
Danger - building works
In the days when the heat from the water cooled main frame heated the site and before PCs, we had a new building build next door. One day the machine digging the foundations put it through the main water supply for the site (I heard the effect was like a whale blowing). This meant no main frame, no air conditioning. As this was a "modern building" this also meant there were no opening windows ( or outside awareness panels as they were called). It took a week to fix, and the boyd odour smell was not pleasant.
In the last 10 years I visited a company who had taken High Availability very seriously, including dual network providers etc. The both came in the front of the building - in through the same hole in the wall. This didn't help when someone did some work in the road and cut though both network cables.
I also heard of the case when a French company had two network suppliers to cross France. One cable headed north, one cable headed south - what could go wrong? What happened was both cables were rerouted to use the common network infrastructure across France. At the other end, one cable went north, and down to the site, and the other went south, and then up to the site. Im sure youve guessed what happened. Something happened to the common network infrastructure and so both networks were broken.
Re: Danger - building works
"the boyd odour smell was not pleasant"
I used to work in building called Boyd House that did indeed have its own unique aroma.
Re: Danger - building works
HA links ending up in the same infrastructure half a mile away is par for the course but it does allow us to claim 'not our fault' and point at telco who provided the 'HA' at great cost.
While working for a large french company in the 90s I was amazed to discover that we we in the process of laying direct site to site fibre in new conduits under 1.5Km of suburban Paris streets (senior company execs being good chums with senior local government officials from way back).
The fibre was only for a remote data centre backup facility and comms backup should a shovel meet the main comms lines at either site, neither of which hosted the head office.
It must have been a Friday
"I got the carpet guy tucking it under the rug. We are good!"
And it was crawling up on Pub:30
Done and dusted!
About ten years back
A colleague was starting up a system and found that the network cable to one of the racks was broken.
As the rack needed to work the next day for the building "soft opening", they put in a temporary patch cable between the processor in the rack and the patchbay the other side of the room.
They noted it in the report and formally requested that the network cabling guys fix the in-conduit cable when they were due to come back to site the next week to fix any issues discovered during system startup.
Two years later, I went to the site and found that patch cable still in place - at about waist height, strung across the middle of the room...
I assume limbo was popular?
Re: About ten years back
I once worked with a company who was replacing their mishmash of network kit. The company specified to BT/Virgin/NTL/whoever that the WAN line should terminate within 6ft of the Cisco router (so they could use the Cisco supplied 6ft cable)
After a few cases of "I'm 6ft fingertip-to-fingertip and I can touch both the line jack and the router", "yes, but you're standing in the gangway!" they decided it was easier to supply 5m cables than go through the palaver of booking jobs with BT/Virgin/NTL/whoever to move the network termination to a sensible position
Luxury
I worked at a UK council where an RS232 would have been heaven.
Picture an IBM machine spitting out paper tape in to an empty printer paper box, once the box was full the end was feed in to an ICL DME machine paper tape reader.
So one writing and one reading at the same time, the operators had to test run any programs to ensure the box was big enough as the ICL machine read faster than the IBM machine wrote.
Now that’s real data transfer.
Re: Luxury
If the writer could have kept up with the reader you know what would have happened - a tape crossing the room, probably at waist height to cause as much obstruction and risk of being broken as possible.
Re: Luxury
Pre-made paper tape? Sheer exuberance. When I were lad used to go to mill to mix paper pulp...
Re: Luxury
Remember Altair DOS & Basic on paper tape. Loaded through ASR-33 teletype after keying in boot loader via front panel switches. Made the Honeywell DPS-6 and Dec PDP boxes look like heaven sent...
Re: Luxury
"Made the Honeywell DPS-6 and Dec PDP boxes look like heaven sent..."
I still think that DEC PDP11 kit was the best teaching platform ever.
Re: Luxury
Tend to agree. All our work was in assembler on the PDP 11/03 and 11/34. The tech guys wrote their own mini operating system hosted on RT-11 to support text processing for newspaper editing and production.
Re: Luxury
I still think that DEC PDP11 kit was the best teaching platform ever.
I really should fire mine up again, after I check the PSU caps. I could do with more storage, but Q-Bus SCSI cards are like hen's teeth, and similarly priced. The existing RD52 still works, but 30MB is a bit tight these days...
Re: Luxury
30MB! We had RL01...who would ever need more than 5MB per cartridge...
Remember serial breakout boxes?
They were about the size of a phone and an inch thick. You'd open it like a book and it'd have an RS-232 connector on each side, one female and one male. It had pins that broke out the connections that you could jumper as appropriate.
Great for quickly finding out if you needed to swap pins 2-3, for example. You'd then make a cable that duplicated the jumper arrangement.
Man, I also miss mid-80s Radio Shack. That place was heaven back then.
Re: Remember serial breakout boxes?
Yep, see above!
I'm wondering where mine is as I'm sure I wouldn't have disposed of it (I never throw out IT kit!)
Re: Remember serial breakout boxes?
"it'd have an RS-232 connector"
Or any other data cable connector (RS422, GPIB, etc.). The good ones come with red and green LED pairs on each pin. I still have a pile of 'em. They come in handy occasionally.
Re: Remember serial breakout boxes?
I am sure mine must still be around somewhere, although it was a little smaller. An open case with a 25 way connector on each end and 25 pins sticking up. Jumper leads could be added as necessary. The last resort in my serial toolkit, which also included a long straight-through cable (1-1, 2-2, ... 25-25), short adapters for common requirements such as null-modem or 25-9 serial, gender-benders. If two boxes could talk via a common protocol through a 9 or 25 pin D socket, I could probably connect them.
Re: Remember serial breakout boxes?
Interfakers...?
I can remember my first boss using one to troubleshoot serial - but then it was replaced by Dec MMJ cables which were rollovers - so just missed those pleasures...!
Re: Remember serial breakout boxes?
I still have a couple in a desk drawer. And the bit of veroboard with a 25-pin D-type on each end and a selection of LEDs and resistors to show line state. It must be getting on for 40 years old now, best debugging tool I ever made.
Then again I've only recently scrapped an HP 4951C protocol analyzer, complete with pods for RS 232/423 and serial synchronous comms. Nice kit, but none of our recent serial lines ran at less than 2Mbit/s, and it topped out at 64kbit/s.
Re: Remember serial breakout boxes?
'They were about the size of a phone and an inch thick.'
The size of a then-phone or a now-phone?
Re: Remember serial breakout boxes?
Not only do I remember them I even used one a month ago or so. Together with a couple of DB-25 to DB-9 adapters.
I remember things differently ...
"There was no email or messaging back then"
Yes, there was. Maybe not in program loaders disguised as operating systems, but that doesn't mean they didn't exist. And even with those toy computers, the BBS world was tiny, but there. If you had a modem, of course.
"nor was there the remote access"
Yes, there was. TehIntraWebTubes was well over a decade old by then.
"hasty sending of a patch"
Yes, there was. (Sometimes the "hasty" was via motorcycle ... good bandwidth, but bad latency.)
"A small business computer was size of a washer and dryer and cost around $100,000."
My Heath H11 had a 16 bit PDP11 CPU, was about the size of three modern large desk-side computers and cost me just under $3,500 (I had a lot of accessories and maxed it out on RAM ... the base kit was about $1300). It was my xmas gift to myself in 1977.
Re: I remember things differently ...
Yes, there was. (Sometimes the "hasty" was via motorcycle ... good bandwidth, but bad latency.)
Don't forget the occasional packet loss due to a crash.
Oh god
RS232... the memories... the noise... you wouldn't understand.. you weren't there.
Take a fresh faced slightly(at this stage) cynical cockroach, add in a PC, and a bunch of Fanuc robots, both have RS232 interfaces so it should be easy to to link them together (breaks down sobbing again)
Yupp wire 2 to pin 3 etc etc etc and we have the first cable made up, plug it in and nothing, change the PC port setup, nothing, change it agaain , nothing ..hmm.
yupp 9600 baud... I know... no flow control.... WTF is an 087 alarm? ahh buffer overrun, TEN BYTE BUFFER YOU ARE KIDDING ME!!!, ok says here uses software flow control when buffer is full, set the PC to software mode.... yay got 12 bytes in the machine before it stopped... hmm and hardware flow control to resume sending!!!!! what the *&%%*&%*&%(^*(&ing idiot designed this?
(starts drinking)
OK.... checks sources on the internet , all of them go LOL when asked about using windows to setup the rs232 port to talk to Fanuc machines
Several days later after writing a small java program to directly control the serial port .... and we have success.... wait for the xoff.... then wait for CTS line to change before sending more data (remembers his cheering and happy dance, then collapses sobbing again)
Then the boss comes up and says "You taken 4 days to make ONE zarking wire.... what the zark have you been doing all this time?"
(closes the door and resumes his isolation with his plastic dinosaurs)
RS232 is alive and well
In the calibration business, there's a lot of RS232 connections still in use. Typically they're for adjusting an instrument or changing its settings, if it doesn't have a display and buttons. Easy to use, too - plug USB-to-RS232 adapter into computer, fire up HyperTerminal or PuTTY, initiate link to device, and refer to the manual for the commands.
The big difference seems to be that the RS232 cable doesn't stay plugged in all the time.
Long RS232 cabes
I have two stories about long-run RS232 cables.
The first was when I worked at a UK educational establishment. Pretty much everything was jury-rigged there, because they did not want to pay professionals to do things like laying cables. Anyway, my PDP11 was having some cables run down the hall to the lecturers offices, where their newly bought BBC Micro's were to act as terminals (and, in fact, were also connected to the floor Econet). The cables were laid, and loosely tied to a convenient support. Everything worked fine. One of the other technicians then decided ti tidy it up, and proceeded to tack it to whatever was available using staple gun instead of wire clips....
You can guess. Staple straight through the cable, shorting pins 2 and 3 (as we later found out). No immediate problems, because the BBC micro was not being used. But the PDP11 started getting slower and slower, and after about 4 minutes or so, crashed with a I/O buffer overrun error. Rebooted, and the system was fine for four minutes, and then crashed again.
Finally realized that on the port with the damaged cable, the system was sending out the login banner, which was promptly read back in as input from the terminal (pins 2 and 3 shorted). This generated multiple new login banners, exponentially increasing the amount of traffic until the PDP11 (which was very good at handling character I/O normally), just gave up the ghost,
Once found, rather than laying a new 25 metre cable, the rather embarrassed technician cut the damaged bit of the cable out and spliced in (using screw terminal blocks, I believe) a couple of inches of new cable,
The second story is from a factory floor, where a terminal was in the middle with the cable, the maximum length permitted by the RS232 standard, was run through the roof. Periodically, a couple of times a day, the computer it was attached to reported TTY Hog messages, and promptly shut the terminal down. I was giving remote support to a VAR, and after several days getting them running diagnostics on the port and checking the cable for damage or shorts, I asked where the cable was routed. They said that they had run it down the existing cable runs with everything else. After a few seconds thinking, I asked what else was fun down the runs. "Oh", they said. "Pretty much everything". Apparently, this included power for the electric motors that ran large industrial hoists for moving things around the factory floor. "OK", I said, "Is there any chance that the ports shutting down happened at the same time that these hoists were operating,,,,".
Turns out the motors were very dirty, and drew a lot of current. The rapid current spikes generated by the motors was being picked up as interference on the RS232 cable, which was not good quality shielded cable. It would have worked fine in an office environment, but in an industrial environment, it was not up to the task. The system tried it's best to make sense of the noise but failed, shutting down the port. Anyway, they actually relocated the terminal to a different part of the factory floor with a much shorter cable run and good quality cable, and I never heard from them again.
Many years ago I needed to get an RS232 connection from the basement to the 1st floor of the office I was working in. At the time there was a mass of twisted pair cables that ran around the building, used to hook all the phones up. I was able to isolate a couple of unused pairs and hooked up the RX, TX and GND. The hack worked a charm and the data flowed.
Fast forward 3 months. The workers came into the office one Monday morning to find that the phone system was not working. Going on the fact that I was the last person "messing" with the system, the boss decided that my hack was responsible for the failure (bear in mind it had been working fine for 3 months). So I set about unwiring my makeshift RS232 cable. And surprise, surprise, it failed to make any difference. The non-working phone system remained non-working. So finally the boss had to bite the bullet and call in a proper phone engineer to sort the problem out. It took another day but the phones started working again. And nothing more was said.
But here's the thing, I knew the phone engineer outside of the office and we ended up bumping into each other later that year and decided to go for a pint. I asked him about the phone job and he laughed- Apparently, the boss had decided to change the carpet in his office. So he decided to do it on the cheap and he and his father-in-law had come in on the Saturday, pulled out the old carpet and replaced it. The fault with the phones? They'd used a stapler to fix the carpet and managed to put a staple through the phone lines. Not once, but many times.
Proper lash up
Probably still there.