Panic at the Cisco tech, thanks to ancient IOS syntax helper that outsmarted itself
- Reference: 1733727486
- News link: https://www.theregister.co.uk/2024/12/09/who_me/
- Source link:
This week's hero is a network tech whom we will Regomize as "Sherlock." Back in the 1990s he was working as an IT manager for a business that had a number of regional hubs, each linked to HQ over a 128K link. Smaller regional offices were linked to their hubs with 64K connections.
Cisco knows that netadmins are a busy lot, and one feature of the command line for its IOS is an autocomplete that means typing some characters of a command followed by the TAB key offers tells the OS to complete whatever syntax you're typing.
[1]
Typing sh -TAB can therefore be a shortcut to the command show , thus saving two whole keystrokes and valuable microseconds for greater productivity.
[2]
[3]
The version Sherlock used back in the day had a similar feature that he discovered could cause problems when driving boxes over a wide area network. So when he typed net sh ip xxx to get settings for a remote box, his network connection disappeared. Attempts to re-establish the link were to no avail – no link existed.
A hasty examination of the Cisco manual revealed the issue: over a network connection, the abbreviation sh corresponds not to show but to shut . Sherlock had destroyed his connection because in the 1990s, when IOS was young, no-one had thought to ensure that the same shortcut didn't mean two entirely different commands in different contexts.
[4]NetAdmin learns that wooden chocks, unlike swipe cards, open doors when networks can't
[5]Network engineer chose humiliation over a night on the datacenter floor
[6]Undergrad thought he had mastered Unix in weeks. Then he discovered rm -rf
[7]The sad tale of the Alpha massacre
Understanding the issue was one thing – fixing it another. Sherlock managed to find a modem with which he could dial back in to the remote office, and successfully established a connection that way. He still had to re-establish the link to head office though.
He tried the command net open ip xxx , but to no avail. There was still no connection. And the manual was no help this time either.
[8]
In desperation he consulted an engineer friend – let's call him "Watson" – and asked if he knew how to detangle this conundrum.
Watson thought for a moment and asked if he had un-shut the network connection. Sherlock thought he had – isn't that what open is for?
No, it seems not. In this ancient edition of IOS, before open would make a connection, users had to reverse the shut condition. So the appropriate command was net no sh ip xxx .
[9]
With that done, Sherlock was able to continue his work, with a valuable lesson learned.
Ever had a moment when something you did resulted in something you didn't want to happen, and reading the manual didn't help? Tell us about it in [10]an email to Who, Me? and we may feature your misadventure on some future Monday. ®
Get our [11]Tech Resources
[1] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/networks&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2Z1bN13KFsntpXb-3spwIEwAAAM0&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/networks&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Z1bN13KFsntpXb-3spwIEwAAAM0&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/networks&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33Z1bN13KFsntpXb-3spwIEwAAAM0&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[4] https://www.theregister.com/2024/12/02/who_me/
[5] https://www.theregister.com/2024/11/25/who_me/
[6] https://www.theregister.com/2024/11/18/who_me/
[7] https://www.theregister.com/2024/11/11/who_me/
[8] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/networks&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Z1bN13KFsntpXb-3spwIEwAAAM0&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[9] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/networks&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33Z1bN13KFsntpXb-3spwIEwAAAM0&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[10] mailto:whome@theregister.com
[11] https://whitepapers.theregister.com/
I guess it had to be Sherlock...
if he were Mycroft then this particular episode would never have occurred.
I didn't notice whether Watson, not being the brains of the firm, found the solution or someone read the manual.
net no sh ip xxx -- No shit Sherlock.
Network engineers never were my favourite IT people partly because of this sort of brain damage. "No shut = open" verges on Newspeak double plus ungood.
Re: I guess it had to be Sherlock...
Just wait till you hear about "switchport trunk allow vlan" sans add.
Good times.
Re: I guess it had to be Sherlock...
Well that's my submission into "Who, me?" scuppered!
Re: I guess it had to be Sherlock...
> Network engineers never were my favourite IT people partly because of this sort of brain damage.
That's why good ones earn a packet....
as a former "cisco network manager" i can relate to this; been there, seen it, got the tee-shirt !!!
ios is great - until you screw up. one command is all it takes to bring an entire global network down (don't ask how i know that one !!)
Context matters
”sh
One should note here that the command to do something on a Cisco is never ”sh”, there’s always a full, expanded command (”show”, ”shutdown”, or even ”shim” in some places) being executed (even if you don’t see it at the prompt). It’s the console cowboy’s responsibility to know what he’s typing. And believe me, I’ve mistyped a lot of IOS/XE/XR commands over years.
Juniper’s JunOS CLI, on the other hand, auto-expands the commands as you type so you’ll know what the router thinks you meant to do before you hit enter. (You still need to know what you’re doing, though!)
"Ever had a moment when [..] reading the manual didn't help"
Oh yeah. It was way back when Lotus Notes was Release 4. LotusScript had just appeared and I had been tasked for creating a Content Management System for an international customer.
Now, you need to understand two things : the first is that, in those days, a Notes database was limited to 2GB in size. The second thing is a little quirk in the system when you're cycling through all the documents (or records, for you RDBMS people) in a particular view.
The manual says (still to this day) that using view.getnextdocument(olddocument) will give you the next document in the view, which it pretty much does. What the manual does not say is that the old document is not deleted from memory when you do that.
So, in those days, it wasn't much of an issue because Notes databases did not contain hundreds of thousands of documents.
But then R5 came out, and the 2GB cap was lifted. And oh boy, did those databases start filling up. That is when I learned, the hard way, that cycling through 85K+ documents, with attachments, in a view could crash the script (thankfully, not the server). The bug report came in from the customer, and I spent days trying to understand why because the script never crashed on the same document (thank God computers are supposed to work on zeros and ones - God only knows what would happen if fuzzy logic were to be used).
In any case, after much head-scratching, I finally clicked that the script always failed after more than 80K documents had been processed. It took me another few minutes and then I wrote this :
Set olddoc = doc
Set doc = view.getNextDocument(olddoc)
Delete olddoc
With that, magically the script always completed successfully from then on (Delete removes from RAM, not from database).
That is one lesson I have never forgotten.
Backward double-negative logic is a predictor of massive future industry dinosaurs.
Fred Brooks called IBM's OS/360 a "multi-million dollar mistake" back when that was a lot of money. Now called Z/OS, its backward COND statement ("Don't run this step if the previous step's return code was not N) is still the mainstay of the financial systems across the world.
Pedant
“Typing sh-TAB can therefore be a shortcut to the command show, thus saving two whole keystrokes and valuable microseconds for greater productivity.”
Well, really only one keystroke…
Re: Pedant
I had the same thought, but then I remembered that it will also include the space after 'show' or 'shut', so it's still a net two keystrokes, after all.
Sherlock-adjacent icon -->
Ouch... This felt like reading my own story from 1996. Done exactly the same by trying to get the interface information using the same cursed "sh" shortcut.
This sort of crap trap is the reason I got into the habit of typing commands in full - much to the amusement of my colleagues.
Nothing to with networks, but operating a 1.5 m (diameter) infrared telescope at an Italian observatory as an undergrad, I was rather horrified that when I typed in the wrong coordinates, and the system asked me to verify the coordinates were right (Y/N), my response "N", resulted in a cheerful "Then I go!" followed by the humming of drive motors to swing the telescope into the requested position. This could result in the liquid nitrogen and, worse, liquid helium to be poured out of the cryostat of the infrared spectrograph we were testing, potentially causing major damage. Happily nothing bad happened, but I did learn from the Italians running the place that any key except Ctrl-D would be interpreted as "Yes". Brilliant!
Suffice to say I was not impressed by their UI design.
With something like that the UI should always be designed to 'failsafe'. For the given example it should have been that ANYTHING other than a single key combination (e.g. Ctrl-Y) would be taken as a 'no'. It should never have been set so that the 'default' result was the potentially dangerous condition.
Cisco IOS - its the reason I never went very deeply into configuring network equipment. The few times I had contact with IOS resulted in much hair pulling and the sort of language you don't want your boss to hear.