News: 1713872706

  ARM Give a man a fire and he's warm for a day, but set fire to him and he's warm for the rest of his life (Terry Pratchett, Jingo)

Voyager 1 regains sanity after engineers patch around problematic memory

(2024/04/23)


NASA's Voyager 1 spacecraft has begun returning usable engineering data after engineers devised a way to work around a damaged memory chip.

It is the first time the spacecraft has returned usable data since it began [1]babbling nonsensically in 2023. The problem was [2]eventually traced to a single chip responsible for storing a portion of the Flight Data Subsystem (FDS) memory.

The chip's failure made science and engineering data from the probe unusable, and posed a challenge for engineers. The chip contained some of the FDS computer's code, but simply shifting that code elsewhere wasn't an option – no single location was large enough to hold it.

[3]

The solution was to break the code into sections, tweak them so they still functioned as a whole, and store them in different places in the FDS. Any references to the code's location would also need to be updated.

[4]

[5]

The spacecraft is currently more than 15 billion miles (24 billion kilometers) from Earth, meaning that a radio signal from Earth takes approximately 22.5 hours to reach it and the same again to get a response.

[6]NASA tries to jog Voyager 1's memory from 15 billion miles away

[7]Voyager 1 starts making sense again after months of babble

[8]Work to resolve binary babble from Voyager 1 is ongoing

[9]NASA engineers scratch heads as Voyager 1 starts spouting cosmic gibberish

Engineers started with the code responsible for packaging the spacecraft's engineering data and sent it to its new location on April 18. On April 20, they had confirmation – [10]the fix was successful , and engineering data began flowing from the spacecraft for the first time in five months.

The team will relocate the rest of the affected software over the next few weeks, meaning that science data should also start being returned from the veteran probe.

Dr Garry Hunt, one of the original Voyager scientists, told The Register he was "thrilled" by the news but cautioned that it would take a little time to get the spacecraft set up again for science, thanks in part to the lengthy communication time.

[11]

He also told us he was not surprised by the achievement: "JPL engineers are brilliant."

The news will help lift some of the gloom hanging over the Jet Propulsion Laboratory (JPL) in the wake of [12]NASA's budget woes . However, it is also a reminder of the extraordinary longevity and forward-thinking of past missions. Voyager 1 was, after all, launched in 1977 and there is every chance that the spacecraft will still be functioning more than 50 years after its launch. ®

Get our [13]Tech Resources



[1] https://www.theregister.com/2023/12/14/engineers_work_to_fix_voyager/

[2] https://www.theregister.com/2024/04/15/voyager_engineers_prepare_fix/

[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_offbeat/science&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2ZifbIugU2Hv5oW@iE3H6OQAAAJc&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0

[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_offbeat/science&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZifbIugU2Hv5oW@iE3H6OQAAAJc&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0

[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_offbeat/science&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZifbIugU2Hv5oW@iE3H6OQAAAJc&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0

[6] https://www.theregister.com/2024/04/15/voyager_engineers_prepare_fix/

[7] https://www.theregister.com/2024/03/14/voyager_1_not_dead/

[8] https://www.theregister.com/2024/02/08/engineers_work_to_revive_voyager/

[9] https://www.theregister.com/2023/12/14/engineers_work_to_fix_voyager/

[10] https://www.jpl.nasa.gov/news/nasas-voyager-1-resumes-sending-engineering-updates-to-earth

[11] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_offbeat/science&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZifbIugU2Hv5oW@iE3H6OQAAAJc&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0

[12] https://www.theregister.com/2024/04/16/nasa_msr_mission_update/

[13] https://whitepapers.theregister.com/



phuzz

That's some bloody impressive remote troubleshooting.

>>>> beers all round

Sam not the Viking

Human ingenuity --->

I wonder if AI was involved?

Assuming not, I wonder what it would have recommended: "More adverts before downloads."

On the other hand, could Copilot have been 'helping' with the transmission? "Looks like you're trying to send data. Would you like me to cock that up for you?"

Locomotion69

Fixing a failed device 47 years after deployment... Well done!

Built to last.

It is rocket science after all.

This is why...

John Robson

... we over engineer things which are "difficult" to service.

Re: This is why...

Sam not the Viking

Knowing that things will go wrong.

Really good news

Will Godfrey

In these times we need such positive reports.

In space

Forget It

non one can hear you debug

Difficult to comprehend that...

Oh Matron!

the entire code base is probably less than an empty Word Document file.

Re: Difficult to comprehend that...

John Riddoch

According to [1]NASA , there's about 68KB of memory in total. The FDS have 8198 16 bit words, so about 16KB for each FDS computer. I've created an empty Word doc which is actually just under 12KB as .docx and 26KB as .doc, so it depends on the format you want to save it as. Word used to be a lot worse I recall, so it also depends on which version you're using...

Regardless of that comparison, 8198 words doesn't give a lot of wiggle room to move stuff about...

[1] https://voyager.jpl.nasa.gov/frequently-asked-questions/

Re: Difficult to comprehend that...

Neil 44

Not sure what the processor(s) will be, but many of that generation (I'm thinking Z80, as that's what I learnt) traded "jump" space with functionality. Eg, the Z80 had a JR instruction was 2 bytes and could jump +-128 bytes. Whereas the JP instruction took 3 bytes - but could jump anywhere in the 64kb address space.

JR was your friend if you had either limited memory or code that needed to relocate as it didn't contain an absolute address

Bingo!

Mike 137

" The solution was to break the code into sections, tweak them so they still functioned as a whole, and store them in different places in the FDS. "

A colleague and I used this approach in the very early eighties, to upgrade ROM-based applications for Acorn computers (fitting 4k EPROMs). We looked for unused spaces, or spaces that could be released by simplifying existing code, that were long enough for a few new instructions plus a jump in and out, and broke the new code into fragments to fit.

Nice that the technique hasn't been forgotten in this age of coding while ignoring the hardware.

Doctor Syntax

Fixed by spaghetti coding!

Locomotion69

I would call this an perfect example of position independent modular coding !

Lord Elpuss

Epic engineering chaps & chapesses. Have a >>>. And a stiff whisky for the person who actually has to press SEND, knowing that any cockup is likely to be utterly irreversible.

TonyMurphy

The human race is doing a lot of dumb things these days, but occasionally. for briefest of instants, it shines bright with ingenuity, skill and brilliance,

"The problem was eventually traced to a single chip"

Pascal Monett

Wow. That's never something you're going to read in a Borkzilla report, now is it ?

Well done to NASA engineers. Once again they have snatched victory from the jaws of defeat.

I hope they can carry this on for a long time yet.

MichaelGordon

Given the age of the hardware it's probably not that susceptible to radiation, even if it's not specifically rad-hardened, but get beyond the heliosphere and you're likely to get hit by a few very high energy particles. It'd be interesting to know if that's what happened here or if it's just the hardware showing its age.

Fan-bleeding-tastic work!

Michael H.F. Wilkinson

I remember watching the Voyagers being launched on TV decades ago, and avidly followed their progress past the outer planets and beyond. BIG thumbs up to the brilliant people at NASA for fixing this.

Just look at their whiteboard!

TJ1

No DevOps here - maturity and hundreds of years of experience!

Take a look at the scribbles on their whiteboard - looks like they have the signal clock (SC freq) and data bit pulse width calculations along with some amplitude compression recovery.

https://www.jpl.nasa.gov/news/nasas-voyager-1-resumes-sending-engineering-updates-to-earth

Nice to see them putting the ACE back in hACkEr !

The connection between the language in which we think/program and the problems
and solutions we can imagine is very close. For this reason restricting
language features with the intent of eliminating programmer errors is at best
dangerous.
-- Bjarne Stroustrup in "The C++ Programming Language"