Former NSA cyberspy's not-so-secret hobby: Hacking Christmas lights
- Reference: 1735133233
- News link: https://www.theregister.co.uk/2024/12/25/joyce_christmas_lights/
- Source link:
As the former head of the NSA's Tailored Access Operations squad – the people who crack systems and gather intelligence for the US government – Joyce was also the friendly public face of the agency. The agency didn't come out of the Edward Snowden affair with a great reputation when the ex-NSA contractor-turned-whistleblower made public the existence of the [1]NSA data collection programs back in [2]June 2013 . Many in the security industry were peeved at the agency's disregard for privacy and the accepted norms under which people assumed it operated.
Joyce was part of a campaign to make the NSA acceptable again, and he was doing a good job. We covered [3]his talk at the first Enigma security conference, and it was unusually frank – most talks by agency personnel are about as exciting as watching paint dry.
[4]
But the Shmoocon talk (see below) was a personal matter. It turns out Joyce is a big fan of Christmas, and of hacking the seasonal strings of lights that hang off so many American houses during the holiday season. As you can see from the video below, he's serious about it and applies all the rigor normally used to break into adversaries' networks to put on a show.
[5]
[6]
[7]Youtube Video
Next year, at the last Shmoocon, Joyce will reprise his talk and offer updates on how to turn your house into a light show that will either delight, or irritate, the neighbors. He spoke to The Register ahead of time to explain the hobby.
[8]
The Register : The first and most obvious question: why? What got you into this?
Joyce : We had a family tradition where we drove around looking at Christmas lights on Christmas Eve with the kids. And we passed a house only a few miles from us that had computer-controlled LEDs synchronized to music and a little radio station you tuned to. And it just captivated me.
It was not elegant, right? It was over the top and gaudy, and just really made me happy. I said "I think I could do that," meaning I have the technical chops to achieve it. And [Joyce's wife] said, "yes you can," and I took that as license to mean, "yes, you can do it." And so when boxes started arriving in the mail in February and March, she's like, "what the hell is this?"
[9]
The Register : In terms of work time, how long does a setup like this take?
Joyce : The startup was enormous if you've seen the images. I could cover the house and put things in the yard, and there are a bunch of different props that are created. If I knew how much work it was I would not have started.
But now the incremental work is not so bad. It takes me about three and a half days to get things set up. I hope for a weather window, and the first day I start is somewhere right before Thanksgiving because the first thing I do is the very top of the roof, up on the outside of the house. The second day is the next layer of things on the house, and then after that, if we have bad weather, I can do things on the ground and around the yard without much concern – but I don't want to be on the roof in bad weather and wind.
[10]Youtube Video
The Register : A senior person in the NSA ordering huge amounts of electronic equipment from China didn't set off any red flags?
Joyce : None of the compute comes from China, just the LED strings themselves. I would applaud somebody if they could supply chain that.
I do take a little more care in the control system itself. It's not connected to the internet and is a standalone network – because I do have friends who have interesting hobbies and would love to change my display and make it say some interesting things.
The Register : You've said that the things that impress many people aren't the lights themselves but the radio station that synchronizes with them. How's that set up?
Joyce : It's probably on the hairy edge of what the FCC would approve, but I make sure it's not interfering with anything in the neighborhood. It's probably the least technical part of the whole creation. You plug an audio signal in and it just spits it out as a radio channel. I did custom make the antenna to make sure that there's good reception.
The Register : What about the software, do you code it yourself?
Joyce : I don't write the software. I use an open source product called xLights that will let you map the songs to your display. And there's a group of people out there who write that software and improve it constantly. I'm very grateful to them and donate to their project every year to help keep them going. But much like me, they do it as a labor of love.
Then there's some software that runs on Raspberry Pis that controls the flat panel displays that are hung on the outside of the house, and also pipes all the pixel commands in real time out to all the lights across the whole show. That also is open source and part of this hobbyist community.
[11]Take a former NSA head hacker, a Raspberry Pi, weird Kiwi radios and what do you get?
[12]Asda hits the brakes on tech tweaks to avoid festive fiasco
[13]Will Windows Insiders find Recall lurking under the Christmas tree?
[14]NSA's top hacking boss explains how to protect your network from his attack squads
The Register : Is the Christmas lights hobbyist community that large?
Joyce : Well, every year xLights put out a song and they put out the light sequence to it, and then all these people map them to their display and record their house. Then they cut together a video of all of these different houses doing the same song. And there's thousands of people that do that. It got so big. I did it for a couple of years, but I stopped doing it just because it's so big.
The Register : Are you still using the same controller board?
Joyce : No, they're out in the cold, and they get a lot of thermal cycling. That makes them a little flaky. The technology has advanced now. Now it's all surface mount technology, and they include better debugging to help you figure out where your issues are. And there's a Raspberry Pi as the main controller.
The Register : And for connections – wired or wireless?
Joyce : I still do Ethernet. You know, wireless introduces delays and I worry about interference and the signal. Also I still don't want the mischievous friends coming over and messing with my wireless. They can roll up with a cable of Ethernet and jack into the local network there, but the security cameras outside would spot them.
[15]Youtube Video
The Register : What do the neighbors think of all this?
Joyce : I'm trying to keep my neighbors reasonably happy. They all enjoy the light show to an extent, but these days they can't get in their driveway because the traffic is backed up. You know, that makes for unhappy neighbors. I had one family and was getting the vibes that they weren't that crazy about it, but they've had children and the grandchildren love it, and I think it reminded them of the joy it brings to other visitors.
When there are low lying clouds, you can be a mile away from the house and you'll see the clouds change color, so it's always going to be nice thinking, "Yeah, I did that."
The Register : What advice would you give to people that want to get into this?
Joyce : Have an understanding partner.
Seriously, go to the xLights website. There's a forum there that will link you to the YouTube community. There are great tutorials on YouTube, and there's a bunch of very active Facebook groups, and on the xLights site you'll find something people refer to as the xLights Zoom Room. It's a group of people who volunteer their time and they sit on Zoom calls and help people work through the technical issues they have.
The Register : Can you give us a rough idea of what the setup on something like this would cost?
Joyce : Oh no, I'm from NSA. I know how to keep a secret.
Joyce also has a [16]Twitter account devoted to his hobby. For security (and traffic) reasons, we won't be publishing his physical address. ®
It's spreading to Europe …
American-style Christmas lights aren't as popular in Europe, but they're catching on, especially as hackers get in on the game.
The Register spoke to a British security operations center manager and volunteer admin for xLights, who developed a passion for fiddling with lighting after seeing the 2018 talk, and there are about 1,300 enthusiasts in the scene in Europe. This year's spectacle [17]looks superb , but there are issues that Joyce doesn't have to deal with.
Regulation is a problem, Paul Glavin explained. As of next year, US manufacturers might not be able to sell the control systems for such lighting displays in the UK. So a grassroots startup [18]Buildalightshow is building its own control boards to brighten up the Solstice night.
"We now have a UK-based board - it's called a [19]Baldrik board. The network that they operate on is called the turnip network. It's just that kind of humor behind it [Ed: This is a Blackadder reference, as they have a cunning plan]. So it just gives you a bit of a giggle."
And while Joyce can broadcast Christmas carols via radio, that's not really allowed in the UK, so Glavin has speakers on the lawn instead. As a result, he only runs the lights for an hour each day to avoid being impolite to the neighbors and to cut down on traffic, but the project raises money for local school meals.
"The main reason I do it is the love others have for it. I've just come in from the 17:30 showing, which had kids dancing and singing along to the songs," he said on Monday. Given last year's [20]soundtrack , that's understandable.
"I had parents telling me this is their x year seeing the lights and this was the only thing their daughter wanted to see this Christmas, other families have mentioned this year that it's their family tradition; so no matter how hard it gets to put the show on, how much I hate being up a ladder, how much I curse whichever storm has ripped through the garden and damaged yet more props, or more LEDs have failed, I have to keep it going for the local community."
Get our [21]Tech Resources
[1] https://www.theregister.com/2014/08/26/new_snowden_leak_nsa_massively_expanded_metadata_sharing_in_200607/
[2] https://www.theregister.com/2013/06/14/companies_share_data_with_spies/
[3] https://www.theregister.com/2016/01/28/nsas_top_hacking_boss_explains_how_to_protect_your_network_from_his_minions/
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_security/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2Z26eWSqfLBQIO550D__vkgAAAQ4&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_security/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Z26eWSqfLBQIO550D__vkgAAAQ4&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[6] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_security/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33Z26eWSqfLBQIO550D__vkgAAAQ4&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[7] https://www.youtube.com/watch?v=VjYejkIKpBU
[8] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_security/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Z26eWSqfLBQIO550D__vkgAAAQ4&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[9] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_security/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33Z26eWSqfLBQIO550D__vkgAAAQ4&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[10] https://vimeo.com/898763905
[11] https://www.theregister.com/2018/01/22/rob_joyce_hacking/
[12] https://www.theregister.com/2024/12/03/asda_festive_system_freeze/
[13] https://www.theregister.com/2024/11/12/recall_windows_insiders_december/
[14] https://www.theregister.com/2016/01/28/nsas_top_hacking_boss_explains_how_to_protect_your_network_from_his_minions/
[15] https://vimeo.com/898770960
[16] https://x.com/RGB_Lights?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Eauthor
[17] https://www.youtube.com/watch?v=YlsMzeT9-FU
[18] https://buildalightshow.com/
[19] https://en.wikipedia.org/wiki/Baldrick
[20] https://www.youtube.com/watch?v=KyIWrWqK0Os
[21] https://whitepapers.theregister.com/
Re: "that's not really allowed in the UK"
He could stream it on an open wi-fi network!
Re: "that's not really allowed in the UK"
Bluetooth! (for the greater technical & usability challenge)
Re: "that's not really allowed in the UK"
The Christmas neighbour from hell.
Best joke ever :D
" I'm from NSA. I know how to keep a secret. " Snowden anyone ?
Re: Best joke ever :D
«" I'm from NSA. I know how to keep a secret. " Snowden anyone ?»
Knowing "how to" and "doing so" are fortunately not quite the same thing.
Not sure whether the chap is daft or just eccentric in the best British tradition but at least mostly harmless and doubtless brings enormous Yuletide joy and delight to the young and not so young of his community
The lavish house decoration with Xmas lights is fairly recent in AU probably occurred only after the advent of cheap led lights. Those with incandescent bulbs were often wired in series and plugged directly into the 240VAC mains so not really for outdoor use. The ones running off transformers 24V (?) were quite expensive I think.
I am surprised that a manufacturer hasn't incorporated a controller/interface for a low speed serial protocol like 1-wire into the leds and power supply which could make these dynamic displays a lot easier to implement.
They have
The most popular is probably the WS2812B chip, which is an RGB LED with a built-in driver.
The chain is basically a shift register, data is clocked in at one end and each chip extracts the three bytes it needs, then passing the rest on regenerating the signal.
The trouble with these is that the protocol is very susceptible to noise, so runs between chips have to be kept very short, perhaps five metres at most. So the data run from the controller to the base of a tree is usually ANSI E1.11 or E1.31, then a protocol convertor.
There's a variety of similar chips using the same or similar protocols.
I have this mental image
of a Chinese manufacturer being teased by his colleagues because all he makes is LEDs, and they only sell at ten for a cent... ah but, he replies, every year I see a hundred bilion...
This is my festive, ice-covered house.
I call it my Snow-Den.
When the thaw comes, Snow-Den leaks.
Can != May != Should
I said "I think I could do that," meaning I have the technical chops to achieve it. And [Joyce's wife] said, "yes you can," and I took that as license to mean, "yes, you can do it."
And that's the root of the data slurping explained.
"Hacking" Christmas lights
Not the story I expected it to be. I thought the NSA folk were actually "hacking" Christmas lights. Setting some rando's lights to rapid strobe level flashing, or something devious like that. I did not expect a legit story about setting up synchronized Christmas lights.
Re: "Hacking" Christmas lights
It's absolutely trivial.
These systems are close relatives of SCADA - wide open, with no electronic security whatsoever - which is why they don't use Wifi or connect it to the Internet.
Some might find it surprising that home enthusiasts have a greater understanding of the limitations of their systems.
Re: "Hacking" Christmas lights
It being the NSA, I was envisioning more along the lines of planting bugged bulbs on the nation's Christmas trees, in hopes of picking up secrets divulged in less-than-sober moments.
Check Out
Dave Plummer on YouTube, he loves doing this stuff.
Warning: He's an EX-Microsoft hack, but got out before the turd-chase started there.
Just stop flashing!!!
I would make do with just being able to tell my cheap chinese Christmas lights strings (for my Xmas tree, outdoor trees, draped over the garage, etc) to come on without bloody flashing !!
They all seem to be similar - they have a controller built into the power plug and (mostly) have a remote control. Either switch can be used to cycle through a variety of annoying patterns. But there is no way to set the startup to be my desired pattern (actually just on - no flashing, fixed colour).
I would be happy to have something that could either replace the power supply/controller or the remote control. Including a timer (so I could replace the mains timer I use for each string) would be even better.
Anyone know of any such gear?
Re: Just stop flashing!!!
Pop the lid and it'll probably be obvious how to disable the flashy.
Re: Just stop flashing!!!
May be overly obvious - but most lights manufactured in the last few years have a built-in timer. Turn them on at,say, 4pm, choose your wave/flash/steady pattern and then just leave to their own devices. They'll turn themselves off after 8 hours and come back on after 16 hours - ie 4pm - with the settings you selected earlier.
Been used as a trope in a Charlie Higgson novel
Where the hero uses them as trojan horse into the villains network. * I didn't think it was real.
Kind of looks like it isn't.
*Interesting plot. Bad people doing bad things for good motives. Bad people finding out there are worse people. Good people doing bad things for good motives.
"that's not really allowed in the UK"
No kidding. 98.5 FM - that Rob uses - is smack in the middle of Radio 1's fat range. You'll have an angry Greg James at your door if you smear your Xmas songs all over his Xmas songs.
Appreciate the interview. This is one of those hobbies I enjoy other people spending vast amounts of time on :)