Tick-tock, Facebook: Not a reference to that short vid horsepuckey but a literal open-source timekeeper
- Reference: 1628698147
- News link: https://www.theregister.co.uk/2021/08/11/facebook_clock/
- Source link:
Basically, it's a PCI Express card that uses a Global Navigation Satellite System (GNSS) and oscillator to provide an authoritative source of time. Stick the card into a vacant slot and hey presto – an ordinary server can become a time appliance.
Timelords
Facebook has been working on the problem for a while. While users might sometimes wonder at the seeming randomness of their timeline, keeping server time synchronised and accurate is important when it comes to infrastructure management and distributed databases.
Back in March 2020 the social media giant confirmed it was in the process of switching over to a new timekeeping service based on the Network Time Protocol (NTP.) The update, according to Facebook, increased the accuracy of timekeeping in its infrastructure from 10 milliseconds to 100 microseconds.
However, the new architecture uses a Stratum 1. This component is linked to a source of time such as a [3]GNSS or a caesium clock . Facebook's idea is that, rather than depend on something like time.facebook.com for their Stratum 1, which is an one that might suffer a wobble if connectivity were lost, how about companies use a Time Appliance instead?
[4]
The approach is hardly new, and such appliances have been available off-the-shelf for years. Problems, however, include a distinct lean toward a proprietary world which, in turn, can make things expensive and more difficult to support.
[5]Open Compute Project to design open silicon and optics in Strategy 2.0
[6]Facebook takes bold stance on privacy – of its ads: Independent transparency research blocked
[7]Er, no, we would like to continue suing Facebook, US state AGs tell courts
[8]Facebook tries to save face by recalling itch-inducing Oculus Quest 2 VR headset foam
Thus Facebook has designed and open sourced a Time Card. The PCI Express card includes a GNSS receiver to provide Time of Day (ToD) and one pulse per second (PPS.) An oscillator (such as an atomic clock) backs things up in the event the GNSS signal is lost and an off-the-shelf network card (the NVIDIA Mellanox ConnectX-6 Dx was Facebook's initial choice) sends the time out over the network.
It is undoubtedly a neat bit of tech and free from vendor lock-in. The specifications, source code and schematics can all be found on GitHub should one wish to roll one's own. The driver for the Time Card lurks in the Linux kernel from version 5.15, or can be built on 5.12 or later.
[9]
Hardware-wise, pretty much any PC will do the job so long as it has enough spare PCIe slots and we reckon somebody handy with a soldering iron, the necessary components and access to PCB printing could make a decent fist of making their own Time Appliance for considerably less than buying one off-the-shelf.
A weekend project, perhaps? It'd certainly be an improvement than sitting around and, er, scrolling through Facebook. ®
Get our [10]Tech Resources
[1] https://www.theregister.com/2021/08/04/facebook_research_lockdown/
[2] https://engineering.fb.com/2021/08/11/open-source/time-appliance/
[3] https://www.theregister.com/2019/12/03/register_lecture_times_up_for_gps_atomic_clocks_to_the_rescue/
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/servers&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2YRRInjgWqjzWbQ3A6hp8eQAAAAM&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[5] https://www.theregister.com/2021/08/09/open_compute_strategy_2/
[6] https://www.theregister.com/2021/08/04/facebook_research_lockdown/
[7] https://www.theregister.com/2021/07/28/states_facebook_antitrust/
[8] https://www.theregister.com/2021/07/27/facebook_recalls_oculus_quest/
[9] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/servers&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YRRInjgWqjzWbQ3A6hp8eQAAAAM&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[10] https://whitepapers.theregister.com/
You mock, but rubidium boxes with plenty of life left are only £150 or so, it's unlikely to be the limiting factor here.
www.ebay.co.uk/itm/224319541159
(The CSAC module that this design uses is spiffy, being small, low power an decent spec, but doesn't seem to be purchasable. That might just be an export control thing, mind)
Just the ticket!
Ideal for the datacenter, with its clear view of the sky.
Why?
FB time appliance? Will FB slurp user data via a NTP protocol?
Re: Why?
As you say, why?
Even Microsoft, finally uses NTP for it's clock.
This makes no sense, much like FaceBook.
So they've opened the source code, except for the interesting bit.
Oh, interesting. I'd love to see how they did the fpga, I've a bit of a time-nuts thing going on here, and the published specs look good.
I've got plenty of suitable FPGA boards, GNSS boards and a handy rubidium oscillator, fancy having a go. Somewhat disappointed that there's a bitstream file, and that's it.
To be slightly fair, there's a bit of pdf documentation of the FPGA, but that's pushing 'open source' beyond breaking point.
Oh well. Teasers.
Re: So they've opened the source code, except for the interesting bit.
Wouldn't FPGA have more jitter than a bit of old 74xx and a good crystal?
Re: So they've opened the source code, except for the interesting bit.
Good grief, no. FPGAs are quite good at this sort of thing. For a start, you can clock them faster than the edge speed of a 74xx, and you can build tunable tapped delay lines by stringing logic carefully in straight lines. Single nanoseconds are fair game, better with effort. These are all things I was looking forward to seeing how they'd implemented.
Rubidium oscillators are a bit jittery at this level, a good quartz oscillator is less jittery but less stable longer term.
I suppose it depends how accurate you need to be. For most people, just synchronising the OS's clock to NTP every now and then will be perfectly sufficient, but there must be further use-cases. When I worked in ILR, we only needed accuracy to (say) half a second, in order to be able to synchronise between local and network content (put the fader up for IRN at the right moment), but to do so we had a receiver in the racks which synchronised to Rugby or DSF, that receiver itself had a laser-trimmed quartz crystal in case of loss of the LW signals, and it sent out pulses over a serial link to wall clocks in the studios, each of which had a standard (if fairly high-quality) quartz movement which could certainly keep them close enough to Rugby for several hours - possibly days, given how stable the studio temperatures were - just in case the serial link went down. No GPS (or at least, no civilian GPS) in those days, but even if there had been I suspect we'd have had a similar fail-safe cascade.
As for the Time Appliance, I like the casual throw-away phrase used for backup:
An oscillator (such as an atomic clock) backs things up
Yeah, I have a couple of those in a box in the attic somewhere, gimme a moment...
M.