I joined the NTP Pool, but not with my Stratum 1 Pi

(jeffgeerling.com)

88 points | by fortran77 5 hours ago

7 comments

  • Avamander 2 hours ago
    For the price and complexity of building a good Linux-based time server I'd heavily recommend looking at FPGA-based solutions instead. x86 platforms also have PCIe PTM and TGPIO options available[0], which are tad better than the common ARM equivalents and GPIO interrupts.

    In general Pi-s are relatively bad for timekeeping. Not just the NIC (which can be replaced by hats with their own), there are also oscillator replacement mods[2] to work around this to some extent. Though all this still leaves many other sources of timekeeping error.

    So if the goal is low-cost precise timekeeping, one can assemble something FPGA-based[2] for quite a bit less money, that can reach very low noise figures. You can spend the remaining money on the GNSS antenna and receiver instead.

    I'm also sure a FPGA-based approach could be taken even further with some external timekeeping chips available and the FPGA could just do the NTP/PTP protocol stuff.

    [0]: https://www.youtube.com/watch?v=zxK0_j2Ah2I [1]: https://raspberrypi.stackexchange.com/questions/74482/switch... [2]: https://fy.chalmers.se/~f96hajo/fakernet/board_Arty_A7-35-GP...

    • jeffbee 2 hours ago
      Back when I thought this was true, I put together an x86 platform with PTM and a fancy NIC and it would hard-lock the bus to such an extent that I had to physically unplug it to recover. There are a lot of pieces required for a complete solution and I'd caution against a categorical statement like "x86 has PTM".
      • Avamander 1 hour ago
        As I said, tad better, not perfect. I'd say finding reasonably-priced x86 hardware with that functionality is an even bigger hurdle.
        • geerlingguy 45 minutes ago
          It all depends on the gains you're looking for. Almost any solution with PPS input can get you under 1µs, even on Linux through GPIO.

          If you want to chase lower, you need a hardware-disciplined clock. Either through FPGA, or in the case of my Time Pi, I have a PTP NIC with a PHC disciplined from the ublox F9T—which should be within 5-20ns of GPS time. This can be measured on higher-end boards like the TimeHAT by comparing PPS out from the GNSS module, PPS from the NIC, and PPS from a separate GPSDO, along with a timestamp comparison tool like the tinyGTC or TAPR TICC: https://tapr.org/product/tapr-ticc/

          The annoying thing is there are few PC hardware combos that include the relevant PPS and frequency outputs to allow actual verification of your time in hardware. Software can show numbers but even on my Pi dashboard, the sub-20ns sync reported is between the system clock and the NIC PHC... but not currently accounting for the delay when the kernel reads the PHC over PCIe.

          Pi could support PTM on its PCIe bus, that would be awesome. Overkill for 99.9% of its market though!

  • lightswitch05 2 hours ago
    Great write up! I too run a stratum 1 and am part of the pool. Except, I have way less interesting hardware. One benefit on going cheap is having multiple stratum 1s then you can have some redundancy and have more interesting local peers to compare results against. I do see a more expensive setup in my future, though.

    Anyways, I use NTPSec- which is the default these days on Debian when you apt-get install NTP. I’ve not tried Chrony, but I see it mentioned all the time.

    I’m curious what others here are running and their thoughts on the pros and cons?

    Edit: I forgot to add, one of my favorite parts of being in the pool is basically a free uptime monitor of my ISP. There have been multiple cases where I see a blip in my offset then a full blown ISP outage within 78 hours of the blip, then a return to my normal offsets after to outage has resolved.

    • geerlingguy 2 hours ago
      In the NTP Pool community, I've seen a few people mention CenterClick NTP servers; I'm happy to do my own DIY builds (Pi or otherwise), but this looks like a nice option for SMBs or time-nuts who don't want to manage their own hardware: https://centerclick.com/ntp/

      There are also a lot of people who buy old Spectracom or other 'enterprise' time servers since they can be had for low hundreds of dollars with some nice features like rubidium or OCXO oscillators, so they can hold over time much more accurately when GPS is down, or upstream NTP providers are unreachable.

      And regarding chrony, it's the default in many distros now, I like some of its built-in monitoring and clock adjustments, it just feels easier to manage if you want more than just base NTP features. It also supports hardware timestamping if you have network hardware that has it, which IIRC NTPd doesn't? I haven't messed with systemd-timesyncd or ntpd-rs, so can't say much on them.

      • unsnap_biceps 1 hour ago
        I have around a dozen CenterClick 270s at various locations and I think generally I have one lock up every 6 months or so. A PoE power cycle on the switch recovers it, so overall I've been really happy with them. I believe they're a one-person shop.
        • cinntaile 34 minutes ago
          What do you use them for?
    • nahikoa 2 hours ago
      Some of the cool kids are moving to https://github.com/pendulum-project/ntpd-rs
    • Palomides 2 hours ago
      I think most people run chrony or ntpd-rs; not part of the pool, but I've been using chrony for a while, and it seems like ntpd-rs's support for PTP (which I "need") is still under development
  • bityard 1 hour ago
    I thought I remembered reading that they did not allow VPSes to be part of the pool because of the historical issues around VMs and timing, combined with the fact that the VPS operator could make host-level changes at any time that could impact the accuracy of the host. But now I'm not finding anything about that. Maybe they have enough monitoring and automation that running nptd on bare metal is not a strict requirement anymore?
    • walrus01 1 hour ago
      I don't recall the details but one of the historical problems with VPSes is that people were trying to use things like openvz VPSes which are more like a glorified chroot/jail, rather than something like a QEMU/KVM VPS (or Xen PVH, or Xen HVM with virtualization extensions).

      I still personally wouldn't try to run an NTP server on a VM unless I also owned/controlled the bare metal hardware the hypervisor is running on.

      • geerlingguy 52 minutes ago
        KVM allows chrony to adjust the clock, which give much better stability. With other VMs you can have a very jumpy clock and poor time service.

        Some decent discussion on this topic (though with broken links to a separate NTP Pool discussion) here: https://a.osmarks.net/content/serverfault.com_en_all_2022-11...

        • walrus01 47 minutes ago
          I'm not sure if I would call answers from 2016 'recent' as KVM linux-on-linux has advanced significantly in the years since then, particularly with improvements focused on performance in something like a 6.1.x kernel in a theoretical scenario like a debian trixie host (proxmox 9.2 or whatever) and a trixie or fedora 44/45 guest.
          • geerlingguy 27 minutes ago
            Touche, but the main point is KVM is fine for NTP timing. I don't think I'd try to virtualize a PTP server, that still needs to be closer to the physical hardware.
  • omoikane 2 hours ago
    > https://www.jeffgeerling.com/blog/2026/i-joined-the-ntp-pool...

    I wonder why the wall clock on top is 10 seconds ahead of the bottom one?

    • geerlingguy 57 minutes ago
      Bottom one is NTP, updating I think once or twice a day. The top one is using WWVB radio time, but my studio is a little bit of a faraday cage, so to get it to sync I have to bring it near the front door and point the back toward the West overnight (STL to Ft. Collins).

      I have a solution to that I'm working on—Pico W that grabs time from NTP then broadcasts about 1m with a tiny antenna so I can set my wall clock and watch (the one with the green band in the picture).

    • undersuit 1 hour ago
      Relativity. /s
  • zodwick 1 hour ago
    One of those posts which truly hits you in the face and calls you an idiot for how much you don't know. Definitely a rabbit hole I'm going down this week. Thanks
  • jeffrallen 2 hours ago
    PSA: The Swiss government has NTP servers on ntp{11,12,13}.metas.ch.

    METAS is the Federal Institute of Metrology.

  • aaronax 2 hours ago
    What speed did you choose in the pool dashboard? How many request per second are you serving, and do you have a goal for that?
    • geerlingguy 2 hours ago
      I started at 25 Mbit to see how things would go, and found the Oxtrys VPS I have the public server running on could handle that without issue.

      Over the past few weeks I bumped it up to 50, then 100, and now it's at 250 Mbit. At 100 Mbit I was seeing 180-250 requests/sec. At 250 (just switched earlier this morning) I'm seeing 500-600 requests/sec.

      I'm monitoring things on the server with chronytop[1], a tool I whipped up with Claude since I didn't see any other concise dashboards and hated `watch`ing chronyc sources to get any kind of updated data.

      [1] https://github.com/geerlingguy/chronytop