[Help] Configuring Headscale server and Tailscale clients

Hi!

I'm trying to set up Headscale (as a self-hosted Tailnet control server) on my desktop and Tailscale clients on all my computers (including my desktop) so that I can SSH to any of my machines regardless of what network I'm on.

I've configured the Headscale server on my desktop but I'm not sure if I've done it right because I can't successfully run the Tailscale client (also on my desktop).

Apologies in advance: I have very shallow understanding of networking: I vaguely know about IP addresses, domain names, DNS, VPNs, etc. but I have no clue about how things work under the hood, so I'll provide as many details though they might be irrelevant.

I have a domain name (anonymized to example.com) purchased at Namecheap and I've set up dynamic DNS using ddclient:

{...}: {
  services.ddclient = {
    enable = true;
    protocol = "namecheap";
    username = "example.com";
    passwordFile = "/path/to/secret";
    server = "dynamicdns.park-your-domain.com";
    use = "web, web=dynamicdns.park-your-domain.com/getip";
    domains = [
      "@.example.com"
    ];
  };
}

When I check my public IP using e.g. https://whatismyipaddress.com/, the IP address matches the one reported by Namecheap for that domain (example.com), so I assume dynamic DNS is working.

Here's my Headscale configuration:

let
  domain = "example.com";
in {
  services.headscale = {
    enable = true;
    address = "0.0.0.0";
    port = 8080;
    user = "my-username";
    settings = {
      server_url = "https://headscale.${domain}";
      dns_config = {
        base_domain = domain;
        magic_dns = true;
        domains = ["headscale.${domain}"];
        nameservers = [
          "1.1.1.1"
          "9.9.9.9"
        ];
      };
      ip_prefixes = [
        "100.64.0.0/10"
      ];
      logtail.enabled = false;
    };
  };

  # DERP port (https://tailscale.com/kb/1082/firewall-ports)
  networking.firewall.allowedUDPPorts = [3478];
}

Here's my Tailscale configuration:

{config, ...}: {
  services.tailscale = {
    enable = true;
    openFirewall = true;
    useRoutingFeatures = "client";
  };
  networking.firewall.trustedInterfaces = [config.services.tailscale.interfaceName];
}

I added a Headscale user by running headscale users create my-username and now headscale users list outputs:

2024-04-14T02:17:02-04:00 WRN Warning: dns_config.domains is set, but no nameservers are configured. Ignoring domains.
2024-04-14T02:17:02-04:00 WRN Warning: dns_config.domains is set, but no nameservers are configured. Ignoring domains.
ID | Name         | Created
1  | my-username  | 2024-04-14 04:41:28

I'm not sure why I'm getting that warning. I can successfully ping example.com...

1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 2.160/2.160/2.160/0.000 m

...but I can't ping headscale.example.com:

ping: headscale.example.com: Name or service not known

When I try to run tailscale up --login-server https://headscale.example.com --operator my-username, it gets stuck and there's no output. If I run journalctl -u tailscaled.service, I get:

Apr 14 02:22:47 desktop tailscaled[2663]: control: LoginInteractive -> regen=true
Apr 14 02:22:47 desktop tailscaled[2663]: control: doLogin(regen=true, hasUrl=false)
Apr 14 02:22:47 desktop tailscaled[2663]: [RATELIMIT] format("control: trying bootstrapDNS(%q, %q) for %q ...") (8 dropped)
Apr 14 02:22:47 desktop tailscaled[2663]: control: trying bootstrapDNS("derp7.tailscale.com", "167.179.89.145") for "headscale.example.com" ...
Apr 14 02:22:47 desktop tailscaled[2663]: control: trying bootstrapDNS("derp3.tailscale.com", "2400:6180:0:d1::67d:8001") for "headscale.example.com" ...
Apr 14 02:22:50 desktop tailscaled[2663]: control: trying bootstrapDNS("derp1d.tailscale.com", "165.22.33.71") for "headscale.example.com" ...
Apr 14 02:22:50 desktop tailscaled[2663]: control: trying bootstrapDNS("derp5.tailscale.com", "2001:19f0:5801:10b7:5400:2ff:feaa:284c") for "headscale.example.com" ...
Apr 14 02:22:51 desktop tailscaled[2663]: control: trying bootstrapDNS("derp4d.tailscale.com", "134.122.94.167") for "headscale.example.com" ...
Apr 14 02:22:51 desktop tailscaled[2663]: control: trying bootstrapDNS("derp1c.tailscale.com", "2604:a880:800:10::7a0:e001") for "headscale.example.com" ...
Apr 14 02:22:51 desktop tailscaled[2663]: [RATELIMIT] format("control: trying bootstrapDNS(%q, %q) for %q ...")
Apr 14 02:22:53 desktop tailscaled[2663]: Received error: fetch control key: Get "https://headscale.example.com/key?v=88": failed to resolve "headscale.example.com": no DNS fallback candidates rema

Is there anything obviously wrong with my configuration?

Any troubleshooting advice/pointers to relevant documentation would be appreciated. I've spent several hours trying to debug this with my limited knowledge but at this point I'd rather ask an expert :)

Thanks!

EDIT: on the advice of u/bubblegumpuma, I've added a reverse proxy:

services.caddy = {
    enable = true;
    virtualHosts."headscale.${domain}".extraConfig = ''
      reverse_proxy * 127.0.0.1:${toString port}
    '';
  };

(I've also made headscale and caddy both run as the root user)

Unfortunately, I'm still running into the same issue.