blog: detail my next-generation wireguard setup (#317)

* blog: detail my next-generation wireguard setup

Signed-off-by: Christine Dodrill <me@christine.website>

* blog/my-wireguard-setup: is it a good idea to microwave this?

Signed-off-by: Christine Dodrill <me@christine.website>

1 files changed, 212 insertions, 0 deletions

diff --git a/blog/my-wireguard-setup-2021-02-06.markdown b/blog/my-wireguard-setup-2021-02-06.markdown
new file mode 100644
index 0000000..8d6fb04
--- /dev/null
+++ b/blog/my-wireguard-setup-2021-02-06.markdown
@@ -0,0 +1,212 @@
+---
+title: "My Automagic NixOS Wireguard Setup"
+date: 2021-02-06
+tags:
+ - wireguard
+ - nixos
+ - tailscale
+---
+
+# My Automagic NixOS WireGuard Setup
+
+It's been a while since I went into detail about how my [Site to Site
+Wireguard](/blog/series/site-to-site-wireguard) setup works. I've had a lot of
+time to think about how I can improve it since then, and I think I've come to a
+new setup that I'm happy with. I've replaced all of the manual setup,
+copying/pasting and more with a unified [network metadata
+file](https://github.com/Xe/nixos-configs/blob/master/ops/metadata/hosts.toml)
+and some generators that consume it. Here's my logic, influences and the details
+about how I implemented it.
+
+When I worked at [IMVU](https://secure.imvu.com/) one of the most useful
+services was the asset database. This database ended up being a giant bag of
+state that a lot of the other SRE services consumed. This was used by the
+machine provisioner, DHCP server and the configuration management. My personal
+infrastructure isn't quite big enough yet to justify setting up a whole database
+for tracking it all, however I think I have a happy middle path with a file
+called `hosts.toml`.
+
+At a high level it contains the following information:
+
+- IP subnets that I use across my infrastructure
+- Descriptions of the logical subnets they fall into (usually based on physical
+  location with a few special exceptions)
+- Host information including SSH/wireguard pubkeys
+
+Here's a random host description from `hosts.toml`:
+
+```toml
+[hosts.shachi]
+network = "hexagone"
+ip_addr = "192.168.0.177"
+ssh_pubkey = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIL3Jt26HXD7mLNjg+B+pB5+fXtxEmMeR6Bqv1Z5/819n"
+
+[hosts.shachi.wireguard]
+pubkey = "S8XgS18Z8xiKwed6wu9FE/JEp1a/tFRemSgfUl3JPFw="
+port = 51820
+addrs = { v4 = "10.77.2.8", v6 = "ed22:a601:31ef:e676:e9bd" }
+```
+
+This includes enough information for me to do the following things:
+
+- Set up prometheus monitoring probes
+- Send this host [encrypted secrets using its SSH host
+  key](/blog/nixos-encrypted-secrets-2021-01-20)
+- Configure the Wireguard tunnel that my machines use to talk to eachother
+
+I also have two functions that generate [peer
+configs](https://search.nixos.org/options?channel=20.09&from=0&size=50&sort=relevance&query=networking.wireguard.interfaces.%3Cname%3E.peers)
+from this metadata,
+[roamPeer](https://github.com/Xe/nixos-configs/blob/master/ops/metadata/peers.nix#L5-L17)
+and
+[serverPeer](https://github.com/Xe/nixos-configs/blob/master/ops/metadata/peers.nix#L18-L33).
+
+The main difference between these two functions is that serverPeer allows me to
+tell the target machine to actively reach out to the peer, whereas roamPeer sets
+up config for the other end to connect to that peer. This allows me to stick a
+machine behind a NAT firewall and still have it connect to the network.
+
+I have two main peerlists based on the location of the machine in question:
+
+```nix
+# expected peer lists
+hexagone = [
+  # cloud
+  (serverPeer lufta)
+  (serverPeer firgu)
+  (serverPeer kahless)
+  # hexagone
+  (serverPeer chrysalis)
+  (serverPeer keanu)
+  (serverPeer shachi)
+  (serverPeer genza)
+];
+
+cloud = [
+  # cloud
+  (serverPeer lufta)
+  (serverPeer firgu)
+  (serverPeer kahless)
+  # hexagone
+  (roamPeer chrysalis)
+  (roamPeer keanu)
+  (roamPeer shachi)
+  (roamPeer genza)
+];
+```
+
+Inside `hexagone`, all of the machines can freely contact eachother. These IP
+addresses aren't very useful for cloud servers, so those servers get a roaming
+peer config.
+
+Now that I have these peer lists all I need to do is generate the base Wireguard
+config for that machine. At a minimum we need to set the following:
+
+- IP addresses
+- The Wireguard private key location
+- The Wireguard listen port
+- The list of peers
+
+So we do this in the very imaginatively named function `interfaceInfo`:
+
+```nix
+interfaceInfo = { network, wireguard, ... }:
+  peers:
+  let
+    net = metadata.networks."${network}";
+    v6subnet = net.ula;
+  in {
+    ips = [
+      "${metadata.common.ula}:${wireguard.addrs.v6}/128"
+      "${metadata.common.gua}:${wireguard.addrs.v6}/128"
+      "${wireguard.addrs.v4}/32"
+    ];
+    privateKeyFile = "/root/wireguard-keys/private";
+    listenPort = wireguard.port;
+    inherit peers;
+  };
+```
+
+`interfaceInfo` takes host information from `hosts.toml` and combines it with a
+peerlist in order to tell NixOS all it needs to set up the Wireguard interface.
+With this information plus the peerlists from before, we can set up host
+configurations:
+
+```nix
+hosts = {
+  # hexagone
+  chrysalis = interfaceInfo chrysalis hexagone;
+  keanu = interfaceInfo keanu hexagone;
+  shachi = interfaceInfo shachi hexagone;
+  genza = interfaceInfo genza hexagone;
+
+  # cloud
+  lufta = interfaceInfo lufta cloud;
+  firgu = interfaceInfo firgu cloud;
+};
+```
+
+And then I can set up a `akua.nix` file in the host configuration folder that
+looks something like this:
+
+```nix
+{ config, pkgs, ... }:
+
+let metadata = pkgs.callPackage ../../ops/metadata/peers.nix { };
+in {
+  networking.wireguard.interfaces.akua =
+    metadata.hosts."${config.networking.hostName}";
+    
+  within.secrets.wg-privkey = {
+    source = ./secrets/wg.privkey;
+    dest = "/root/wireguard-keys/private";
+    owner = "root";
+    group = "root";
+    permissions = "0400";
+  };
+}
+```
+
+Then when I push to my machines next, the new Wireguard config will be pushed
+across the network, seamlessly integrating any new machine into the mesh.
+
+[Wait, you have other machines like an iPad, iPhone and MacBook and I didn't see
+you detail those anywhere in this network. How do you manage Wireguard for
+them?](conversation://Mara/hmm)
+
+I don't!
+
+I actually use Tailscale's [subnet
+routing](https://tailscale.com/kb/1019/subnets) to handle this. I have my tower
+at home expose a route for `10.77.0.0/16` and then it all works out
+automagically. Sure it doesn't expose _everything_ if my tower goes and stays
+down, however in that case I'm probably going to just make one of my cloud
+servers into the subnet router.
+
+<small>Small disclaimer: Tailscale is my employer. I am not speaking for them
+with this section. I use them for this because it solves the problem I have
+with this so well that I don't have to care about this anymore. Seriously this
+has removed so much manual process from my Wireguard networks it's not even
+funny. I was a Tailscale user before I was a Tailscale employee.</small>
+
+[Is it really a good idea to include those Wireguard public keys in a public git
+repo like that?](conversation://Mara/hmm)
+
+They are _public_ keys, however I have no idea if it really is a good idea or
+not. It hasn't gotten me hacked yet (as far as I'm aware), so there's probably
+not much of a practical issue.
+
+My logic behind making my NixOS config repo a public one is to act as an example
+for others to take inspiration from. I also wanted to make it harder for me to
+let the config drift. It also gives me a bunch of fodder for this blog.
+
+---
+
+This is basically what my setup has turned into. It's super easy to manage now.
+If I want to add machines to the network, I just generate a new wirguard keypair,
+modify `hosts.toml` and then push out the config to the network. That's it. It's
+beautiful and I love it.
+
+Feel free to take inspiration from this setup. I'm sure you can do it in a nicer
+way somehow (maybe put the metadata table into the nix file itself? that way it
+would work on NixOS stable), but this works amazingly for my needs.