path: root/blog/we-have-go-2.markdown

---
title: We Already Have Go 2
date: 2022-05-25
tags:
 - golang
 - generics
 - context
 - modules
---

I've been using Go since Go 1.4. Since I started using Go then (2014-2015 ish),
I’ve seen the language evolve significantly. The Go I write today is roughly the
same Go as the Go I wrote back when I was still learning the language, but the
toolchain has changed in ways that make it so much nicer in practice. Here are
the biggest things that changed how I use Go on a regular basis:

* The compiler rewrite in Go
* Go modules
* The context package
* Generics

This is a good thing. Go has had a lot of people use it. My career would not
exist in its current form without Go. My time in the Go community has been
_catalytic_ to my career goals and it’s made me into the professional I am
today. Without having met the people I did in the Go slack, I would probably not
have gotten as lucky as I have as consistently as I have.

Releasing a "Go 2" has become a philosophical and political challenge due to the
forces that be. "Go 2" has kind of gotten the feeling of "this is never going to
happen, is it?" with how the political forces within and without the Go team are
functioning. They seem to have been incrementally releasing new features and
using version gating in `go.mod` to make it easier on people instead of a big
release with breaking changes all over the standard library.

This is pretty great and I am well in favour of this approach, but with all of
the changes that have built up there really should be a Go 2 by this point. If
only to make no significant changes and tag what we have today as Go 2.

<xeblog-conv name="Cadey" mood="coffee">Take everything I say here with a grain
of salt the size of east Texas. I am not an expert in programming language
design and I do not pretend to be one on TV. I am also not a member of the Go
team nor do I pretend to be one or see myself becoming one in the
future.<br /><br />If you are on the Go team and think that something I said
here is demonstrably wrong, please [contact me](/contact) so I can correct it. I
have tried to contain my personal feelings or observations about things to these
conversation snippets.</xeblog-conv>

This is a look back at the huge progress that has been made since Go 1 released
and what I'd consider to be the headline features of Go 2.
This is a whirlwind tour of the huge progress in improvement to the Go compiler,
toolchain, and standard library, including what I'd consider to be the headline
features of Go 2. I highly encourage you read this fairly large post in chunks
because it will feel like _a lot_ if you read it all at once.

## The Compiler Rewrite in Go

When the Go compiler was first written, it was written in C because the core Go
team has a background in Plan 9 and C was its lingua franca. However as a result
of either it being written in C or the design around all the tools it was
shelling out to, it wasn’t easy to cross compile Go programs. If you were
building windows programs on a Mac you needed to do a separate install of Go
from source with other targets enabled. This worked, but it wasn’t the default
and eventually the Go compiler rewrite in Go changed this so that Go could cross
compile natively with no extra effort required.

<xeblog-conv name="Cadey" mood="enby">This has been such an amazingly productive
part of the Go toolchain that I was shocked that Go didn’t have this out of the
gate at version 1. Most people that use Go today don’t know that there was a
point where Go didn’t have the easy to use cross-compiling superpower it
currently has, and I think that is a more sure marker of success than anything
else.</xeblog-conv>

<xeblog-conv name="Mara" mood="happy">The cross compliation powers are why
Tailscale uses Go so extensively throughout its core product. Every Tailscale
client is built on the same Go source tree and everything is in lockstep with
eachother, provided people actually update their apps. This kind of thing would
be at the least impossible or at the most very difficult in other languages like
Rust or C++.</xeblog-conv>

This one feature is probably at the heart of more CI flows, debian package
releases and other workflows than we can know. It's really hard to understate
how simple this kind of thing makes distributing software for other
architectures, especially given that macOS has just switched over to aarch64
CPUs.

Having the compiler be self-hosting does end up causing a minor amount of
grief for people wanting to bootstrap a Go compiler from absolute source code
on a new Linux distribtion (and slightly more after the minimum Go compiler
version to compile Go will be raised to Go 1.17 with the release of Go 1.19
in about 6 months from the time of this post being written). This isn't too
big of a practical issue given how fast the compiler builds, but it is a
nonzero amount of work. The bootstrapping can be made simpler with
[gccgo](https://gcc.gnu.org/onlinedocs/gccgo/), a GCC frontend that is mostly
compatible with the semantics and user experience of the Go compiler that
Google makes.

Another key thing porting the compiler to Go unlocks is the ability to compile
Go packages in parallel. Back when the compiler was written in C, the main point
of parallelism was the fact that each Go package was compiled in parallel. This
lead to people splitting up bigger packages into smaller sub-packages in order
to speedhack the compiler. Having the compiler be written in Go means that the
compiler can take advantage of Go features like its dead-simple concurrency
primitives to spread the load out across all the cores on the machine.

<xeblog-conv name="Mara" mood="hacker">The Go compiler is fast sure, but
over a certain point having each package be compiled in a single-threaded manner
adds up and can make build times slow. This was a lot worse when things like the
AWS, GCP and Kubernetes client libraries had everything in one big package.
Building those packages could take minutes, which is very long in Go
time.</xeblog-conv>

## Go Modules

In Go's dependency model, you have a folder that contains all your Go code
called the `GOPATH`. The `GOPATH` has a few top level folders that have a
well-known meaning in the Go ecosystem:

* bin: binary files made by `go install` or `go get` go here
* pkg: intermediate compiler state goes here
* src: Go packages go here

`GOPATH` has one major advantage: it is ruthlessly easy to understand the
correlation between the packages you import in your code to their locations on
disk.

If you need to see what `within.website/ln` is doing, you go to
`GOPATH/src/within.website/ln`. The files you are looking for are somewhere in
there. You don’t have to really understand how the package manager works (mostly
because there isn’t one). If you want to hack something up you just go to the
folder and add the changes you want to see.

You can delete all of the intermediate compiler state easily in one fell swoop.
Just delete the `pkg` folder and poof, it’s all gone. This was great when you
needed to free up a bunch of disk space really quickly because over months the
small amount of incremental compiler state can really add up.

The Go compiler would fetch any missing packages from the internet at build time
so things Just Worked™️. This makes it utterly trivial to check out a project and
then build/run it. That combined with `go get` to automatically just figure
things out and install them made installing programs written in Go so easy that
it’s almost magic. This combined with Go's preference for making static binaries
as much as possible meant that even if the user didn't have Go installed you could
easily make a package to hand off to your users.

The GOPATH was conceptually simple to reason about. Go code goes in the GOPATH. The
best place for it was in the GOPATH. There's no reason to put it anywhere else.
Everything was organized into its place and it was lovely.

This wasn’t perfect though. There were notable flaws in this setup that were
easy to run into in practice:

* There wasn't a good way to make sure that everyone was using the _same copies_
  of every library. People did add vendoring tools later to check that everyone
  was using the same copies of every package, but this also introduced problems
  when one project used one version of a dependency and another project used
  another in ways that were mutually incompatible.
* The process to get the newest version of a dependency was to grab the latest
  commit off of the default branch of that git repo. There was support for SVN,
  mercurial and fossil, but in practice Git was the most used one so it’s almost
  not worth mentioning the other version control systems. This also left you at
  the mercy of other random people having good code security sense and required
  you to audit your dependencies, but this is fairly standard across ecosystems.
* Dependency names were case sensitive on Linux but not on Windows or macOS.
  Arguably this is a "Windows and macOS are broken for backwards compatibility
  reasons" thing, but this did bite me at random times without warning.
* If the wrong random people deleted their GitHub repos, there's a chance your
  builds could break unless your GOPATH had the packages in it already. Then you
  could share that with your coworkers or the build machine somehow, maybe even
  upload those packages to a git repository to soft-fork it.
* The default location for the GOPATH created a folder in your home directory.

<xeblog-conv name="Cadey" mood="coffee">Yeah, yeah, this default was added later
but still people complained about having to put the GOPATH somewhere at first.
Having to choose a place to put all the Go code they would use seemed like a big
choice that people really wanted solid guidance and defaults on. After a while
they changed this to default to `~/go` (with an easy to use command to influence
the defaults without having to set an environment variable). I don't personally
understand the arguments people have for wanting to keep their home directory
"clean", but their preferences are valid regardless.</xeblog-conv>

Overall I think GOPATH was a net good thing for Go. It had its downsides, but as
far as these things go it was a very opinionated place to start from. This is
something typical to Go (much to people's arguments), but the main thing that it
focused on was making Go conceptually simple. There's not a lot going on there.
You have code in the folder and then that's where the Go compiler looks for
other code. It's a very lightweight approach to things that a lot of other
languages could learn a lot from. It's great for monorepos because it basically
treats all your Go code as one big monorepo. So many other languages don’t
really translate well to working in a monorepo context like Go does.

### Vendoring

That making sure everyone had the same versions of everything problem ended up
becoming a big problem in practice. I'm assuming that the original intent of the
GOPATH was to be similar to how Google's internal monorepo worked, where
everyone clones and deals with the entire GOPATH in source control. You'd then
have to do GOPATH juggling between monorepos, but the intent was to have
everything in one big monorepo anyways, so this wasn't thought of as much of a
big deal in practice. It turns out that people in fact did not want to treat Go
code this way, in practice this conflicted with the dependency model that Go
encouraged people to use with how people consume libraries from GitHub or other
such repository hosting sites.

The main disconnect between importing from a GOPATH monorepo and a Go library
off of GitHub is that when you import from a monorepo with a GOPATH in it, you
need to be sure to import the repository path and not the path used inside the
repository. This sounds weird but this means you'd import
`github.com/Xe/x/src/github.com/Xe/x/markov` instead of
`github.com/Xe/x/markov`. This means that things need to be extracted _out of_
monorepos and reformatted into "flat" repos so that you can only grab the one
package you need. This became tedious in practice.

In Go 1.5 (the one where they rewrote the compiler in Go) they added support for
[vendoring code into your
repo](https://medium.com/@freeformz/go-1-5-s-vendor-experiment-fd3e830f52c3).
The idea here was to make it easy to get closer to the model that the Go authors
envisioned for how people should use Go. Go code should all be in one big happy
repo and everything should have its place in your GOPATH. This combined with
other tools people made allowed you to vendor all of your dependencies into a
`vendor` folder and then you could do whatever you wanted from there.

One of the big advantages of the `vendor` folder was that you could clone your
git repo, create a new process namespace and then run tests without a network
stack. Everything would work offline and you wouldn't have to worry about
external state leaking in. Not to mention removing the angle of someone deleting
their GitHub repos causing a huge problem for your builds.

<xeblog-conv name="Mara" mood="happy">Save tests that require internet access or
a database engine!</xeblog-conv>

This worked for a very long time. People were able to vendor their code into
their repos and everything was better for people using Go. However the most
critical oversight with the `vendor` folder approach was that the Go team didn't
create an official tool to manage that `vendor` folder. They wanted to let tools
like `godep` and `glide` handle that. This is kind of a reasonable take, Go
comes from a very Google culture where this kind of problem doesn't happen, so
as a result they probably won't be able to come up with something that meets the
needs of the outside world very easily.

<xeblog-conv name="Cadey" mood="enby">I can't speak for how `godep` or `glide`
works, I never really used them enough to have a solid opinion. I do remember
using [`vendor`](https://github.com/bmizerany/vendor) in my own projects though.
That had no real dependency resolution algorithm to speak of because it assumed
that you had everything working locally when you vendored the code.</xeblog-conv>

### `dep`

After a while the Go team worked with people in the community to come up with an
"official experiment" in tracking dependencies called `dep`. `dep` was a tool
that used some more fancy computer science maths to help developers declare
dependencies for projects in a way like you do in other ecosystems. When `dep`
was done thinking, it emitted a bunch of files in `vendor` and a lockfile in
your repository. This worked really well and when I was working at Heroku this
was basically our butter and bread for how to deal with Go code.

<xeblog-conv name="Cadey" mood="enby">It probably helped that my manager was on
the team that wrote `dep`.</xeblog-conv>

One of the biggest advantages of `dep` over other tools was the way that it
solved versioning. It worked by having each package declare
[constraints](https://golang.github.io/dep/docs/the-solver.html) in the ranges
of versions that everything requires. This allowed it to do some fancy
dependency resolution math similar to how the solvers in `npm` or `cargo` work.

This worked fantastically in the 99% case. There were some fairly easy to
accidentally get yourself in cases where you could make the solver loop
infinitely though, as well as ending up in a state where you have mutually
incompatible transient dependencies without any real way around it.

<xeblog-conv name="Mara" mood="hacker">`npm` and `cargo` work around this by
letting you use multiple versions of a single dependency in a
project.</xeblog-conv>

However these cases were really really rare, only appearing in much, much larger
repositories. I don't think I practically ran into this, but I'm sure someone
reading this right now found themselves in `dep` hell and probably has a hell of
a war story around it.

### vgo and Modules

This lead the Go team to come up with a middle path between the unrestricted
madness of GOPATH and something more maximal like `dep`. They eventually called
this Go modules and the core reasons for it are outlined in [this series of
technical posts](https://research.swtch.com/vgo).

<xeblog-conv name="Mara" mood="hacker">These posts are a very good read and I'd
highly suggest reading them if you've never seem then before. It outlines the
problem space and the justification for the choices that Go modules ended up
using. I don't agree with all of what is said there, but overall it's well
worth reading at least once if you want to get an idea of the inspirations
that lead to Go modules.</xeblog-conv>

Apparently the development of Go modules came out as a complete surprise, 
even to the core developer team of `dep`. I'm fairly sure this lead my 
manager to take up woodworking as his main non work side hobby, I can only
wonder about the kind of resentment this created for other parts of the 
`dep` team. They were under the impression that `dep` was going to be the
future of the ecosystem (likely under the subcommand `go dep`) and then had
the rug pulled out from under their feet.

<xeblog-conv name="Cadey" mood="coffee">The `dep` team was as close as we've
gotten for having people in the _actual industry_ using Go _in production_
outside of Google having a real voice in how Go is used in the real world. I
fear that we will never have this kind of thing happen again.<br /><br />It's
also worth noting that the fallout of this lead to the core `dep` team leaving
the Go community.</xeblog-conv>

<xeblog-conv name="Mara" mood="hmm">Well, Google has to be using Go modules in
their monorepo, right? If that's the official build system for Go it makes sense
that they'd be dogfooding it hard enough that they'd need to use the tool in the
same way that everyone else did.</xeblog-conv>

<xeblog-conv name="Numa" mood="delet">lol nope. They use an overcomplicated
bazel/blaze abomination that has developed in parallel to their NIH'd source
control server. Google doesn't have to deal with the downsides of Go modules
unless it's in a project like Kubernetes. It's easy to imagine that they just
don't have the same problems that everyone else does due to how weird Google
prod is. Google only has problems that Google has, and statistically your
company is NOT Google.</xeblog-conv>

Go modules does solve one very critical problem for the Go ecosystem though: it
allows you to have the equivalent of the GOPATH but with multiple versions of
dependencies in it. It allows you to have `within.website/ln@v0.7` and
`within.website/ln@0.9` as dependencies for _two different projects_ without
having to vendor source code or do advanced GOPATH manipulation between
projects. It also adds cryptographic checksumming for each Go module that you
download from the internet, so that you can be sure the code wasn't tampered
with in-flight. They also created a cryptographic checksum comparison server so
that you could ask a third party to validate what it thinks the checksum is so
you can be sure that the code isn't tampered with on the maintainer's side. This
also allows you to avoid having to shell out to `git` every time you fetch a
module that someone else has fetched before. Companies could run their own Go
module proxy and then use that to provide offline access to Go code fetched from
the internet.

<xeblog-conv name="Mara" mood="hmm">Wait, couldn't this allow Google to see the
source code of all of your Go dependencies? How would this intersect with
private repositories that shouldn't ever be on anything but work
machines?</xeblog-conv>

<xeblog-conv name="Cadey" mood="coffee">Yeah, this was one of the big privacy
disadvantages out of the gate with Go modules. I think that in practice the
disadvantages are limited, but still the fact that it defaults to phoning home
to Google every time you run a Go build without all the dependencies present
locally is kind of questionable. They did make up for this with the checksum
verification database a little, but it's still kinda sus.<br /><br />I'm not
aware of any companies I've worked at running their own internal Go module
caching servers, but I ran my own for a very long time.</xeblog-conv>

The earliest version of Go modules basically was a glorified `vendor` folder
manager named `vgo`. This worked out amazingly well and probably made
prototyping this a hell of a lot easier. This worked well enough that we used
this in production for many services at Heroku. We had no real issues with it
and most of the friction was with the fact that most of the existing ecosystem
had already been using `dep` or `glide`.

<xeblog-conv name="Mara" mood="hacker">There was a bit of interoperability glue
that allowed `vgo` to parse the dependency definitions in `dep`, `godep` and
`glide`. This still exists today and helps `go mod init` tell what dependencies
to import into the Go module to aid migration.</xeblog-conv>

If they had shipped this in prod, it probably would have been a huge success. It
would also let people continue to use `dep`, `glide` and `godep`, but just doing
that would also leave the ecosystem kinda fragmented. You’d need to have code
for all 4 version management systems to parse their configuration files and
implement algorithms that would be compatible with the semantics of all of them.
It would work and the Go team is definitely smart enough to do it, but in
practice it would be a huge mess.

This also solved the case-insensitive filesystem problem with
[bang-casing](https://go.dev/ref/mod#goproxy-protocol). This allows them to
encode the capital letters in a path in a way that works on macOS and Windows
without having to wor