aboutsummaryrefslogtreecommitdiff
path: root/lume/src/talks/2023/carcinization-rust.mdx
diff options
context:
space:
mode:
authorXe Iaso <me@xeiaso.net>2024-02-16 19:57:04 -0800
committerXe Iaso <me@xeiaso.net>2024-02-16 19:57:04 -0800
commit736781f3262ee5d4838683c0ba6ceba2ad422fd7 (patch)
tree86d60696d0e7a505326724e228b970c360b36ded /lume/src/talks/2023/carcinization-rust.mdx
parente5a4b5a337a74ca3c7459f4ee1aacc19be016ff7 (diff)
downloadxesite-736781f3262ee5d4838683c0ba6ceba2ad422fd7.tar.xz
xesite-736781f3262ee5d4838683c0ba6ceba2ad422fd7.zip
talks: reorganize into folders without breaking links
Signed-off-by: Xe Iaso <me@xeiaso.net>
Diffstat (limited to 'lume/src/talks/2023/carcinization-rust.mdx')
-rw-r--r--lume/src/talks/2023/carcinization-rust.mdx170
1 files changed, 170 insertions, 0 deletions
diff --git a/lume/src/talks/2023/carcinization-rust.mdx b/lume/src/talks/2023/carcinization-rust.mdx
new file mode 100644
index 0000000..57f45cb
--- /dev/null
+++ b/lume/src/talks/2023/carcinization-rust.mdx
@@ -0,0 +1,170 @@
+---
+title: "[talk] The carcinization of Go programs"
+date: 2023-03-24
+basename: ../wazero-lightning-2023
+slides_link: https://drive.google.com/file/d/1ANxRJPzNeKbLogZz0wCH_o9E9jNLypja/view?usp=sharing
+tags:
+ - wasm
+ - wazero
+ - rust
+ - golang
+skip_ads: true
+---
+
+<xeblog-talk-warning></xeblog-talk-warning>
+
+<xeblog-video path="talks/2023/wazero-lightning"></xeblog-video>
+
+## Transcript
+
+<XeblogConv standalone name="Cadey" mood="enby">
+ This is a lightning talk version of [this
+ post](https://xeiaso.net/blog/carcinization-golang).
+</XeblogConv>
+
+<XeblogSlide name="2023/wazero-lightning/01" essential></XeblogSlide>
+
+Computers are complicated. Programming computers is even more
+complicated. Sometimes you have a square hole, but the square peg you
+need is written in Rust and the rest of your program is written in Go.
+Linking these two worlds together like this is a painful process
+fraught with peril, fear, terror, utter torment, and lemon-scented
+moist towelettes.
+
+<XeblogSlide name="2023/wazero-lightning/02"></XeblogSlide>
+
+Normally if you want to call a Rust function from Go, you need to
+expose a C interface in Rust, and then bind to that C interface with
+cgo. This does work. It's a thing you can do, but it only works the
+most reliably at very small scales. The main problem with doing this
+is that it breaks people's semantic expectations of how tools should
+work.
+
+<XeblogSlide name="2023/wazero-lightning/03"></XeblogSlide>
+
+When Rust compiles `.so` files that link against your program, they
+also link against system libraries. This isn't normally a problem,
+except if you have people like me around that use a hipster distro
+where everything is different. You also have to maintain a separate
+dot s o file for each OS and CPU architecture combination you support.
+Your build step is no longer `go build`, it's in Makefile land.
+
+<XeblogSlide name="2023/wazero-lightning/04"></XeblogSlide>
+
+So, imagine a world where you could just put one binary blob into your
+git repo, and have that Just Work everywhere. Without making your
+build more complicated than `go build`. Without configuration when
+people are building the code. Imagine how much easier that would be.
+
+<XeblogSlide name="2023/wazero-lightning/05"></XeblogSlide>
+
+I bet you can guess where I'm going with this, I'm talking about the
+carcinization of Go programs via WebAssembly. This is how I snuck Rust
+into a Go shop.
+
+<XeblogConv name="Mara" mood="hacker">
+ The "carcinization" refers to the evolutionary tendency of programs becoming
+ either crabs or trees when time stretches to infinity. If you can imagine
+ library use as evolution, then this joke makes more sense.
+</XeblogConv>
+
+<XeblogSlide name="2023/wazero-lightning/06"></XeblogSlide>
+
+Why?
+
+<XeblogSlide name="2023/wazero-lightning/07"></XeblogSlide>
+
+Mastodon. Mastodon uses a protocol called ActivityPub to federate
+posts. ActivityPub posts are formatted in HTML. Reading Mastodon posts
+from the API returns HTML. We want to show these posts off in a Slack
+channel, and Slack doesn't support using your own HTML. It has its own
+bespoke markup format called Slackdown because of course it does.
+
+<XeblogSlide name="2023/wazero-lightning/08"></XeblogSlide>
+
+There was nothing off of the shelf to handle this in Go. I assume that
+this problem is fairly novel. Anything that was close to this just
+made atrocities out of the text in ways that I couldn't customize.
+
+<XeblogConv name="Aoi" mood="wut">
+ I guess some solutions could be out there, but just locked in closed-source
+ repos.
+</XeblogConv>
+
+<XeblogSlide name="2023/wazero-lightning/09"></XeblogSlide>
+
+Then I remembered a Rust library that I use on my blog:
+[lol_html](https://docs.rs/lol_html/latest/lol_html/). lol_html is a
+streaming HTML parser/rewriter. I use it on my blog for all my HTML
+shortcodes and I know it lets you mangle HTML into other formats with
+element handlers. Mastodon has a list of HTML tags it will send out
+over the API, so I used that to assemble a little test program in
+Rust.
+
+<XeblogSlide name="2023/wazero-lightning/10"></XeblogSlide>
+
+When I wrote this program, I made it in a fairly naiive way. I took
+HTML over standard input and had it spit out slackdown on standard
+output.
+
+This idea just so happens to be one of the core parts of the Unix
+philosophy: programs should be filters that take input in one form and
+then transform it to output in another form. This made it trivial to
+test against arbitrary HTML from the Mastodon API and get things down
+to the output format I wanted.
+
+<XeblogSlide name="2023/wazero-lightning/11"></XeblogSlide>
+
+Then comes integration. I already knew I didn't want to deal with the
+surreal horror that is dynamically linking Rust code into Go (but I
+could figure it out if I needed to), so on a lark I decided to just
+try compiling it to WebAssembly with [WASI](https://wasi.dev) and see
+if that works.
+
+<XeblogSlide name="2023/wazero-lightning/12"></XeblogSlide>
+
+It did. I then applied some optimizations and got it down to a 200
+kilobyte ball of mud that did exactly what I needed. Then all I needed
+to do was glue it into the Go program with Wazero.
+
+<XeblogSlide name="2023/wazero-lightning/13"></XeblogSlide>
+
+Wazero made this trivial. I overrode the input and output buffers,
+then had it kick things off as needed. At first my code was very lazy
+and slow, but it worked. The only build step was `go build`, just like
+I wanted. I committed the WASM blob to git and shipped the hack. It
+worked perfectly.
+
+<XeblogSlide name="2023/wazero-lightning/14"></XeblogSlide>
+
+The last part was making it faster. With some guidance from #wazero on
+Slack, I managed to get each call of this function down to two hundred
+microseconds. That's faster than the equivalent code in Go using its
+[HTML parsing library](https://pkg.go.dev/golang.org/x/net/html) that
+I only found out existed about a week ago.
+
+<XeblogSlide name="2023/wazero-lightning/15"></XeblogSlide>
+
+And now this atrocity is shipped to production and holds together the
+Mastodon post announcement service. Most people aren't aware that it's
+a thing, and it runs fast enough that nobody really cares that it's a
+programming turducken of Go, Rust and WebAssembly. It works perfectly
+and it wouldn't be possible without the efforts of the Wazero team.
+
+<XeblogSlide name="2023/wazero-lightning/16" essential></XeblogSlide>
+
+I'm Xe Iaso, and I hoped you enjoyed my talk about programming crimes.
+
+I do developer relations at Tailscale as the Archmage of
+Infrastructure. If you have any questions, please don't hesitate to
+reach out at [wazero2023@xeserv.us](mailto:wazero2023@xeserv.us). I'm
+more than happy to answer them.
+
+Congrats to the Wazero team for the 1.0 release! Be well, all!
+
+## With apologies to
+
+- Renee French
+- The Rust community
+- The Wazero team
+- Hbomberguy