diff options
| author | Xe Iaso <me@xeiaso.net> | 2025-03-20 15:06:58 -0400 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2025-03-20 15:06:58 -0400 |
| commit | c47347ff76f115f56676f5ecad0032bfcb98a03d (patch) | |
| tree | 53b17e30a83dab1f96acf64ba83655340d8e1a5f /docs | |
| parent | 240159e921449236c79e3ae8c7160dfe8ee1b40c (diff) | |
| download | anubis-c47347ff76f115f56676f5ecad0032bfcb98a03d.tar.xz anubis-c47347ff76f115f56676f5ecad0032bfcb98a03d.zip | |
add docs site based on docusarus (#35)
* add docs site based on docusarus
Closes #2
Signed-off-by: Xe Iaso <me@xeiaso.net>
* docs: deploy to aeacus
Signed-off-by: Xe Iaso <me@xeiaso.net>
* ready for merge
Signed-off-by: Xe Iaso <me@xeiaso.net>
* docs: fix anubis port
Signed-off-by: Xe Iaso <me@xeiaso.net>
---------
Signed-off-by: Xe Iaso <me@xeiaso.net>
Diffstat (limited to 'docs')
44 files changed, 20842 insertions, 2 deletions
diff --git a/docs/.dockerignore b/docs/.dockerignore new file mode 100644 index 0000000..13ad2b9 --- /dev/null +++ b/docs/.dockerignore @@ -0,0 +1,23 @@ +# Dependencies +/node_modules + +# Production +/build + +# Generated files +.docusaurus +.cache-loader + +# Misc +.DS_Store +.env.local +.env.development.local +.env.test.local +.env.production.local + +npm-debug.log* +yarn-debug.log* +yarn-error.log* + +# Kubernetes manifests +/manifest
\ No newline at end of file diff --git a/docs/.gitignore b/docs/.gitignore new file mode 100644 index 0000000..b2d6de3 --- /dev/null +++ b/docs/.gitignore @@ -0,0 +1,20 @@ +# Dependencies +/node_modules + +# Production +/build + +# Generated files +.docusaurus +.cache-loader + +# Misc +.DS_Store +.env.local +.env.development.local +.env.test.local +.env.production.local + +npm-debug.log* +yarn-debug.log* +yarn-error.log* diff --git a/docs/Dockerfile b/docs/Dockerfile new file mode 100644 index 0000000..0faa826 --- /dev/null +++ b/docs/Dockerfile @@ -0,0 +1,10 @@ +FROM node AS build + +WORKDIR /app +COPY . . + +RUN npm ci && npm run build + +FROM nginx:alpine +COPY --from=build /app/build /usr/share/nginx/html +LABEL org.opencontainers.image.source="https://github.com/TecharoHQ/anubis"
\ No newline at end of file diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 0000000..0c6c2c2 --- /dev/null +++ b/docs/README.md @@ -0,0 +1,41 @@ +# Website + +This website is built using [Docusaurus](https://docusaurus.io/), a modern static website generator. + +### Installation + +``` +$ yarn +``` + +### Local Development + +``` +$ yarn start +``` + +This command starts a local development server and opens up a browser window. Most changes are reflected live without having to restart the server. + +### Build + +``` +$ yarn build +``` + +This command generates static content into the `build` directory and can be served using any static contents hosting service. + +### Deployment + +Using SSH: + +``` +$ USE_SSH=true yarn deploy +``` + +Not using SSH: + +``` +$ GIT_USER=<Your GitHub username> yarn deploy +``` + +If you are using GitHub pages for hosting, this command is a convenient way to build the website and push to the `gh-pages` branch. diff --git a/docs/blog/2019-05-28-first-blog-post.md b/docs/blog/2019-05-28-first-blog-post.md new file mode 100644 index 0000000..d3032ef --- /dev/null +++ b/docs/blog/2019-05-28-first-blog-post.md @@ -0,0 +1,12 @@ +--- +slug: first-blog-post +title: First Blog Post +authors: [slorber, yangshun] +tags: [hola, docusaurus] +--- + +Lorem ipsum dolor sit amet... + +<!-- truncate --> + +...consectetur adipiscing elit. Pellentesque elementum dignissim ultricies. Fusce rhoncus ipsum tempor eros aliquam consequat. Lorem ipsum dolor sit amet diff --git a/docs/blog/2019-05-29-long-blog-post.md b/docs/blog/2019-05-29-long-blog-post.md new file mode 100644 index 0000000..eb4435d --- /dev/null +++ b/docs/blog/2019-05-29-long-blog-post.md @@ -0,0 +1,44 @@ +--- +slug: long-blog-post +title: Long Blog Post +authors: yangshun +tags: [hello, docusaurus] +--- + +This is the summary of a very long blog post, + +Use a `<!--` `truncate` `-->` comment to limit blog post size in the list view. + +<!-- truncate --> + +Lorem ipsum dolor sit amet, consectetur adipiscing elit. Pellentesque elementum dignissim ultricies. Fusce rhoncus ipsum tempor eros aliquam consequat. Lorem ipsum dolor sit amet + +Lorem ipsum dolor sit amet, consectetur adipiscing elit. Pellentesque elementum dignissim ultricies. Fusce rhoncus ipsum tempor eros aliquam consequat. Lorem ipsum dolor sit amet + +Lorem ipsum dolor sit amet, consectetur adipiscing elit. Pellentesque elementum dignissim ultricies. Fusce rhoncus ipsum tempor eros aliquam consequat. Lorem ipsum dolor sit amet + +Lorem ipsum dolor sit amet, consectetur adipiscing elit. Pellentesque elementum dignissim ultricies. Fusce rhoncus ipsum tempor eros aliquam consequat. Lorem ipsum dolor sit amet + +Lorem ipsum dolor sit amet, consectetur adipiscing elit. Pellentesque elementum dignissim ultricies. Fusce rhoncus ipsum tempor eros aliquam consequat. Lorem ipsum dolor sit amet + +Lorem ipsum dolor sit amet, consectetur adipiscing elit. Pellentesque elementum dignissim ultricies. Fusce rhoncus ipsum tempor eros aliquam consequat. Lorem ipsum dolor sit amet + +Lorem ipsum dolor sit amet, consectetur adipiscing elit. Pellentesque elementum dignissim ultricies. Fusce rhoncus ipsum tempor eros aliquam consequat. Lorem ipsum dolor sit amet + +Lorem ipsum dolor sit amet, consectetur adipiscing elit. Pellentesque elementum dignissim ultricies. Fusce rhoncus ipsum tempor eros aliquam consequat. Lorem ipsum dolor sit amet + +Lorem ipsum dolor sit amet, consectetur adipiscing elit. Pellentesque elementum dignissim ultricies. Fusce rhoncus ipsum tempor eros aliquam consequat. Lorem ipsum dolor sit amet + +Lorem ipsum dolor sit amet, consectetur adipiscing elit. Pellentesque elementum dignissim ultricies. Fusce rhoncus ipsum tempor eros aliquam consequat. Lorem ipsum dolor sit amet + +Lorem ipsum dolor sit amet, consectetur adipiscing elit. Pellentesque elementum dignissim ultricies. Fusce rhoncus ipsum tempor eros aliquam consequat. Lorem ipsum dolor sit amet + +Lorem ipsum dolor sit amet, consectetur adipiscing elit. Pellentesque elementum dignissim ultricies. Fusce rhoncus ipsum tempor eros aliquam consequat. Lorem ipsum dolor sit amet + +Lorem ipsum dolor sit amet, consectetur adipiscing elit. Pellentesque elementum dignissim ultricies. Fusce rhoncus ipsum tempor eros aliquam consequat. Lorem ipsum dolor sit amet + +Lorem ipsum dolor sit amet, consectetur adipiscing elit. Pellentesque elementum dignissim ultricies. Fusce rhoncus ipsum tempor eros aliquam consequat. Lorem ipsum dolor sit amet + +Lorem ipsum dolor sit amet, consectetur adipiscing elit. Pellentesque elementum dignissim ultricies. Fusce rhoncus ipsum tempor eros aliquam consequat. Lorem ipsum dolor sit amet + +Lorem ipsum dolor sit amet, consectetur adipiscing elit. Pellentesque elementum dignissim ultricies. Fusce rhoncus ipsum tempor eros aliquam consequat. Lorem ipsum dolor sit amet diff --git a/docs/blog/2021-08-01-mdx-blog-post.mdx b/docs/blog/2021-08-01-mdx-blog-post.mdx new file mode 100644 index 0000000..0c4b4a4 --- /dev/null +++ b/docs/blog/2021-08-01-mdx-blog-post.mdx @@ -0,0 +1,24 @@ +--- +slug: mdx-blog-post +title: MDX Blog Post +authors: [slorber] +tags: [docusaurus] +--- + +Blog posts support [Docusaurus Markdown features](https://docusaurus.io/docs/markdown-features), such as [MDX](https://mdxjs.com/). + +:::tip + +Use the power of React to create interactive blog posts. + +::: + +{/* truncate */} + +For example, use JSX to create an interactive button: + +```js +<button onClick={() => alert('button clicked!')}>Click me!</button> +``` + +<button onClick={() => alert('button clicked!')}>Click me!</button> diff --git a/docs/blog/2021-08-26-welcome/docusaurus-plushie-banner.jpeg b/docs/blog/2021-08-26-welcome/docusaurus-plushie-banner.jpeg Binary files differnew file mode 100644 index 0000000..11bda09 --- /dev/null +++ b/docs/blog/2021-08-26-welcome/docusaurus-plushie-banner.jpeg diff --git a/docs/blog/2021-08-26-welcome/index.md b/docs/blog/2021-08-26-welcome/index.md new file mode 100644 index 0000000..349ea07 --- /dev/null +++ b/docs/blog/2021-08-26-welcome/index.md @@ -0,0 +1,29 @@ +--- +slug: welcome +title: Welcome +authors: [slorber, yangshun] +tags: [facebook, hello, docusaurus] +--- + +[Docusaurus blogging features](https://docusaurus.io/docs/blog) are powered by the [blog plugin](https://docusaurus.io/docs/api/plugins/@docusaurus/plugin-content-blog). + +Here are a few tips you might find useful. + +<!-- truncate --> + +Simply add Markdown files (or folders) to the `blog` directory. + +Regular blog authors can be added to `authors.yml`. + +The blog post date can be extracted from filenames, such as: + +- `2019-05-30-welcome.md` +- `2019-05-30-welcome/index.md` + +A blog post folder can be convenient to co-locate blog post images: + + + +The blog supports tags as well! + +**And if you don't want a blog**: just delete this directory, and use `blog: false` in your Docusaurus config. diff --git a/docs/blog/authors.yml b/docs/blog/authors.yml new file mode 100644 index 0000000..8bfa5c7 --- /dev/null +++ b/docs/blog/authors.yml @@ -0,0 +1,23 @@ +yangshun: + name: Yangshun Tay + title: Front End Engineer @ Facebook + url: https://github.com/yangshun + image_url: https://github.com/yangshun.png + page: true + socials: + x: yangshunz + github: yangshun + +slorber: + name: Sébastien Lorber + title: Docusaurus maintainer + url: https://sebastienlorber.com + image_url: https://github.com/slorber.png + page: + # customize the url of the author page at /blog/authors/<permalink> + permalink: '/all-sebastien-lorber-articles' + socials: + x: sebastienlorber + linkedin: sebastienlorber + github: slorber + newsletter: https://thisweekinreact.com diff --git a/docs/blog/tags.yml b/docs/blog/tags.yml new file mode 100644 index 0000000..bfaa778 --- /dev/null +++ b/docs/blog/tags.yml @@ -0,0 +1,19 @@ +facebook: + label: Facebook + permalink: /facebook + description: Facebook tag description + +hello: + label: Hello + permalink: /hello + description: Hello tag description + +docusaurus: + label: Docusaurus + permalink: /docusaurus + description: Docusaurus tag description + +hola: + label: Hola + permalink: /hola + description: Hola tag description diff --git a/docs/docs/CHANGELOG.md b/docs/docs/CHANGELOG.md new file mode 100644 index 0000000..2015670 --- /dev/null +++ b/docs/docs/CHANGELOG.md @@ -0,0 +1,31 @@ +--- +sidebar_position: 999 +--- + +# Changelog + +All notable changes to this project will be documented in this file. + +The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), +and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). + +## [Unreleased] + +- Documentation has been moved to https://anubis.techaro.lol/ with sources in docs/ + +## 1.13.0 + +- Proof-of-work challenges are drastically sped up [#19](https://github.com/TecharoHQ/anubis/pull/19) +- Docker images are now built with the timestamp set to the commit timestamp +- The README now points to TecharoHQ/anubis instead of Xe/x +- Images are built using ko instead of `docker buildx build` + [#13](https://github.com/TecharoHQ/anubis/pull/13) + +## 1.12.1 + +- Phrasing in the `<noscript>` warning was replaced from its original placeholder text to + something more suitable for general consumption + ([fd6903a](https://github.com/TecharoHQ/anubis/commit/fd6903aeed315b8fddee32890d7458a9271e4798)). +- Footer links on the check page now point to Techaro's brand + ([4ebccb1](https://github.com/TecharoHQ/anubis/commit/4ebccb197ec20d024328d7f92cad39bbbe4d6359)) +- Anubis was imported from [Xe/x](https://github.com/Xe/x). diff --git a/docs/docs/admin/_category_.json b/docs/docs/admin/_category_.json new file mode 100644 index 0000000..4b7ba50 --- /dev/null +++ b/docs/docs/admin/_category_.json @@ -0,0 +1,8 @@ +{ + "label": "Administrative guides", + "position": 40, + "link": { + "type": "generated-index", + "description": "Tradeoffs and considerations you may want to keep in mind when using Anubis." + } +}
\ No newline at end of file diff --git a/docs/docs/admin/caveats-gitea-forgejo.mdx b/docs/docs/admin/caveats-gitea-forgejo.mdx new file mode 100644 index 0000000..6569a7a --- /dev/null +++ b/docs/docs/admin/caveats-gitea-forgejo.mdx @@ -0,0 +1,34 @@ +--- +title: When using Caddy with Gitea/Forgejo +--- + +Gitea/Forgejo relies on the reverse proxy setting the `X-Real-Ip` header. Caddy does not do this out of the gate. Modify your Caddyfile like this: + +```python +ellenjoe.int.within.lgbt { + # ... + # diff-remove + reverse_proxy http://localhost:3000 + # diff-add + reverse_proxy http://localhost:3000 { + # diff-add + header_up X-Real-Ip {remote_host} + # diff-add + } + # ... +} +``` + +Ensure that Gitea/Forgejo have `[security].REVERSE_PROXY_TRUSTED_PROXIES` set to the IP ranges that Anubis will appear from. Typically this is sufficient: + +```ini +[security] +REVERSE_PROXY_TRUSTED_PROXIES = 127.0.0.0/8,::1/128 +``` + +However if you are running Anubis in a separate Pod/Deployment in Kubernetes, you may have to adjust this to the IP range of the Pod space in your Container Networking Interface plugin: + +```ini +[security] +REVERSE_PROXY_TRUSTED_PROXIES = 10.192.0.0/12 +``` diff --git a/docs/docs/design/_category_.json b/docs/docs/design/_category_.json new file mode 100644 index 0000000..65764e8 --- /dev/null +++ b/docs/docs/design/_category_.json @@ -0,0 +1,8 @@ +{ + "label": "Design", + "position": 10, + "link": { + "type": "generated-index", + "description": "How Anubis is designed and the tradeoffs it makes." + } +}
\ No newline at end of file diff --git a/docs/docs/design/how-anubis-works.mdx b/docs/docs/design/how-anubis-works.mdx new file mode 100644 index 0000000..5f5cfde --- /dev/null +++ b/docs/docs/design/how-anubis-works.mdx @@ -0,0 +1,121 @@ +--- +sidebar_position: 1 +title: How Anubis works +--- + +Anubis uses a proof-of-work challenge to ensure that clients are using a modern browser and are able to calculate SHA-256 checksums. Anubis has a customizable difficulty for this proof-of-work challenge, but defaults to 5 leading zeroes. + +```mermaid +--- +title: Challenge generation and validation +--- + +flowchart TD + Backend("Backend") + Fail("Fail") + + style PresentChallenge color:#FFFFFF, fill:#AA00FF, stroke:#AA00FF + style ValidateChallenge color:#FFFFFF, fill:#AA00FF, stroke:#AA00FF + style Backend color:#FFFFFF, stroke:#00C853, fill:#00C853 + style Fail color:#FFFFFF, stroke:#FF2962, fill:#FF2962 + + subgraph Server + PresentChallenge("Present Challenge") + ValidateChallenge("Validate Challenge") + end + + subgraph Client + Main("main.mjs") + Worker("Worker") + end + + Main -- Request challenge --> PresentChallenge + PresentChallenge -- Return challenge & difficulty --> Main + Main -- Spawn worker --> Worker + Worker -- Successful challenge --> Main + Main -- Validate challenge --> ValidateChallenge + ValidateChallenge -- Return cookie --> Backend + ValidateChallenge -- If anything is wrong --> Fail +``` + +### Challenge presentation + +Anubis decides to present a challenge using this logic: + +- User-Agent contains `"Mozilla"` +- Request path is not in `/.well-known`, `/robots.txt`, or `/favicon.ico` +- Request path is not obviously an RSS feed (ends with `.rss`, `.xml`, or `.atom`) + +This should ensure that git clients, RSS readers, and other low-harm clients can get through without issue, but high-risk clients such as browsers and AI scraper bots will get blocked. + +```mermaid +--- +title: Challenge presentation logic +--- + +flowchart LR + Request("Request") + Backend("Backend") + %%Fail("Fail") + PresentChallenge("Present +challenge") + HasMozilla{"Is browser +or scraper?"} + HasCookie{"Has cookie?"} + HasExpired{"Cookie expired?"} + HasSignature{"Has valid +signature?"} + RandomJitter{"Secondary +screening?"} + POWPass{"Proof of +work valid?"} + + style PresentChallenge color:#FFFFFF, fill:#AA00FF, stroke:#AA00FF + style Backend color:#FFFFFF, stroke:#00C853, fill:#00C853 + %%style Fail color:#FFFFFF, stroke:#FF2962, fill:#FF2962 + + Request --> HasMozilla + HasMozilla -- Yes --> HasCookie + HasMozilla -- No --> Backend + HasCookie -- Yes --> HasExpired + HasCookie -- No --> PresentChallenge + HasExpired -- Yes --> PresentChallenge + HasExpired -- No --> HasSignature + HasSignature -- Yes --> RandomJitter + HasSignature -- No --> PresentChallenge + RandomJitter -- Yes --> POWPass + RandomJitter -- No --> Backend + POWPass -- Yes --> Backend + PowPass -- No --> PresentChallenge + PresentChallenge -- Back again for another cycle --> Request +``` + +### Proof of passing challenges + +When a client passes a challenge, Anubis sets an HTTP cookie named `"within.website-x-cmd-anubis-auth"` containing a signed [JWT](https://jwt.io/) (JSON Web Token). This JWT contains the following claims: + +- `challenge`: The challenge string derived from user request metadata +- `nonce`: The nonce / iteration number used to generate the passing response +- `response`: The hash that passed Anubis' checks +- `iat`: When the token was issued +- `nbf`: One minute prior to when the token was issued +- `exp`: The token's expiry week after the token was issued + +This ensures that the token has enough metadata to prove that the token is valid (due to the token's signature), but also so that the server can independently prove the token is valid. This cookie is allowed to be set without triggering an EU cookie banner notification; but depending on facts and circumstances, you may wish to disclose this to your users. + +### Challenge format + +Challenges are formed by taking some user request metadata and using that to generate a SHA-256 checksum. The following request headers are used: + +- `Accept-Encoding`: The content encodings that the requestor supports, such as gzip. +- `Accept-Language`: The language that the requestor would prefer the server respond in, such as English. +- `X-Real-Ip`: The IP address of the requestor, as set by a reverse proxy server. +- `User-Agent`: The user agent string of the requestor. +- The current time in UTC rounded to the nearest week. +- The fingerprint (checksum) of Anubis' private ED25519 key. + +This forms a fingerprint of the requestor using metadata that any requestor already is sending. It also uses time as an input, which is known to both the server and requestor due to the nature of linear timelines. Depending on facts and circumstances, you may wish to disclose this to your users. + +### JWT signing + +Anubis uses an ed25519 keypair to sign the JWTs issued when challenges are passed. Anubis will generate a new ed25519 keypair every time it starts. At this time, there is no way to share this keypair between instance of Anubis, but that will be addressed in future versions. diff --git a/docs/docs/index.mdx b/docs/docs/index.mdx new file mode 100644 index 0000000..2803c5c --- /dev/null +++ b/docs/docs/index.mdx @@ -0,0 +1,28 @@ +--- +sidebar_position: 1 +title: Anubis +--- + +<img + width={256} + src="/img/happy.webp" + alt="A smiling chibi dark-skinned anthro jackal with brown hair and tall ears looking victorious with a thumbs-up" +/> + + + + + + + +Anubis [weighs the soul of your connection](https://en.wikipedia.org/wiki/Weighing_of_souls) using a sha256 proof-of-work challenge in order to protect upstream resources from scraper bots. + +This program is designed to help protect the small internet from the endless storm of requests that flood in from AI companies. Anubis is as lightweight as possible to ensure that everyone can afford to protect the communities closest to them. + +Anubis is a bit of a nuclear response. This will result in your website being blocked from smaller scrapers and may inhibit "good bots" like the Internet Archive. You can configure [bot policy definitions](./policies) to explicitly allowlist them and we are working on a curated set of "known good" bots to allow for a compromise between discoverability and uptime. + +## Support + +If you run into any issues running Anubis, please [open an issue](https://github.com/TecharoHQ/anubis/issues/new?template=Blank+issue) and tag it with the Anubis tag. Please include all the information I would need to diagnose your issue. + +For live chat, please join the [Patreon](https://patreon.com/cadey) and ask in the Patron discord in the channel `#anubis`. diff --git a/docs/docs/installation.mdx b/docs/docs/installation.mdx new file mode 100644 index 0000000..35d1252 --- /dev/null +++ b/docs/docs/installation.mdx @@ -0,0 +1,135 @@ +--- +sidebar_position: 20 +title: Setting up Anubis +--- + +Anubis is meant to sit between your reverse proxy (such as Nginx or Caddy) and your target service. One instance of Anubis must be used per service you are protecting. + +Anubis is shipped in the Docker repo [`ghcr.io/techarohq/anubis`](https://github.com/TecharoHQ/anubis/pkgs/container/anubis). The following tags exist for your convenience: + +| Tag | Meaning | +| :------------------ | :--------------------------------------------------------------------------------------------------------------------------------- | +| `latest` | The latest [tagged release](https://github.com/TecharoHQ/anubis/releases), if you are in doubt, start here. | +| `v<version number>` | The Anubis image for [any given tagged release](https://github.com/TecharoHQ/anubis/tags) | +| `main` | The current build on the `main` branch. Only use this if you need the latest and greatest features as they are merged into `main`. | +| `pr-<number>` | The build associated with PR `#<number>`. Only use this for debugging issues fixed by a PR. | + +Other methods to install Anubis may exist, but the Docker image is currently the only supported method. + +The Docker image runs Anubis as user ID 1000 and group ID 1000. If you are mounting external volumes into Anubis' container, please be sure they are owned by or writable to this user/group. + +Anubis has very minimal system requirements. I suspect that 128Mi of ram may be sufficient for a large number of concurrent clients. Anubis may be a poor fit for apps that use WebSockets and maintain open connections, but I don't have enough real-world experience to know one way or another. + +Anubis uses these environment variables for configuration: + +| Environment Variable | Default value | Explanation | +| :------------------- | :------------------------- | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `BIND` | `:8923` | The TCP port that Anubis listens on. | +| `DIFFICULTY` | `5` | The difficulty of the challenge, or the number of leading zeroes that must be in successful responses. | +| `METRICS_BIND` | `:9090` | The TCP port t |
