diff options
| author | Xe Iaso <me@xeiaso.net> | 2025-04-02 20:03:28 -0400 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2025-04-03 00:03:28 +0000 |
| commit | 21a9d77788653f30fef4f3b37e8163a833f1a5d9 (patch) | |
| tree | a7a0efc6263c4af705a2f36556374d66387856ee | |
| parent | 266d8c0cc25f9d93ea7da87eb199bc87e41c653e (diff) | |
| download | anubis-21a9d77788653f30fef4f3b37e8163a833f1a5d9.tar.xz anubis-21a9d77788653f30fef4f3b37e8163a833f1a5d9.zip | |
data/botPolicies: allow iMessage scraper by default (#178)
Signed-off-by: Xe Iaso <me@xeiaso.net>
| -rw-r--r-- | data/botPolicies.json | 6 | ||||
| -rw-r--r-- | docs/docs/CHANGELOG.md | 1 | ||||
| -rw-r--r-- | internal/test/playwright_test.go | 6 |
3 files changed, 13 insertions, 0 deletions
diff --git a/data/botPolicies.json b/data/botPolicies.json index d0e27a2..25a7b77 100644 --- a/data/botPolicies.json +++ b/data/botPolicies.json @@ -344,6 +344,12 @@ ] }, { + "_comment": "This has been reverse-engineered through making iMessage's preview function hit a URL that prints the user-agent in the server logs.", + "name": "iMessage preview", + "user_agent_regex": ".*facebookexternalhit/1\\.1 Facebot Twitterbot/1\\.0$", + "action": "ALLOW" + }, + { "name": "us-artificial-intelligence-scraper", "user_agent_regex": "\\+https\\://github\\.com/US-Artificial-Intelligence/scraper", "action": "DENY" diff --git a/docs/docs/CHANGELOG.md b/docs/docs/CHANGELOG.md index 11c85ca..273d8d9 100644 --- a/docs/docs/CHANGELOG.md +++ b/docs/docs/CHANGELOG.md @@ -11,6 +11,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +- Allow iMessage's link preview fetcher through Anubis by default. - Added a periodic cleanup routine for the decaymap that removes expired entries, ensuring stale data is properly pruned. - Added a no-store Cache-Control header to the challenge page - Hide the directory listings for Anubis' internal static content diff --git a/internal/test/playwright_test.go b/internal/test/playwright_test.go index ae4022e..88368c7 100644 --- a/internal/test/playwright_test.go +++ b/internal/test/playwright_test.go @@ -64,6 +64,12 @@ var ( userAgent: "Mozilla/5.0 (compatible; Kagibot/1.0; +https://kagi.com/bot)", }, { + name: "iMessageScraper", + action: actionAllow, + realIP: placeholderIP, + userAgent: "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_1) AppleWebKit/601.2.4 (KHTML, like Gecko) Version/9.0.1 Safari/601.2.4 facebookexternalhit/1.1 Facebot Twitterbot/1.0", + }, + { name: "unknownAgent", action: actionAllow, realIP: placeholderIP, |