fix(config): remove trailing newlines in regexes (#373)v1.17.0-beta31.17.0-beta2

Closes #372

Fun YAML fact of the day:

What is the difference between how these two expressions are parsed?

```yaml
foo: >
  bar
```

```yaml
foo: >-
  bar
```

They are invisible in yaml, but when you evaluate them to JSON the
difference is obvious:

```json
{
  "foo": "bar\n"
}
```

```json
{
  "foo": "bar"
}
```

User-Agent strings, URL path values, and HTTP headers _do_ end in
newlines in HTTP/1.1 wire form, but that newline is usually stripped
before the server actually handles it. Also HTTP/2 is a thing and does
not terminate header values with newlines.

This change makes Anubis more aggressively detect mistaken uses of the
yaml `>` operator and nudges the user into using the yaml `>-` operator
which does not append the trailing newline.

I had honestly forgotten about this YAML behavior because it wasn't
relevant for so long. Oops! Glad I released a beta.

Whenever you get into this state, Anubis will throw a config parsing
error and then give you a message hinting at the folly of your ways.

```
config.Bot: regular expression ends with newline (try >- instead of > in yaml)
```

Big thanks to https://yaml-multiline.info, this helped me realize my
folly instantly.

@aiverson, this is official permission to say "told you so".

Signed-off-by: Xe Iaso <me@xeiaso.net>

2 files changed, 38 insertions, 0 deletions

diff --git a/lib/policy/config/testdata/bad/regex_ends_newline.json b/lib/policy/config/testdata/bad/regex_ends_newline.json
new file mode 100644
index 0000000..14c7fa9
--- /dev/null
+++ b/lib/policy/config/testdata/bad/regex_ends_newline.json
@@ -0,0 +1,21 @@
+{
+  "bots": [
+    {
+      "name": "user-agent-ends-newline",
+      "user_agent_regex": "Mozilla\n",
+      "action": "CHALLENGE"
+    },
+    {
+      "name": "path-ends-newline",
+      "path_regex": "^/evil/.*$\n",
+      "action": "CHALLENGE"
+    },
+    {
+      "name": "headers-ends-newline",
+      "headers_regex": {
+        "CF-Worker": ".*\n"
+      },
+      "action": "CHALLENGE"
+    }
+  ]
+}
diff --git a/lib/policy/config/testdata/bad/regex_ends_newline.yaml b/lib/policy/config/testdata/bad/regex_ends_newline.yaml
new file mode 100644
index 0000000..1f0ae85
--- /dev/null
+++ b/lib/policy/config/testdata/bad/regex_ends_newline.yaml
@@ -0,0 +1,17 @@
+bots:
+- name: user-agent-ends-newline
+  # Subtle bug: this ends with a newline
+  user_agent_regex: >
+    Mozilla
+  action: CHALLENGE
+- name: path-ends-newline
+  # Subtle bug: this ends with a newline
+  path_regex: >
+    ^/evil/.*$
+  action: CHALLENGE
+- name: headers-ends-newline
+  # Subtle bug: this ends with a newline
+  headers_regex:
+    CF-Worker: >
+      .*
+  action: CHALLENGE
\ No newline at end of file