Support query-parameter auth (refs #5)

CONTRIBUTING.md

@@ -91,9 +91,29 @@ and if it wants a word before the key in that header:
 "auth": {"header": "authorization", "prefix": "Token"}
 ```
 
-which sends `authorization: Token <key>`. That is the whole credential: one
-key, one header. APIs that need OAuth, a browser login, token refresh, a
-signed request or a second secret cannot be connectors.
+which sends `authorization: Token <key>`. If the API takes the key only as
+a request parameter:
+
+```json
+"auth": {"query": "token"}
+```
+
+sends `?token=<key>` on every request and no credential header. Sprites
+replaces any `token` parameter the Sprite put in the query string rather
+than adding a second one, so a Sprite can never pick the value the API
+sees, and the API never receives the parameter twice. Two things follow
+from the key travelling in the URL:
+
+- Say so in a notice (step 2, last section): URLs are logged more readily
+  than headers, on the vendor's side too, and users deciding whether to
+  connect should know.
+- If the API echoes the request URL in its error bodies, add
+  `"gateway_error_messages": false` to the manifest so those bodies are not
+  shown to users. Check this against a wrong key before you submit.
+
+That is the whole credential: one key, in one header or one query parameter.
+APIs that need OAuth, a browser login, token refresh, a signed request or a
+second secret cannot be connectors.
 
 ### Which requests are allowed
 
@@ -156,8 +176,9 @@ section).
 }
 ```
 
-- `method` is `GET` or `POST`; `path` may carry a query string; `label` is a
-  short name for what the request is ("account", "balance", "search").
+- `method` is `GET` or `POST`; `path` may carry a query string (with query
+  auth, the key is added to it); `label` is a short name for what the
+  request is ("account", "balance", "search").
 - `json` is a request body for `POST` probes, sent as JSON:
   `"json": {"query": "sprites", "numResults": 1}`.
 - `expect` is what a successful response looks like. A 200 only counts when
@@ -247,8 +268,10 @@ section).
 
 `receive_timeout`, `account_id_hash_length`, `hooks`, `validation`,
 `gateway_error_messages` and `playground` are set by the Sprites team when
-needed. Leave them out unless a reviewer asks. Some of the committed
-connectors carry them; that is not a cue to copy.
+needed. Leave them out unless a reviewer asks, with one exception: a
+query-auth connector whose API echoes request URLs sets
+`gateway_error_messages` to `false` (see "How the key is sent"). Some of
+the committed connectors carry them; that is not a cue to copy.
 
 ## Step 3: add the icons
 
@@ -287,8 +310,10 @@ ACME_KEY=... node scripts/probe.mjs acme --key-env ACME_KEY
 This sends the probe exactly as Sprites will and prints each `expect`
 assertion with its result. The name is looked up under `connectors/` and
 then `examples/`. The key is read from the environment variable
-you name and is never printed. If the response is not 200, it prints the
-reason the `errors` rules give it.
+you name and is never printed; with query auth it is part of the request
+URL, and the script replaces it with `[redacted]` in everything it prints.
+If the response is not 200, it prints the reason the `errors` rules give
+it.
 
 If an assertion fails, look at the real response and adjust `expect` until
 every line reads `ok`. Then try a wrong key and confirm the probe fails.
@@ -298,8 +323,9 @@ ACME_KEY=... node scripts/probe.mjs acme --key-env ACME_KEY --write-fixture
 ```
 
 also records the response as `fixtures/acme/probe_200.json` at the
-repository root, with your key and any credential headers removed. A
-recording is optional and is not part of the connector: a connector
+repository root, with any credential headers removed and every string that
+holds your key, as typed or URL-encoded, dropped from the headers and body.
+A recording is optional and is not part of the connector: a connector
 directory holds only its four files, and the checks refuse anything else in
 it. When present, a recording shows reviewers what the assertions were
 written against, and the checks confirm it is named `probe_<status>.json`
@@ -329,7 +355,8 @@ These are limits of the format, so a reviewer cannot waive them:
 - Run code: no custom validation, request rewriting, response parsing or
   request signing.
 - OAuth or any browser flow, token refresh, JWT exchange or multi-part
-  credentials. One API key only.
+  credentials. One API key only, sent as a bearer token, a fixed header or
+  a query parameter.
 - A user-chosen hostname, or any secret beyond the key.
 - Open wildcards on resource roots. Every route is listed by method, and a
   `:param` matches one identifier.

scripts/lib/credential.mjs

@@ -0,0 +1,106 @@
+// Where the key travels, shared by scripts/probe.mjs and the manifest
+// checks. A credential is {header: [name, value]} or {query: [name, value]}.
+// The manifest's `auth` picks one: "bearer" and {"header": name, "prefix"?:
+// text} send a header; {"query": name} puts name=<key> in the query string,
+// replacing every parameter of that name the request already carried, so
+// the caller can never choose the value the API sees. A query credential
+// also appears in the request URL, so anything that prints or records a URL
+// must pass it through redactText first.
+
+// A query parameter name that survives the URL unencoded and cannot smuggle
+// a second parameter or an escape: no separators, whitespace or percent.
+export const PARAM_NAME = /^[^\s=&%]+$/;
+
+export const AUTH_FORMS = 'auth must be "bearer", {"header": name, "prefix"?: text} or {"query": name}';
+
+// Null when `auth` is one of the three forms, otherwise why not. The schema
+// says the same; this gives the checks a message in words.
+export function authProblem(auth) {
+  if (auth === "bearer") return null;
+  if (auth && typeof auth === "object" && !Array.isArray(auth)) {
+    const keys = Object.keys(auth).sort().join(",");
+    const text = (v) => typeof v === "string" && v.length > 0;
+    if (keys === "header" && text(auth.header)) return null;
+    if (keys === "header,prefix" && text(auth.header) && text(auth.prefix)) return null;
+    if (keys === "query") {
+      if (typeof auth.query === "string" && PARAM_NAME.test(auth.query)) return null;
+      return `auth.query must be a parameter name without whitespace, =, & or %, got ${JSON.stringify(auth.query)}`;
+    }
+  }
+  return `${AUTH_FORMS}, got ${JSON.stringify(auth)}`;
+}
+
+export function credential(auth, key) {
+  const why = authProblem(auth);
+  if (why) throw new TypeError(why);
+  if (auth === "bearer") return { header: ["authorization", `Bearer ${key}`] };
+  if (typeof auth.header === "string") return { header: [auth.header, auth.prefix ? `${auth.prefix} ${key}` : key] };
+  return { query: [auth.query, key] };
+}
+
+// x-www-form-urlencoded: letters, digits, - . _ ~ stay, a space becomes +,
+// every other byte is %XX.
+export function encodeWwwForm(text) {
+  let out = "";
+  for (const byte of Buffer.from(String(text), "utf8")) {
+    const c = String.fromCharCode(byte);
+    if (/[A-Za-z0-9._~-]/.test(c)) out += c;
+    else if (c === " ") out += "+";
+    else out += "%" + byte.toString(16).toUpperCase().padStart(2, "0");
+  }
+  return out;
+}
+
+function decodeWwwForm(text) {
+  try {
+    return decodeURIComponent(text.replace(/\+/g, " "));
+  } catch {
+    return text;
+  }
+}
+
+// `query` (no leading ?) with every parameter named `name` removed and
+// name=value appended, URL-encoded. Other pairs keep the caller's encoding;
+// only their keys are decoded for the comparison, so %74oken is token too.
+export function queryString(query, name, value) {
+  const kept = query.split("&").filter((pair) => pair !== "" && decodeWwwForm(pair.split("=", 1)[0]) !== name);
+  return [...kept, `${encodeWwwForm(name)}=${encodeWwwForm(value)}`].join("&");
+}
+
+// Places a credential on a request: a header credential goes first in
+// `headers`, a query credential replaces its parameter in `url` (a full URL
+// or a path, with or without a query string).
+export function apply(url, headers, cred) {
+  if (cred.header) return { url, headers: [cred.header, ...headers] };
+  const [name, value] = cred.query;
+  const at = url.indexOf("?");
+  const [base, query] = at === -1 ? [url, ""] : [url.slice(0, at), url.slice(at + 1)];
+  return { url: `${base}?${queryString(query, name, value)}`, headers };
+}
+
+// The key as typed and as it appears URL-encoded in a query string, both
+// with + and with %20 for a space.
+export function keyForms(key) {
+  return [...new Set([key, encodeWwwForm(key), encodeURIComponent(key)])].filter((form) => form !== "");
+}
+
+// One case-insensitive pattern over every spelling of the key, so an API
+// that echoes the URL with lowercase or mixed-case percent escapes
+// (%2b for %2B) still matches. Over-redaction is acceptable; a key that
+// survives is not. Null for an empty key, which matches nothing.
+export function keyMatcher(key) {
+  const forms = keyForms(key).map((form) => form.replace(/[.*+?^${}()|[\]\\]/g, "\\$&"));
+  return forms.length ? new RegExp(forms.join("|"), "gi") : null;
+}
+
+export function holdsKey(text, key) {
+  const matcher = keyMatcher(key);
+  return matcher !== null && matcher.test(String(text));
+}
+
+// `text` with every spelling of the key replaced, for messages and request
+// lines that may quote a URL.
+export function redactText(text, key) {
+  const matcher = keyMatcher(key);
+  return matcher === null ? String(text) : String(text).replace(matcher, "[redacted]");
+}

scripts/lib/manifest.mjs

@@ -1,10 +1,12 @@
 // Manifest checks the JSON Schema cannot express, shared by
 // scripts/check-connectors.mjs and the unit tests: the probe and every
 // example must be routes the manifest forwards, expectations must be
-// well-formed, and path_param must compile. Returns problems as strings;
-// an empty list means the manifest passes.
+// well-formed, path_param must compile and auth must be one of the forms
+// the probe runner can send. Returns problems as strings; an empty list
+// means the manifest passes.
 import { compilePathParam, pathAllowed } from "./routes.mjs";
 import { problem as expectationProblem } from "./expect.mjs";
+import { authProblem } from "./credential.mjs";
 
 // The gateway matches routes on the path alone; a probe or example may carry
 // a query string for the request.
@@ -27,6 +29,10 @@ function expectProblems(at, expectations) {
 
 export function manifestProblems(manifest) {
   const problems = [];
+  if (manifest.auth !== undefined) {
+    const why = authProblem(manifest.auth);
+    if (why) problems.push(why);
+  }
   const routes = Array.isArray(manifest.allowed_paths) ? manifest.allowed_paths : [];
   let pathParam;
   if (manifest.path_param !== undefined) {