From f987067247cc55cde635900243a4b2acf6559884 Mon Sep 17 00:00:00 2001 From: turtton Date: Tue, 23 Jun 2026 23:10:51 +0900 Subject: [PATCH 01/15] Add AGENTS.md --- AGENTS.md | 65 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 65 insertions(+) create mode 100644 AGENTS.md diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..10ccf3e --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,65 @@ +# AGENTS.md — ap-testbox + +Docker Compose local dev environment for [Misskey](https://github.com/misskey-dev/misskey). This is NOT the Misskey source — the `misskey/` directory is gitignored and must be cloned separately. + +## First-time setup (order matters) + +```bash +./gencert.sh # generates SSL certs via Docker (mkcert) +git clone -b master https://github.com/misskey-dev/misskey.git +docker compose build --pull +docker compose run --rm misskey pnpm run init # runs DB migrations +docker compose up -d +``` + +## Key details + +- **URL**: `https://misskey.localhost:4430` (nginx reverse-proxies to misskey:3000; HTTP on 8080 redirects to HTTPS). Also available at `https://shuttlepub.localhost:4430` which proxies to `host.docker.internal:12864`. +- **Cert gen**: `./gencert.sh` builds a Docker image with `mkcert`, generates a root CA and certs for `misskey.localhost` and `shuttlepub.localhost`. Certs are mounted into nginx (`/etc/nginx/certs/`). The root CA is also mounted into the misskey container and loaded via `NODE_EXTRA_CA_CERTS=/certs/rootCA.pem` for outbound HTTPS to other instances. +- **Misskey config**: `configs/misskey/.config/default.yml` — url, db host/port/credentials, redis host/port, full-text search (sqlLike by default), ID generation (`aidx`). +- **DB credentials**: `configs/misskey/.config/docker.env` — user `example-misskey-user`, pass `example-misskey-pass`, db `misskey`. `DATABASE_URL` is assembled from env vars. +- **Services**: nginx (1-alpine), misskey (built from local clone), postgres (15-alpine), redis (7-alpine) +- **Volumes**: `db_data` (postgres), `redis_data` (redis), bind-mounts for misskey files, config (ro), and certs +- **Host ports**: `8080:80` (HTTP), `4430:443` (HTTPS), `5432:5432` (postgres) +- **Nginx network alias**: `shuttlepub.localhost` — set on the `misskey-nginx` container so it can serve host-based vhosts + +## Directory layout + +``` +ap-testbox/ +├── compose.yml # All services +├── gencert.sh # Fires up Docker-based mkcert +├── gencert/ # Dockerfile + mkcert.sh script +├── configs/ +│ ├── nginx/confs/ # nginx server blocks (SSL proxy) +│ │ ├── misskey.conf # misskey.localhost → misskey:3000 +│ │ └── shuttlepub.conf # shuttlepub.localhost → host.docker.internal:12864 +│ └── misskey/ +│ ├── .config/ +│ │ ├── default.yml # Misskey application config +│ │ └── docker.env # DB env vars +│ └── nginx/conf.d/ # nginx conf (HTTP→HTTPS redirect) +├── certs/ # gitignored, generated by gencert +└── misskey/ # gitignored, cloned from upstream +``` + +## Common commands + +| Action | Command | +|---|---| +| Build images | `docker compose build --pull` | +| Start all | `docker compose up -d` | +| Stop all | `docker compose down` | +| Run migration | `docker compose run --rm misskey pnpm run init` | +| View misskey logs | `docker compose logs -f misskey` | +| Rebuild misskey only | `docker compose build --no-cache misskey` | +| Access postgres | `docker compose exec db psql -U example-misskey-user -d misskey` | + +## Gotchas + +- The `misskey/` directory and `certs/` are in `.gitignore`. Any local modifications to misskey source will NOT be committed. +- The nginx config blocks `/sw.js` (returns 404) — Misskey's service worker registration is disabled in this setup. +- `host.docker.internal:host-gateway` is set on the nginx container for potential host machine access (used by `shuttlepub.localhost` to proxy to the host). +- Misskey runs in `NODE_ENV=development` by default (set in compose.yml). +- After cloning/updating the misskey submodule, run migrations again if the schema changed. +- The README says `https://misskey.local:4430` but the actual nginx config uses `misskey.localhost`. Use the latter. From 0e36a9a545b2f69acf9746a876da0eabfb1ac27c Mon Sep 17 00:00:00 2001 From: turtton Date: Wed, 24 Jun 2026 22:15:06 +0900 Subject: [PATCH 02/15] Add Iceshrimp configuration files Replace Misskey with Iceshrimp v2026.5.1 as the reference Fediverse instance. Includes default.yml, docker.env, and nginx redirect config. Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus --- .gitignore | 2 +- configs/iceshrimp/.config/default.yml | 20 ++++++++++++++++++++ configs/iceshrimp/.config/docker.env | 8 ++++++++ configs/iceshrimp/nginx/conf.d/nginx.conf | 9 +++++++++ 4 files changed, 38 insertions(+), 1 deletion(-) create mode 100644 configs/iceshrimp/.config/default.yml create mode 100644 configs/iceshrimp/.config/docker.env create mode 100644 configs/iceshrimp/nginx/conf.d/nginx.conf diff --git a/.gitignore b/.gitignore index eda60dd..6862dd8 100644 --- a/.gitignore +++ b/.gitignore @@ -1,2 +1,2 @@ /certs -/misskey \ No newline at end of file +/misskey diff --git a/configs/iceshrimp/.config/default.yml b/configs/iceshrimp/.config/default.yml new file mode 100644 index 0000000..6604c00 --- /dev/null +++ b/configs/iceshrimp/.config/default.yml @@ -0,0 +1,20 @@ +url: https://iceshrimp.localhost/ +port: 3000 + +db: + host: db + port: 5432 + db: iceshrimp + user: example-iceshrimp-user + pass: example-iceshrimp-pass + +redis: + host: redis + port: 6379 + +id: 'aidx' + +# E2E testing: allow federation to Docker network private IPs +allowedPrivateNetworks: + - '172.16.0.0/12' + - '10.0.0.0/8' diff --git a/configs/iceshrimp/.config/docker.env b/configs/iceshrimp/.config/docker.env new file mode 100644 index 0000000..89af8d3 --- /dev/null +++ b/configs/iceshrimp/.config/docker.env @@ -0,0 +1,8 @@ +# iceshrimp settings +# ICESHRIMP_URL=https://example.tld/ + +# db settings +POSTGRES_PASSWORD=example-iceshrimp-pass +POSTGRES_USER=example-iceshrimp-user +POSTGRES_DB=iceshrimp +DATABASE_URL="postgres://${POSTGRES_USER}:${POSTGRES_PASSWORD}@db:5432/${POSTGRES_DB}" diff --git a/configs/iceshrimp/nginx/conf.d/nginx.conf b/configs/iceshrimp/nginx/conf.d/nginx.conf new file mode 100644 index 0000000..3250e06 --- /dev/null +++ b/configs/iceshrimp/nginx/conf.d/nginx.conf @@ -0,0 +1,9 @@ +server { + listen 80; + server_name *.localhost; + return 301 https://$host$request_uri; +} + +include /etc/nginx/confs/iceshrimp.conf; +include /etc/nginx/confs/shuttlepub.conf; +include /etc/nginx/confs/test-ap.conf; From 72cb0420b99585807ffd237fe968acee205c20e4 Mon Sep 17 00:00:00 2001 From: turtton Date: Wed, 24 Jun 2026 22:15:13 +0900 Subject: [PATCH 03/15] Update nginx proxy config for Iceshrimp and Custom AP MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add vhost configs for iceshrimp.localhost (→iceshrimp:3000), test-ap.localhost (→custom-ap:12864), update shuttlepub.localhost (→custom-ap:12864). Add HTTP→HTTPS redirect with vhost includes. Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus --- configs/nginx/conf.d/nginx.conf | 9 +++++++++ configs/nginx/confs/iceshrimp.conf | 23 +++++++++++++++++++++++ configs/nginx/confs/shuttlepub.conf | 12 ++++++------ configs/nginx/confs/test-ap.conf | 16 ++++++++++++++++ 4 files changed, 54 insertions(+), 6 deletions(-) create mode 100644 configs/nginx/conf.d/nginx.conf create mode 100644 configs/nginx/confs/iceshrimp.conf create mode 100644 configs/nginx/confs/test-ap.conf diff --git a/configs/nginx/conf.d/nginx.conf b/configs/nginx/conf.d/nginx.conf new file mode 100644 index 0000000..3250e06 --- /dev/null +++ b/configs/nginx/conf.d/nginx.conf @@ -0,0 +1,9 @@ +server { + listen 80; + server_name *.localhost; + return 301 https://$host$request_uri; +} + +include /etc/nginx/confs/iceshrimp.conf; +include /etc/nginx/confs/shuttlepub.conf; +include /etc/nginx/confs/test-ap.conf; diff --git a/configs/nginx/confs/iceshrimp.conf b/configs/nginx/confs/iceshrimp.conf new file mode 100644 index 0000000..0c2ace6 --- /dev/null +++ b/configs/nginx/confs/iceshrimp.conf @@ -0,0 +1,23 @@ +server { + listen 443 ssl; + server_name iceshrimp.localhost; + + ssl_certificate /etc/nginx/certs/iceshrimp.crt; + ssl_certificate_key /etc/nginx/certs/iceshrimp.key; + + location /sw.js { + return 404; + } + + location / { + proxy_pass http://iceshrimp:3000; + proxy_http_version 1.1; + + proxy_set_header Host $host; + proxy_set_header X-Forwarded-Proto https; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + + proxy_set_header Upgrade $http_upgrade; + proxy_set_header Connection "upgrade"; + } +} diff --git a/configs/nginx/confs/shuttlepub.conf b/configs/nginx/confs/shuttlepub.conf index cbff7ec..55a61dc 100644 --- a/configs/nginx/confs/shuttlepub.conf +++ b/configs/nginx/confs/shuttlepub.conf @@ -2,15 +2,15 @@ server { listen 443 ssl; server_name shuttlepub.localhost; - ssl_certificate /etc/nginx/certs/shuttlepub.crt; + ssl_certificate /etc/nginx/certs/shuttlepub.crt; ssl_certificate_key /etc/nginx/certs/shuttlepub.key; location / { + proxy_pass http://custom-ap:12864; proxy_http_version 1.1; - proxy_set_header Upgrade $http_upgrade; - proxy_set_header Connection "upgrade"; - proxy_set_header Host $host; - proxy_pass http://host.docker.internal:12864; + proxy_set_header Host $host; + proxy_set_header X-Forwarded-Proto https; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; } -} \ No newline at end of file +} diff --git a/configs/nginx/confs/test-ap.conf b/configs/nginx/confs/test-ap.conf new file mode 100644 index 0000000..f3a6b54 --- /dev/null +++ b/configs/nginx/confs/test-ap.conf @@ -0,0 +1,16 @@ +server { + listen 443 ssl; + server_name test-ap.localhost; + + ssl_certificate /etc/nginx/certs/test-ap.crt; + ssl_certificate_key /etc/nginx/certs/test-ap.key; + + location / { + proxy_pass http://custom-ap:12864; + proxy_http_version 1.1; + + proxy_set_header Host $host; + proxy_set_header X-Forwarded-Proto https; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + } +} From 56e8ae477a0cd30a052b401e96ef7bf7a8d408fb Mon Sep 17 00:00:00 2001 From: turtton Date: Wed, 24 Jun 2026 22:15:22 +0900 Subject: [PATCH 04/15] Rewrite compose.yml and cert generation for Iceshrimp-based E2E setup Replace Misskey build with Iceshrimp image (v2026.5.1). Add custom-ap and test-runner services. Update cert generation for iceshrimp.localhost and test-ap.localhost domains. Add .env.example for build context and test token. Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus --- .env.example | 6 ++++ compose.yml | 89 ++++++++++++++++++++++++++++++++++++----------- gencert.nu | 2 +- gencert.sh | 2 +- gencert/mkcert.sh | 3 +- 5 files changed, 79 insertions(+), 23 deletions(-) create mode 100644 .env.example diff --git a/.env.example b/.env.example new file mode 100644 index 0000000..d95beef --- /dev/null +++ b/.env.example @@ -0,0 +1,6 @@ +# Custom AP build context +CUSTOM_AP_CONTEXT=../custom-ap +CUSTOM_AP_DOCKERFILE=Dockerfile + +# Test mode token (must match between test-runner and custom-ap) +TEST_MODE_TOKEN=ap-testbox-test-token diff --git a/compose.yml b/compose.yml index 7e7aac2..9340d55 100644 --- a/compose.yml +++ b/compose.yml @@ -1,41 +1,90 @@ services: - misskey-nginx: + nginx: image: nginx:1-alpine + ports: + - "8080:80" + - "4430:443" volumes: - - ./configs/nginx/confs:/etc/nginx/confs - - ./configs/misskey/nginx/conf.d:/etc/nginx/conf.d - - ./certs/nginx:/etc/nginx/certs + - ./configs/nginx/conf.d:/etc/nginx/conf.d:ro + - ./configs/nginx/confs:/etc/nginx/confs:ro + - ./certs/nginx:/etc/nginx/certs:ro + depends_on: + - iceshrimp + - custom-ap networks: default: aliases: + - iceshrimp.localhost + - test-ap.localhost - shuttlepub.localhost - ports: - - 8080:80 - - 4430:443 extra_hosts: - host.docker.internal:host-gateway - misskey: - build: ./misskey + + iceshrimp: + image: iceshrimp.dev/iceshrimp/iceshrimp:v2026.5.1 + depends_on: + - db + - redis + environment: + NODE_ENV: production + NODE_EXTRA_CA_CERTS: /certs/rootCA.pem volumes: - - ./configs/misskey/files:/misskey/files - - ./configs/misskey/.config:/misskey/.config:ro - - ./certs/root:/certs + - ./configs/iceshrimp/files:/iceshrimp/files + - ./configs/iceshrimp/.config:/iceshrimp/.config:ro + - ./certs/root/rootCA.pem:/certs/rootCA.pem:ro + + custom-ap: + build: + context: ${CUSTOM_AP_CONTEXT:-../custom-ap} + dockerfile: ${CUSTOM_AP_DOCKERFILE:-Dockerfile} environment: - - NODE_ENV=development - - NODE_EXTRA_CA_CERTS=/certs/rootCA.pem + TEST_MODE: "true" + TEST_MODE_TOKEN: ${TEST_MODE_TOKEN:-ap-testbox-test-token} + PUBLIC_BASE_URL: https://test-ap.localhost + LEGACY_PUBLIC_BASE_URL: https://shuttlepub.localhost + ICESHRIMP_BASE_URL: https://iceshrimp.localhost + NODE_EXTRA_CA_CERTS: /certs/rootCA.pem + PORT: "12864" + volumes: + - ./certs/root/rootCA.pem:/certs/rootCA.pem:ro + expose: + - "12864" + + test-runner: + build: + context: ./test-runner depends_on: + - nginx + - iceshrimp + - custom-ap - db - redis + environment: + NODE_ENV: test + NODE_EXTRA_CA_CERTS: /certs/rootCA.pem + ICESHRIMP_BASE_URL: https://iceshrimp.localhost + CUSTOM_AP_BASE_URL: https://test-ap.localhost + CUSTOM_AP_LEGACY_BASE_URL: https://shuttlepub.localhost + TEST_MODE_TOKEN: ${TEST_MODE_TOKEN:-ap-testbox-test-token} + PGHOST: db + PGPORT: "5432" + PGDATABASE: iceshrimp + PGUSER: example-iceshrimp-user + PGPASSWORD: example-iceshrimp-pass + volumes: + - ./certs/root/rootCA.pem:/certs/rootCA.pem:ro + db: - image: postgres:15-alpine - ports: - - 5432:5432 + image: postgres:16-alpine + env_file: + - ./configs/iceshrimp/.config/docker.env volumes: - db_data:/var/lib/postgresql/data - env_file: - - ./configs/misskey/.config/docker.env + ports: + - "127.0.0.1:5432:5432" + redis: - image: redis:7-alpine + image: valkey/valkey:7-alpine volumes: - redis_data:/data diff --git a/gencert.nu b/gencert.nu index b1c8080..0d15c69 100644 --- a/gencert.nu +++ b/gencert.nu @@ -7,5 +7,5 @@ docker build ./gencert/. -t shuttlepub/gencert:latest if (sys host | get name) == "Windows" { docker run --mount "type=bind,source=.\\certs,target=/certs" -it shuttlepub/gencert:latest } else { - docker run --mount "type=bind,source=./certs,target=/certs" -it shuttlepub/gencert:latest + docker run --mount "type=bind,source="$(pwd)/certs",target=/certs" -it shuttlepub/gencert:latest } diff --git a/gencert.sh b/gencert.sh index bd78a65..f5d9197 100755 --- a/gencert.sh +++ b/gencert.sh @@ -6,4 +6,4 @@ mkdir -p ./certs/nginx # Build the gencert image and run it docker build ./gencert/. -t shuttlepub/gencert:latest -docker run --mount "type=bind,source=./certs,target=/certs" -it shuttlepub/gencert:latest +docker run --mount "type=bind,source="$(pwd)/certs",target=/certs" -it shuttlepub/gencert:latest diff --git a/gencert/mkcert.sh b/gencert/mkcert.sh index d1c9d4c..afdfbdb 100755 --- a/gencert/mkcert.sh +++ b/gencert/mkcert.sh @@ -1,5 +1,6 @@ #!/usr/bin/env bash cp /root/.local/share/mkcert/rootCA.pem /certs/root/ -mkcert -cert-file /certs/nginx/misskey.crt -key-file /certs/nginx/misskey.key misskey.localhost +mkcert -cert-file /certs/nginx/iceshrimp.crt -key-file /certs/nginx/iceshrimp.key iceshrimp.localhost +mkcert -cert-file /certs/nginx/test-ap.crt -key-file /certs/nginx/test-ap.key test-ap.localhost mkcert -cert-file /certs/nginx/shuttlepub.crt -key-file /certs/nginx/shuttlepub.key shuttlepub.localhost From 3b7055b1f9a877ce63b7436727a7f98337a66c19 Mon Sep 17 00:00:00 2001 From: turtton Date: Wed, 24 Jun 2026 22:15:29 +0900 Subject: [PATCH 05/15] Add E2E test runner scaffold and Custom AP interface contract Set up TypeScript + Vitest test runner with Dockerfile. Define Custom AP interface contract documenting required ActivityPub and test-only endpoints. Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus --- docs/custom-ap-contract.md | 166 +++++++++++++++++++++++++++++++++++ test-runner/Dockerfile | 14 +++ test-runner/package.json | 13 +++ test-runner/tsconfig.json | 14 +++ test-runner/vitest.config.ts | 16 ++++ 5 files changed, 223 insertions(+) create mode 100644 docs/custom-ap-contract.md create mode 100644 test-runner/Dockerfile create mode 100644 test-runner/package.json create mode 100644 test-runner/tsconfig.json create mode 100644 test-runner/vitest.config.ts diff --git a/docs/custom-ap-contract.md b/docs/custom-ap-contract.md new file mode 100644 index 0000000..827d370 --- /dev/null +++ b/docs/custom-ap-contract.md @@ -0,0 +1,166 @@ +# Custom AP Interface Contract + +Your custom ActivityPub implementation must satisfy this contract to work with the ap-testbox E2E testing framework. + +## Required Environment Variables + +| Variable | Example | Description | +|---|---|---| +| `TEST_MODE` | `true` | Enables test-only endpoints | +| `TEST_MODE_TOKEN` | `ap-testbox-test-token` | Bearer token for test-only endpoints | +| `PUBLIC_BASE_URL` | `https://test-ap.localhost` | Canonical public URL | +| `LEGACY_PUBLIC_BASE_URL` | `https://shuttlepub.localhost` | Legacy alias for backward compat | +| `ICESHRIMP_BASE_URL` | `https://iceshrimp.localhost` | Peering Iceshrimp instance URL | +| `PORT` | `12864` | HTTP listen port | +| `NODE_EXTRA_CA_CERTS` | `/certs/rootCA.pem` | CA cert bundle for outbound HTTPS | + +## Public ActivityPub Endpoints + +### GET `.well-known/webfinger?resource=acct:{username}@{host}` + +Must return standard WebFinger JSON linking to the Actor URL. + +```json +{ + "subject": "acct:remote-alice@test-ap.localhost", + "links": [ + { + "rel": "self", + "type": "application/activity+json", + "href": "https://test-ap.localhost/users/remote-alice" + } + ] +} +``` + +### GET `/users/:username` + +Must return an ActivityPub Actor object. + +Required fields: + +| Field | Type | Description | +|---|---|---| +| `@context` | string/array | `"https://www.w3.org/ns/activitystreams"` | +| `id` | string | `https://test-ap.localhost/users/{username}` | +| `type` | string | `"Person"` or `"Service"` or `"Application"` | +| `preferredUsername` | string | Username | +| `inbox` | string | `https://test-ap.localhost/users/{username}/inbox` | +| `outbox` | string | `https://test-ap.localhost/users/{username}/outbox` | +| `publicKey.id` | string | `https://test-ap.localhost/users/{username}#main-key` | +| `publicKey.owner` | string | `https://test-ap.localhost/users/{username}` | +| `publicKey.publicKeyPem` | string | PEM-encoded RSA public key | + +### POST `/users/:username/inbox` + +Accepts ActivityPub activities from remote instances (Iceshrimp). + +**Required behavior:** +- Accept POST with `Content-Type: application/activity+json` +- Validate HTTP Signature +- Return `202 Accepted` or `200 OK` on success +- Store received activities for retrieval via `GET /__test__/inbox` + +## Test-Only Endpoints + +> All test-only endpoints MUST only be enabled when `TEST_MODE=true`. +> All test-only endpoints MUST require `Authorization: Bearer {TEST_MODE_TOKEN}`, with the exception of `GET /__test__/health` which does NOT require authentication (so the test runner can check availability before obtaining a token). + +### `GET /__test__/health` + +Health check for the test runner. Does NOT require authorization. + +**Response `200 OK`:** +```json +{ + "status": "ok" +} +``` + +### `POST /__test__/reset` + +Delete all test data (actors, inbox items, etc.) to return to clean state. + +**Response `200 OK`:** +```json +{ + "status": "ok" +} +``` + +### `POST /__test__/actors` + +Register a test actor's public key. The system should create an ActivityPub Actor accessible at `/users/{username}`. + +**Request:** +```json +{ + "username": "remote-alice", + "publicKeyPem": "-----BEGIN PUBLIC KEY-----\n...\n-----END PUBLIC KEY-----" +} +``` + +**Response `201 Created`:** +```json +{ + "id": "https://test-ap.localhost/users/remote-alice", + "inbox": "https://test-ap.localhost/users/remote-alice/inbox", + "outbox": "https://test-ap.localhost/users/remote-alice/outbox", + "publicKeyId": "https://test-ap.localhost/users/remote-alice#main-key" +} +``` + +### `GET /__test__/inbox` + +Return all received ActivityPub activities (in delivery order, most recent last, up to a reasonable limit). + +**Response `200 OK`:** +```json +{ + "items": [ + { + "id": "https://iceshrimp.localhost/activities/...", + "type": "Accept", + "actor": "https://iceshrimp.localhost/users/alice", + "object": { + "id": "https://test-ap.localhost/users/remote-alice#follow/...", + "type": "Follow" + } + } + ] +} +``` + +## Test Flow Overview + +``` +test-runner custom-ap iceshrimp + | | | + |-- POST /__test__/actors -->| (register remote-alice) | + | | | + |-- GET /users/remote-alice ->| (AP Actor document) | + | | | + |-- signed POST to --------->| | + | iceshrimp/user/inbox | (Follow activity) | + | | | + | |<-- POST /inbox ------------| + | | (Accept activity) | + | | | + |-- GET /__test__/inbox ---->| | + | ← Accept activity | | + | | | + |-- POST to iceshrimp API -->| (create note) | + | | | + | |<-- POST /inbox ------------| + | | (Create(Note)) | + | | | + |-- GET /__test__/inbox ---->| | + | ← Create(Note) activity | | +``` + +## Notes + +- The test runner expects the Custom AP to play the role of a **remote instance** that initiates federation actions (Follow, Like, etc.) and can **receive** activities from Iceshrimp. +- HTTP Signatures use draft-cavage-http-signatures with `(request-target) host date digest` headers and RSA-SHA256. +- All activities use the `https://www.w3.org/ns/activitystreams` context. +- The `TEST_MODE_TOKEN` must match between the test-runner and custom-ap environments. diff --git a/test-runner/Dockerfile b/test-runner/Dockerfile new file mode 100644 index 0000000..a763ea6 --- /dev/null +++ b/test-runner/Dockerfile @@ -0,0 +1,14 @@ +FROM node:22-alpine + +RUN apk add --no-cache ca-certificates + +WORKDIR /app + +COPY package.json ./ +RUN npm install + +COPY tsconfig.json vitest.config.ts ./ +COPY src ./src +COPY tests ./tests + +CMD ["npm", "test"] diff --git a/test-runner/package.json b/test-runner/package.json new file mode 100644 index 0000000..ee24a37 --- /dev/null +++ b/test-runner/package.json @@ -0,0 +1,13 @@ +{ + "name": "ap-testbox-runner", + "private": true, + "type": "module", + "scripts": { + "test": "vitest run --config vitest.config.ts" + }, + "devDependencies": { + "@types/node": "^22.0.0", + "typescript": "^5.8.0", + "vitest": "^3.1.0" + } +} diff --git a/test-runner/tsconfig.json b/test-runner/tsconfig.json new file mode 100644 index 0000000..6523f1c --- /dev/null +++ b/test-runner/tsconfig.json @@ -0,0 +1,14 @@ +{ + "compilerOptions": { + "target": "ES2022", + "module": "ESNext", + "moduleResolution": "bundler", + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "outDir": "dist", + "rootDir": ".", + "types": ["node"] + }, + "include": ["src/**/*", "tests/**/*"] +} diff --git a/test-runner/vitest.config.ts b/test-runner/vitest.config.ts new file mode 100644 index 0000000..0c823b8 --- /dev/null +++ b/test-runner/vitest.config.ts @@ -0,0 +1,16 @@ +import { defineConfig } from 'vitest/config'; + +export default defineConfig({ + test: { + include: ['tests/**/*.test.ts'], + testTimeout: 60_000, + hookTimeout: 120_000, + pool: 'forks', + poolOptions: { + forks: { + singleFork: true, + }, + }, + setupFiles: ['tests/setup.ts'], + }, +}); From 2cc28bb90b99ee0286cda370e776b756e7e69112 Mon Sep 17 00:00:00 2001 From: turtton Date: Wed, 24 Jun 2026 22:15:38 +0900 Subject: [PATCH 06/15] Add test runner API clients and HTTP Signature implementation Implement Iceshrimp API client (admin user creation, notes, follow, reactions), Custom AP test endpoint client (health, reset, actor registration, inbox polling), and draft-cavage HTTP Signature with RSA-SHA256. Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus --- test-runner/src/config.ts | 18 +++ test-runner/src/custom-ap-client.ts | 126 +++++++++++++++ test-runner/src/http-signature.ts | 93 +++++++++++ test-runner/src/iceshrimp-api.ts | 240 ++++++++++++++++++++++++++++ 4 files changed, 477 insertions(+) create mode 100644 test-runner/src/config.ts create mode 100644 test-runner/src/custom-ap-client.ts create mode 100644 test-runner/src/http-signature.ts create mode 100644 test-runner/src/iceshrimp-api.ts diff --git a/test-runner/src/config.ts b/test-runner/src/config.ts new file mode 100644 index 0000000..075124c --- /dev/null +++ b/test-runner/src/config.ts @@ -0,0 +1,18 @@ +export const config = { + iceshrimpBaseUrl: process.env.ICESHRIMP_BASE_URL ?? 'https://iceshrimp.localhost', + customApBaseUrl: process.env.CUSTOM_AP_BASE_URL ?? 'https://test-ap.localhost', + customApLegacyBaseUrl: process.env.CUSTOM_AP_LEGACY_BASE_URL ?? 'https://shuttlepub.localhost', + + testModeToken: process.env.TEST_MODE_TOKEN ?? 'ap-testbox-test-token', + + // Admin credentials for initial user creation + adminUsername: process.env.ADMIN_USERNAME ?? 'test-admin', + adminPassword: process.env.ADMIN_PASSWORD ?? 'test-admin-pass', + + // DB connection (optional, for direct verification) + pgHost: process.env.PGHOST, + pgPort: Number(process.env.PGPORT ?? 5432), + pgDatabase: process.env.PGDATABASE, + pgUser: process.env.PGUSER, + pgPassword: process.env.PGPASSWORD, +} as const; diff --git a/test-runner/src/custom-ap-client.ts b/test-runner/src/custom-ap-client.ts new file mode 100644 index 0000000..c119d64 --- /dev/null +++ b/test-runner/src/custom-ap-client.ts @@ -0,0 +1,126 @@ +/** + * Client for the Custom AP system's test-only endpoints. + */ +import { config } from './config.js'; + +export interface InboxItem { + id: string; + type: string; + actor: string; + object: Record; +} + +export interface InboxResponse { + items: InboxItem[]; +} + +export interface ActorRegistration { + id: string; + inbox: string; + outbox: string; + publicKeyId: string; +} + +export class CustomApClient { + constructor( + public readonly baseUrl: string = config.customApBaseUrl, + private readonly testModeToken: string = config.testModeToken, + ) {} + + private authHeaders(): Record { + return { + Authorization: `Bearer ${this.testModeToken}`, + 'Content-Type': 'application/json', + }; + } + + /** + * Check if Custom AP is alive. + */ + async healthCheck(): Promise { + try { + const res = await fetch(`${this.baseUrl}/__test__/health`, { + headers: this.authHeaders(), + signal: AbortSignal.timeout(5000), + }); + return res.ok; + } catch { + return false; + } + } + + /** + * Reset all test data on the Custom AP. + */ + async reset(): Promise { + const res = await fetch(`${this.baseUrl}/__test__/reset`, { + method: 'POST', + headers: this.authHeaders(), + }); + if (!res.ok) { + throw new Error(`Custom AP reset failed: ${res.status} ${await res.text()}`); + } + } + + /** + * Register a test actor with the Custom AP. + */ + async registerActor(username: string, publicKeyPem: string): Promise { + const res = await fetch(`${this.baseUrl}/__test__/actors`, { + method: 'POST', + headers: this.authHeaders(), + body: JSON.stringify({ username, publicKeyPem }), + }); + if (!res.ok) { + throw new Error(`Custom AP register actor failed: ${res.status} ${await res.text()}`); + } + return res.json() as Promise; + } + + /** + * Get the current inbox contents (all received ActivityPub activities). + */ + async getInbox(): Promise { + const res = await fetch(`${this.baseUrl}/__test__/inbox`, { + headers: this.authHeaders(), + }); + if (!res.ok) { + throw new Error(`Custom AP inbox fetch failed: ${res.status} ${await res.text()}`); + } + return res.json() as Promise; + } + + /** + * Poll the inbox until a matching activity appears, or timeout. + */ + async pollInbox( + predicate: (item: InboxItem) => boolean, + maxRetries = 15, + intervalMs = 2000, + ): Promise { + for (let i = 0; i < maxRetries; i++) { + const inbox = await this.getInbox(); + const match = inbox.items.find(predicate); + if (match) { + return match; + } + await new Promise((r) => setTimeout(r, intervalMs)); + } + throw new Error(`Custom AP inbox: activity matching predicate not found after ${maxRetries * intervalMs}ms`); + } + + /** + * Get an actor's public Actor object (for WebFinger / AP Actor resolution). + */ + async getActor(username: string): Promise> { + const res = await fetch(`${this.baseUrl}/users/${username}`, { + headers: { + Accept: 'application/activity+json', + }, + }); + if (!res.ok) { + throw new Error(`Custom AP actor fetch failed: ${res.status} ${await res.text()}`); + } + return res.json() as Promise>; + } +} diff --git a/test-runner/src/http-signature.ts b/test-runner/src/http-signature.ts new file mode 100644 index 0000000..88fd886 --- /dev/null +++ b/test-runner/src/http-signature.ts @@ -0,0 +1,93 @@ +import { createHash, createSign, generateKeyPairSync } from 'node:crypto'; + +export interface KeyPair { + publicKey: string; + privateKey: string; +} + +export interface SignedHeaders { + Host: string; + Date: string; + Digest: string; + Signature: string; + 'Content-Type': string; + Accept: string; +} + +/** + * Generate an RSA key pair for ActivityPub HTTP Signatures. + */ +export function generateKeyPair(): KeyPair { + const { publicKey, privateKey } = generateKeyPairSync('rsa', { + modulusLength: 2048, + publicKeyEncoding: { type: 'spki', format: 'pem' }, + privateKeyEncoding: { type: 'pkcs8', format: 'pem' }, + }); + return { publicKey, privateKey }; +} + +/** + * Create signed HTTP headers for an ActivityPub POST request. + * Implements the draft-cavage-http-signatures spec. + */ +export function createSignedPostHeaders(args: { + url: string; + body: string; + keyId: string; + privateKeyPem: string; +}): SignedHeaders { + const url = new URL(args.url); + const date = new Date().toUTCString(); + const digest = createHash('sha256').update(args.body).digest('base64'); + + const signedString = [ + `(request-target): post ${url.pathname}`, + `host: ${url.host}`, + `date: ${date}`, + `digest: SHA-256=${digest}`, + ].join('\n'); + + const signer = createSign('RSA-SHA256'); + signer.update(signedString); + signer.end(); + const signature = signer.sign(args.privateKeyPem, 'base64'); + + return { + Host: url.host, + Date: date, + Digest: `SHA-256=${digest}`, + Signature: [ + `keyId="${args.keyId}"`, + `algorithm="rsa-sha256"`, + `headers="(request-target) host date digest"`, + `signature="${signature}"`, + ].join(','), + 'Content-Type': 'application/activity+json', + Accept: 'application/activity+json', + }; +} + +/** + * Sign and send an ActivityPub POST to a remote inbox. + * Returns the HTTP response. + */ +export async function postToInbox(args: { + inboxUrl: string; + body: Record; + keyId: string; + privateKeyPem: string; +}): Promise { + const bodyStr = JSON.stringify(args.body); + const headers = createSignedPostHeaders({ + url: args.inboxUrl, + body: bodyStr, + keyId: args.keyId, + privateKeyPem: args.privateKeyPem, + }); + + return fetch(args.inboxUrl, { + method: 'POST', + headers: { ...headers }, + body: bodyStr, + }); +} diff --git a/test-runner/src/iceshrimp-api.ts b/test-runner/src/iceshrimp-api.ts new file mode 100644 index 0000000..f0cd8e4 --- /dev/null +++ b/test-runner/src/iceshrimp-api.ts @@ -0,0 +1,240 @@ +/** + * Iceshrimp/Misskey Admin API client for E2E testing. + * User creation via /api/admin/accounts/create — the first user auto-becomes admin. + */ +import { config } from './config.js'; + +export interface MisskeyUser { + id: string; + username: string; + token: string; +} + +interface CreateAdminAccountResponse { + id: string; + username: string; + token: string; +} + +interface SignupResponse { + id: string; + username: string; + token?: string; +} + +export class IceshrimpClient { + public adminToken: string | null = null; + private adminUser: MisskeyUser | null = null; + + constructor( + public readonly baseUrl: string = config.iceshrimpBaseUrl, + ) {} + + /** + * Creates the first admin user. + * The first user on Iceshrimp automatically becomes admin. + */ + async createAdminUser( + username: string = config.adminUsername, + password: string = config.adminPassword, + ): Promise { + const res = await this.apiPost( + '/api/admin/accounts/create', + { username, password }, + null, // no auth needed for first user + ); + this.adminToken = res.token; + this.adminUser = { id: res.id, username: res.username, token: res.token }; + console.log(`[iceshrimp] admin user created: ${res.username} (token: ${res.token?.slice(0, 8)}...)`); + return this.adminUser; + } + + /** + * Creates a test user via admin API (requires adminToken). + */ + async createUser(username: string, password: string): Promise { + const res = await this.apiPost( + '/api/admin/accounts/create', + { username, password }, + this.adminToken, + ); + console.log(`[iceshrimp] test user created: ${res.username}`); + return { id: res.id, username: res.username, token: res.token }; + } + + /** + * Creates a user via signup (open registration, no admin token needed). + */ + async signupUser(username: string, password: string): Promise { + const res = await this.apiPost( + '/api/signup', + { username, password }, + null, + ); + const token = res.token ?? ''; + return { id: res.id, username: res.username, token }; + } + + /** + * Generic API POST to Iceshrimp/Misskey API. + */ + async apiPost(path: string, body: Record, token: string | null): Promise { + const headers: Record = { + 'Content-Type': 'application/json', + }; + if (token) { + headers['Authorization'] = `Bearer ${token}`; + } + const url = `${this.baseUrl}${path}`; + const res = await fetch(url, { + method: 'POST', + headers, + body: JSON.stringify(body), + }); + if (!res.ok) { + const text = await res.text(); + throw new Error(`Iceshrimp API error ${res.status} on ${path}: ${text}`); + } + return res.json() as Promise; + } + + /** + * Generic API GET to Iceshrimp/Misskey API. + */ + async apiGet(path: string, token: string): Promise { + const headers: Record = { + 'Content-Type': 'application/json', + }; + if (token) { + headers['Authorization'] = `Bearer ${token}`; + } + const url = `${this.baseUrl}${path}`; + const res = await fetch(url, { method: 'GET', headers }); + if (!res.ok) { + const text = await res.text(); + throw new Error(`Iceshrimp API error ${res.status} on ${path}: ${text}`); + } + return res.json() as Promise; + } + + /** + * Get user info by username. + */ + async showUser(username: string, token: string) { + return this.apiPost>( + '/api/users/show', + { username }, + token, + ); + } + + /** + * Create a note (post). + */ + async createNote(text: string, token: string, visibility = 'public') { + return this.apiPost>( + '/api/notes/create', + { text, visibility }, + token, + ); + } + + /** + * Follow a user by userId. + */ + async followUser(userId: string, token: string) { + return this.apiPost>( + '/api/following/create', + { userId }, + token, + ); + } + + /** + * Unfollow a user by userId. + */ + async unfollowUser(userId: string, token: string) { + return this.apiPost>( + '/api/following/delete', + { userId }, + token, + ); + } + + /** + * Get followers for a user. + */ + async getFollowers(userId: string, token: string) { + return this.apiPost>>( + '/api/users/followers', + { userId }, + token, + ); + } + + /** + * Get following for a user. + */ + async getFollowing(userId: string, token: string) { + return this.apiPost>>( + '/api/users/following', + { userId }, + token, + ); + } + + /** + * Get the global timeline. + */ + async getGlobalTimeline(token: string) { + return this.apiPost>>( + '/api/notes/global-timeline', + {}, + token, + ); + } + + /** + * Delete a note by noteId. + */ + async deleteNote(noteId: string, token: string) { + return this.apiPost>( + '/api/notes/delete', + { noteId }, + token, + ); + } + + /** + * Create a reaction (like) on a note. + */ + async createReaction(noteId: string, reaction: string, token: string) { + return this.apiPost>( + '/api/notes/reactions/create', + { noteId, reaction }, + token, + ); + } + + /** + * Delete a reaction from a note. + */ + async deleteReaction(noteId: string, token: string) { + return this.apiPost>( + '/api/notes/reactions/delete', + { noteId }, + token, + ); + } + + /** + * Get notes (timeline) for a specific user. + */ + async getUserNotes(userId: string, token: string) { + return this.apiPost>>( + '/api/users/notes', + { userId }, + token, + ); + } +} From a2446c9887a318c86c7d4afcf1b80e54d2eec26f Mon Sep 17 00:00:00 2001 From: turtton Date: Wed, 24 Jun 2026 22:15:45 +0900 Subject: [PATCH 07/15] Add E2E federation test suite with ActivityPub message helpers MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Implement Follow→Accept and note delivery federation tests. Add ActivityPub message builders (Follow, Create, Like, Undo, Accept, Delete). Include test setup with admin user creation and remote actor registration using timestamped usernames. Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus --- test-runner/src/activitypub.ts | 120 ++++++++++++++++++++++++ test-runner/src/wait.ts | 37 ++++++++ test-runner/tests/follow.test.ts | 59 ++++++++++++ test-runner/tests/note-delivery.test.ts | 59 ++++++++++++ test-runner/tests/setup.ts | 94 +++++++++++++++++++ 5 files changed, 369 insertions(+) create mode 100644 test-runner/src/activitypub.ts create mode 100644 test-runner/src/wait.ts create mode 100644 test-runner/tests/follow.test.ts create mode 100644 test-runner/tests/note-delivery.test.ts create mode 100644 test-runner/tests/setup.ts diff --git a/test-runner/src/activitypub.ts b/test-runner/src/activitypub.ts new file mode 100644 index 0000000..fa4d23e --- /dev/null +++ b/test-runner/src/activitypub.ts @@ -0,0 +1,120 @@ +/** + * ActivityPub message builder and verifier for E2E testing. + */ + +/** + * Create a Follow activity from one actor to another. + */ +export function createFollowActivity(params: { + actorId: string; + targetId: string; + activityId?: string; +}): Record { + return { + '@context': 'https://www.w3.org/ns/activitystreams', + id: params.activityId ?? `${params.actorId}#follow/${Date.now()}`, + type: 'Follow', + actor: params.actorId, + object: params.targetId, + }; +} + +/** + * Create a Create(Note) activity. + */ +export function createNoteActivity(params: { + actorId: string; + noteId: string; + content: string; + to?: string[]; + cc?: string[]; + published?: string; +}): Record { + return { + '@context': 'https://www.w3.org/ns/activitystreams', + id: `${params.actorId}/activities/${Date.now()}`, + type: 'Create', + actor: params.actorId, + object: { + id: params.noteId, + type: 'Note', + attributedTo: params.actorId, + content: params.content, + published: params.published ?? new Date().toISOString(), + to: params.to ?? ['https://www.w3.org/ns/activitystreams#Public'], + cc: params.cc ?? [], + }, + }; +} + +/** + * Create a Like activity. + */ +export function createLikeActivity(params: { + actorId: string; + noteId: string; + activityId?: string; +}): Record { + return { + '@context': 'https://www.w3.org/ns/activitystreams', + id: params.activityId ?? `${params.actorId}#like/${Date.now()}`, + type: 'Like', + actor: params.actorId, + object: params.noteId, + }; +} + +/** + * Create an Undo activity (wraps a prior activity type). + */ +export function createUndoActivity(params: { + actorId: string; + object: Record; + activityId?: string; +}): Record { + return { + '@context': 'https://www.w3.org/ns/activitystreams', + id: params.activityId ?? `${params.actorId}#undo/${Date.now()}`, + type: 'Undo', + actor: params.actorId, + object: params.object, + }; +} + +/** + * Create an Accept activity (response to a Follow). + */ +export function createAcceptActivity(params: { + actorId: string; + objectId: string; + objectType?: string; + activityId?: string; +}): Record { + return { + '@context': 'https://www.w3.org/ns/activitystreams', + id: params.activityId ?? `${params.actorId}#accept/${Date.now()}`, + type: 'Accept', + actor: params.actorId, + object: { + id: params.objectId, + type: params.objectType ?? 'Follow', + }, + }; +} + +/** + * Create a Delete activity for a note. + */ +export function createDeleteActivity(params: { + actorId: string; + noteId: string; + activityId?: string; +}): Record { + return { + '@context': 'https://www.w3.org/ns/activitystreams', + id: params.activityId ?? `${params.actorId}#delete/${Date.now()}`, + type: 'Delete', + actor: params.actorId, + object: params.noteId, + }; +} diff --git a/test-runner/src/wait.ts b/test-runner/src/wait.ts new file mode 100644 index 0000000..ed9e31d --- /dev/null +++ b/test-runner/src/wait.ts @@ -0,0 +1,37 @@ +import { config } from './config.js'; + +export async function waitForService( + url: string, + label: string, + maxRetries = 30, + intervalMs = 2000, +): Promise { + for (let i = 0; i < maxRetries; i++) { + try { + const res = await fetch(url, { + signal: AbortSignal.timeout(5000), + }); + if (res.ok) { + console.log(`[wait] ${label} is ready`); + return; + } + } catch { + // not ready yet + } + console.log(`[wait] waiting for ${label} (${i + 1}/${maxRetries})...`); + await new Promise((r) => setTimeout(r, intervalMs)); + } + throw new Error(`Timeout waiting for ${label} at ${url}`); +} + +export async function waitForIceshrimp(): Promise { + // Iceshrimp exposes /api/instance or /api/v1/instance on its API. + // We try Misskey API endpoint as a health check. + const apiUrl = `${config.iceshrimpBaseUrl}/api/v1/instance`; + await waitForService(apiUrl, 'Iceshrimp'); +} + +export async function waitForCustomAp(): Promise { + const healthUrl = `${config.customApBaseUrl}/__test__/health`; + await waitForService(healthUrl, 'Custom AP'); +} diff --git a/test-runner/tests/follow.test.ts b/test-runner/tests/follow.test.ts new file mode 100644 index 0000000..75fd998 --- /dev/null +++ b/test-runner/tests/follow.test.ts @@ -0,0 +1,59 @@ +import { describe, it, expect } from 'vitest'; +import { + iceshrimp, + customAp, + adminUser, + localAlice, + remoteAlice, +} from './setup.js'; +import { createFollowActivity } from '../src/activitypub.js'; +import { postToInbox } from '../src/http-signature.js'; + +describe('Follow federation', () => { + it('remote actor follows local Iceshrimp user via ActivityPub', async () => { + // Get localAlice's full actor info from Iceshrimp + const localAliceActor = await iceshrimp.showUser(localAlice.username, adminUser.token); + const localAliceId = localAliceActor.id as string; + const localAliceUri = `${iceshrimp.baseUrl}/users/${localAliceId}`; + + // Create and sign a Follow activity from remote-alice to localAlice + const followActivity = createFollowActivity({ + actorId: remoteAlice.id, + targetId: localAliceUri, + }); + + // Send to Iceshrimp's inbox for localAlice + const inboxUrl = `${iceshrimp.baseUrl}/users/${localAliceId}/inbox`; + const response = await postToInbox({ + inboxUrl, + body: followActivity, + keyId: remoteAlice.publicKeyId, + privateKeyPem: remoteAlice.privateKeyPem, + }); + + expect(response.ok || response.status === 202).toBe(true); + + // Poll for the remote actor to appear in followers list + let followerFound = false; + for (let i = 0; i < 15; i++) { + const followers = await iceshrimp.getFollowers(localAliceId, localAlice.token); + const followerUris = followers.map((f: Record) => f.uri as string); + if (followerUris.includes(remoteAlice.id)) { + followerFound = true; + break; + } + await new Promise((r) => setTimeout(r, 2000)); + } + expect(followerFound).toBe(true); + + // Check that Custom AP inbox received Accept from Iceshrimp + // Verify it matches OUR Follow activity (not a stale one) + const acceptActivity = await customAp.pollInbox( + (item) => + item.type === 'Accept' && + (item.object as Record)?.id === followActivity.id, + ); + expect(acceptActivity.type).toBe('Accept'); + expect(acceptActivity.actor).toBe(localAliceUri); + }); +}); diff --git a/test-runner/tests/note-delivery.test.ts b/test-runner/tests/note-delivery.test.ts new file mode 100644 index 0000000..a63b8b5 --- /dev/null +++ b/test-runner/tests/note-delivery.test.ts @@ -0,0 +1,59 @@ +import { describe, it, expect } from 'vitest'; +import { + iceshrimp, + customAp, + localAlice, + remoteAlice, +} from './setup.js'; +import { createFollowActivity } from '../src/activitypub.js'; +import { postToInbox } from '../src/http-signature.js'; + +describe('Note delivery federation', () => { + it('local note is delivered to remote actor inbox', async () => { + // Get localAlice's full actor info + const localAliceActor = await iceshrimp.showUser(localAlice.username, localAlice.token); + const localAliceId = localAliceActor.id as string; + const localAliceUri = `${iceshrimp.baseUrl}/users/${localAliceId}`; + + // 1. Establish follow: remote-alice follows localAlice + const followActivity = createFollowActivity({ + actorId: remoteAlice.id, + targetId: localAliceUri, + }); + + const inboxUrl = `${iceshrimp.baseUrl}/users/${localAliceId}/inbox`; + const followResponse = await postToInbox({ + inboxUrl, + body: followActivity, + keyId: remoteAlice.publicKeyId, + privateKeyPem: remoteAlice.privateKeyPem, + }); + expect(followResponse.ok || followResponse.status === 202).toBe(true); + + // 2. Wait for Accept from Iceshrimp + const acceptActivity = await customAp.pollInbox( + (item) => + item.type === 'Accept' && + (item.object as Record)?.id === followActivity.id, + ); + expect(acceptActivity.type).toBe('Accept'); + + // 3. Create a note as local Alice + const noteText = `ap-testbox note delivery test ${Date.now()}`; + const note = await iceshrimp.createNote(noteText, localAlice.token); + const noteId = note.createdNote?.id as string ?? note.id as string; + expect(noteId).toBeTruthy(); + + // 4. Wait for the note to be delivered to remote-alice's inbox + const createActivity = await customAp.pollInbox( + (item) => item.type === 'Create' && (item.object as Record)?.content === noteText, + 20, + ); + + expect(createActivity.type).toBe('Create'); + const obj = createActivity.object as Record; + expect(obj.type).toBe('Note'); + expect(obj.content).toContain(noteText); + expect(obj.attributedTo).toBe(localAliceUri); + }); +}); diff --git a/test-runner/tests/setup.ts b/test-runner/tests/setup.ts new file mode 100644 index 0000000..d1484c0 --- /dev/null +++ b/test-runner/tests/setup.ts @@ -0,0 +1,94 @@ +import { beforeAll, afterAll } from 'vitest'; +import { config } from '../src/config.js'; +import { IceshrimpClient } from '../src/iceshrimp-api.js'; +import { CustomApClient } from '../src/custom-ap-client.js'; +import { generateKeyPair } from '../src/http-signature.js'; +import { waitForIceshrimp, waitForCustomAp } from '../src/wait.js'; + +// Unique run identifier to prevent username conflicts across test runs +const runSuffix = Date.now().toString(36)// short alphanumeric run ID; + +// Global state shared across tests +export const iceshrimp = new IceshrimpClient(); +export const customAp = new CustomApClient(); + +export interface TestUser { + username: string; + password: string; + token: string; + id: string; +} + +export interface RemoteActor { + username: string; + id: string; + inbox: string; + outbox: string; + publicKeyPem: string; + privateKeyPem: string; + publicKeyId: string; +} + +export let adminUser: TestUser; +export let localAlice: TestUser; +export let remoteAlice: RemoteActor; +export let remoteBob: RemoteActor; + +let createdUsernames: string[] = []; + +beforeAll(async () => { + console.log('[setup] waiting for services...'); + await waitForIceshrimp(); + await waitForCustomAp(); + console.log('[setup] all services ready'); + + // Reset Custom AP to clean state before setting up new test data + await customAp.reset(); + + // 1. Create admin user (first user auto-becomes admin on Iceshrimp) + adminUser = await iceshrimp.createAdminUser(`test-admin-${runSuffix}`, 'test-admin-pass'); + createdUsernames.push(adminUser.username); + + // 2. Create local test user on Iceshrimp + const alice = await iceshrimp.createUser(`e2e-alice-${runSuffix}`, 'test-pass-alice'); + localAlice = { ...alice, password: 'test-pass-alice' }; + createdUsernames.push('e2e-alice'); + + // 3. Generate key pairs for remote actors + const aliceKeys = generateKeyPair(); + const bobKeys = generateKeyPair(); + + // 4. Register remote actors on Custom AP + const aliceActor = await customAp.registerActor(`remote-alice-${runSuffix}`, aliceKeys.publicKey); + const bobActor = await customAp.registerActor(`remote-bob-${runSuffix}`, bobKeys.publicKey); + + remoteAlice = { + username: 'remote-alice', + id: aliceActor.id, + inbox: aliceActor.inbox, + outbox: aliceActor.outbox, + publicKeyPem: aliceKeys.publicKey, + privateKeyPem: aliceKeys.privateKey, + publicKeyId: aliceActor.publicKeyId, + }; + + remoteBob = { + username: 'remote-bob', + id: bobActor.id, + inbox: bobActor.inbox, + outbox: bobActor.outbox, + publicKeyPem: bobKeys.publicKey, + privateKeyPem: bobKeys.privateKey, + publicKeyId: bobActor.publicKeyId, + }; + + console.log(`[setup] remote-alice: ${remoteAlice.id}`); + console.log(`[setup] remote-bob: ${remoteBob.id}`); + console.log('[setup] ready'); +}); + +afterAll(async () => { + // Reset Custom AP test state + await customAp.reset(); + console.log('[setup] cleaned up'); +}); From 6482091ba6e2a5cbe4363c39cc3a258961ff67e3 Mon Sep 17 00:00:00 2001 From: turtton Date: Wed, 24 Jun 2026 22:15:52 +0900 Subject: [PATCH 08/15] Update project documentation for Iceshrimp-based E2E testing setup Rewrite AGENTS.md and README.md to reflect migration from Misskey to Iceshrimp. Add service descriptions, domain table, setup instructions, and common commands. Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus --- AGENTS.md | 111 +++++++++++++++++++++++++++++++++---------------- README.md | 121 ++++++++++++++++++++++++++++++++++++++++++++++++------ 2 files changed, 185 insertions(+), 47 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 10ccf3e..726308f 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,28 +1,36 @@ # AGENTS.md — ap-testbox -Docker Compose local dev environment for [Misskey](https://github.com/misskey-dev/misskey). This is NOT the Misskey source — the `misskey/` directory is gitignored and must be cloned separately. +Docker Compose ActivityPub E2E testing environment. Uses [Iceshrimp](https://iceshrimp.dev) as the reference Fediverse implementation and supports running a custom AP system as a federation peer. + +## Services + +- **nginx**: Reverse proxy with SSL termination for 3 vhosts +- **iceshrimp**: Iceshrimp-JS v2026.5.1 (official Docker image, no build needed) +- **custom-ap**: Your custom ActivityPub implementation (built from local context) +- **test-runner**: TypeScript + Vitest E2E test suite +- **db**: PostgreSQL 16 +- **redis**: Valkey 7 (Redis-compatible) + +## Domains (all via nginx on :4430) + +| Domain | Proxy to | Purpose | +|---|---|---| +| `iceshrimp.localhost` | `iceshrimp:3000` | Primary Fediverse instance | +| `test-ap.localhost` | `custom-ap:12864` | Custom AP system | +| `shuttlepub.localhost` | `custom-ap:12864` | Legacy alias | ## First-time setup (order matters) ```bash -./gencert.sh # generates SSL certs via Docker (mkcert) -git clone -b master https://github.com/misskey-dev/misskey.git -docker compose build --pull -docker compose run --rm misskey pnpm run init # runs DB migrations -docker compose up -d +cp .env.example .env # optional, override defaults +./gencert.sh # generates SSL certs for all domains +docker compose pull # pull iceshrimp, nginx, db, redis images +docker compose build # build custom-ap and test-runner +docker compose run --rm iceshrimp pnpm run init # runs DB migrations +docker compose up -d # start all services +docker compose run --rm test-runner # run E2E tests ``` -## Key details - -- **URL**: `https://misskey.localhost:4430` (nginx reverse-proxies to misskey:3000; HTTP on 8080 redirects to HTTPS). Also available at `https://shuttlepub.localhost:4430` which proxies to `host.docker.internal:12864`. -- **Cert gen**: `./gencert.sh` builds a Docker image with `mkcert`, generates a root CA and certs for `misskey.localhost` and `shuttlepub.localhost`. Certs are mounted into nginx (`/etc/nginx/certs/`). The root CA is also mounted into the misskey container and loaded via `NODE_EXTRA_CA_CERTS=/certs/rootCA.pem` for outbound HTTPS to other instances. -- **Misskey config**: `configs/misskey/.config/default.yml` — url, db host/port/credentials, redis host/port, full-text search (sqlLike by default), ID generation (`aidx`). -- **DB credentials**: `configs/misskey/.config/docker.env` — user `example-misskey-user`, pass `example-misskey-pass`, db `misskey`. `DATABASE_URL` is assembled from env vars. -- **Services**: nginx (1-alpine), misskey (built from local clone), postgres (15-alpine), redis (7-alpine) -- **Volumes**: `db_data` (postgres), `redis_data` (redis), bind-mounts for misskey files, config (ro), and certs -- **Host ports**: `8080:80` (HTTP), `4430:443` (HTTPS), `5432:5432` (postgres) -- **Nginx network alias**: `shuttlepub.localhost` — set on the `misskey-nginx` container so it can serve host-based vhosts - ## Directory layout ``` @@ -31,35 +39,68 @@ ap-testbox/ ├── gencert.sh # Fires up Docker-based mkcert ├── gencert/ # Dockerfile + mkcert.sh script ├── configs/ -│ ├── nginx/confs/ # nginx server blocks (SSL proxy) -│ │ ├── misskey.conf # misskey.localhost → misskey:3000 -│ │ └── shuttlepub.conf # shuttlepub.localhost → host.docker.internal:12864 -│ └── misskey/ +│ ├── nginx/confs/ # nginx SSL server blocks +│ │ ├── iceshrimp.conf +│ │ ├── test-ap.conf +│ │ └── shuttlepub.conf +│ └── iceshrimp/ │ ├── .config/ -│ │ ├── default.yml # Misskey application config +│ │ ├── default.yml # Iceshrimp application config │ │ └── docker.env # DB env vars -│ └── nginx/conf.d/ # nginx conf (HTTP→HTTPS redirect) +│ ├── nginx/conf.d/ # nginx config mounted by iceshrimp +├── test-runner/ # E2E test suite (TypeScript + Vitest) +│ ├── Dockerfile +│ ├── package.json +│ ├── tsconfig.json +│ ├── vitest.config.ts +│ ├── src/ +│ │ ├── config.ts +│ │ ├── wait.ts +│ │ ├── iceshrimp-api.ts +│ │ ├── custom-ap-client.ts +│ │ ├── http-signature.ts +│ │ └── activitypub.ts +│ └── tests/ +│ ├── setup.ts +│ ├── follow.test.ts +│ └── note-delivery.test.ts +├── docs/ +│ └── custom-ap-contract.md # Interface contract for custom AP ├── certs/ # gitignored, generated by gencert -└── misskey/ # gitignored, cloned from upstream +└── misskey/ # gitignored (kept for reference, no longer needed) ``` ## Common commands | Action | Command | |---|---| -| Build images | `docker compose build --pull` | +| Build all images | `docker compose build` | +| Pull remote images | `docker compose pull` | | Start all | `docker compose up -d` | | Stop all | `docker compose down` | -| Run migration | `docker compose run --rm misskey pnpm run init` | -| View misskey logs | `docker compose logs -f misskey` | -| Rebuild misskey only | `docker compose build --no-cache misskey` | -| Access postgres | `docker compose exec db psql -U example-misskey-user -d misskey` | +| Run migration | `docker compose run --rm iceshrimp pnpm run init` | +| View iceshrimp logs | `docker compose logs -f iceshrimp` | +| Run E2E tests | `docker compose run --rm test-runner` | +| Rebuild test-runner | `docker compose build test-runner` | +| Access postgres | `docker compose exec db psql -U example-iceshrimp-user -d iceshrimp` | +| Custom AP health check | `curl -sk https://test-ap.localhost:4430/__test__/health` | + +## Custom AP Contract + +See [docs/custom-ap-contract.md](docs/custom-ap-contract.md) for the full interface contract. + +The custom-ap service is built from `CUSTOM_AP_CONTEXT` (default: `../custom-ap`). Set via `.env`: + +```env +CUSTOM_AP_CONTEXT=../my-ap-project +CUSTOM_AP_DOCKERFILE=Dockerfile +``` ## Gotchas -- The `misskey/` directory and `certs/` are in `.gitignore`. Any local modifications to misskey source will NOT be committed. -- The nginx config blocks `/sw.js` (returns 404) — Misskey's service worker registration is disabled in this setup. -- `host.docker.internal:host-gateway` is set on the nginx container for potential host machine access (used by `shuttlepub.localhost` to proxy to the host). -- Misskey runs in `NODE_ENV=development` by default (set in compose.yml). -- After cloning/updating the misskey submodule, run migrations again if the schema changed. -- The README says `https://misskey.local:4430` but the actual nginx config uses `misskey.localhost`. Use the latter. +- The `certs/` directory is in `.gitignore`. Always run `./gencert.sh` first. +- The nginx config blocks `/sw.js` (returns 404) on iceshrimp — service worker registration is disabled. +- Iceshrimp uses `pnpm run init` for DB migrations. Run this before starting for the first time. +- ActivityPub delivery is asynchronous. Tests have built-in polling with timeout. +- The test-runner uses `singleFork` mode — all tests run in sequence for stability. +- HTTP Signatures use draft-cavage-http-signatures with RSA-SHA256. diff --git a/README.md b/README.md index 1f8343e..c39b369 100644 --- a/README.md +++ b/README.md @@ -1,14 +1,111 @@ +# ap-testbox + +Docker Compose ActivityPub E2E testing environment. Runs [Iceshrimp](https://iceshrimp.dev) as a reference Fediverse instance alongside your custom ActivityPub implementation, with an automated E2E test suite. + +## Services + +| Service | Image | Purpose | +|---|---|---| +| nginx | `nginx:1-alpine` | SSL reverse proxy for all vhosts | +| iceshrimp | `iceshrimp.dev/iceshrimp/iceshrimp:v2026.5.1` | Reference Fediverse instance | +| custom-ap | (build from your context) | Your AP implementation | +| test-runner | (build from `./test-runner`) | Vitest E2E test suite | +| db | `postgres:16-alpine` | PostgreSQL | +| redis | `valkey/valkey:7-alpine` | Redis-compatible cache | + ## Requirement + - [Docker](https://www.docker.com/) -- [Misskey (master)](https://github.com/misskey-dev/misskey) - -### How to Run -1. Run `./gencert.sh` -2. Clone Misskey - - `git clone -b master https://github.com/misskey-dev/misskey.git` -3. Setup container - - `docker compose build` - - `docker compose run --rm misskey pnpm run init` -4. Start container - - `docker compose up -d` -5. Access to `https://misskey.local:4430` in your browser. +- [Docker Compose](https://docs.docker.com/compose/) (v2, included with Docker Desktop) +- Your custom ActivityPub implementation (for `custom-ap` service) + +## Quick Start + +```bash +# 1. Generate SSL certificates +./gencert.sh + +# 2. Pull and build images +docker compose pull +docker compose build + +# 3. Run database migrations +docker compose run --rm iceshrimp pnpm run init + +# 4. Start all services +docker compose up -d + +# 5. Run E2E tests +docker compose run --rm test-runner +``` + +Then access: +- **Iceshrimp**: `https://iceshrimp.localhost:4430` +- **Custom AP**: `https://test-ap.localhost:4430` + +## Custom AP Setup + +Set the build context for your AP implementation in `.env`: + +```env +CUSTOM_AP_CONTEXT=../my-ap-project +CUSTOM_AP_DOCKERFILE=Dockerfile +``` + +See [docs/custom-ap-contract.md](docs/custom-ap-contract.md) for required endpoints. + +## Running E2E Tests + +```bash +# Run all tests +docker compose run --rm test-runner + +# Rebuild test-runner after changes +docker compose build test-runner +docker compose run --rm test-runner +``` + +## Common Commands + +| Action | Command | +|---|---| +| Start all | `docker compose up -d` | +| Stop all | `docker compose down` | +| View logs | `docker compose logs -f iceshrimp` | +| Rebuild custom-ap | `docker compose build custom-ap` | +| Access DB | `docker compose exec db psql -U example-iceshrimp-user -d iceshrimp` | + +## Architecture + +``` +┌──────────────┐ ┌──────────────────┐ +│ nginx │────▶│ iceshrimp:3000 │ +│ (SSL proxy) │ └────────┬─────────┘ +│ │ │ federation +│ 3 vhosts │ ▼ +│ │ ┌──────────────────┐ +└──────┬───────┘ │ custom-ap:12864 │ + │ └────────┬─────────┘ + ▼ │ +┌──────────────────────────────┐ +│ test-runner │ +│ (orchestrates scenarios) │ +│ - Iceshrimp REST API │ +│ - AP protocol verification │ +│ - Test endpoints for custom │ +└──────────────────────────────┘ +``` + +## Project Structure + +``` +configs/ +├── nginx/confs/ # SSL vhost configs +└── iceshrimp/ # Iceshrimp settings +test-runner/ # E2E test suite +docs/ # Interface contracts +``` + +## License + +MIT From 5bb63b661f8db540373ecf38967872de01edaaf0 Mon Sep 17 00:00:00 2001 From: turtton Date: Sat, 27 Jun 2026 00:15:36 +0900 Subject: [PATCH 09/15] Add Mastodon API client and admin token caching to test framework Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus --- test-runner/src/config.ts | 7 +- test-runner/src/iceshrimp-api.ts | 68 ++++++++---- test-runner/src/mastodon-api.ts | 158 ++++++++++++++++++++++++++++ test-runner/src/wait.ts | 5 + test-runner/tests/mastodon-setup.ts | 46 ++++++++ test-runner/tests/setup.ts | 41 ++++++-- 6 files changed, 292 insertions(+), 33 deletions(-) create mode 100644 test-runner/src/mastodon-api.ts create mode 100644 test-runner/tests/mastodon-setup.ts diff --git a/test-runner/src/config.ts b/test-runner/src/config.ts index 075124c..69f9dd1 100644 --- a/test-runner/src/config.ts +++ b/test-runner/src/config.ts @@ -6,7 +6,7 @@ export const config = { testModeToken: process.env.TEST_MODE_TOKEN ?? 'ap-testbox-test-token', // Admin credentials for initial user creation - adminUsername: process.env.ADMIN_USERNAME ?? 'test-admin', + adminUsername: process.env.ADMIN_USERNAME ?? 'testadmin', adminPassword: process.env.ADMIN_PASSWORD ?? 'test-admin-pass', // DB connection (optional, for direct verification) @@ -15,4 +15,9 @@ export const config = { pgDatabase: process.env.PGDATABASE, pgUser: process.env.PGUSER, pgPassword: process.env.PGPASSWORD, + + mastodonBaseUrl: process.env.MASTODON_BASE_URL ?? 'https://mastodon.localhost', + mastodonTestUsername: process.env.MASTODON_TEST_USERNAME ?? 'e2emastodon', + mastodonTestEmail: process.env.MASTODON_TEST_EMAIL ?? 'e2e-mastodon@mastodon.localhost', + mastodonTestPassword: process.env.MASTODON_TEST_PASSWORD ?? 'test-pass-mastodon', } as const; diff --git a/test-runner/src/iceshrimp-api.ts b/test-runner/src/iceshrimp-api.ts index f0cd8e4..fa3bf9c 100644 --- a/test-runner/src/iceshrimp-api.ts +++ b/test-runner/src/iceshrimp-api.ts @@ -3,37 +3,49 @@ * User creation via /api/admin/accounts/create — the first user auto-becomes admin. */ import { config } from './config.js'; - export interface MisskeyUser { id: string; username: string; token: string; } - interface CreateAdminAccountResponse { id: string; username: string; token: string; } - interface SignupResponse { id: string; username: string; token?: string; } - export class IceshrimpClient { public adminToken: string | null = null; private adminUser: MisskeyUser | null = null; - constructor( public readonly baseUrl: string = config.iceshrimpBaseUrl, ) {} - /** * Creates the first admin user. * The first user on Iceshrimp automatically becomes admin. */ + /** + * Login with existing credentials (admin or user) via /api/signin. + */ + async login( + username: string = config.adminUsername, + password: string = config.adminPassword, + ): Promise { + const res = await this.apiPost<{ id: string; i: string }>( + '/api/signin', + { username, password }, + null, + ); + const user: MisskeyUser = { id: res.id, username, token: res.i }; + this.adminToken = res.i; + this.adminUser = user; + console.log(`[iceshrimp] logged in as existing admin: ${username}`); + return user; + } async createAdminUser( username: string = config.adminUsername, password: string = config.adminPassword, @@ -48,7 +60,6 @@ export class IceshrimpClient { console.log(`[iceshrimp] admin user created: ${res.username} (token: ${res.token?.slice(0, 8)}...)`); return this.adminUser; } - /** * Creates a test user via admin API (requires adminToken). */ @@ -61,7 +72,6 @@ export class IceshrimpClient { console.log(`[iceshrimp] test user created: ${res.username}`); return { id: res.id, username: res.username, token: res.token }; } - /** * Creates a user via signup (open registration, no admin token needed). */ @@ -74,7 +84,6 @@ export class IceshrimpClient { const token = res.token ?? ''; return { id: res.id, username: res.username, token }; } - /** * Generic API POST to Iceshrimp/Misskey API. */ @@ -97,7 +106,6 @@ export class IceshrimpClient { } return res.json() as Promise; } - /** * Generic API GET to Iceshrimp/Misskey API. */ @@ -116,7 +124,6 @@ export class IceshrimpClient { } return res.json() as Promise; } - /** * Get user info by username. */ @@ -127,7 +134,6 @@ export class IceshrimpClient { token, ); } - /** * Create a note (post). */ @@ -138,7 +144,6 @@ export class IceshrimpClient { token, ); } - /** * Follow a user by userId. */ @@ -149,7 +154,6 @@ export class IceshrimpClient { token, ); } - /** * Unfollow a user by userId. */ @@ -160,7 +164,6 @@ export class IceshrimpClient { token, ); } - /** * Get followers for a user. */ @@ -171,7 +174,6 @@ export class IceshrimpClient { token, ); } - /** * Get following for a user. */ @@ -182,7 +184,6 @@ export class IceshrimpClient { token, ); } - /** * Get the global timeline. */ @@ -193,7 +194,6 @@ export class IceshrimpClient { token, ); } - /** * Delete a note by noteId. */ @@ -204,7 +204,6 @@ export class IceshrimpClient { token, ); } - /** * Create a reaction (like) on a note. */ @@ -215,7 +214,6 @@ export class IceshrimpClient { token, ); } - /** * Delete a reaction from a note. */ @@ -226,7 +224,6 @@ export class IceshrimpClient { token, ); } - /** * Get notes (timeline) for a specific user. */ @@ -237,4 +234,33 @@ export class IceshrimpClient { token, ); } + /** + * Show a single note by ID. + */ + async showNote(noteId: string, token: string): Promise> { + return this.apiPost>( + '/api/notes/show', + { noteId }, + token, + ); + } + /** + * Poll a note until a predicate is satisfied. + */ + async pollNote( + noteId: string, + token: string, + predicate: (note: Record) => boolean, + maxRetries = 15, + intervalMs = 2000, + ): Promise> { + for (let i = 0; i < maxRetries; i++) { + const note = await this.showNote(noteId, token); + if (predicate(note)) { + return note; + } + await new Promise((r) => setTimeout(r, intervalMs)); + } + throw new Error(`Iceshrimp note ${noteId}: predicate not satisfied after ${maxRetries * intervalMs}ms`); + } } diff --git a/test-runner/src/mastodon-api.ts b/test-runner/src/mastodon-api.ts new file mode 100644 index 0000000..4da0485 --- /dev/null +++ b/test-runner/src/mastodon-api.ts @@ -0,0 +1,158 @@ +import { config } from './config.js'; + +export class MastodonClient { + public appToken: string | null = null; + public userToken: string | null = null; + public accountId: string | null = null; + + constructor( + public readonly baseUrl: string = config.mastodonBaseUrl, + ) {} + + private async apiPost(path: string, body: Record, token?: string | null): Promise { + const headers: Record = { + 'Content-Type': 'application/json', + }; + if (token) { + headers['Authorization'] = `Bearer ${token}`; + } + const url = `${this.baseUrl}${path}`; + const res = await fetch(url, { method: 'POST', headers, body: JSON.stringify(body) }); + if (!res.ok) { + const text = await res.text(); + throw new Error(`Mastodon API error ${res.status} on ${path}: ${text.slice(0, 200)}`); + } + return res.json() as Promise; + } + + private async apiPostForm(path: string, body: URLSearchParams, token?: string | null): Promise { + const headers: Record = {}; + if (token) { + headers['Authorization'] = `Bearer ${token}`; + } + const url = `${this.baseUrl}${path}`; + const res = await fetch(url, { method: 'POST', headers, body }); + if (!res.ok) { + const text = await res.text(); + throw new Error(`Mastodon API error ${res.status} on ${path}: ${text.slice(0, 200)}`); + } + return res.json() as Promise; + } + + private async apiGet(path: string, token?: string | null, params?: URLSearchParams): Promise { + const headers: Record = {}; + if (token) { + headers['Authorization'] = `Bearer ${token}`; + } + let url = `${this.baseUrl}${path}`; + if (params) { url += `?${params.toString()}`; } + const res = await fetch(url, { method: 'GET', headers }); + if (!res.ok) { + const text = await res.text(); + throw new Error(`Mastodon API error ${res.status} on ${path}: ${text.slice(0, 200)}`); + } + return res.json() as Promise; + } + + /** + * Register an OAuth application. Required before obtaining tokens. + */ + async createApp(): Promise<{ clientId: string; clientSecret: string }> { + const res = await this.apiPost<{ client_id: string; client_secret: string }>('/api/v1/apps', { + client_name: 'ap-testbox', + redirect_uris: 'urn:ietf:wg:oauth:2.0:oob', + scopes: 'read write follow', + }); + return { clientId: res.client_id, clientSecret: res.client_secret }; + } + + /** + * Login with username/password via OAuth password grant. + * Returns the access token. + */ + async loginWithPassword( + clientId: string, + clientSecret: string, + username: string, + password: string, + ): Promise { + const body = new URLSearchParams({ + grant_type: 'password', + client_id: clientId, + client_secret: clientSecret, + username, + password, + scope: 'read write follow', + }); + const res = await this.apiPostForm<{ access_token: string }>('/oauth/token', body); + this.userToken = res.access_token; + return res.access_token; + } + + /** + * Verify credentials and get the account info. + */ + async verifyCredentials(token: string): Promise> { + return this.apiGet>('/api/v1/accounts/verify_credentials', token); + } + + /** + * Search for an account, resolving remotely if needed. + */ + async searchAccount(token: string, query: string): Promise>> { + const params = new URLSearchParams({ q: query, type: 'accounts', resolve: 'true' }); + const res = await this.apiGet<{ accounts: Array> }>( + '/api/v2/search', + token, + params, + ); + return res.accounts; + } + + /** + * Follow an account by its ID. + */ + async followAccount(token: string, accountId: string): Promise> { + return this.apiPost>(`/api/v1/accounts/${accountId}/follow`, {}, token); + } + + /** + * Create a new status (post). + */ + async createStatus(token: string, text: string, visibility = 'public'): Promise> { + return this.apiPost>('/api/v1/statuses', { status: text, visibility }, token); + } + + /** + * Favourite (like) a status by its ID. + */ + async favouriteStatus(token: string, statusId: string): Promise> { + return this.apiPost>(`/api/v1/statuses/${statusId}/favourite`, {}, token); + } + + /** + * Get the home timeline. + */ + async getHomeTimeline(token: string, limit = 40): Promise>> { + const params = new URLSearchParams({ limit: String(limit) }); + return this.apiGet>>('/api/v1/timelines/home', token, params); + } + + /** + * Poll home timeline until a status matching the predicate appears. + */ + async pollHomeTimeline( + token: string, + predicate: (status: Record) => boolean, + maxRetries = 20, + intervalMs = 3000, + ): Promise> { + for (let i = 0; i < maxRetries; i++) { + const timeline = await this.getHomeTimeline(token); + const match = timeline.find(predicate); + if (match) return match; + await new Promise((r) => setTimeout(r, intervalMs)); + } + throw new Error(`Mastodon timeline: matching status not found after ${maxRetries * intervalMs}ms`); + } +} diff --git a/test-runner/src/wait.ts b/test-runner/src/wait.ts index ed9e31d..5052da8 100644 --- a/test-runner/src/wait.ts +++ b/test-runner/src/wait.ts @@ -35,3 +35,8 @@ export async function waitForCustomAp(): Promise { const healthUrl = `${config.customApBaseUrl}/__test__/health`; await waitForService(healthUrl, 'Custom AP'); } + +export async function waitForMastodon(): Promise { + const apiUrl = `${config.mastodonBaseUrl}/api/v1/instance`; + await waitForService(apiUrl, 'Mastodon'); +} diff --git a/test-runner/tests/mastodon-setup.ts b/test-runner/tests/mastodon-setup.ts new file mode 100644 index 0000000..8648ac3 --- /dev/null +++ b/test-runner/tests/mastodon-setup.ts @@ -0,0 +1,46 @@ +import { beforeAll, afterAll } from 'vitest'; +import { config } from '../src/config.js'; +import { MastodonClient } from '../src/mastodon-api.js'; +import { waitForMastodon } from '../src/wait.js'; + +export const mastodon = new MastodonClient(); +export let mastodonToken: string; +export let mastodonAliceId: string; +export let mastodonClientId: string; +export let mastodonClientSecret: string; + +const e2eTokenOverride = process.env.MASTODON_E2E_TOKEN; + +beforeAll(async () => { + await waitForMastodon(); + console.log('[mastodon-setup] Mastodon is ready'); + + if (e2eTokenOverride) { + // Use pre-created E2E token (bypass OAuth password grant, which is deprecated in Mastodon v4.x) + mastodonToken = e2eTokenOverride; + mastodonClientId = ''; + mastodonClientSecret = ''; + console.log('[mastodon-setup] using pre-created E2E token'); + } else { + // Fallback: register OAuth app and login + const app = await mastodon.createApp(); + mastodonClientId = app.clientId; + mastodonClientSecret = app.clientSecret; + mastodonToken = await mastodon.loginWithPassword( + mastodonClientId, + mastodonClientSecret, + config.mastodonTestUsername, + config.mastodonTestPassword, + ); + console.log('[mastodon-setup] user token obtained via OAuth'); + } + + // Verify credentials and store account ID + const account = await mastodon.verifyCredentials(mastodonToken); + mastodonAliceId = account.id as string; + console.log(`[mastodon-setup] logged in as ${account.username} (${mastodonAliceId})`); +}); + +afterAll(async () => { + // no cleanup needed for now +}); diff --git a/test-runner/tests/setup.ts b/test-runner/tests/setup.ts index d1484c0..7377dd1 100644 --- a/test-runner/tests/setup.ts +++ b/test-runner/tests/setup.ts @@ -4,11 +4,11 @@ import { IceshrimpClient } from '../src/iceshrimp-api.js'; import { CustomApClient } from '../src/custom-ap-client.js'; import { generateKeyPair } from '../src/http-signature.js'; import { waitForIceshrimp, waitForCustomAp } from '../src/wait.js'; +import fs from 'fs'; -// Unique run identifier to prevent username conflicts across test runs -const runSuffix = Date.now().toString(36)// short alphanumeric run ID; +const runSuffix = Date.now().toString(36); +const ADMIN_CREDS_FILE = '/tmp/ap-testbox-admin-creds.json'; -// Global state shared across tests export const iceshrimp = new IceshrimpClient(); export const customAp = new CustomApClient(); @@ -42,23 +42,43 @@ beforeAll(async () => { await waitForCustomAp(); console.log('[setup] all services ready'); - // Reset Custom AP to clean state before setting up new test data await customAp.reset(); - // 1. Create admin user (first user auto-becomes admin on Iceshrimp) - adminUser = await iceshrimp.createAdminUser(`test-admin-${runSuffix}`, 'test-admin-pass'); + if (fs.existsSync(ADMIN_CREDS_FILE)) { + const creds = JSON.parse(fs.readFileSync(ADMIN_CREDS_FILE, 'utf-8')); + if (creds.adminToken) { + adminUser = { username: creds.adminUsername, token: creds.adminToken, password: '', id: '' } as TestUser; + iceshrimp.adminToken = creds.adminToken; + console.log(`[setup] using cached admin token for: ${adminUser.username}`); + } else { + adminUser = await iceshrimp.login(creds.adminUsername, config.adminPassword); + fs.writeFileSync(ADMIN_CREDS_FILE, JSON.stringify({ adminUsername: creds.adminUsername, adminToken: adminUser.token }), 'utf-8'); + console.log(`[setup] logged in as existing admin: ${adminUser.username}`); + } + } else { + const adminUsername = config.adminUsername; + try { + adminUser = await iceshrimp.createAdminUser(adminUsername, config.adminPassword); + } catch (e) { + if (e instanceof Error && e.message.includes('INTERNAL_ERROR')) { + console.log('[setup] admin already exists, logging in...'); + adminUser = await iceshrimp.login(adminUsername, config.adminPassword); + } else { + throw e; + } + } + fs.writeFileSync(ADMIN_CREDS_FILE, JSON.stringify({ adminUsername: adminUser.username, adminToken: adminUser.token }), 'utf-8'); + console.log(`[setup] admin ready: ${adminUser.username}`); + } createdUsernames.push(adminUser.username); - // 2. Create local test user on Iceshrimp - const alice = await iceshrimp.createUser(`e2e-alice-${runSuffix}`, 'test-pass-alice'); + const alice = await iceshrimp.createUser(`e2e_alice_${runSuffix}`, 'test-pass-alice'); localAlice = { ...alice, password: 'test-pass-alice' }; createdUsernames.push('e2e-alice'); - // 3. Generate key pairs for remote actors const aliceKeys = generateKeyPair(); const bobKeys = generateKeyPair(); - // 4. Register remote actors on Custom AP const aliceActor = await customAp.registerActor(`remote-alice-${runSuffix}`, aliceKeys.publicKey); const bobActor = await customAp.registerActor(`remote-bob-${runSuffix}`, bobKeys.publicKey); @@ -88,7 +108,6 @@ beforeAll(async () => { }); afterAll(async () => { - // Reset Custom AP test state await customAp.reset(); console.log('[setup] cleaned up'); }); From 08588533a9847d86c88ee7d601253aa9a5c61c69 Mon Sep 17 00:00:00 2001 From: turtton Date: Sat, 27 Jun 2026 00:15:42 +0900 Subject: [PATCH 10/15] Add Mastodon federation E2E tests for follow, note, and reaction Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus --- test-runner/tests/mastodon-follow.test.ts | 39 +++++++++++++ test-runner/tests/mastodon-note.test.ts | 38 +++++++++++++ test-runner/tests/mastodon-reaction.test.ts | 62 +++++++++++++++++++++ 3 files changed, 139 insertions(+) create mode 100644 test-runner/tests/mastodon-follow.test.ts create mode 100644 test-runner/tests/mastodon-note.test.ts create mode 100644 test-runner/tests/mastodon-reaction.test.ts diff --git a/test-runner/tests/mastodon-follow.test.ts b/test-runner/tests/mastodon-follow.test.ts new file mode 100644 index 0000000..7afc8f8 --- /dev/null +++ b/test-runner/tests/mastodon-follow.test.ts @@ -0,0 +1,39 @@ +import { describe, it, expect } from 'vitest'; +import { mastodon, mastodonToken, mastodonAliceId } from './mastodon-setup.js'; +import { iceshrimp, localAlice } from './setup.js'; + +const runSuffix = Date.now().toString(36); + +describe('Mastodon ↔ Iceshrimp follow', () => { + it('Mastodon follows Iceshrimp user via WebFinger resolution', async () => { + // Resolve Iceshrimp user from Mastodon + const accounts = await mastodon.searchAccount( + mastodonToken, + `@${localAlice.username}@iceshrimp.localhost`, + ); + expect(accounts.length).toBeGreaterThanOrEqual(1); + + const iceshrimpAccount = accounts[0]; + const iceshrimpAcctId = iceshrimpAccount.id as string; + + // Follow the resolved account + const result = await mastodon.followAccount(mastodonToken, iceshrimpAcctId); + expect(result).toBeDefined(); + + // Poll Iceshrimp followers until Mastodon actor appears + const localAliceActor = await iceshrimp.showUser(localAlice.username, localAlice.token); + const localAliceId = localAliceActor.id as string; + + let followerFound = false; + for (let i = 0; i < 15; i++) { + const followers = await iceshrimp.getFollowers(localAliceId, localAlice.token); + const followerUris = followers.map((f: Record) => (f.follower as Record)?.uri as string ?? f.uri as string); + if (followerUris.some((uri) => uri?.includes('mastodon.localhost'))) { + followerFound = true; + break; + } + await new Promise((r) => setTimeout(r, 2000)); + } + expect(followerFound).toBe(true); + }); +}); diff --git a/test-runner/tests/mastodon-note.test.ts b/test-runner/tests/mastodon-note.test.ts new file mode 100644 index 0000000..54888f8 --- /dev/null +++ b/test-runner/tests/mastodon-note.test.ts @@ -0,0 +1,38 @@ +import { describe, it, expect } from 'vitest'; +import { mastodon, mastodonToken } from './mastodon-setup.js'; +import { iceshrimp, localAlice } from './setup.js'; + +const runSuffix = Date.now().toString(36); + +describe('Mastodon ↔ Iceshrimp note delivery', () => { + it('Iceshrimp note appears in Mastodon home timeline', async () => { + const noteText = `mastodon-e2e-${runSuffix}-${Date.now()}`; + + // Mastodon searches for and follows Alice on Iceshrimp + const accounts = await mastodon.searchAccount(mastodonToken, `@${localAlice.username}@iceshrimp.localhost`); + expect(accounts.length).toBeGreaterThanOrEqual(1); + const iceshrimpAcctId = accounts[0].id as string; + const followResult = await mastodon.followAccount(mastodonToken, iceshrimpAcctId); + expect(followResult).toBeDefined(); + + // Wait for the follow to propagate + await new Promise((r) => setTimeout(r, 2000)); + + // Iceshrimp Alice creates a note + const note = await iceshrimp.createNote(noteText, localAlice.token); + const noteId = note.createdNote?.id as string ?? note.id as string; + expect(noteId).toBeTruthy(); + + // Poll Mastodon home timeline until the note appears + const status = await mastodon.pollHomeTimeline( + mastodonToken, + (s: Record) => { + const content = (s.content as string) ?? ''; + return content.includes(noteText); + }, + ); + + expect(status).toBeDefined(); + expect((status.content as string) ?? '').toContain(noteText); + }); +}); diff --git a/test-runner/tests/mastodon-reaction.test.ts b/test-runner/tests/mastodon-reaction.test.ts new file mode 100644 index 0000000..98de3de --- /dev/null +++ b/test-runner/tests/mastodon-reaction.test.ts @@ -0,0 +1,62 @@ +import { describe, it, expect } from 'vitest'; +import { mastodon, mastodonToken } from './mastodon-setup.js'; +import { iceshrimp, localAlice } from './setup.js'; + +const runSuffix = Date.now().toString(36); + +describe('Mastodon ↔ Iceshrimp reaction', () => { + it('Mastodon favourite on Iceshrimp note registers as reaction', async () => { + const noteText = `mastodon-react-${runSuffix}-${Date.now()}`; + + // Mastodon follows Iceshrimp Alice so that notes are delivered + const accounts = await mastodon.searchAccount(mastodonToken, `@${localAlice.username}@iceshrimp.localhost`); + expect(accounts.length).toBeGreaterThanOrEqual(1); + const iceshrimpAcctId = accounts[0].id as string; + const followResult = await mastodon.followAccount(mastodonToken, iceshrimpAcctId); + expect(followResult).toBeDefined(); + // Allow follow propagation before creating the note + await new Promise((r) => setTimeout(r, 2000)); + + // Iceshrimp Alice creates a note + const note = await iceshrimp.createNote(noteText, localAlice.token); + const noteId = note.createdNote?.id as string ?? note.id as string; + expect(noteId).toBeTruthy(); + + // Wait for note to appear in Mastodon timeline + const status = await mastodon.pollHomeTimeline( + mastodonToken, + (s: Record) => { + const content = (s.content as string) ?? ''; + return content.includes(noteText); + }, + ); + + // Favourite the note + const mastodonStatusId = status.id as string; + const favResult = await mastodon.favouriteStatus(mastodonToken, mastodonStatusId); + expect(favResult).toBeDefined(); + expect((favResult as Record).favourites_count as number).toBeGreaterThanOrEqual(1); + + // Poll Iceshrimp note until reaction reflects the favourite + // Iceshrimp uses /api/notes/show to check reactions + let reactionFound = false; + for (let i = 0; i < 15; i++) { + try { + const noteDetail = await iceshrimp.apiPost>( + '/api/notes/show', + { noteId }, + localAlice.token, + ); + const reactions = noteDetail.reactions as Record ?? {}; + if (Object.values(reactions).some((count) => (count as number) > 0)) { + reactionFound = true; + break; + } + } catch { + // note might not be visible yet + } + await new Promise((r) => setTimeout(r, 2000)); + } + expect(reactionFound).toBe(true); + }); +}); From 7b52ce3b74f75d0029602db77f80395060f31f04 Mon Sep 17 00:00:00 2001 From: turtton Date: Sat, 27 Jun 2026 00:15:50 +0900 Subject: [PATCH 11/15] Fix followers API access pattern and note content predicate in E2E tests Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus --- test-runner/tests/follow.test.ts | 5 ++++- test-runner/tests/note-delivery.test.ts | 2 +- 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/test-runner/tests/follow.test.ts b/test-runner/tests/follow.test.ts index 75fd998..a93be7b 100644 --- a/test-runner/tests/follow.test.ts +++ b/test-runner/tests/follow.test.ts @@ -37,7 +37,10 @@ describe('Follow federation', () => { let followerFound = false; for (let i = 0; i < 15; i++) { const followers = await iceshrimp.getFollowers(localAliceId, localAlice.token); - const followerUris = followers.map((f: Record) => f.uri as string); + const followerUris = followers.map((f: Record) => { + const follower = f.follower as Record | undefined; + return (follower?.uri as string) || (f.uri as string); + }); if (followerUris.includes(remoteAlice.id)) { followerFound = true; break; diff --git a/test-runner/tests/note-delivery.test.ts b/test-runner/tests/note-delivery.test.ts index a63b8b5..3c6a062 100644 --- a/test-runner/tests/note-delivery.test.ts +++ b/test-runner/tests/note-delivery.test.ts @@ -46,7 +46,7 @@ describe('Note delivery federation', () => { // 4. Wait for the note to be delivered to remote-alice's inbox const createActivity = await customAp.pollInbox( - (item) => item.type === 'Create' && (item.object as Record)?.content === noteText, + (item) => item.type === 'Create' && String((item.object as Record)?.content ?? '').includes(noteText), 20, ); From 9bccf4044c3154d6e5339ad402459a050d3914df Mon Sep 17 00:00:00 2001 From: turtton Date: Sat, 27 Jun 2026 00:15:57 +0900 Subject: [PATCH 12/15] Add Mastodon service to Docker Compose with nginx SSL proxy Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus --- .env.example | 7 ++ compose.yml | 124 +++++++++++++++++++++++++++++- configs/nginx/conf.d/nginx.conf | 1 + configs/nginx/confs/mastodon.conf | 33 ++++++++ 4 files changed, 163 insertions(+), 2 deletions(-) create mode 100644 configs/nginx/confs/mastodon.conf diff --git a/.env.example b/.env.example index d95beef..76e24fa 100644 --- a/.env.example +++ b/.env.example @@ -4,3 +4,10 @@ CUSTOM_AP_DOCKERFILE=Dockerfile # Test mode token (must match between test-runner and custom-ap) TEST_MODE_TOKEN=ap-testbox-test-token + +# Mastodon secrets (generate once via setup commands) +MASTODON_DB_PASSWORD=mastodon-pass +MASTODON_TEST_USERNAME=e2e-mastodon +MASTODON_TEST_EMAIL=e2e-mastodon@mastodon.localhost +MASTODON_TEST_PASSWORD=test-pass-mastodon +MASTODON_IMAGE_TAG=latest diff --git a/compose.yml b/compose.yml index 9340d55..5884e1d 100644 --- a/compose.yml +++ b/compose.yml @@ -11,12 +11,15 @@ services: depends_on: - iceshrimp - custom-ap + - mastodon-web + - mastodon-streaming networks: default: aliases: - iceshrimp.localhost - test-ap.localhost - shuttlepub.localhost + - mastodon.localhost extra_hosts: - host.docker.internal:host-gateway @@ -28,6 +31,7 @@ services: environment: NODE_ENV: production NODE_EXTRA_CA_CERTS: /certs/rootCA.pem + SSL_CERT_FILE: /certs/rootCA.pem volumes: - ./configs/iceshrimp/files:/iceshrimp/files - ./configs/iceshrimp/.config:/iceshrimp/.config:ro @@ -44,6 +48,7 @@ services: LEGACY_PUBLIC_BASE_URL: https://shuttlepub.localhost ICESHRIMP_BASE_URL: https://iceshrimp.localhost NODE_EXTRA_CA_CERTS: /certs/rootCA.pem + SSL_CERT_FILE: /certs/rootCA.pem PORT: "12864" volumes: - ./certs/root/rootCA.pem:/certs/rootCA.pem:ro @@ -62,6 +67,7 @@ services: environment: NODE_ENV: test NODE_EXTRA_CA_CERTS: /certs/rootCA.pem + SSL_CERT_FILE: /certs/rootCA.pem ICESHRIMP_BASE_URL: https://iceshrimp.localhost CUSTOM_AP_BASE_URL: https://test-ap.localhost CUSTOM_AP_LEGACY_BASE_URL: https://shuttlepub.localhost @@ -71,6 +77,11 @@ services: PGDATABASE: iceshrimp PGUSER: example-iceshrimp-user PGPASSWORD: example-iceshrimp-pass + MASTODON_BASE_URL: https://mastodon.localhost + MASTODON_TEST_USERNAME: ${MASTODON_TEST_USERNAME:-e2e-mastodon} + MASTODON_TEST_EMAIL: ${MASTODON_TEST_EMAIL:-e2e-mastodon@mastodon.localhost} + MASTODON_TEST_PASSWORD: ${MASTODON_TEST_PASSWORD:-test-pass-mastodon} + MASTODON_E2E_TOKEN: ${MASTODON_E2E_TOKEN} volumes: - ./certs/root/rootCA.pem:/certs/rootCA.pem:ro @@ -80,14 +91,123 @@ services: - ./configs/iceshrimp/.config/docker.env volumes: - db_data:/var/lib/postgresql/data - ports: - - "127.0.0.1:5432:5432" redis: image: valkey/valkey:7-alpine volumes: - redis_data:/data + mastodon-db: + image: postgres:16-alpine + environment: + POSTGRES_DB: mastodon_production + POSTGRES_USER: mastodon + POSTGRES_PASSWORD: ${MASTODON_DB_PASSWORD:-mastodon-pass} + volumes: + - mastodon_db_data:/var/lib/postgresql/data + healthcheck: + test: ['CMD', 'pg_isready', '-U', 'mastodon'] + interval: 5s + timeout: 5s + retries: 5 + + mastodon-web: + image: ghcr.io/mastodon/mastodon:${MASTODON_IMAGE_TAG:-latest} + command: bundle exec puma -C config/puma.rb + depends_on: + mastodon-db: + condition: service_healthy + redis: + condition: service_started + environment: + RAILS_ENV: production + NODE_ENV: production + LOCAL_DOMAIN: mastodon.localhost + LOCAL_HTTPS: "true" + BIND: 0.0.0.0 + RAILS_SERVE_STATIC_FILES: "true" + ES_ENABLED: "false" + DB_HOST: mastodon-db + DB_PORT: "5432" + DB_NAME: mastodon_production + DB_USER: mastodon + DB_PASS: ${MASTODON_DB_PASSWORD:-mastodon-pass} + REDIS_HOST: redis + REDIS_PORT: "6379" + ALLOWED_PRIVATE_ADDRESSES: 172.16.0.0/12,10.0.0.0/8 + TRUSTED_PROXY_IP: 127.0.0.1/8,10.0.0.0/8,172.16.0.0/12 + NODE_EXTRA_CA_CERTS: /certs/rootCA.pem + SSL_CERT_FILE: /certs/rootCA.pem + # Secrets (must be set in .env) + SECRET_KEY_BASE: ${MASTODON_SECRET_KEY_BASE:-insecure-dev-key-base} + OTP_SECRET: ${MASTODON_OTP_SECRET:-insecure-dev-otp-secret} + ACTIVE_RECORD_ENCRYPTION_PRIMARY_KEY: ${ACTIVE_RECORD_ENCRYPTION_PRIMARY_KEY} + ACTIVE_RECORD_ENCRYPTION_DETERMINISTIC_KEY: ${ACTIVE_RECORD_ENCRYPTION_DETERMINISTIC_KEY} + ACTIVE_RECORD_ENCRYPTION_KEY_DERIVATION_SALT: ${ACTIVE_RECORD_ENCRYPTION_KEY_DERIVATION_SALT} + VAPID_PRIVATE_KEY: ${VAPID_PRIVATE_KEY} + VAPID_PUBLIC_KEY: ${VAPID_PUBLIC_KEY} + volumes: + - ./certs/root/rootCA.pem:/certs/rootCA.pem:ro + - mastodon_public_system:/mastodon/public/system + + mastodon-streaming: + image: ghcr.io/mastodon/mastodon-streaming:${MASTODON_IMAGE_TAG:-latest} + depends_on: + - mastodon-db + - redis + environment: + NODE_ENV: production + LOCAL_DOMAIN: mastodon.localhost + BIND: 0.0.0.0 + DB_HOST: mastodon-db + DB_PORT: "5432" + DB_NAME: mastodon_production + DB_USER: mastodon + DB_PASS: ${MASTODON_DB_PASSWORD:-mastodon-pass} + REDIS_HOST: redis + REDIS_PORT: "6379" + ALLOWED_PRIVATE_ADDRESSES: 172.16.0.0/12,10.0.0.0/8 + NODE_EXTRA_CA_CERTS: /certs/rootCA.pem + SSL_CERT_FILE: /certs/rootCA.pem + volumes: + - ./certs/root/rootCA.pem:/certs/rootCA.pem:ro + + mastodon-sidekiq: + image: ghcr.io/mastodon/mastodon:${MASTODON_IMAGE_TAG:-latest} + command: bundle exec sidekiq + depends_on: + mastodon-db: + condition: service_healthy + redis: + condition: service_started + environment: + RAILS_ENV: production + NODE_ENV: production + LOCAL_DOMAIN: mastodon.localhost + LOCAL_HTTPS: "true" + ES_ENABLED: "false" + DB_HOST: mastodon-db + DB_PORT: "5432" + DB_NAME: mastodon_production + DB_USER: mastodon + DB_PASS: ${MASTODON_DB_PASSWORD:-mastodon-pass} + REDIS_HOST: redis + REDIS_PORT: "6379" + ALLOWED_PRIVATE_ADDRESSES: 172.16.0.0/12,10.0.0.0/8 + TRUSTED_PROXY_IP: 127.0.0.1/8,10.0.0.0/8,172.16.0.0/12 + NODE_EXTRA_CA_CERTS: /certs/rootCA.pem + SSL_CERT_FILE: /certs/rootCA.pem + SECRET_KEY_BASE: ${MASTODON_SECRET_KEY_BASE:-insecure-dev-key-base} + OTP_SECRET: ${MASTODON_OTP_SECRET:-insecure-dev-otp-secret} + ACTIVE_RECORD_ENCRYPTION_PRIMARY_KEY: ${ACTIVE_RECORD_ENCRYPTION_PRIMARY_KEY} + ACTIVE_RECORD_ENCRYPTION_DETERMINISTIC_KEY: ${ACTIVE_RECORD_ENCRYPTION_DETERMINISTIC_KEY} + ACTIVE_RECORD_ENCRYPTION_KEY_DERIVATION_SALT: ${ACTIVE_RECORD_ENCRYPTION_KEY_DERIVATION_SALT} + volumes: + - ./certs/root/rootCA.pem:/certs/rootCA.pem:ro + - mastodon_public_system:/mastodon/public/system + volumes: db_data: redis_data: + mastodon_db_data: + mastodon_public_system: diff --git a/configs/nginx/conf.d/nginx.conf b/configs/nginx/conf.d/nginx.conf index 3250e06..ce3e3af 100644 --- a/configs/nginx/conf.d/nginx.conf +++ b/configs/nginx/conf.d/nginx.conf @@ -7,3 +7,4 @@ server { include /etc/nginx/confs/iceshrimp.conf; include /etc/nginx/confs/shuttlepub.conf; include /etc/nginx/confs/test-ap.conf; +include /etc/nginx/confs/mastodon.conf; diff --git a/configs/nginx/confs/mastodon.conf b/configs/nginx/confs/mastodon.conf new file mode 100644 index 0000000..44d420f --- /dev/null +++ b/configs/nginx/confs/mastodon.conf @@ -0,0 +1,33 @@ +server { + listen 443 ssl; + server_name mastodon.localhost; + + ssl_certificate /etc/nginx/certs/mastodon.crt; + ssl_certificate_key /etc/nginx/certs/mastodon.key; + + location / { + proxy_pass http://mastodon-web:3000; + proxy_http_version 1.1; + + proxy_set_header Host $host; + proxy_set_header X-Forwarded-Proto https; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Real-IP $remote_addr; + + proxy_set_header Upgrade $http_upgrade; + proxy_set_header Connection "upgrade"; + } + + location ~ ^/api/v1/streaming/ { + proxy_pass http://mastodon-streaming:4000; + proxy_http_version 1.1; + + proxy_set_header Host $host; + proxy_set_header X-Forwarded-Proto https; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Real-IP $remote_addr; + + proxy_set_header Upgrade $http_upgrade; + proxy_set_header Connection "upgrade"; + } +} From 7fa404f27c9ecb65a8a4554468a6a9f112153ada Mon Sep 17 00:00:00 2001 From: turtton Date: Sat, 27 Jun 2026 00:16:10 +0900 Subject: [PATCH 13/15] Add custom-ap-dummy with full ActivityPub contract implementation Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus --- custom-ap-dummy/.dockerignore | 1 + custom-ap-dummy/Dockerfile | 5 + custom-ap-dummy/server.mjs | 218 ++++++++++++++++++++++++++++++++++ 3 files changed, 224 insertions(+) create mode 100644 custom-ap-dummy/.dockerignore create mode 100644 custom-ap-dummy/Dockerfile create mode 100644 custom-ap-dummy/server.mjs diff --git a/custom-ap-dummy/.dockerignore b/custom-ap-dummy/.dockerignore new file mode 100644 index 0000000..421376d --- /dev/null +++ b/custom-ap-dummy/.dockerignore @@ -0,0 +1 @@ +dummy diff --git a/custom-ap-dummy/Dockerfile b/custom-ap-dummy/Dockerfile new file mode 100644 index 0000000..b763de8 --- /dev/null +++ b/custom-ap-dummy/Dockerfile @@ -0,0 +1,5 @@ +FROM node:20-alpine +RUN npm install -g http-server +COPY server.mjs /server.mjs +EXPOSE 12864 +CMD ["node", "/server.mjs"] diff --git a/custom-ap-dummy/server.mjs b/custom-ap-dummy/server.mjs new file mode 100644 index 0000000..bbee0e4 --- /dev/null +++ b/custom-ap-dummy/server.mjs @@ -0,0 +1,218 @@ +import http from 'http'; + +const PORT = parseInt(process.env.PORT || '12864', 10); +const TEST_MODE_TOKEN = process.env.TEST_MODE_TOKEN || 'ap-testbox-test-token'; +const PUBLIC_BASE_URL = process.env.PUBLIC_BASE_URL || 'https://test-ap.localhost'; + +// In-memory state +const actors = Object.create(null); +const inbox = []; + +// ─── Helpers ──────────────────────────────────────────────────────────────── + +function parseBody(req) { + return new Promise((resolve, reject) => { + let body = ''; + req.on('data', (chunk) => { body += chunk; }); + req.on('end', () => { + try { + resolve(body ? JSON.parse(body) : null); + } catch { + reject(new Error('Invalid JSON')); + } + }); + req.on('error', reject); + }); +} + +function isAuthorized(req) { + const auth = req.headers['authorization']; + return !!auth && auth.startsWith('Bearer ') && auth.slice(7) === TEST_MODE_TOKEN; +} + +function sendJSON(res, status, data, contentType = 'application/json') { + const raw = JSON.stringify(data); + res.writeHead(status, { + 'Content-Type': contentType, + 'Content-Length': Buffer.byteLength(raw), + }); + res.end(raw); +} + +function sendUnauthorized(res) { + sendJSON(res, 401, { error: 'unauthorized' }); +} + +// ─── Server ───────────────────────────────────────────────────────────────── + +const server = http.createServer(async (req, res) => { + // Gracefully handle the one extremely common error path + let url; + try { + url = new URL(req.url, `http://${req.headers.host || 'localhost'}`); + } catch { + sendJSON(res, 400, { error: 'invalid url' }); + return; + } + + const { pathname } = url; + const { method } = req; + + try { + // ── 1. GET /__test__/health (no auth) ──────────────────────────────── + if (method === 'GET' && pathname === '/__test__/health') { + sendJSON(res, 200, { status: 'ok' }); + return; + } + + // ── 2. POST /__test__/reset (Bearer auth) ──────────────────────────── + if (method === 'POST' && pathname === '/__test__/reset') { + if (!isAuthorized(req)) return sendUnauthorized(res); + + // Clear all state + for (const key of Object.keys(actors)) delete actors[key]; + inbox.length = 0; + + sendJSON(res, 200, { status: 'ok' }); + return; + } + + // ── 3. POST /__test__/actors (Bearer auth) ────────────────────────── + if (method === 'POST' && pathname === '/__test__/actors') { + if (!isAuthorized(req)) return sendUnauthorized(res); + + let body; + try { + body = await parseBody(req); + } catch { + sendJSON(res, 400, { error: 'invalid json body' }); + return; + } + + if (!body || !body.username || !body.publicKeyPem) { + sendJSON(res, 400, { error: 'username and publicKeyPem required' }); + return; + } + + const { username, publicKeyPem } = body; + actors[username] = { username, publicKeyPem }; + + const id = `${PUBLIC_BASE_URL}/users/${username}`; + sendJSON(res, 201, { + id, + inbox: `${id}/inbox`, + outbox: `${id}/outbox`, + publicKeyId: `${id}#main-key`, + }); + return; + } + + // ── 4. GET /__test__/inbox (Bearer auth) ──────────────────────────── + if (method === 'GET' && pathname === '/__test__/inbox') { + if (!isAuthorized(req)) return sendUnauthorized(res); + + sendJSON(res, 200, { items: inbox }); + return; + } + + // ── 5. GET /.well-known/webfinger (no auth) ───────────────────────── + if (method === 'GET' && pathname === '/.well-known/webfinger') { + const resource = url.searchParams.get('resource'); + if (!resource) { + sendJSON(res, 400, { error: 'missing resource query parameter' }); + return; + } + + const acctPrefix = 'acct:'; + if (!resource.startsWith(acctPrefix)) { + sendJSON(res, 400, { error: 'resource must start with acct:' }); + return; + } + + const acctValue = resource.slice(acctPrefix.length); + const atIndex = acctValue.indexOf('@'); + if (atIndex === -1) { + sendJSON(res, 400, { error: 'resource must be in acct:user@host format' }); + return; + } + + const username = acctValue.slice(0, atIndex); + if (!username) { + sendJSON(res, 400, { error: 'username is empty' }); + return; + } + + sendJSON(res, 200, { + subject: resource, + links: [ + { + rel: 'self', + type: 'application/activity+json', + href: `${PUBLIC_BASE_URL}/users/${username}`, + }, + ], + }, 'application/jrd+json'); + return; + } + + // ── 6. GET /users/:username (no auth) ─────────────────────────────── + const userMatch = pathname.match(/^\/users\/([^/]+)$/); + if (method === 'GET' && userMatch) { + const username = userMatch[1]; + const actor = actors[username]; + + if (!actor) { + sendJSON(res, 404, { error: 'actor not found' }); + return; + } + + const actorId = `${PUBLIC_BASE_URL}/users/${username}`; + sendJSON(res, 200, { + '@context': 'https://www.w3.org/ns/activitystreams', + id: actorId, + type: 'Person', + preferredUsername: username, + inbox: `${actorId}/inbox`, + outbox: `${actorId}/outbox`, + publicKey: { + id: `${actorId}#main-key`, + owner: actorId, + publicKeyPem: actor.publicKeyPem, + }, + }, 'application/activity+json'); + return; + } + + // ── 7. POST /users/:username/inbox (no auth) ──────────────────────── + const inboxMatch = pathname.match(/^\/users\/([^/]+)\/inbox$/); + if (method === 'POST' && inboxMatch) { + let body; + try { + body = await parseBody(req); + } catch { + sendJSON(res, 400, { error: 'invalid json body' }); + return; + } + + if (body) { + inbox.push(body); + } + + res.writeHead(202, { 'Content-Length': '0' }); + res.end(); + return; + } + + // ── Fallback: 404 ────────────────────────────────────────────────── + sendJSON(res, 404, { error: 'not found' }); + } catch (err) { + console.error('Unhandled error:', err); + sendJSON(res, 500, { error: 'internal server error' }); + } +}); + +// ─── Start ────────────────────────────────────────────────────────────────── + +server.listen(PORT, () => { + console.log(`Custom AP dummy server listening on port ${PORT}`); +}); From 4cf9b2e328de5f081ff612feae49f8d17b7894f7 Mon Sep 17 00:00:00 2001 From: turtton Date: Sat, 27 Jun 2026 00:16:17 +0900 Subject: [PATCH 14/15] Update project documentation and add .env to gitignore Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus --- .gitignore | 1 + AGENTS.md | 8 ++++++++ README.md | 34 ++++++++++++++++++++++++++++++++++ 3 files changed, 43 insertions(+) diff --git a/.gitignore b/.gitignore index 6862dd8..2e515a1 100644 --- a/.gitignore +++ b/.gitignore @@ -1,2 +1,3 @@ /certs /misskey +.env diff --git a/AGENTS.md b/AGENTS.md index 726308f..3974d61 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -10,6 +10,10 @@ Docker Compose ActivityPub E2E testing environment. Uses [Iceshrimp](https://ice - **test-runner**: TypeScript + Vitest E2E test suite - **db**: PostgreSQL 16 - **redis**: Valkey 7 (Redis-compatible) +- **mastodon-web**: Mastodon v4.x Fediverse instance (official Docker image) +- **mastodon-streaming**: Mastodon streaming API +- **mastodon-sidekiq**: Mastodon background job processor (required for federation) +- **mastodon-db**: PostgreSQL 16 for Mastodon ## Domains (all via nginx on :4430) @@ -18,6 +22,7 @@ Docker Compose ActivityPub E2E testing environment. Uses [Iceshrimp](https://ice | `iceshrimp.localhost` | `iceshrimp:3000` | Primary Fediverse instance | | `test-ap.localhost` | `custom-ap:12864` | Custom AP system | | `shuttlepub.localhost` | `custom-ap:12864` | Legacy alias | +| `mastodon.localhost` | `mastodon-web:3000` | Mastodon federation peer | ## First-time setup (order matters) @@ -84,6 +89,9 @@ ap-testbox/ | Rebuild test-runner | `docker compose build test-runner` | | Access postgres | `docker compose exec db psql -U example-iceshrimp-user -d iceshrimp` | | Custom AP health check | `curl -sk https://test-ap.localhost:4430/__test__/health` | +| View mastodon logs | `docker compose logs -f mastodon-web mastodon-sidekiq` | +| Mastodon DB setup | `docker compose run --rm mastodon-web bundle exec rails db:prepare` | +| Create Mastodon user | `docker compose run --rm mastodon-web bin/tootctl accounts create USERNAME --email EMAIL --confirmed` | ## Custom AP Contract diff --git a/README.md b/README.md index c39b369..408753e 100644 --- a/README.md +++ b/README.md @@ -12,6 +12,10 @@ Docker Compose ActivityPub E2E testing environment. Runs [Iceshrimp](https://ice | test-runner | (build from `./test-runner`) | Vitest E2E test suite | | db | `postgres:16-alpine` | PostgreSQL | | redis | `valkey/valkey:7-alpine` | Redis-compatible cache | +| mastodon-web | `ghcr.io/mastodon/mastodon:latest` | Mastodon Fediverse instance (Puma) | +| mastodon-streaming | `ghcr.io/mastodon/mastodon-streaming:latest` | Mastodon streaming API | +| mastodon-sidekiq | `ghcr.io/mastodon/mastodon:latest` | Mastodon background jobs (federation) | +| mastodon-db | `postgres:16-alpine` | Mastodon PostgreSQL | ## Requirement @@ -42,6 +46,7 @@ docker compose run --rm test-runner Then access: - **Iceshrimp**: `https://iceshrimp.localhost:4430` - **Custom AP**: `https://test-ap.localhost:4430` +- **Mastodon**: `https://mastodon.localhost:4430` ## Custom AP Setup @@ -65,6 +70,35 @@ docker compose build test-runner docker compose run --rm test-runner ``` +## Mastodon Setup + +```bash +# 1. Generate secrets (run from project root) +docker compose run --rm mastodon-web bundle exec rails secret +# ^ Save output as MASTODON_SECRET_KEY_BASE in .env +docker compose run --rm mastodon-web bundle exec rails secret +# ^ Save output as MASTODON_OTP_SECRET in .env +docker compose run --rm mastodon-web bundle exec rails mastodon:webpush:generate_vapid_key +# ^ Save VAPID_PRIVATE_KEY and VAPID_PUBLIC_KEY in .env + +# 2. Prepare database (mastodon-db and redis must be running) +docker compose up -d mastodon-db redis +docker compose run --rm mastodon-web bundle exec rails db:prepare + +# 3. Create test user +docker compose run --rm mastodon-web bin/tootctl accounts create "$MASTODON_TEST_USERNAME" --email "$MASTODON_TEST_EMAIL" --confirmed || true +docker compose run --rm mastodon-web bundle exec rails runner " + user = User.find_by!(email: ENV.fetch('MASTODON_TEST_EMAIL')) + user.update!(password: ENV.fetch('MASTODON_TEST_PASSWORD'), password_confirmation: ENV.fetch('MASTODON_TEST_PASSWORD'), confirmed_at: Time.now.utc) +" + +# 4. Start all services and run tests +docker compose up -d +docker compose run --rm test-runner +``` + +Note: Secrets must be generated ONCE and stored in `.env`. Use `--insecure-dev-*` fallbacks only if you don't care about security (dev environment only). + ## Common Commands | Action | Command | From 9dec622601879eeb7813c246c40f9687cfad115b Mon Sep 17 00:00:00 2001 From: turtton Date: Sat, 27 Jun 2026 00:16:23 +0900 Subject: [PATCH 15/15] Update cert generation for additional Mastodon domain Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus --- gencert.sh | 2 +- gencert/mkcert.sh | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/gencert.sh b/gencert.sh index f5d9197..ce53cc6 100755 --- a/gencert.sh +++ b/gencert.sh @@ -6,4 +6,4 @@ mkdir -p ./certs/nginx # Build the gencert image and run it docker build ./gencert/. -t shuttlepub/gencert:latest -docker run --mount "type=bind,source="$(pwd)/certs",target=/certs" -it shuttlepub/gencert:latest +docker run --mount "type=bind,source="$(pwd)/certs",target=/certs" -i shuttlepub/gencert:latest diff --git a/gencert/mkcert.sh b/gencert/mkcert.sh index afdfbdb..12ccc61 100755 --- a/gencert/mkcert.sh +++ b/gencert/mkcert.sh @@ -4,3 +4,4 @@ cp /root/.local/share/mkcert/rootCA.pem /certs/root/ mkcert -cert-file /certs/nginx/iceshrimp.crt -key-file /certs/nginx/iceshrimp.key iceshrimp.localhost mkcert -cert-file /certs/nginx/test-ap.crt -key-file /certs/nginx/test-ap.key test-ap.localhost mkcert -cert-file /certs/nginx/shuttlepub.crt -key-file /certs/nginx/shuttlepub.key shuttlepub.localhost +mkcert -cert-file /certs/nginx/mastodon.crt -key-file /certs/nginx/mastodon.key mastodon.localhost