Commit Graph

5 Commits

Author SHA1 Message Date
khannurien
14e3ab125b v3: fix the privacy and security findings of the comment pass — private dumps' broadcasts, mentions and attachments reached people who can't see them, and a few smaller holes. A dump's every write now goes through ws-service's broadcastDumpChange(before, after), which decides the audience: a private dump's creation, edits and deletion reach its owner's sockets alone (its deletion id went to everyone before), going private is dump_deleted for everyone else and dump_updated for the owner (whose other tabs dropped it), going public dump_created for everyone else. Mentions in a private dump, a comment on one, or a private playlist's description notify no one until it goes public (mentions are deduplicated per context, so going public notifies them once), and a moderator's edit of a comment, chat message or playlist mentions in the author's name. A link preview of a private playlist is refused, as one of a private dump already was: the preview is stored in the linking dump, which others may see. Attachments record the type of the resource that links them (migration 0018 backfills it); one linked by a private dump or playlist, or a comment on a private dump, is served only with its owner's media token, which Markdown now adds to attachment URLs, and one whose resource is gone answers 404 and is purged at startup. The Bandcamp provider claimed bandcamp.com.evil.example (the pattern wasn't anchored) and now reuses bandcamp-tralbum's anchored one; the YouTube provider took any site's ?list= for a YouTube playlist and missed m., music., youtube-nocookie and /live/ links, and now shares canonical-url's rules. Sign-in does the same scrypt work for an unknown username as for a wrong password, so timing no longer tells which exist. Usernames and emails are unique regardless of case (migration 0017, which refuses to run over existing clashes and names them): "Bob" and "bob" could both sign up and a mention picked one, emails had no constraint, and a reset request in another case silently sent nothing. The Open Graph tags escape the site name and build their URLs on PUBLIC_URL and FRONTEND_URL instead of the request's origin, which a proxy may rewrite.
Error statuses follow one rule: APIException takes a status and derives the code from it (BAD_REQUEST 400, UNAUTHORIZED 401, FORBIDDEN 403, NOT_FOUND 404, CONFLICT 409, RATE_LIMITED 429, SERVER_ERROR 500, BAD_GATEWAY 502), so a code can't contradict its status. That fixed real bugs: a wrong current password on change-password answered 401, on which authFetch signs the user out; editing or deleting someone else's comment or chat message answered 401 rather than 403; one invalid dump edit answered 422. A taken username is 409, like a taken email.

Along the way: the database is created from schema.sql by connection.ts when the file doesn't exist, so every task works on a fresh checkout (dev and serve never ran db/init.ts, which is gone); the startup purge compared ISO dates with SQLite's datetime() as text and kept anything from the current UTC day, and now compares ISO cutoffs; the seeded admin's hash comes from lib/password.ts (split out of jwt.ts, which needs the JWT secret) rather than a copy of its format; avatar, thumbnail and playlist-cover uploads go through replaceUploadedFile, which writes beside the old file and renames over it before the record announces it (a failed avatar write lost the old avatar; a thumbnail was announced before it existed); upload.ts derives its size message from MAX_IMAGE_SIZE_BYTES and stops re-exporting the upload folders; ALLOWED_IMAGE_MIMES, which duplicated detectImageMime, is gone; the SPA fallback no longer calls next() after answering; a reply must answer a comment on the same dump (a foreign or unknown parent was a 500 or a cross-dump thread). 211 backend and 628 frontend tests pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 15:19:07 +00:00
khannurien
d392be7cb8 v3: api/ gets one layout rule and keeps to it, and whatever crosses the wire is defined once, in shared/ — api/model/interfaces.ts and src/model.ts declared the same 40 types by hand and had drifted: the error envelope, email required on one side and optional on the other, Invitee/InviteTreeEntry and LinkPreview/UrlPreviewResponse for one shape each. The validation limits, the role permissions, the chat length cap, the @mention pattern and the Bandcamp and YouTube player responses were copied too, behind "keep in sync" comments. shared/ now holds them once, one file per domain (api, category, chat, comment, dump, follow, mention, notification, permissions, player, playlist, search, user, validation, ws), and imports nothing from either half. The server sends Dates; the client reads Wire<T>, the same type with every Date an ISO string, and turns it back with src/utils/shared/deserialize.ts. APIErrorCode becomes a const object, since the frontend's erasableSyntaxOnly rejects an enum. The shared types showed two real gaps: the avatar upload and GET /api/users/me return the full self user and the frontend read them as public users, and a stored session may predate media tokens, so the client keeps it as its own Session type. The backend rule: each folder holds one kind of module, each file is named after its domain. routes/ parse and answer; services/ hold the logic and every SQL query (lib/auth.ts's token check moves to user-service); remote/ is the only place that fetches from other sites — the guarded fetch and the HTML scraping out of rich-content-service, public-address, the Bandcamp and YouTube resolvers, the link providers, the image proxy (fetch, checks and cache, out of routes/preview.ts) and the Twemoji glyphs (out of lib/site-icons.ts). model/ holds per table a row type, its guard and its mappers; validators/ the request guards; db/connection.ts the handle and transaction(), db/startup.ts the purges and default admin that db.ts ran on import, called at the same moment. Tests mirror the source one file per module, test/<layer>/<module>.test.ts, with shared/'s under shared/test/ (deno task test runs both); the test that covered the bandcamp and soundcloud providers together is two now, and the backend can() has its own, so 202 tests. deno lint src/ is clean too: window.* becomes globalThis.*, and test bodies and act() callbacks that never awaited lose their async. The README states the rule. No behaviour changes; the catalogues only move their source references.
All checks were successful
Build and Publish Docker Image / build-and-push (push) Successful in 50s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-27 20:19:48 +00:00
khannurien
de5abeceb2 v3: a performance pass over both halves — indexed queries, streamed files, cached previews, and feeds that stop re-rendering on every tick — a review turned up whole-file reads, unindexed scans and render storms; each fix comes with a test. Database: migration 0016 adds the indexes the hot queries lacked — dumps by created_at, (user_id, created_at) and slug, votes by (user_id, created_at), playlists by slug, users by lower(username) for @mentions, and the child side of every cascading foreign key (comment replies, a user's comments, follows both ways, chat messages, invites, reset tokens); the feeds, dump and playlist pages, upvotes and follower fan-out all plan as index searches now, with no temp-table sort. SQLite runs in WAL with synchronous=NORMAL and a busy timeout, and a transaction() helper wraps the multi-row writes (a dump and its categories, category sets, backlinks, attachment links, playlist reorders, follower notifications) so they commit once instead of once a row. Lists no longer run a COUNT(*) per page: they fetch limit + 1 rows and hasMore comes from the extra one; total stays only on search, the one place it's shown. Files: /api/files read the whole upload (up to 50 MB) for every range request a player makes while seeking; api/lib/serve-file.ts streams the requested bytes from disk, with an ETag and 304s, and serves avatars, attachments, covers and thumbnails too. File and thumbnail URLs carry the file's version (size and last change), so the current one is cached for a year and anything else revalidates — a same-size replacement used to stay stale, and a video's still was cached immutable forever under a URL that never changed. Feeds showed image dumps at full size: /api/thumbnails now makes a 960px preview with ffmpeg (GIFs and unreadable images go out as they are), one run per output however many cards ask at once, and ffmpeg is probed once per process instead of spawned per request. Uploads stream to disk; replacing a file writes beside it and renames over, a hard link keeping the old one for rollback instead of holding both in memory; generated stills and previews go when the file is replaced or the dump deleted, and a custom thumbnail with its dump. Outbound: fetchWithTimeout's deadline now covers the body — a server that answered at once and then dripped could hold a create open indefinitely — and pages are read to 4 MB at most; metadata is cached five minutes so the create form's preview and the post that follows share one fetch (refresh-metadata still goes to the page); proxied images sit in a 32 MB LRU. Broadcasts serialize once rather than per socket, and index.html is kept in memory until the file changes. Frontend: currentTime lived in PlayerContext, so every card re-rendered — and re-parsed its Markdown — four times a second while anything played; the playhead has its own PlayerTimeContext now. The socket context is split three ways (feed, chat via useWSChat, presence via useWSPresence), so typing, presence and chat messages leave the feeds alone. DumpCard, JournalCard and Markdown are memoized; card images use the server preview and load lazily; the feed cache is written when the page is left rather than after every pause in scrolling; the theme context value is memoized.
All checks were successful
Build and Publish Docker Image / build-and-push (push) Successful in 49s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-27 16:35:51 +00:00
khannurien
364c498a24 v3: dependencies upgraded and deno audit clean, every API response typed end to end, no explicit any left — and the site search stops handing out password hashes — deno audit reported 19 advisories (11 high); none remain. nodemailer 9 → 10.0.10 closes four of them (a quadratic address parser, two recipient-domain allow-list bypasses, resolveContent ignoring disableFileAccess/disableUrlAccess); 10's one break is Node ≥ 20, and 10.0.11 is a day too young for the minimum release age. react-router 8.0.1 → 8.4.0 closes the RSC-mode CSRF bypass. Refreshing the lockfile clears the transitive ones that came in through Vite, ESLint and Lingui: postcss, nanoid, picomatch, browserslist, brace-expansion and baseline-browser-mapping. Everything else moves to its latest too — Lingui 6.8 (its SWC plugin 6.7), React 19.3, Vite 8.3.1, react-hook-form 7.89, marked, jose, ESLint 10.11, typescript-eslint 8.70, the @types — and frimousse 0.3 → 0.4, whose declarations only add (custom emoji data resolvers, a cache). TypeScript stays on 6.0: typescript-eslint supports < 6.1, so 7.0 would stop the lint. allowScripts names @swc/core 1.16.2 and drops esbuild, which Vite 8 no longer installs.
All checks were successful
Build and Publish Docker Image / build-and-push (push) Successful in 51s
Search: GET /api/search sent each matching user's full row — password hash and email included — to anyone, signed in or not, because searchUsersGlobal returned User[] and the route passed it through. It returns profiles only now, stripped where the rows are read, and the search test fails on the old code. Types alone wouldn't have caught it (a User is structurally a PublicUser), so the strip is explicit and tested.

Typed routes. Every JSON response is declared as APIResponse<T>, the way the typed routes already did, with the missing shapes named in api/model/interfaces.ts: PublicUser (no hash, no email) and SelfUser (your own profile, with email) plus toPublicUser/toSelfUser in place of ten hand-written destructurings, AuthTokens, AuthResponse, MediaTokenResponse, InviteResponse, AttachmentResponse, SearchResults, LinkPreview, UserSuggestion and Invitee. Request bodies are `unknown` and narrowed by the existing guards, or by a new fieldOf(obj, key) where a route reads one field — no more reading off Oak's `any` or casting to Record<string, unknown>. The password-reset routes answer `{ success: true, data: null }` rather than a bare `{ success: true }`, so every success fits the envelope; the frontend only reads `success`.

Socket protocol: routes/ws.ts had its own send() taking Record<string, unknown>, so nothing checked what it sent. It takes ServerToClientMessage now, which turned up two mismatches: `pong` was missing from the union although the server answers every client ping with one, and vote_ack carried a `success: true` no type declares and no client reads. The union has PongMessage and vote_ack lost the field.

No explicit any. The YouTube playlist parse reads ytInitialData as `unknown` through a small at(node, …path) that checks each step, instead of `type Json = any`. The backend test helpers type the response body as the server's envelope: request<T>() and get/post/put/patch/del<T>() name what `data` holds, res.data throws with the server's message when the call didn't succeed and res.error when it didn't fail, and each test says which API type it expects, so a test reading a field the route doesn't send no longer compiles. Assertions that a field is absent are `assertFalse("passwordHash" in …)` rather than reading a field the type says isn't there. The socket tests' messages are ServerToClientMessage, with next()/all() keyed by type so each returns that message's shape.

No double casts in src/ either. window.YT and onYouTubeIframeAPIReady are declared on Window; the test's fake player is a class forwarding to the spies the tests inspect. WSContext, FollowContext and PlayerContext export their defaults (defaultWSValue, defaultFollowValue, defaultPlayerValue), which the tests import instead of reading React's private _currentValue. handleMentionKeyDown takes a MentionKey — the key fields it reads — so the test builds a plain object; useUserDumpFeed declares sentinelRef as the callback ref it is. In src/test/setup.ts the IntersectionObserver fake implements IntersectionObserver, and report(isIntersecting, top?) builds complete entries in place of partial ones cast in five files; the Animation fake implements Animation; the canvas getContext stub goes in through defineProperty. The XHR fake's last request is optional behind a getter that fails if none was made, the keyframes are checked with matchers, and the roleless legacy session is built by leaving the role out. One test changes meaning slightly: "leaving the view…" relied on entries with no position at all, and now leaves and re-enters at the same height, which is what it describes. The one cast left is in api/test/ws.test.ts: deno.json's "dom" lib hides Deno's WebSocket overload that takes handshake headers, and dropping that lib would break the editor's view of src/; it's commented and names the real signature.

Also: the notifications test answered a Response-typed promise with a plain object, which failed `tsc -b`; it uses the fetch stub's deferred() now. ESLint didn't read the `deno-lint-ignore no-explicit-any` comments, so `eslint .` failed on three files; those anys are gone. `deno fmt` over api/ and src/ (not the compiled src/locales/*.js); the reflow left all 332 message ids intact, and the .po files only have their source line references updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-27 16:05:29 +00:00
khannurien
25601edaf8 v3: the backend gets a test suite, and the bugs it turned up are fixed — deno task test runs 186 tests over api/ in about 18s, against a throwaway database and uploads directory (GERBEUR_DB_PATH and GERBEUR_UPLOADS_DIR now override the defaults; api/test/setup.ts sets them before any app module loads), with requests sent through the whole middleware stack in-process via app.handle, outbound fetches and DNS stubbed, and the socket tests on a real Oak listener since an upgraded request has no response for handle() to return. It covers auth (token purposes, revocation after a password change, live roles, login rate limiting, invites claimed once even under parallel registrations, password resets), dumps end to end (create, privacy, by-url matching on the canonical key, backlinks, categories, edits, file replacement, thumbnails, refresh, delete), file serving headers and ranges, attachments, avatars, comments and likes, playlists, follows and feeds, notifications and mentions, categories, search, chat cursors and read markers, link previews and the SSRF guard on every redirect hop, the Bandcamp and YouTube resolvers, migrations (a fresh schema is left untouched; the data rewrites in 0004/0005 and 0010–0013 do what they say), CORS, OG tags, static caching, and the WebSocket protocol. Fixed along the way: an unknown category id on a new dump returned 400 but kept the dump row, and for a file dump deleted the file while keeping the row; an edit or file replacement with one was half-applied before the 400 — categories are now checked before anything is written. A suffix range (bytes=-3) was read as an end offset and got a 416 instead of the last three bytes. Reordering a playlist accepted a list naming a member twice. verifyPassword threw on a stored hash without a salt instead of answering false. And a request with no Accept-Encoding, which RFC 9110 lets a server treat as accepting anything, was sent the brotli sibling; curl and bots now get the plain file.
All checks were successful
Build and Publish Docker Image / build-and-push (push) Successful in 46s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-27 12:31:05 +00:00