21 Commits

Author	SHA1	Message	Date
khannurien	14e3ab125b	v3: fix the privacy and security findings of the comment pass — private dumps' broadcasts, mentions and attachments reached people who can't see them, and a few smaller holes. A dump's every write now goes through ws-service's broadcastDumpChange(before, after), which decides the audience: a private dump's creation, edits and deletion reach its owner's sockets alone (its deletion id went to everyone before), going private is dump_deleted for everyone else and dump_updated for the owner (whose other tabs dropped it), going public dump_created for everyone else. Mentions in a private dump, a comment on one, or a private playlist's description notify no one until it goes public (mentions are deduplicated per context, so going public notifies them once), and a moderator's edit of a comment, chat message or playlist mentions in the author's name. A link preview of a private playlist is refused, as one of a private dump already was: the preview is stored in the linking dump, which others may see. Attachments record the type of the resource that links them (migration 0018 backfills it); one linked by a private dump or playlist, or a comment on a private dump, is served only with its owner's media token, which Markdown now adds to attachment URLs, and one whose resource is gone answers 404 and is purged at startup. The Bandcamp provider claimed bandcamp.com.evil.example (the pattern wasn't anchored) and now reuses bandcamp-tralbum's anchored one; the YouTube provider took any site's ?list= for a YouTube playlist and missed m., music., youtube-nocookie and /live/ links, and now shares canonical-url's rules. Sign-in does the same scrypt work for an unknown username as for a wrong password, so timing no longer tells which exist. Usernames and emails are unique regardless of case (migration 0017, which refuses to run over existing clashes and names them): "Bob" and "bob" could both sign up and a mention picked one, emails had no constraint, and a reset request in another case silently sent nothing. The Open Graph tags escape the site name and build their URLs on PUBLIC_URL and FRONTEND_URL instead of the request's origin, which a proxy may rewrite. ... Error statuses follow one rule: APIException takes a status and derives the code from it (BAD_REQUEST 400, UNAUTHORIZED 401, FORBIDDEN 403, NOT_FOUND 404, CONFLICT 409, RATE_LIMITED 429, SERVER_ERROR 500, BAD_GATEWAY 502), so a code can't contradict its status. That fixed real bugs: a wrong current password on change-password answered 401, on which authFetch signs the user out; editing or deleting someone else's comment or chat message answered 401 rather than 403; one invalid dump edit answered 422. A taken username is 409, like a taken email. Along the way: the database is created from schema.sql by connection.ts when the file doesn't exist, so every task works on a fresh checkout (dev and serve never ran db/init.ts, which is gone); the startup purge compared ISO dates with SQLite's datetime() as text and kept anything from the current UTC day, and now compares ISO cutoffs; the seeded admin's hash comes from lib/password.ts (split out of jwt.ts, which needs the JWT secret) rather than a copy of its format; avatar, thumbnail and playlist-cover uploads go through replaceUploadedFile, which writes beside the old file and renames over it before the record announces it (a failed avatar write lost the old avatar; a thumbnail was announced before it existed); upload.ts derives its size message from MAX_IMAGE_SIZE_BYTES and stops re-exporting the upload folders; ALLOWED_IMAGE_MIMES, which duplicated detectImageMime, is gone; the SPA fallback no longer calls next() after answering; a reply must answer a comment on the same dump (a foreign or unknown parent was a 500 or a cross-dump thread). 211 backend and 628 frontend tests pass. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-09-29 15:19:07 +00:00
khannurien	0d4e344d77	v3: comments say what the code can't — the documentation pass wanted JSDoc on every export and every field of an exported interface, which filled the code with restatements ("The dump.", "Whether it is loading.", "The props of X.", "Renders the X.") and module docs that told a reader nothing ("The hot and new feeds, and the props every main feed of the index takes."); some were wrong as well as vague. Every comment in api/, shared/ and src/ is rewritten under one rule, now stated in the README's "Comments" section: every module and export still has a JSDoc, and it gives context (who uses it and where), usage (what a caller must know: a precondition, what it leaves to the caller, a side effect) or its place in the chain (where its input comes from, where its output goes); a field, parameter, private function or local gets one only when it adds to its parent's, and context about a whole type goes on the type; a type's doc is phrased from the value's side ("Returned to Y by X when Z", "Passed to Y by X"); full sentences, as short as they can be, no label-colon-list; // goes wherever the code isn't obvious and says why, and the section banners and narration go. Every claim was checked against the code, which turned up some forty comments that were wrong: DumpCard's claimed it was only used by the hot and new feeds, the WebSocket welcome claimed to be a socket's first message (presence comes first), UpdateUserRequest said null removes a description (it keeps it), deletedDumpIds leaves out that it also holds dumps made private, THEME_COLOR claimed to mirror index.html's, and the like. The routers' JSDoc now says where they are mounted and what becomes of a thrown APIException. doc-lint.ts lets a field without a doc through; a module or export without one still fails. No code changes; 204 backend and 628 frontend tests pass, and doc:lint, deno check, deno lint and tsc are clean. ... Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-09-29 14:31:14 +00:00
khannurien	42e0356d21	v3: the docs follow deno doc's grain — the index page was a hand-kept list of some 250 links into deno doc's output layout (./api/main.ts/index.html), policed by mod.test.ts; deno doc has no module index of its own beyond all_symbols.html, which lists every module by path with its summary. mod.ts is now a prose overview of the three parts and their layers, linking that list once, and mod.test.ts is gone. Re-exporting everything from mod.ts, as deno doc's tutorial does for a library, was tried and doesn't fit: export * drops the 18 routers (default exports) and makes 17 names ambiguous, and deno 2.9 renders export * as namespaces with the root's module doc and nested ones empty, losing every module doc. A router's page showed one line: its endpoint list sat on the default symbol, whose summary was cut at the colon introducing it; the list is now the module doc of all 18 routers, and the router's JSDoc says so. 22 summaries ended in a colon (the routers, BandcampPlayer, migration 0016, useDumpListSync, usePlaylistListSync) and read cut short wherever deno doc shows a first paragraph alone; they end in a full stop, doc-lint.ts now fails on a colon summary, and the README states the rule. No behaviour changes; 204 backend tests pass. ... Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-09-28 19:28:24 +00:00
khannurien	d62e4a9883	v3: the backend is documented for deno doc, one way in every file, and the docs build and publish themselves — comments were in three shapes: file headers as // lines, as an untagged /** */ (which deno doc drops, only an @module block counts) or missing; 43 exports described by a // deno doc can't see, two JSDoc blocks sitting on the wrong declaration and one duplicated; endpoints commented in 5 of 18 route files, three ways, on router.get() calls no doc tool reads. The rule, applied to every non-test module of api/, migrations included: each opens with a /** … @module */ block; every export has JSDoc, and so does every field of an exported interface; a comment describing a declaration is /** */, // only notes code; functions open with a verb, everything else with a noun phrase; parameters are named in the prose, {@link} only for symbols in scope; whatever throws to its caller says so with @throws (for an APIException, the statuses and when); a router's JSDoc lists its endpoints, method and path, who may call it, what it does. deno doc --lint goes from 528 errors to none in api/ (shared/ keeps its 385; it is next). The lint also wanted explicit types on config.ts's settings and a few return types, JsonLdResult exported, and the routers typed Router. deno task doc builds the HTML reference into docs/ from every module of api/ and shared/ but the tests, deno task doc:lint lints it; the index page is the module doc of a root mod.ts (deno.json's exports), which introduces the backend, shared/ and the frontend to come and links every module, and mod.test.ts fails when one is missing. The docs workflow builds it on every push to main that touches them and publishes it to the gh-pages branch, like publish-assistant's. Two defects found along the way are fixed: a private playlist's cover was served to anyone who had its id or slug, and now needs its owner's media token, as private dump files do (playlistImageUrl adds it for private playlists only, so public cover URLs stay the same for everyone); og.ts and static.ts each filled index.html's placeholders, and now share lib/index-html.ts, so the OG middleware picks up a rebuilt index.html instead of keeping the first it read. The README states the documentation rule. 205 backend and 628 frontend tests pass. ... Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-09-27 21:07:40 +00:00
khannurien	d392be7cb8	v3: api/ gets one layout rule and keeps to it, and whatever crosses the wire is defined once, in shared/ — api/model/interfaces.ts and src/model.ts declared the same 40 types by hand and had drifted: the error envelope, email required on one side and optional on the other, Invitee/InviteTreeEntry and LinkPreview/UrlPreviewResponse for one shape each. The validation limits, the role permissions, the chat length cap, the @mention pattern and the Bandcamp and YouTube player responses were copied too, behind "keep in sync" comments. shared/ now holds them once, one file per domain (api, category, chat, comment, dump, follow, mention, notification, permissions, player, playlist, search, user, validation, ws), and imports nothing from either half. The server sends Dates; the client reads Wire<T>, the same type with every Date an ISO string, and turns it back with src/utils/shared/deserialize.ts. APIErrorCode becomes a const object, since the frontend's erasableSyntaxOnly rejects an enum. The shared types showed two real gaps: the avatar upload and GET /api/users/me return the full self user and the frontend read them as public users, and a stored session may predate media tokens, so the client keeps it as its own Session type. The backend rule: each folder holds one kind of module, each file is named after its domain. routes/ parse and answer; services/ hold the logic and every SQL query (lib/auth.ts's token check moves to user-service); remote/ is the only place that fetches from other sites — the guarded fetch and the HTML scraping out of rich-content-service, public-address, the Bandcamp and YouTube resolvers, the link providers, the image proxy (fetch, checks and cache, out of routes/preview.ts) and the Twemoji glyphs (out of lib/site-icons.ts). model/ holds per table a row type, its guard and its mappers; validators/ the request guards; db/connection.ts the handle and transaction(), db/startup.ts the purges and default admin that db.ts ran on import, called at the same moment. Tests mirror the source one file per module, test/<layer>/<module>.test.ts, with shared/'s under shared/test/ (deno task test runs both); the test that covered the bandcamp and soundcloud providers together is two now, and the backend can() has its own, so 202 tests. deno lint src/ is clean too: window.* becomes globalThis.*, and test bodies and act() callbacks that never awaited lose their async. The README states the rule. No behaviour changes; the catalogues only move their source references. ... Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-09-27 20:19:48 +00:00
khannurien	de5abeceb2	v3: a performance pass over both halves — indexed queries, streamed files, cached previews, and feeds that stop re-rendering on every tick — a review turned up whole-file reads, unindexed scans and render storms; each fix comes with a test. Database: migration 0016 adds the indexes the hot queries lacked — dumps by created_at, (user_id, created_at) and slug, votes by (user_id, created_at), playlists by slug, users by lower(username) for @mentions, and the child side of every cascading foreign key (comment replies, a user's comments, follows both ways, chat messages, invites, reset tokens); the feeds, dump and playlist pages, upvotes and follower fan-out all plan as index searches now, with no temp-table sort. SQLite runs in WAL with synchronous=NORMAL and a busy timeout, and a transaction() helper wraps the multi-row writes (a dump and its categories, category sets, backlinks, attachment links, playlist reorders, follower notifications) so they commit once instead of once a row. Lists no longer run a COUNT(*) per page: they fetch limit + 1 rows and hasMore comes from the extra one; total stays only on search, the one place it's shown. Files: /api/files read the whole upload (up to 50 MB) for every range request a player makes while seeking; api/lib/serve-file.ts streams the requested bytes from disk, with an ETag and 304s, and serves avatars, attachments, covers and thumbnails too. File and thumbnail URLs carry the file's version (size and last change), so the current one is cached for a year and anything else revalidates — a same-size replacement used to stay stale, and a video's still was cached immutable forever under a URL that never changed. Feeds showed image dumps at full size: /api/thumbnails now makes a 960px preview with ffmpeg (GIFs and unreadable images go out as they are), one run per output however many cards ask at once, and ffmpeg is probed once per process instead of spawned per request. Uploads stream to disk; replacing a file writes beside it and renames over, a hard link keeping the old one for rollback instead of holding both in memory; generated stills and previews go when the file is replaced or the dump deleted, and a custom thumbnail with its dump. Outbound: fetchWithTimeout's deadline now covers the body — a server that answered at once and then dripped could hold a create open indefinitely — and pages are read to 4 MB at most; metadata is cached five minutes so the create form's preview and the post that follows share one fetch (refresh-metadata still goes to the page); proxied images sit in a 32 MB LRU. Broadcasts serialize once rather than per socket, and index.html is kept in memory until the file changes. Frontend: currentTime lived in PlayerContext, so every card re-rendered — and re-parsed its Markdown — four times a second while anything played; the playhead has its own PlayerTimeContext now. The socket context is split three ways (feed, chat via useWSChat, presence via useWSPresence), so typing, presence and chat messages leave the feeds alone. DumpCard, JournalCard and Markdown are memoized; card images use the server preview and load lazily; the feed cache is written when the page is left rather than after every pause in scrolling; the theme context value is memoized. ... Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-09-27 16:35:51 +00:00
khannurien	2ad8ae73d5	v3: the API stops trusting what it's handed — a review of api/ turned up a server anyone could crash, uploads that ran script on the app's origin, and a server that would fetch its own network on request, and this closes them along with the smaller holes around them. A WebSocket message that wasn't an object (a bare null will do) threw inside the listener, which Deno treats as uncaught and exits on; messages are now shape-checked and the handler can't throw out. Uploaded files kept the MIME type the uploader's browser claimed, so an .html or .svg dump opened from /api/files ran with the session token in reach; only images, audio, video, PDF and plain text are served inline now, the rest download, and every file carries nosniff and a sandboxing CSP (PDFs aside, which Chrome won't render sandboxed). The image proxy passed image/svg+xml straight through the same way and now takes raster formats only. Outbound fetches for previews, the proxy and Bandcamp checked the hostname against a partial list — no 169.254.169.254, no 0.0.0.0, an IPv6 check that never matched a bracketed hostname, nothing for names that resolve inward, and redirects followed blindly; every hop now resolves and is refused if any address is private or reserved, redirects are walked by hand, the curl fallback is pinned to the checked IP, and /api/preview needs a login. PATCH /users/me accepted a new password without the current one and now takes only the description and email; a wrong password at login answered 500 because the handler caught its own 401, and now says 401. Changing or resetting a password now ends every older session through users.tokens_valid_after (migration 0015) and an iat on every token, with change-password handing back fresh tokens the modal keeps, and all token checks go through one authenticate() that also refuses deleted users. Invites and reset links are claimed atomically before the async hash rather than marked after, so one invite no longer opens several accounts, and an email address belongs to one account. File and thumbnail URLs carry a media-only token rather than the session token, so a URL in a log exposes files and not the account; older sessions fetch one on load. Private things stop leaking into broadcasts: private playlist updates go to their owner, votes and likes on someone else's private dump 404 and their counts go to the owner alone, and a private dump can't be slipped into a playlist. Smaller: selfProvider only claims this site's own host, so github.com/users/bob stops rendering our bob; the chat read marker can't be parked in the future on an empty chat; dump mutations use the resolved id so a slug works; multipart dumps get the title and comment limits; logins and reset requests are rate-limited through a shared token bucket; the password compare is constant-time; ffmpeg reads only local files in known video containers. ... Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-09-24 20:07:33 +00:00
khannurien	73b80af08e	v3: the chat stops dropping things and holds up to long messages — a message sent while the socket was down vanished with the draft already cleared, and a rejected one fared no better since the client ignored any error that wasn't about a vote or a like. Sends now carry a nonce and sit dimmed in the list until their echo retires them; the server's refusal, an offline socket or ten seconds of silence turns one into a failed row with retry, edit and discard, and a failed row that turns up in the history after a reconnect is retired as having landed after all. Failed edits and deletes show in a bar above the composer. The live stream only goes to signed-in sockets, matching the history behind auth; sends, edits and deletes share a per-user token bucket (typing and read markers get their own, and are just dropped past it); and moderation re-reads the role from the database on every check rather than trusting the one a week-old socket captured. Unread now lives on the server as users.chat_read_at (migration 0014, existing users start caught up), so the count survives a reload and every tab and device clears together; opening onto unread history lands on a "new" divider rather than the bottom, and marks it read only once you've scrolled down to it. After a reconnect the chat re-reads everything from its oldest loaded message and takes that as the truth, edits and deletions included, or reloads the latest page if too much happened. History pages on (created_at, id) so a shared millisecond can't hide a message at a boundary. In the modal: a run of messages from one author shares a header with the time in the gutter on hover, mentions and replies to you get an accent edge, a pill counts what arrived while you were scrolled up, the draft survives closing and reloading, relative times tick, reply previews are flattened from markdown, owners can delete their own messages, ↑ in an empty composer edits your last one, and there's a proper loading and retry state instead of a flash of "No messages yet". Long messages: the composer and edit box grow only so far and then scroll, instead of squeezing the list away; the counter shows only near the limit and sits above the box, since .comment-reply-textarea's padding shorthand had quietly cancelled the room kept for it inside; TextEditor sizes itself before paint, so opening an edit animates to the right height instead of dipping and snapping; and the list follows the bottom through any change in its content's height while pinned there, so ending an edit returns you to the end. A mention notification opens the chat over the page you're on rather than the home page. ... Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-09-22 20:36:49 +00:00
khannurien	fb8364e24d	v3: reworked posting — three panels, drafts, duplicate detection, upload progress ... Posting a dump was a single form where the important choices were the easiest to miss. It is now three panels: link or file, why & where, playlists. Composition: - No more URL/File toggle. An empty panel offers both ways in at once and the kind follows what you actually did; a file dropped anywhere in the modal is accepted, not just on the zone. - Categories and visibility get their own panel instead of a disclosure that read as optional, and the primary button stays "Next" until they've been seen. Visibility carries a real label now. - The draft (link, title, why, categories, visibility) is mirrored to localStorage on every change and restored on reopen, so Escape or a stray backdrop click costs nothing. Only an attached file can't be restored, so that is the one case that asks before closing. - URL dumps can carry a poster-supplied title instead of being stuck with whatever the page scraped, editable right under the preview. - Multipart uploads go through XHR so there is a real progress bar and a percentage on the button, rather than 50 MB of silence. Duplicates: - New dumps.url_canonical column (+ index, backfilled by 0013) holding a lossy key that ignores scheme, www., trailing slashes, tracking parameters and YouTube share shapes. GET /api/dumps/by-url reads it, and the create form warns "already dumped by X" while it fetches the preview. Never blocking. Fixes: - /api/preview now reports whether the page was actually reached: a failed fetch still yields a hostname-only stub, so a dead link and a page without metadata used to render identically. - The Web Share Target never worked. The manifest posts to "/", but the index redirect dropped the query string, so every Android share landed on the feed with nothing pre-filled. - File dumps no longer take the extension into their title. - The link field no longer autofocuses on touch, where it raised a keyboard over the modal. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TiAPtJZeCYYk8rKehtLUQU	2026-09-08 14:40:37 +00:00
khannurien	7303509b6f	v3: fix migration 0011 mistaking real og:images for favicons ... 0011 tested for "favicon" anywhere in the path, so mirtitles.org's declared og:image — /wp-content/uploads/2022/04/mir-logo-favicon.png — was moved into faviconUrl and its thumbnailUrl cleared. Without a thumbnail, hasThumbnail() is false and the journal mosaic demoted the dump from an image card to a pull-quote. A blanket .svg match did the same to real artwork (tanibis.net's header.svg). 0012 judges an icon by its filename rather than the whole path: named favicon…/apple-touch-icon…/icon…, living in an icons/ directory, or ending in .ico. That spares mir-logo-favicon.png and also catches icon_SEARCH.png, which 0011 missed. It runs in both directions, and is deliberately narrow when reversing — a value is only promoted back to thumbnailUrl if it matched 0011's rule but not this one, so rows written by a normal fetch aren't disturbed. On the dev database: 4 thumbnails restored, 1 icon reclassified, real favicons left alone, and a second pass rewrites nothing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-08-11 19:51:49 +00:00
khannurien	1cb904d2cf	v3: generated placeholder thumbnails for pages with no preview image, and a fix for hotlink-protected ones ... Thumbnails were failing in two different ways that both ended as an empty box. Cloudflare hotlink protection answers a cross-site Referer with 403, so images we had extracted correctly (dles.aukspot.com's og:image among them) never rendered — every onError handler set display:none and swallowed it. The new Thumbnail component loads with referrerPolicy="no-referrer", retries once through /api/proxy-image for hosts that reject an empty referrer too, and only then falls back to a placeholder. Separately, the extraction cascade ended at the page's icon and then a guessed /favicon.ico, so thumbnailUrl was almost never empty — just a 16x16 icon cover-cropped into a 128x72 box. It now stops at real artwork, with faviconUrl and accentColor (theme-color / msapplication-TileColor / mask-icon) as their own fields. An absent thumbnailUrl finally means "no artwork", which is what makes the placeholder possible: the site's own color mixed into the theme surface, with its favicon centered on it, or its initial. Contrast holds for any third-party color by construction rather than by luminance math, so nyt only has to set --thumb-tint-strength to 0% to stay monochrome and geocities only has to raise it. Missing accents fall back to a stable hostname-derived hue, so rows saved before this get a tint with no backfill. Migration 0011 reclassifies favicon-shaped thumbnailUrls on existing dumps. The journal mosaic keeps its pull-quote and text fallbacks — the placeholder appears there only to repair a broken image. Also fixed: refresh silently overwrote good metadata with a failure stub, the refresh button swallowed every error, refresh never broadcast the update, extractBestIcon ranked SVG icons below 16x16 PNGs, and shared links with no artwork carried no og:image at all. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-08-11 19:43:55 +00:00
khannurien	0e138be6df	v3: youtube embeds now honour the timestamp in the source url, with a migration backfilling existing dumps ... Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-08-08 11:47:26 +00:00
Vincent Lannurien	d2d086831e	v3: chatbox improvements (typing indicators, answer to previous messages, backlog days separators, scroll to last message position, visual tweaks)	2026-06-30 12:27:03 +02:00
khannurien	124866ad4f	v3: added a chatbox	2026-06-29 20:25:52 +00:00
khannurien	6c4f410d9b	v3: added a chatbox	2026-06-29 20:25:10 +00:00
Vincent Lannurien	f171027673	v3: added customizable site emoji (logo), small visual tweaks	2026-06-29 15:12:25 +02:00
khannurien	fae25f3e6c	v3: added site-wide categories, added admin category management, various visual fixes	2026-06-28 16:23:44 +00:00
khannurien	810faaf21a	v3: added a roles/permissions system, added user management tab on user profile	2026-06-28 06:21:23 +00:00
khannurien	4687a6b515	v3: add backlinks between dumps (via either dump url or dump target url mentioned through dump description and dump comments)	2026-06-21 20:30:03 +00:00
khannurien	cf988ae608	v3: allow users to edit dump thumbnail, fixed some linter errors in the frontend	2026-06-21 15:41:40 +00:00
khannurien	3d03c9e69c	v3: fixed docker image, updated locales	2026-06-21 14:38:17 +00:00