18 Commits

Author	SHA1	Message	Date
khannurien	c3594e4c27	v4: what the server and the client both state is declared once — several rules were written out in two or more files, some already drifting apart. A link's site name (its host without www.) was computed in five places, now shared/url.ts's siteHost; dumpFileVersion, which the frontend and the server had to keep equal by hand, is one function in shared/dump.ts; the Markdown flattening of the chat's reply previews and the digest's quotes is shared/markdown.ts's markdownToPlainText, so the digest also keeps an image's alt text; an attachment's URL and how to find one in a text are shared/attachment.ts, which the editor, Markdown and attachment-service read, its id pattern the new UUID_RE that the file and thumbnail routes check ids with. On the server, the Bandcamp, YouTube playlist and link preview caches use one TtlCache (api/lib/ttl-cache.ts) instead of three copies of the same expiry and eviction; the link previews' theme-color and the emails' theme tokens read colours with one parser (api/lib/css-color.ts), and a transparent theme-color now gives no tint; the WebSocket route sends through ws-service's send, the welcome included, so it is type-checked; fetch.ts strips an IPv6 literal's brackets with public-address's bareHostname; and a category's slug is checked by the same validateSlug as a dump's, with its own length. The migrations keep their frozen copies, as they document. 272 backend tests and 677 frontend tests pass, and deno check, tsc, eslint and doc:lint are clean. ... Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-09 13:16:52 +00:00
khannurien	4468438ba7	v4: every page and API URL follows one of two rules, written down in API.md — pages are a hierarchy where @name is a user and any other root word a category or an app page, tabs and views are plain segments and a default view is the bare path; API resources are /api/<plural>/<key>, a user's key being an id or @name, the caller's things under /api/me, media a sub-resource, lookups by another key a query filter. ~ is gone: the feed is /, /hot, /new, /journal, /follow and /music/hot, a profile /@bob with its overview and the tabs dumps, upvoted, playlists (her own only), follow (users, and /follow/playlists), invitees and manage (role and name, for user:manage), which replace the separate UserDumps, UserUpvoted and UserPlaylists pages; /settings holds the account (now with the username) and /manage the category manager, both linked from the user menu and one's profile. Dumps and playlists live under their owner, /@bob/dumps/<slug> and /@bob/playlists/<slug>, named by slug alone: migration 0021 re-slugs them to the bare slugify(title), unique per owner and kind (the oldest keeps it, clashes get -2), renames the categories whose slug v4 reserves (hot, follow, settings…), and rewrites this site's v3 links in dump notes, comments, chat, and playlist and profile descriptions. A slug changes only when edited, through the edit forms' new link field, which proposes slugify(title) as the title changes; a username changes from /settings or, for user:manage, from /@name/manage. Every slug or name left behind stays an alias of its item or user for good (dump_slug_aliases, playlist_slug_aliases, user_aliases), so old links redirect without chaining and nobody else can take one; slugs change 3 times a day per item, names once in 30 days, moderators exempt. An alias, or a name in another case, answers the item with its current path: the pages put it in their URL (useCanonicalPath, without fetching again), and the OG middleware answers a 301. shared/paths.ts builds and parses every page path, for the client's links, the emails' (digest-service lost its own builders and its id fallbacks, and links /follow and /settings) and the server's matchers (OG tags, self previews, backlinks, self-url); shared/slug.ts holds slugify and the format categories share. Dumps and playlists carry ownerUsername and a required slug, and notifications join their users' current names and their items' paths each time they are sent or read, so renames reach them and a deleted item's has no link. The API was renamed per the plan's table: POST /api/users and /api/sessions, /api/password-resets, /api/me (with settings, password, avatar, media-token, playlists?contains=, follow, feed?from=, notifications), /api/users/:key/{dumps,upvoted,playlists,follow/users,follow/playlists,invitees,avatar} and by path /api/users/:key/{dumps,playlists}/:slug, /api/dumps?url= and ?category=, /api/dumps/:id/{file,thumbnail,metadata}, /api/playlists/:id/cover, /api/remote/{preview,image,bandcamp,youtube} (one router), and /api/unsubscribe, the v3 POST /api/users/unsubscribe staying for the digests already sent. A stored default feed tab "followed" reads as "follow" once. dumps.slug and playlists.slug stay nullable, since SQLite can't add NOT NULL without rebuilding tables others reference; every writer sets them and unique (user_id, slug) indexes hold them. 254 backend tests (new: migration 0021 on a v3 fixture, slug aliases, swaps and rate limits, username renames, aliases and limits, reads by path and their privacy, the OG 301s, both unsubscribe paths, v4-shaped digest links with no ids, notification joins, shared paths and slugs) and 668 frontend tests (new: the profile's tabs and follow views, Settings, Manage, the link field, canonical URLs on the dump, playlist and profile pages, bare default feed paths) pass, and tsc, eslint, deno check, deno lint, deno fmt and doc:lint are clean; the migration was also run on a copy of the development database. ... Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-06 16:08:59 +00:00
khannurien	f3f07ed1e3	v3: every doc opens with the role, then the mechanism, then the details — the docs were written bottom-up: what a thing does first ("Declares the three contexts WSProvider fills…"), details that only make sense once its purpose is known next, and the reason it exists last ("…so that a feed doesn't re-render"), so a reader had nothing to hang the first sentences on. The README's rule caused it, since a module doc "opened with what the module does"; it now opens with the module's role in the app, and the value-side phrasing of a type ("Returned to Y by X") comes once the role is said, with WSContext as the before and after. WSContext.ts is rewritten that way throughout, and so are the docs elsewhere that buried their purpose in a trailing "so that" (the migrations, og, image-proxy, public-address, useFading, useFeedCache, the drafts), opened on a generic verb that hid it ("Holds", "Declares", "Loads", "Defines", "Types": the contexts and providers, the shared types, the models, the hooks), or were label fragments ("The page of a dump…", "A row of tabs."). A function whose verb is its action ("Stores session"), a props or result type, and a protocol message ("Sent by X when Y") already say their role and are left as they are. Only comments changed, and the catalogs only move their references; deno fmt, deno lint and doc:lint are clean. ... Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-09-29 21:42:08 +00:00
khannurien	0d4e344d77	v3: comments say what the code can't — the documentation pass wanted JSDoc on every export and every field of an exported interface, which filled the code with restatements ("The dump.", "Whether it is loading.", "The props of X.", "Renders the X.") and module docs that told a reader nothing ("The hot and new feeds, and the props every main feed of the index takes."); some were wrong as well as vague. Every comment in api/, shared/ and src/ is rewritten under one rule, now stated in the README's "Comments" section: every module and export still has a JSDoc, and it gives context (who uses it and where), usage (what a caller must know: a precondition, what it leaves to the caller, a side effect) or its place in the chain (where its input comes from, where its output goes); a field, parameter, private function or local gets one only when it adds to its parent's, and context about a whole type goes on the type; a type's doc is phrased from the value's side ("Returned to Y by X when Z", "Passed to Y by X"); full sentences, as short as they can be, no label-colon-list; // goes wherever the code isn't obvious and says why, and the section banners and narration go. Every claim was checked against the code, which turned up some forty comments that were wrong: DumpCard's claimed it was only used by the hot and new feeds, the WebSocket welcome claimed to be a socket's first message (presence comes first), UpdateUserRequest said null removes a description (it keeps it), deletedDumpIds leaves out that it also holds dumps made private, THEME_COLOR claimed to mirror index.html's, and the like. The routers' JSDoc now says where they are mounted and what becomes of a thrown APIException. doc-lint.ts lets a field without a doc through; a module or export without one still fails. No code changes; 204 backend and 628 frontend tests pass, and doc:lint, deno check, deno lint and tsc are clean. ... Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-09-29 14:31:14 +00:00
khannurien	d62e4a9883	v3: the backend is documented for deno doc, one way in every file, and the docs build and publish themselves — comments were in three shapes: file headers as // lines, as an untagged /** */ (which deno doc drops, only an @module block counts) or missing; 43 exports described by a // deno doc can't see, two JSDoc blocks sitting on the wrong declaration and one duplicated; endpoints commented in 5 of 18 route files, three ways, on router.get() calls no doc tool reads. The rule, applied to every non-test module of api/, migrations included: each opens with a /** … @module */ block; every export has JSDoc, and so does every field of an exported interface; a comment describing a declaration is /** */, // only notes code; functions open with a verb, everything else with a noun phrase; parameters are named in the prose, {@link} only for symbols in scope; whatever throws to its caller says so with @throws (for an APIException, the statuses and when); a router's JSDoc lists its endpoints, method and path, who may call it, what it does. deno doc --lint goes from 528 errors to none in api/ (shared/ keeps its 385; it is next). The lint also wanted explicit types on config.ts's settings and a few return types, JsonLdResult exported, and the routers typed Router. deno task doc builds the HTML reference into docs/ from every module of api/ and shared/ but the tests, deno task doc:lint lints it; the index page is the module doc of a root mod.ts (deno.json's exports), which introduces the backend, shared/ and the frontend to come and links every module, and mod.test.ts fails when one is missing. The docs workflow builds it on every push to main that touches them and publishes it to the gh-pages branch, like publish-assistant's. Two defects found along the way are fixed: a private playlist's cover was served to anyone who had its id or slug, and now needs its owner's media token, as private dump files do (playlistImageUrl adds it for private playlists only, so public cover URLs stay the same for everyone); og.ts and static.ts each filled index.html's placeholders, and now share lib/index-html.ts, so the OG middleware picks up a rebuilt index.html instead of keeping the first it read. The README states the documentation rule. 205 backend and 628 frontend tests pass. ... Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-09-27 21:07:40 +00:00
khannurien	d392be7cb8	v3: api/ gets one layout rule and keeps to it, and whatever crosses the wire is defined once, in shared/ — api/model/interfaces.ts and src/model.ts declared the same 40 types by hand and had drifted: the error envelope, email required on one side and optional on the other, Invitee/InviteTreeEntry and LinkPreview/UrlPreviewResponse for one shape each. The validation limits, the role permissions, the chat length cap, the @mention pattern and the Bandcamp and YouTube player responses were copied too, behind "keep in sync" comments. shared/ now holds them once, one file per domain (api, category, chat, comment, dump, follow, mention, notification, permissions, player, playlist, search, user, validation, ws), and imports nothing from either half. The server sends Dates; the client reads Wire<T>, the same type with every Date an ISO string, and turns it back with src/utils/shared/deserialize.ts. APIErrorCode becomes a const object, since the frontend's erasableSyntaxOnly rejects an enum. The shared types showed two real gaps: the avatar upload and GET /api/users/me return the full self user and the frontend read them as public users, and a stored session may predate media tokens, so the client keeps it as its own Session type. The backend rule: each folder holds one kind of module, each file is named after its domain. routes/ parse and answer; services/ hold the logic and every SQL query (lib/auth.ts's token check moves to user-service); remote/ is the only place that fetches from other sites — the guarded fetch and the HTML scraping out of rich-content-service, public-address, the Bandcamp and YouTube resolvers, the link providers, the image proxy (fetch, checks and cache, out of routes/preview.ts) and the Twemoji glyphs (out of lib/site-icons.ts). model/ holds per table a row type, its guard and its mappers; validators/ the request guards; db/connection.ts the handle and transaction(), db/startup.ts the purges and default admin that db.ts ran on import, called at the same moment. Tests mirror the source one file per module, test/<layer>/<module>.test.ts, with shared/'s under shared/test/ (deno task test runs both); the test that covered the bandcamp and soundcloud providers together is two now, and the backend can() has its own, so 202 tests. deno lint src/ is clean too: window.* becomes globalThis.*, and test bodies and act() callbacks that never awaited lose their async. The README states the rule. No behaviour changes; the catalogues only move their source references. ... Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-09-27 20:19:48 +00:00
khannurien	de5abeceb2	v3: a performance pass over both halves — indexed queries, streamed files, cached previews, and feeds that stop re-rendering on every tick — a review turned up whole-file reads, unindexed scans and render storms; each fix comes with a test. Database: migration 0016 adds the indexes the hot queries lacked — dumps by created_at, (user_id, created_at) and slug, votes by (user_id, created_at), playlists by slug, users by lower(username) for @mentions, and the child side of every cascading foreign key (comment replies, a user's comments, follows both ways, chat messages, invites, reset tokens); the feeds, dump and playlist pages, upvotes and follower fan-out all plan as index searches now, with no temp-table sort. SQLite runs in WAL with synchronous=NORMAL and a busy timeout, and a transaction() helper wraps the multi-row writes (a dump and its categories, category sets, backlinks, attachment links, playlist reorders, follower notifications) so they commit once instead of once a row. Lists no longer run a COUNT(*) per page: they fetch limit + 1 rows and hasMore comes from the extra one; total stays only on search, the one place it's shown. Files: /api/files read the whole upload (up to 50 MB) for every range request a player makes while seeking; api/lib/serve-file.ts streams the requested bytes from disk, with an ETag and 304s, and serves avatars, attachments, covers and thumbnails too. File and thumbnail URLs carry the file's version (size and last change), so the current one is cached for a year and anything else revalidates — a same-size replacement used to stay stale, and a video's still was cached immutable forever under a URL that never changed. Feeds showed image dumps at full size: /api/thumbnails now makes a 960px preview with ffmpeg (GIFs and unreadable images go out as they are), one run per output however many cards ask at once, and ffmpeg is probed once per process instead of spawned per request. Uploads stream to disk; replacing a file writes beside it and renames over, a hard link keeping the old one for rollback instead of holding both in memory; generated stills and previews go when the file is replaced or the dump deleted, and a custom thumbnail with its dump. Outbound: fetchWithTimeout's deadline now covers the body — a server that answered at once and then dripped could hold a create open indefinitely — and pages are read to 4 MB at most; metadata is cached five minutes so the create form's preview and the post that follows share one fetch (refresh-metadata still goes to the page); proxied images sit in a 32 MB LRU. Broadcasts serialize once rather than per socket, and index.html is kept in memory until the file changes. Frontend: currentTime lived in PlayerContext, so every card re-rendered — and re-parsed its Markdown — four times a second while anything played; the playhead has its own PlayerTimeContext now. The socket context is split three ways (feed, chat via useWSChat, presence via useWSPresence), so typing, presence and chat messages leave the feeds alone. DumpCard, JournalCard and Markdown are memoized; card images use the server preview and load lazily; the feed cache is written when the page is left rather than after every pause in scrolling; the theme context value is memoized. ... Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-09-27 16:35:51 +00:00
khannurien	2ad8ae73d5	v3: the API stops trusting what it's handed — a review of api/ turned up a server anyone could crash, uploads that ran script on the app's origin, and a server that would fetch its own network on request, and this closes them along with the smaller holes around them. A WebSocket message that wasn't an object (a bare null will do) threw inside the listener, which Deno treats as uncaught and exits on; messages are now shape-checked and the handler can't throw out. Uploaded files kept the MIME type the uploader's browser claimed, so an .html or .svg dump opened from /api/files ran with the session token in reach; only images, audio, video, PDF and plain text are served inline now, the rest download, and every file carries nosniff and a sandboxing CSP (PDFs aside, which Chrome won't render sandboxed). The image proxy passed image/svg+xml straight through the same way and now takes raster formats only. Outbound fetches for previews, the proxy and Bandcamp checked the hostname against a partial list — no 169.254.169.254, no 0.0.0.0, an IPv6 check that never matched a bracketed hostname, nothing for names that resolve inward, and redirects followed blindly; every hop now resolves and is refused if any address is private or reserved, redirects are walked by hand, the curl fallback is pinned to the checked IP, and /api/preview needs a login. PATCH /users/me accepted a new password without the current one and now takes only the description and email; a wrong password at login answered 500 because the handler caught its own 401, and now says 401. Changing or resetting a password now ends every older session through users.tokens_valid_after (migration 0015) and an iat on every token, with change-password handing back fresh tokens the modal keeps, and all token checks go through one authenticate() that also refuses deleted users. Invites and reset links are claimed atomically before the async hash rather than marked after, so one invite no longer opens several accounts, and an email address belongs to one account. File and thumbnail URLs carry a media-only token rather than the session token, so a URL in a log exposes files and not the account; older sessions fetch one on load. Private things stop leaking into broadcasts: private playlist updates go to their owner, votes and likes on someone else's private dump 404 and their counts go to the owner alone, and a private dump can't be slipped into a playlist. Smaller: selfProvider only claims this site's own host, so github.com/users/bob stops rendering our bob; the chat read marker can't be parked in the future on an empty chat; dump mutations use the resolved id so a slug works; multipart dumps get the title and comment limits; logins and reset requests are rate-limited through a shared token bucket; the password compare is constant-time; ffmpeg reads only local files in known video containers. ... Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-09-24 20:07:33 +00:00
khannurien	1cb904d2cf	v3: generated placeholder thumbnails for pages with no preview image, and a fix for hotlink-protected ones ... Thumbnails were failing in two different ways that both ended as an empty box. Cloudflare hotlink protection answers a cross-site Referer with 403, so images we had extracted correctly (dles.aukspot.com's og:image among them) never rendered — every onError handler set display:none and swallowed it. The new Thumbnail component loads with referrerPolicy="no-referrer", retries once through /api/proxy-image for hosts that reject an empty referrer too, and only then falls back to a placeholder. Separately, the extraction cascade ended at the page's icon and then a guessed /favicon.ico, so thumbnailUrl was almost never empty — just a 16x16 icon cover-cropped into a 128x72 box. It now stops at real artwork, with faviconUrl and accentColor (theme-color / msapplication-TileColor / mask-icon) as their own fields. An absent thumbnailUrl finally means "no artwork", which is what makes the placeholder possible: the site's own color mixed into the theme surface, with its favicon centered on it, or its initial. Contrast holds for any third-party color by construction rather than by luminance math, so nyt only has to set --thumb-tint-strength to 0% to stay monochrome and geocities only has to raise it. Missing accents fall back to a stable hostname-derived hue, so rows saved before this get a tint with no backfill. Migration 0011 reclassifies favicon-shaped thumbnailUrls on existing dumps. The journal mosaic keeps its pull-quote and text fallbacks — the placeholder appears there only to repair a broken image. Also fixed: refresh silently overwrote good metadata with a failure stub, the refresh button swallowed every error, refresh never broadcast the update, extractBestIcon ranked SVG icons below 16x16 PNGs, and shared links with no artwork carried no og:image at all. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-08-11 19:43:55 +00:00
khannurien	88f1eb3d03	v3: rich content meta regex improvements	2026-06-27 14:34:47 +00:00
khannurien	26f5abfa4e	v3: fixed linter error, code format	2026-06-22 17:31:21 +00:00
khannurien	473f1046f3	v3: fix to title extraction regex	2026-06-21 20:40:10 +00:00
khannurien	34933a3d4f	v3: fixed rich content extraction heuristics	2026-04-11 13:13:43 +00:00
khannurien	a69788c15b	v3: localization fixes, char counters & limits on all text fields, ux fixes	2026-04-03 19:47:37 +00:00
khannurien	ca70bdc14b	v3: added opengraph support to the app, wrote README instructions incl. a Docker image	2026-03-26 19:55:48 +00:00
khannurien	fbbbb43258	v3: code quality pass, various bug fixes	2026-03-23 07:47:49 +00:00
khannurien	867e64cb5b	v1 review pass: fixed some minor bugs	2026-03-16 11:08:39 +00:00
khannurien	e88fed4e98	vibe coded v1	2026-03-16 07:34:49 +00:00