vincent/gerbeur
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: vincent:5fd1d364da35699fb236174b3a595d0335a1c3b7
Branches Tags
vincent:main vincent:gh-pages
...
compare: vincent:main
Branches Tags
vincent:gh-pages vincent:main

107 Commits
5fd1d364da ... main

Author SHA1 Message Date
khannurien
eec73e9749 v4: a comment and a chat message are edited in the same field as every text edited in place — each had an editor of its own: a comment's was its reply form, which showed a refused edit as a card and left the focus nowhere once closed, and a chat message's a box of its own with its own buttons and rules. InlineEdit gives its field, InlineEditField, to the components that open it with an action of their own, as these two do with their Edit and ✎: they keep those, a click on someone's message being no request to edit it. The field opens on the text with the caret at its end and as tall as the text, saves on Ctrl+Enter (and on Enter in the chat, where Shift+Enter breaks a line, as in its composer), leaves on Escape, sends nothing for a text left as it was, refuses an empty one, says why a save was refused in one line, and gives the focus back to the button that opened it, unless the caller has put it elsewhere, as the chat does to its composer. CommentForm only posts a comment or a reply now; ChatEditBox keeps only what is the chat's, scrolling itself into the list's view.
All checks were successful
Build and Publish Docker Image / build-and-push (push) Successful in 3m20s
Details
Build and Deploy Static Documentation Site / build-and-push (push) Successful in 44s
Details 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-20 18:16:43 +00:00
khannurien
3e03626cc1 v4: text is edited where it stands through one component, InlineEdit, and a dump's note and a playlist's title and description join its title, a profile's description and email — those three were written apart, each with its own state, keys, buttons, error and CSS, and the note and the playlist could only be changed in their full editors. InlineEdit (components/form) shows the text; a click on it, or the ✎ after it, opens a field holding it, with Save and Cancel under it. A one-line field takes the text's own type in every style, from the class it is handed (dump-title, playlist-detail-title), and sits on the faint plate the text showed under the pointer, so that nothing moves but the buttons that come; a Markdown text opens the editor comments use. Enter saves a line, Ctrl+Enter a Markdown text, Escape backs out, a text left as it was closes without a request, a required one can't be emptied, a refused save says why in one line beside the buttons, and the focus goes back to the ✎. A click on a link in the text, or one that ends a selection, follows the link or keeps the selection. The ✎ is now a button with the action's name, where a div pretended to be one, and the text keeps its own role, a heading or Markdown. The plate is drawn under the text by a pseudo-element, tinted from its colour: the surface colour it had was the card's own, and showed nothing on the dump page. On a playlist's page the title edits in place beside Follow and Edit, which go to the row's end, and the description too; an empty note or description says what goes there to whoever may write it. DescriptionEditor and EmailEditor go. 
The GeoCities marquee stops only for a link the keyboard focused: a link pressed with the pointer took the focus as well, which put the line back at its start and scrolled it, so the link slipped from under the pointer before the button came up and the click was lost. The next page's marquee also carries on from the very point the last one had reached, with the loop it showed: it worked its place out from when the loop began, which ran on while the crawl was paused under the pointer, and showed the feed as it was by then, of another width.

The chat's message field rests on one line, as tall as the Send button: it rested on two, 54 to 62 pixels against the button's 38, and the button sat low beside it. Both take --chat-composer-height.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-20 18:08:53 +00:00
khannurien
b92c674726 v4: the devcontainer has Playwright's browsers — they were installed by hand in the home directory, which the rebuild onto Node 26 left without them, and the screenshots of the dev instances stopped working. The Dockerfile installs Firefox and the Chromium headless shell of npm:playwright@1.63.0, with the system libraries they need, in /opt/ms-playwright (PLAYWRIGHT_BROWSERS_PATH), which the workspace and the instances' services share as part of the image. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-20 18:08:53 +00:00
khannurien
7d0a0e7534 v4: the devcontainer runs Deno 2.9.7 on Node 26 — its image took Deno 2.9.0 and the Node 20 base. The Dockerfile now copies the deno binary from denoland/deno:bin-2.9.7 onto mcr.microsoft.com/devcontainers/typescript-node:26.
All checks were successful
Build and Publish Docker Image / build-and-push (push) Successful in 54s
Details 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-13 13:38:16 +00:00
khannurien
51439f475d v4: the GeoCities marquee scrolls the community feed, what the site's members are doing — it crawled three canned lines, a welcome, "best viewed in Netscape Navigator" and "under construction", which said nothing of the site. The feed is a list any style may show (CommunityFeed.tsx), which GeoCities lays out along its marquee: who is online, the week's dumps, upvotes, comments and chat messages, linking to /community, its most upvoted dump, then the 12 latest things done in public, newest first, each ending on how long ago it was: a public dump posted, a public playlist created, a comment or an upvote on a public dump, someone joining and who invited them. Every name and title links to its page. A dump has one comment line and one upvote line at most, its latest, so that a thread or a popular dump doesn't fill the feed. GET /api/community/feed serves it, signed in only (community-service.ts): the events are read from the tables on each request, the week's figures are those /community already keeps, as of its last refresh. A signed-out visitor has no marquee. CommunityFeedProvider holds the feed around every page, so that it outlives the header each page renders anew, fetches it only while a component shows it, and fetches it again two seconds after the socket last told of a dump, a comment, a vote, a playlist, a rename, a deletion or a reconnection: the socket's messages don't carry a feed's lines, and the server's query stays the one place that decides what the feed holds. Someone joining comes with no message, and shows at the next of those. The crawl goes at 50 pixels a second however much the feed holds, Marquee.tsx measuring one copy for the stylesheet to work the loop's length out from, where a fixed 36 seconds would have sped up with every line. A loop shows what the feed held when it began: text that changed mid-crawl would jump, so what came meanwhile waits for the next loop, which also brings the times up to date. The next page's marquee carries on from where the last one's had got to, rather than from the start. The marquee was hidden from assistive technology as decoration; its first copy is now a labelled list of links, the second, which the seamless loop needs, hidden and out of the tab order. A link that takes the focus stops the crawl and the line scrolls to it, the pointer pauses it as before, and with reduced motion the line shows once, still, follows the feed as it changes and scrolls by hand. The sentences a notification also says, "posted", "upvoted", "commented on", share its translations. The tests' DOM gets an AnimationEvent before React loads (src/test/before-react.ts): finding none, React listened for the animation events under their WebKit names, and a test's "animationiteration" reached no handler.
All checks were successful
Build and Publish Docker Image / build-and-push (push) Successful in 51s
Details
Build and Deploy Static Documentation Site / build-and-push (push) Successful in 26s
Details 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-13 07:38:44 +00:00
khannurien
62e51f1622 v4: the six styles are reworked, each around one idea of its own, and the tab row stops painting the page — a pass over every theme, then the faults it showed in what they share. Brutalist draws every box in ink with a solid offset shadow and sets the palette's colors as flat blocks (a teal tag, an apricot quote, an inked label); its dark scheme keeps that ink darker than the page, on Twilight Indigo, where eggshell lines glared across every frame. GeoCities goes to the desktop of its day: the header is the taskbar, the site's name its Start button, the controls grey bevels, and a marquee crawls under the header (Marquee.tsx, drawn in that style alone), with Comic Neue where the system has no Comic Sans. Melange leaves its browns and Fraunces for a sorbet blend in Bricolage Grotesque, run along the header, through the site's name, where it drifts, and between a journal's blocks; its two schemes are one print and its negative, trading ink and paper while the pink and the plum written on it stay, and they share one fine grain where the light one had a coarse tile that greyed the page. Kanagawa opens the comments and the related dumps on a status line and reads on Lotus's brightest paper; its page is flat. Times draws its heavy rules in a mid grey at night, where ink is near white, and sets a journal cell with no picture as text. In what the styles share: a row of tabs (TabBar) hid the tabs scrolled under its scope switcher, and faded its ends, by painting the page's color over them, which showed as a patch on any page that is a gradient or a picture. The switcher now sits before the row that scrolls, which clips the tabs at its edge, and the fades are a mask (--tab-fade-start and --tab-fade-end); useRoomBeside measures the row through that one more level. A page's title keeps a word whole unless it alone is wider than the line: Firefox sizes a title set with negative letter-spacing a hair narrower than it lays it out, which dropped its last letter to a second line in any style that tightened it, so the rule is the base's and the two styles' copies go. A card holding one paragraph, such as the instances a new account hasn't signed in to on /settings, is spaced by its padding and not by the paragraph's margins as well. The site's name answers the pointer as a card's title does in each style, where two of them left it still. The emails follow the styles' tokens, Melange's blend among them.
All checks were successful
Build and Publish Docker Image / build-and-push (push) Successful in 55s
Details
Build and Deploy Static Documentation Site / build-and-push (push) Successful in 26s
Details 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 19:30:57 +00:00
khannurien
eb4e64683c v4: the Times nameplate sits level with the header's buttons — the name was 7 pixels under their middle on a wide screen and 3 on a narrow one, with twice the paper above its letters as below. The blackletter sets its letters low in its line, an ascender stopping a fifth of an em under the line's top and a descender ending on its bottom, and the nameplate's padding was the heavier above besides. The name now carries 0.22em under its line, which leaves as much below the letters as above at either size, and the nameplate's padding is the same on both sides, 0.6rem and 0.7rem from 860 pixels up. The wide masthead's row is a pixel taller, the narrow header keeps its 60.
All checks were successful
Build and Publish Docker Image / build-and-push (push) Successful in 53s
Details
Build and Deploy Static Documentation Site / build-and-push (push) Successful in 26s
Details 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 16:30:02 +00:00
khannurien
d54ae554a4 v4: the Times style gets a masthead and a face for each job — one condensed serif did everything, buttons and tabs included, under a plain header bar. The site's name is now a blackletter nameplate (Manufacturing Consent), its emoji beside it on the same baseline at the size of a printer's mark. From 860 pixels up it takes the middle of a row of its own, the buttons to its right, and what a page puts in the header, a feed's tabs or the open search field, runs underneath between a hairline and a double rule; a page that puts nothing there keeps one row and a single ink rule. Instrument Serif stays for headlines, at its one weight and larger sizes, Newsreader sets what is read at length (summaries, notes, comments), and Libre Franklin everything the interface says: controls, tabs, datelines, bylines, labels (--font-ui, listed in themes/nyt.css). Tabs are sentence case with an ink underline on the open one. The comments, the related dumps, a profile's sections and a page's groups open on an ink rule with their label in bold capitals under it. A dump's categories are small capital kickers, where they were grey chips. Buttons and fields take a 3px radius, pictures, cards and sheets staying square. The header sits on the page's color in the dark scheme too. In the journal each cell draws its four rules as a 1px shadow in a 1px gap, where it drew its top and left edges: a last row the dumps don't fill was left open on its right, and the row above it unruled over the empty tracks. A page's title keeps a word whole unless it is wider than the line. deno.lock also takes the key format of the Deno that wrote it, with no version changed beyond the three font packages.
All checks were successful
Build and Publish Docker Image / build-and-push (push) Successful in 53s
Details
Build and Deploy Static Documentation Site / build-and-push (push) Successful in 26s
Details 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-12 09:26:09 +00:00
khannurien
3e15e9a4f2 v4: /follow/playlists opens the followed playlists' feed, and a page's test opens it through the app's routes — the path was a 404 since the two views of the followed feed became pages: the route table held each feed tab as an exact path, /follow, so /follow/playlists fell to /:handle/*, which read "follow" as a category and "playlists" as a tab it hasn't. /music/follow/playlists, which goes through that route, worked. The tabs' routes now take what follows them (/${tab}/*), which Feed already reads and refuses when it names no view.
All checks were successful
Build and Publish Docker Image / build-and-push (push) Successful in 57s
Details
Build and Deploy Static Documentation Site / build-and-push (push) Successful in 28s
Details 
No test saw it because each page's test mounted the page on a route pattern of its own, a copy of App's, with stand-in pages where a redirect lands: Index's parsed the path itself, and passed on one the app doesn't serve. renderPage (src/test/render.tsx) now opens a URL through AppRoutes, which App.tsx exports for it, inside the real providers, and every page's test uses it: the page is the one the app routes that URL to, behind its guard, and a redirect is checked by the path it lands on. An entry may be a location, to arrive with the state a link hands over. Every page being loaded with its route, the first query after opening one waits. src/test/App.test.tsx tests the table itself: the paths that are a feed, those that are not found, those that send a guest to sign in and back, and those that send a signed-in user home.

The profile's test held /@ann/dumps/x to be a path the profile answers as not found, which only its own pattern made true: in the app it is a dump's page. It now asks for /@ann/dumps/x/y. The tests of the two guards still mount them around a stand-in child, being tests of the guard alone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-10 19:59:52 +00:00
khannurien
cbb770bff8 v4: a still is scaled from its first frame alone, a second image in its file made the copy flash — an avatar came out as a WebP of two frames looping at 40 ms each, the picture then a blank. A still's file may hold more than one image, a phone's JPEG carrying a gain map or a depth map after the photo, and ffmpeg reads each as a frame: only the JPEG and PNG copies were held to one, the WebP keeping them all as it does a GIF's. scalableImage (lib/upload.ts) now says whether the image is an animation, which a GIF alone is, and scaleImage (lib/ffmpeg.ts) writes one frame of anything else, in every format; the image proxy scales through the same call. The copies already made stay, a copy being current while it has its upload's date: uploads/scaled is to be deleted once on an instance that ran the previous version, everything in it being made again on first request.
All checks were successful
Build and Publish Docker Image / build-and-push (push) Successful in 52s
Details
Build and Deploy Static Documentation Site / build-and-push (push) Successful in 26s
Details 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-10 19:40:14 +00:00
khannurien
d2b7d21824 v4: the page a visit lands on asks for its content while its code loads — a first load went one step after the other: the entry script, then the interface's messages, then the page's chunk and its twenty-odd dependencies, and only then the request for the feed. The built page now carries a script in its head that asks for the messages' chunk at once, beside the entry, picking the language as detectLocale does (preloadCatalog in vite.config.ts; the two must agree). And main.tsx sends the landing page's first request as the app boots (utils/shared/earlyFetch.ts): the feed's first page on / and its tabs, the dump on a dump's page, as the stored session. authFetch hands that response to the page that asks for the same URL, once, to the session it was sent as and within ten seconds, in place of a request of its own; past that the page asks again. feedApiUrl (utils/shared/urls.ts) is the feed's URL for both. A category's feed, a profile, a playlist and the other pages wait for their code as before. On a local build with 150 ms of latency added, the feed's request leaves at 0.69 s where it left at 1.70 s, and the first card shows at 1.59 s where it showed at 1.92 s.
All checks were successful
Build and Publish Docker Image / build-and-push (push) Successful in 51s
Details
Build and Deploy Static Documentation Site / build-and-push (push) Successful in 25s
Details 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-10 19:01:36 +00:00
khannurien
90fc281e90 v4: every image the API serves comes scaled to the size it is shown at — a first visit to the feed loaded 4.8 MB of images out of 5.1 MB in all: an avatar shown at 32 pixels was the upload itself, a 1.4 MB GIF among them, and so were a cover, a custom thumbnail, an attachment and a proxied image, only an image dump having a preview, 960 pixels wide. Every route that answers an image now takes ?w= and sends a WebP copy at most that wide, of the widths shared/image.ts lists, 128, 320, 640 and 960, a width between two taking the one above, and never wider than the image; without it the upload goes out as before, a dump's thumbnail alone being scaled to the widest by default, the custom ones included. serveImage (lib/upload.ts) is where they all end, and the image proxy scales what it fetched the same way, keeping each width in its cache. lib/ffmpeg.ts makes the copies as it made the previews, which they replace: on first request, under uploads/scaled/ at the upload's own path, each given its upload's date and made again when the two differ, so a new avatar or cover needs nobody to delete the old ones. WebP keeps a PNG's transparency and a GIF's animation. A JPEG is turned by its EXIF orientation here, ffmpeg's own rotation of images depending on its version. An animated WebP, which ffmpeg can't read, and an image it fails on go out as they are, a failure being remembered until the upload changes. ?f=compat asks for the copy as a JPEG, or a PNG for what may be transparent: the digest's emails take their thumbnails that way, at 640 pixels, not every mail client reading WebP. On the client, imageProps (utils/shared/urls.ts) gives an image its srcset of those copies and the width it is laid out at, which config/imageSizes.ts states for each place from App.css: Avatar, Thumbnail, which scales an API image and leaves another site's as that site sends it, and the playlists' covers use it. The images of notes and comments still load as uploaded. The copies go with the dump, the custom thumbnail or the attachment they were made from, and generatedMediaSweepJob, a minute after the start then daily, takes those whose upload is gone, the previews under their former name and the temporary files a generation left unfinished. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-10 19:01:36 +00:00
khannurien
6ab398e687 v4: the API's JSON and the page are sent compressed — a page of the feed went out as it was, twenty kilobytes where a fifth does: only the built frontend's files were compressed, at build time, and Deno's server leaves alone a response whose length Oak has set. compressMiddleware (api/middleware/compress.ts), installed after the error middleware and around everything that sets a body, sends an object or array, and a string whose type its route set, in brotli or gzip as the request's Accept-Encoding takes, a coding refused with q=0 not being taken, from a kilobyte up, the threshold the build compresses from. Brotli is at quality 5, the frontend's files, compressed once, staying at 11. Files, streams and bytes, a response already encoded, a HEAD and one marked no-transform go out as they are, and a compressible answer says Vary: Accept-Encoding whether compressed or not. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-10 19:01:06 +00:00
khannurien
1bd5c924b0 v4: Dark Reader is kept off the pages, which lost every background under it — on a phone in dark mode with the extension on, the avatar menu, the cards and the tabs were transparent over Dark Reader's own page colour, in every style. The build gives each light-dark() a fallback for the browsers without it, two custom properties of which one is empty, and Dark Reader's rewrite of the stylesheet drops the empty one under a dark system theme: both halves of every colour then apply at once, which is no colour, so nothing paints a background or a border. index.html now carries <meta name="darkreader-lock">, which the extension obeys by generating nothing; the site has dark palettes of its own, following the system or the user's choice. The dev server never showed it, serving light-dark() as written. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-10 09:26:21 +00:00
khannurien
de223baaa4 v4: the playlists page is reachable at every width, and lists the user's own — below 1150px the header's text link was hidden and nothing replaced it: the avatar menu's "My playlists" led to the user's own, and a guest has no menu. The menu's entry is now "Playlists", to /playlists, at every width, and the page gains a tab after its orders, "Mine", for a signed-in user: their own playlists, private ones too, from GET /api/users/:id/playlists in the default order, narrowed by the category menu and kept live as the others are. It is /playlists?mine=1 (PLAYLISTS_TABS, parsePlaylistsTab, shared/paths.ts), a tab taking the order's place; a guest isn't offered it, and its link sends one to sign in. PlaylistSortTabs takes the tabs it shows, a profile's being the orders alone. The text link shows to a guest at every width, their header giving up its margins under 400px rather than the link, and to a signed-in user where the row has room beside the icons, as measured in every style and both languages: from 560px up to 860px, where the index page's tabs move into the header, and from 1150px as before. A guest in French in the kanagawa style still overflows the header under 340px, by up to 19px.
All checks were successful
Build and Publish Docker Image / build-and-push (push) Successful in 51s
Details
Build and Deploy Static Documentation Site / build-and-push (push) Successful in 26s
Details 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-10 09:14:55 +00:00
khannurien
561cdf98b0 v4: playlists sort by their last dump, their creation or their size, and /playlists is a grid of covers — every list of playlists came newest first, so one fed every week sank under those made since and left empty. GET /api/playlists and /api/users/:key/playlists take ?sort=updated|new|dumps (PLAYLIST_SORTS, shared/playlist.ts), updated being the default: the last to take a dump first, or its creation while it holds none; a new title, cover or description doesn't bring it back up. A playlist carries that date as lastDumpAt, read with its dumpCount by isMemberVisibleTo's rule (VISIBLE_DUMP_FIGURES, the viewer's id as ?1), so the owner's private dumps date and count their playlists for them alone, in the lists and over the socket: playlist_created, playlist_updated and playlist_dumps_updated carry each socket's own. The /playlists page and a profile's playlists tab show the orders as tabs beside the category menu and keep the one picked in ?sort=, the default being the bare path; a tab takes the current history entry's place, and picking a category keeps the order. Live, usePlaylistListSync takes the list's order: a new playlist goes where it puts it, one that rises moves up to its place, as in the updated order when it takes a dump, and one that falls stays until the list is fetched again, its place being perhaps on a page not loaded. The lists in an order of their own, the followed playlists and the search's, pass none, and take no new playlist as before (noNewEntries goes). comparePlaylists is the order on the client, which a test holds to the API's. 
/playlists lays its cards out as tiles, the cover across the top, four to a row and two on a phone, on the wide column: the same PlaylistCard, laid out by its list's class (.playlist-grid), so each theme's card skin carries over. It drops the "public" badge there, every playlist listed being public. On every card, a playlist without a cover shows a tile of its own colour with its initial (ThumbnailPlaceholder) in place of the 📋, and the date is "updated …" of its last dump, or "created …".

A middle, ctrl, cmd or shift click anywhere on a playlist card opens it in a new tab — only its title was a link, the rest navigating by a click handler. useCardLink (hooks/shared) is useDumpCardLink's handlers for any card, which the dump cards now get through it.

A profile tab's query stays with the tab — going from the playlists to the dumps kept the whole query, so a playlist category, and now ?sort=, landed on the dumps' URL. Only the dump lists pass their category to each other; any other change of tab drops the query.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-10 08:51:37 +00:00
khannurien
350e945eb1 v4: a new link between two dumps shows among their related dumps at once — it showed on the next load of the page: the backlink service recorded the link without telling anyone, and the dump page fetched its related dumps once, the socket then only editing or removing the ones it had. relinkBacklinks now sends dump_related_changed when a note or a comment gains or loses a link, naming the dump and those at the other end, and the page showing one of them fetches /related again. A link shows to those who may see both its ends, so each socket is told only of the dumps its user may see, and not at all without one at each end. A private dump that turns public sends it too, for the links it already had, which are new to everyone but its poster. The page also fetches its related dumps again on a reconnection, as it does its comments and playlists. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-10 08:43:55 +00:00
khannurien
dde0232acb v4: a link from one dump's page to another's opens that dump — it reloaded the one on screen: the page stays mounted when its path changes, and for one render the old dump was still loaded under the new path, so useCanonicalPath took that path for a former one and put the old dump's back, which was then fetched again. A related dump's link did it, and so did any dump link in a note or a comment. The page now drops the dump on screen in the render that sees a path naming another, as usePlaylist and useProfile do. The dump handed over by DumpEdit or a title edit goes with it, so that coming back to its path fetches it instead of matching that snapshot and loading forever. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-10 08:34:32 +00:00
khannurien
7dd00068df v4: the header has two playlists links, each to its own page — it had three: the text button and the avatar menu's "Playlists" both led to the user's own playlists, and the menu's "All playlists" to /playlists. The text button now leads to /playlists, every public playlist, and shows to guests too, the page being open to them. The menu keeps one entry, "My playlists", to the user's own. Below 1150px, where the text button is hidden, the header no longer links to /playlists. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-10 08:30:45 +00:00
khannurien
28f910b00e v4: a playlist's dump count is of the dumps the viewer may see — it counted every dump in the playlist, so a card said "3 dumps" over a page listing two, and told everyone how many private dumps a playlist held. The lists (/playlists, a profile's playlists and followed ones, the search, a dump's playlists, the add-to-playlist panel) now count by isMemberVisibleTo's rule, in SQL as VISIBLE_DUMP_COUNT (lib/playlist-members.ts): the public dumps, and for the owner their own private ones. GET /api/playlists and /api/users/:key/follow/playlists read the optional sign-in for it. The community page and the digests count the public dumps, their figures being the same for every reader. Live, playlist_created and playlist_updated carry each socket's own count: broadcastChange builds the owner's message and the others' apart. playlist_dumps_updated loses its dumpCount, a client counting the dumps it is sent. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-10 08:25:24 +00:00
khannurien
ffc071af4c v4: a description shows as text where a list quotes it — a playlist's card printed its description as written, so one holding an image showed "![](/api/attachments/…)", and so did a user's line in the search results. Both now go through markdownToPlainText (shared/markdown.ts), as the chat's reply previews do: the markup goes, a link keeps its text, an image is left out, and a description of nothing but an image leaves no line. The playlist's page and the profile render the Markdown as before. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-10 08:12:44 +00:00
khannurien
1d56c323a4 v4: the community page's figures stay on one line when a label wraps — each figure is a column-reverse flex box stretched to the row's height, and the default justify-content packed it at the bottom, so a label on two lines ("messages dans la tribune") lifted its number above the others. justify-content: flex-end puts each figure at the top of its cell. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-10 08:10:27 +00:00
khannurien
a2ec163ee5 v4: ESLint and deno fmt pass on the whole tree again — html.ts's attribute pattern escaped a "/" its character class doesn't need escaped (no-useless-escape), and a line of journalLayout.test.ts ran past the width.
All checks were successful
Build and Publish Docker Image / build-and-push (push) Successful in 57s
Details
Build and Deploy Static Documentation Site / build-and-push (push) Successful in 25s
Details 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-10 07:43:28 +00:00
khannurien
7c00db2b84 v4: what comes into a live list goes where its date puts it — a dump edited into a category stayed out of the index's feed of it until a reload, and a dump or a playlist made public went on top of the others' lists as if just posted. The index's feed now takes a dump edited into its scope, and orders the dumps that arrive by date, as the server pages them; a created playlist goes by date in the profile's and /playlists' lists, a new one first as before. Both use withByDate (utils/feed/pagedList.ts), which leaves an item older than every one loaded to a later page. The search runs again after a reconnection, for every page of dumps loaded, with its users and playlists; it shares refetchPages with usePagedList's resync. injectDump's doc said a private dump is never broadcast, while its poster gets it. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-10 07:43:28 +00:00
khannurien
308a8299f3 v4: every live update reaches every page that shows what it changed — the client kept each kind of event in one "latest" slot, so two arriving before a render lost the first (two notifications of one dump going public showed one); they now go through a subscription, useWSEvent, and each reaches every subscriber. A playlist made private sent its new title and description to everyone, and kept them on a visitor's page; playlists now follow the dumps' rule, broadcastChange: others get playlist_deleted, and playlist_created when it goes public again. A dump deleted or turned private sends its playlists' dumps again, so their counts and pages follow. Comments on a private dump reach its poster's other tabs. Reading the notifications clears the bell in every tab (notifications_read), and a category change makes every client fetch the categories again (categories_changed). The dump page gives way to a notice when its dump is deleted or made private. A profile gains the user's new dumps, the followed feed those of the users followed and the playlists followed, the search results follow edits and deletions, the index's category feed drops a dump edited out of it, and a renamed user's name follows on cards and comments. After a reconnection, every live list fetches the pages it holds again (usePagedList's resync), and the playlist, the profile, the dump, the categories and the follows are fetched again. The position memory of usePositionAwareSync is now inside the list sync hooks, so a returning item is put back without depending on the order the hooks are called. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-10 07:34:49 +00:00
khannurien
1c5a449d84 v4: API.md says what the URLs are again — it lacked the /playlists pages and GET /api/playlists, a playlist's categories, the ?category= of a user's lists and of the search, and ?chat=<message id>, and its reserved words lacked playlists. It now also states the rules the pages kept without saying: a category is a segment where the page is the category's own and ?category= where it narrows a list on another page; switching tabs takes the place of the history entry, while picking a category or opening another page or an editor adds one; a signed-in page opened signed out comes back after /login. canonical-url.ts named GET /api/dumps/by-url, which is GET /api/dumps?url=. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 22:25:08 +00:00
khannurien
71df0268e3 v4: asking for a password reset link is a page, /reset-password — "Forgot password?" unfolded the request form on /login as a state of the page, which a reload or a link lost, while /reset-password, where the emailed link leads, answered without its token "Invalid link". The page now holds both steps: without a token it asks for the link, with one it sets the new password, and "Forgot password?" links to it. The login page's unfolding panel and its styles are gone. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 22:24:13 +00:00
khannurien
9a215b0cc5 v4: a playlist's editor has a URL, as a dump's has — it opened over the playlist's header as a state of the page, which a reload or a shared link lost, while a dump's editor is /@name/dumps/<slug>/edit. It is now /@name/playlists/<slug>/edit: Edit goes there, and Cancel or Save come back to the page in its place. Both paths render PlaylistDetail at the same place, so the playlist stays loaded, and a new slug keeps the view. Anyone who may not edit it gets the page, a guest the sign-in first. playlistPath takes the view, and parsePagePath reads it, so its link has no preview, as a dump editor's has none. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 22:22:24 +00:00
khannurien
21f4db268e v4: every request about a user, and the dump editor's path, come from one builder each — fourteen places wrote /api/users/<key>/… by hand, some encoding a handle (@alice%40music.example) and some not, a slug never; userApiUrl (src/utils/shared/urls.ts) now writes them all, by the user's id or by @ and the handle a page's path gives, each segment encoded. The editor's path was the dump's page's with /edit added in two places; dumpPath takes the view, as userPath and feedPath do. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 22:18:13 +00:00
khannurien
669e65a3a2 v4: a user from another instance can't be renamed from their profile's manage tab — it offered the name field to every user, and the API refuses a remote one's rename with a 400, since their home names them. The tab now only shows their name, as the settings do for one's own. 
The profile's and the settings' tests of a remote account's links to its home still expected the home's host itself, which 8bcc005 replaced with GET /api/users/:key/home; they now expect that.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 22:15:54 +00:00
khannurien
f1bc23a65e v4: an instance can be named by the address of its pages — sign-in and the trust list took only an instance's issuer, its API's origin, so where the pages live elsewhere the address people see in their browser named no instance: in development, localhost:3000 answered its /.well-known with index.html, and the sign-in failed as "doesn't answer as a gerbeur". resolveIssuer (remote/gerbeur-instance.ts) now reads the metadata at the address given and, when it names another issuer, takes it only if that issuer's own metadata names those pages back as where its /authorize is, so an impostor can't claim one; sign-in, the trust list and the check against this instance go through it. The Vite dev server passes /.well-known on to its API, as whatever serves the pages must in production (README). A handle still carries the API's host. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 22:14:21 +00:00
khannurien
9608520675 v4: a handle with this instance's own host names the local user — another instance writes bob from here as bob@gerbeur.example, so a mention or a link copied from there named no one here: /@bob@gerbeur.example was a 404, and the mention notified nobody. localHandle (shared/validation.ts) drops this instance's host, its API's or its pages', from a handle; resolveUsername reads names by it, so the page, the API and the Open Graph 301 reach bob at /@bob, and so do the mention notifications and the chat's highlight and chime. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 22:11:39 +00:00
khannurien
6dfba07341 v4: the followed feed's two views are pages — "From people" and "From playlists" on a feed's "follow" tab were a state of the page, which a reload, Back or a shared link lost, while a profile's same split is /@name/follow/playlists. They are now /follow and /follow/playlists, and /music/follow/playlists in a category, feedPath taking the view as userPath does (FOLLOW_VIEWS); picking a category keeps it. Their dumps come from GET /api/dumps?follow=users|playlists, signed in, which the feed's word for it names, instead of GET /api/me/feed?from=; an unknown category is a 404 there, as on the rest of the feed, rather than an empty list. 
A feed's tab, as a profile's, a search's or the community page's, now takes the place of the current history entry, since it is a view of the same page; picking a category, another page, still adds one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 22:09:12 +00:00
khannurien
53ceb6bcbc v4: signing in comes back to the page that asked for it — a guest on a signed-in page, the dump editor, /notifications, /community, /settings or /manage, went to /login and, once signed in, to /, so a digest's link opened signed out lost its way; and since the redirect pushed, Back led to it again. RestrictedLoggedIn now replaces the page with /login?next=<the page>, as /authorize already did. A guest on a feed's "follow" tab, which only a link leads to, is sent there the same way, rather than shown the default tab. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 22:05:39 +00:00
khannurien
683944ebe2 v4: a remote user's pages get their link previews and redirects — the Open Graph middleware passed on every path with a dot in it, taking it for a file's, and the handle of a user from a real host has dots (/@bob@music.example), so their profile, dumps and playlists had no tags, and a former name or another case of theirs no 301. Only the dev instances' localhost:8000 escaped it. parsePagePath already passes on any path that doesn't start with @, which no file's does, so the dot check is gone. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 22:04:35 +00:00
khannurien
53515ce46f v4: playlists have categories of their own — a set apart from the dumps', which admins manage on /manage under "Playlist categories", next to "Dump categories". Each category now has a kind, dump or playlist, its slug and name unique within its kind only; migration 0025 rebuilds categories with the column, every existing one becoming a dump category with its dumps, and adds playlist_categories. A playlist's owner files it under any of them in the create and edit forms, and it shows them as chips on its card and its page, each leading to /playlists/<slug>. 
/playlists is the new page of every public playlist, newest first (GET /api/playlists), and /playlists/<slug> a playlist category's, with a menu of the categories, linked from the user menu as "All playlists"; `playlists` is reserved as a dump category's slug, and one that had it is renamed. A profile's Playlists tab narrows to a playlist category with the same menu as its Dumps tab (?category=), and so do the Dumps and Playlists tabs of /search, each with its own kind (category and playlistCategory for GET /api/search), another tab dropping it.

The category service takes the kind throughout, so a dump can't be filed under a playlist category nor the other way round, and categoryJoin is the one filter join the dump, follow and playlist services share. useCategories(kind) gives one kind's set from the one fetch, CategoryChips replaces the chips the dump card and page each drew, useCategoryFilter keeps a list to its category as the socket brings more, and useCategorySelection holds a form's picked categories.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 21:53:16 +00:00
khannurien
681170a517 v4: in French, an upvote is a "validation" of the reco rather than a vote for it — the button reads "Valider", then "Validé", the notification says "<who> a validé <reco>", and the digests' top recos are "les plus validées". The noun stays "vote": the counts, "Retirer le vote" and the live updates warning. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 21:10:22 +00:00
khannurien
93565cff0c v4: deno lint passes again — a Markdown test wrapped its child text in curly braces, which jsx-curly-braces refuses; the text is now written as is. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 21:06:14 +00:00
khannurien
8bcc005140 v4: a remote account's links to its home lead to the home's pages — the settings' "Change it on <host>" button and the profile's "Account on <host>" link went to the host in the handle, which is the home's issuer, its API, not necessarily where its pages are: in development musique's button led to localhost:8000/settings, gerbeur's API, which served a stale build, instead of localhost:3000/settings. GET /api/users/:key/home?path= now redirects to a page of a remote user's home, their profile there by default, at the origin of the home's authorization_endpoint, which its metadata gives (homePageUrl in federation-service.ts), the issuer standing in while the home doesn't answer; a local user 404s, and a path that isn't one 400s. homeUrl links to it. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 21:06:14 +00:00
khannurien
5f2fb4479a v4: the settings' controls line up down the page — each card of /settings was a grid of its own, its label column as wide as its longest label, so the controls began at a different place in every card, and the other instances, a grid inside their card, at yet another. Every card of settings now has a label column of one width, 10rem, a longer label wrapping, and a list such as GrantList, the card's one child, takes the card's rows: their padding, their rules, their columns. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 20:53:29 +00:00
khannurien
2c3ba6ad11 v4: link previews are read once, in one place, and no longer take an error page, a relative image or a mask icon for the page's — each provider fetched its page and scanned it with regular expressions of its own, the generic one some fifteen times, and Bandcamp and SoundCloud read only OG tags. fetchPage (remote/fetch.ts) now fetches every page, follows redirects to the address it was served from, and refuses an error status, so a 404 or a bot challenge is unreachable rather than a preview titled "Just a moment...", and a refresh no longer overwrites good metadata with it. parsePage (remote/html.ts) reads it in one linear pass with a real attribute reader: unquoted values, a > inside quotes, og: tags under name as well as property, and nothing from comments or scripts. Every URL a page names is resolved against where it was served from, so a relative og:image works; entities are decoded in one pass, &eacute; and &rsquo; included, and one that names no character no longer fails the preview (bandcamp-tralbum.ts shares the decoder). The mask icon is no longer the favicon, JSON-LD prefers the page's node over the site's, tracking pixels and lazy placeholders aren't thumbnails, an og:url with a slug keeps the OG tags, and a <title> loses its site suffix. genericPreview is the base of the Bandcamp, SoundCloud and YouTube channel previews; SoundCloud takes www., m. and on.soundcloud.com short links, embedding the page they lead to, and Bandcamp keeps only its own player. The service tidies every preview: text on one line, titles and descriptions capped, only http(s) images and players. RichContent.type is now RichContentType, and the unused videoId is gone. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 20:46:32 +00:00
khannurien
b4936cccf7 devcontainer: update lock file 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 18:23:53 +00:00
khannurien
141e526de7 v4: the instance fields ask for a URL — the login page's field read "Its address, like music.example" and /manage's "An instance's address, like music.example", an example that named no instance of the reader's. They now read "Your gerbeur's URL" and "An instance's URL". 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 18:23:53 +00:00
khannurien
63803c552f v4: a profile's lists of dumps can be narrowed to a category — /@name/dumps and /@name/upvoted showed every dump, with no way to see only those filed under one category, as the index can. The title row of each list now has the index's category menu, and the category picked goes in the URL as ?category=<slug>, which the other list keeps and every other tab drops (a path segment would collide with /@name/dumps/<slug>, a dump's own page). GET /api/users/:key/dumps and /upvoted take ?category= as the followed feed does, an unknown slug answering an empty list; resolveCategoryFilter moves to category-service.ts for both. useUserDumpFeed passes everything the socket changes through the category, so a dump edited out of it leaves and an upvote outside it doesn't come in. CategorySwitcher now reports the category picked to its owner (onSelect) instead of navigating to the index's path itself, and the divider after it in the index's tab row is drawn by CSS, so the profile gets none. useTabParam's setter takes the query string to go to, the current one by default. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 18:23:03 +00:00
khannurien
52a23db504 v4: an instance that takes no users of others still blocks instances its own users may not sign in to — with GERBEUR_FEDERATION_INCOMING=0, /manage hid its list of instances, though a blocked one still kept the instance's users from signing in there, and no admin could block another. The list now stays while federated sign-in goes either way: without incoming sign-ins it shows and adds only blocked instances, the trusted ones kept for when they're back, and says what blocking does there. Where both ways are on, its note now says that users here can't sign in to a blocked instance either. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 16:33:08 +00:00
khannurien
83dc44050e v4: a deployment chooses which ways federated sign-in goes — every instance took users of the others and let its own sign in to them, with no way to turn either off. GERBEUR_FEDERATION_INCOMING=0 keeps users of other instances out: the login and register pages offer no other instance, /manage no list of instances, their sign-ins are refused, and those already in are signed out until it is on again, their homes' events still applied meanwhile. GERBEUR_FEDERATION_OUTGOING=0 keeps the instance's users from signing in elsewhere: its /authorize page lets no instance sign them in, its metadata says so (gerbeur_outgoing), which other instances check before sending anyone there, the settings page lists no instances, and at startup every grant is withdrawn, which signs them out there. Both are on by default and reach the client through index.html. The dev instance gerbeur (:3000) takes no users of the others, while its own still sign in to them. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 16:18:21 +00:00
khannurien
66e402bd91 v4: federated sign-in closes the holes a review found — a callback link made from someone else's sign-in signed the visitor in to that account, a password reset left the user's sessions on other instances open, and events could be lost, reordered or delayed. FederatedSignIn keeps the sign-in's state in sessionStorage (utils/auth/federation.ts) and FederationCallback finishes only a sign-in this tab started. A password reset tells the instances the user signed in to, as a change does. The outbox delivers each instance's events in the order they were made, the first that fails holding back the rest, so a late older update no longer undoes a newer one, and delivers to every instance at once, so a dead one delays none of the others; an event queued during a run is sent by it. 
Receiving, a home's keys that can't be fetched answer 502, which the sender retries, instead of 401, which it dropped; a bad update answers 400 rather than 502; a blocked instance's events answer 403; events are limited to a burst of 120 per instance, and a token naming an unknown key refetches the keys at most once a minute. An avatar must be on its issuer's origin, and its URL is versioned by its file's write time instead of updatedAt, so a profile edit no longer makes clients download it again. An instance that doesn't answer can be blocked, keeping its name, though not trusted.

303 backend tests and 703 frontend tests pass, the federation tests 8 times in a row, and deno check, tsc, eslint, doc:lint and i18n:compile --strict are clean.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 16:05:11 +00:00
Vincent Lannurien
c77e173767 v4: the devcontainer builds and starts again — the docker-outside-of-docker feature failed to install, since the base image is now Debian trixie, where its default moby option isn't supported; it now installs Docker's own packages ("moby": false). Every service of the compose project was also built under one image name, gerbeur-dev, the workspace with the features and the others without, so whichever finished last took the name and the workspace could lose its docker CLI; each service now gets its own image, built from the same cache. 
Checked with the devcontainer CLI's up: the image builds, install exits 0, gerbeur and musique answer on :3000, :3001, :8000 and :8001 (openid-configuration included), and docker ps works as node in the workspace.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 16:39:58 +02:00
khannurien
a25500d833 v4: the devcontainer starts several gerbeur instances side by side, so that signing in from one to another can be tried on one machine — the dev environment ran a single instance. dev/instances.ts lists them (gerbeur on :3000 and :8000, musique on :3001 and :8001), and dev/start.ts starts an API server, restarted on change, and a Vite server for each instance it's given, by one rule: instance <name> keeps api/sql/<name>.db and api/uploads/<name>/, reads .env, and takes its ports and what sets it apart from its entry, with federation allowed over http on localhost. It prefixes each line of output with the instance's name, and stops every server when one stops, such as a Vite server whose port is taken. The devcontainer is now a compose project (.devcontainer/compose.yaml): the workspace, a one-shot install that runs deno install and draws the icons once, and a service per instance running dev/start.ts for it, all on the host's network as before and sharing Deno's cache through a volume. ffmpeg moves from a feature into the Dockerfile, since the instances' services get no features, and docker-outside-of-docker gives the workspace's terminal their logs and restarts. Outside the devcontainer, deno task dev starts them all, or those named. Each Vite server keeps its own dependency cache. The existing dev uploads move to api/uploads/gerbeur/. 
dev/start.ts was checked by hand: musique starting alone on a fresh database, an unknown name refused, a taken port stopping both servers, and Ctrl+C or SIGTERM leaving no server behind. The compose project couldn't be run here (no Docker in the workspace): its YAML parses and resolves as intended, but it is untested; tsc, eslint, deno check, deno fmt, deno lint and doc:lint, which now covers dev/, are clean.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 14:31:25 +00:00
khannurien
c0d03e9aaa v4: users sign in to one gerbeur instance with their account on another — topic instances, each with its own admins, domain and data, had no way to share accounts. Sign-in is federated over a subset of OpenID Connect, the authorization code flow with PKCE: each instance is an identity provider for its own users and a client for the others', and signs what it tells them with an Ed25519 key kept in federation_keys and published at /.well-known/jwks.json, beside its metadata at /.well-known/openid-configuration. Only sign-in is federated: an account from another instance is a local users row linked to (issuer, subject) in user_identities, with no password (password_hash ''), so sessions, roles, permissions, the socket and every foreign key work as before. 
A remote user is name@host here, in users.username, in paths (/@alice@music.example) and in mentions, through one handle rule in shared/validation.ts that paths, MENTION_RE and Markdown all build on; Markdown undoes GFM's email autolink of a handle. Their name, email and avatar come from their home at each sign-in and by its signed events (federation_outbox, delivered by the federation job with backoff): a rename at home leaves the old handle an alias here, and a password change or a withdrawn grant ends their sessions here. Their password, email, avatar and name can't change here (assertLocalUser), nor can they sign in elsewhere from here.

An admin keeps the trusted and blocked instances on /manage (federation:manage): a trusted instance's users sign in from a button on the login page, a blocked one's can't and are signed out at once, and any other's join with an invite from the register page. The /authorize page asks the user once per instance, and /settings lists the instances their account signs in to, each disconnectable; /login?next= brings a signed-out visitor back to /authorize. Emails stay unique among local accounts only, and migration 0024 adds the tables, makes that index partial and reserves authorize and federation as category slugs. remote/gerbeur-instance.ts is the only way to other instances: https and public addresses only, no redirects, no certificate fallback, unless GERBEUR_FEDERATION_ALLOW_PRIVATE lets two instances on one machine talk.

298 backend tests and 702 frontend tests pass, and deno check, deno fmt, deno lint, tsc, eslint, doc:lint and i18n:compile --strict are clean; checked end to end with two built instances on one machine: sign-in through the other's login and consent pages, consent skipped once granted and asked again after disconnecting, a rename and a password change at home reaching the other, and blocking; screenshots of the login, consent, profile, settings and manage pages at 1280 and 390px.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 14:23:00 +00:00
khannurien
2d26d0aada v4: the late-digest test no longer races the clock — it posted its only dump at the current time and sent the digests right after, and a digest's window ends before the run's now: when both fell in the same millisecond, which createUser's cached password hash made common, the dump was outside the week, the digest empty, and the test failed about one run in fifteen. Reproduced with a dump stamped at now (no digest) and one a millisecond earlier (one); the dump is now dated a day back, as "recent" meant. 60 runs of the digest tests, 6 of the backend suite (277 tests) and 4 of the frontend suite (684 tests) pass. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 13:47:56 +00:00
khannurien
fac0e2db28 v4: an audio file's waveform is made once by the server, and the player seeks on one accessible scrubber — every player downloaded the whole file a second time and decoded all of it in the browser to draw 150 bars, so a long FLAC cost tens of megabytes of memory per page, and the waveform could only be clicked: no drag, no keyboard, no slider for a screen reader. GET /api/dumps/:id/waveform now answers an audio dump's peaks, which lib/ffmpeg.ts decodes with ffmpeg on first request, mono at 8 kHz and streamed, keeps beside the thumbnails as the response itself, and serves with an ETag (a ten-minute MP3 takes about a second, then a few hundred bytes). The peaks are taken by shared/waveform.ts's peaksOf, scaled to the loudest so a quiet recording fills the track, and the dump form's preview of a file not yet uploaded decodes it with the same function, at 8 kHz in an OfflineAudioContext. Scrubber replaces Waveform and SeekBar: a role="slider" that seeks where a pointer lets go, follows a drag (the clock shows where it is held), takes arrows, Home and End, and draws the bars as one path with the played part clipped by a nested svg, so it moves smoothly and a timeupdate redraws two attributes instead of 150 rects; audio without a waveform, and video, get a plain bar. MediaPlayer reads whether it plays from the element's events alone, so a click no longer reports the change twice, its handles for PlayerProvider are stable callbacks instead of refs to refs, the drag no longer reattaches every listener, and the <source> children that src made dead are gone, with the mime prop. The file, thumbnail and waveform routes resolve their dump, token and caching through lib/dump-media.ts, and what is generated from an upload is named and removed in lib/ffmpeg.ts, the custom thumbnail by dump-service's customThumbnailPath, instead of paths spelled out in each. The test setup's play() and pause() fire their events as a browser does. 277 backend tests and 684 frontend tests pass, and deno check, tsc, eslint, doc:lint and i18n:compile --strict are clean; checked in the app with a real FLAC on its dump page and in the global player at 1280 and 390px, seeking by keyboard. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 13:34:22 +00:00
khannurien
c709b56911 v4: every message named by an id carries its English in the code — the emails and the four pages that share their words called Lingui with a bare id (i18n._("digest.community"), <Trans id="digest.community" />), so their English lived only in a hand-kept en.po, extract couldn't run with --overwrite, and a page reading the code couldn't tell what it said. The emails now pass descriptors, i18n._({ id, message, values }), the form Lingui reads without a macro, as the server needs. The fifteen messages the digest shares with the community page, Settings, the unsubscribe page and the avatar menu are declared once, as /*i18n*/ descriptors in shared/digest-messages.ts, which those pages translate with useLingui()'s i18n instead of the global one. lingui.config.ts scans shared/ too, and i18n:extract runs with --overwrite, so en.po is written from the code. Every other message already used the macros. The compiled catalogs are unchanged, key for key, in both languages. 272 backend tests and 677 frontend tests pass, and deno check, tsc, eslint, doc:lint and i18n:compile --strict are clean. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 13:19:01 +00:00
khannurien
c3594e4c27 v4: what the server and the client both state is declared once — several rules were written out in two or more files, some already drifting apart. A link's site name (its host without www.) was computed in five places, now shared/url.ts's siteHost; dumpFileVersion, which the frontend and the server had to keep equal by hand, is one function in shared/dump.ts; the Markdown flattening of the chat's reply previews and the digest's quotes is shared/markdown.ts's markdownToPlainText, so the digest also keeps an image's alt text; an attachment's URL and how to find one in a text are shared/attachment.ts, which the editor, Markdown and attachment-service read, its id pattern the new UUID_RE that the file and thumbnail routes check ids with. On the server, the Bandcamp, YouTube playlist and link preview caches use one TtlCache (api/lib/ttl-cache.ts) instead of three copies of the same expiry and eviction; the link previews' theme-color and the emails' theme tokens read colours with one parser (api/lib/css-color.ts), and a transparent theme-color now gives no tint; the WebSocket route sends through ws-service's send, the welcome included, so it is type-checked; fetch.ts strips an IPv6 literal's brackets with public-address's bareHostname; and a category's slug is checked by the same validateSlug as a dump's, with its own length. The migrations keep their frozen copies, as they document. 272 backend tests and 677 frontend tests pass, and deno check, tsc, eslint and doc:lint are clean. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 13:16:52 +00:00
khannurien
134d1a0b97 v4: a moderator's or an admin's @everyone in the chat notifies every user — there was no way to reach everyone at once. The new chat:everyone permission, held by moderators and admins, lets a chat message saying @everyone notify every user but its author and those reading the chat, through notifyEveryone, after notifyMentions, so a user also named is told once, by name; the notification is a user_mentioned with everyone set, which the notifications page and the digest word as "mentioned everyone in the chat". chat_messages.mentions_everyone, sent as ChatMessage.mentionsEveryone, is set from the author's role each time the body is written, so a user's @everyone reaches no one, a moderator's edit of a user's message can't make it reach everyone, and a demoted author's next edit loses it. Only an edit that adds @everyone notifies, so fixing an announcement's typo reaches no one new. The client highlights and chimes by the flag, never by the text; Markdown shows @everyone as a mention only in a flagged message, and never links it to a profile. The chat composer offers @everyone first to those who may say it. No account may be named everyone, in any case: migration 0023 adds the column, flags the moderators' and admins' past @everyone messages, renames a user named everyone to everyone_N and drops an alias everyone. 265 backend tests and 681 frontend tests pass, and deno check, tsc, eslint, doc:lint and i18n:compile --strict are clean; checked in screenshots of the chat at 1280 and 390px, with a moderator's and a user's @everyone and the suggestions open. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 12:54:09 +00:00
khannurien
5f239a144d v4: the community page and the digests' section are titled Communauté in French — "La communauté" read awkwardly as a title. i18n:compile --strict is clean. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 11:52:54 +00:00
khannurien
708d113013 v4: a /community page shows the site's week and month, from the same figures as the digests, and every recurring server task is a job of one scheduler — the digests' top dumps and community section (who joined, the new public playlists, the chat and the totals) existed only in emails. community-service.ts now gathers them for a period from listTopDumps, listNewUsers and listNewPublicPlaylists, each in its own service and returning the API's Dump, PublicUser and Playlist; the digest builds its sections from the same call, its email-only DumpRow and DUMP_COLS replaced by one Dump mapper that networkOf uses too, and links its community section to the page. The page, /community for the last week and /community/monthly for the month, signed in only and linked from the avatar menu, reads GET /api/community?period=, which serves figures kept in memory and recomputed every GERBEUR_COMMUNITY_REFRESH_MINUTES (15 by default). It shows the four figures, the top dumps and the new playlists as live cards, and the new members as UserCards, the follow tab's user row made shared. Its words are the emails' own: the messages both use are renamed from email.digest.* to digest.*, as digest.weekly.off was. api/lib/scheduler.ts's startJobs runs the digests, the community refresh, the WebSocket ping, the typing expiry and the rate-limit sweep, which main.ts starts, so none of them runs on import any more. PERIOD_MS moves to shared/settings.ts. community is reserved as a category slug, and migration 0022 renames a category that had it, with renameReservedCategories moved out of 0021 into reserved-slugs.ts. A group's dump or playlist cards now fill its column. 262 backend tests and 678 frontend tests pass, and deno check, tsc, eslint, doc:lint and i18n:compile --strict are clean; checked in screenshots in Smooth light at 1280 and 390px, Smooth dark with mocked playlists, Brutalist dark at 390px and Kanagawa dark at 1280px. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 11:42:01 +00:00
khannurien
7713c5a1b5 v4: a dump or playlist going public notifies its followers — a private dump notified no one when posted, and nothing told its followers when it was made public: the poster's followers never learnt of it, nor did the followers of the public playlists holding it. Likewise, the dumps added to a playlist while it was private were never notified to its followers. updateDump and replaceFileDump now call notifyIfWentPublic, which notifies the poster's followers (user_dump_posted) and, through the new notifyPlaylistFollowersNowPublic, the followers of each public playlist holding the dump (playlist_dump_added). updatePlaylist calls the same function when a playlist goes public, for each public dump in it, from the bottom up. A follower hears of a dump once both it and the playlist are public; the source keys stop a dump made private and public again from notifying twice. 256 backend tests pass, and deno check, deno fmt and doc:lint are clean. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 10:23:08 +00:00
khannurien
256857f72e v4: a chat mention notification opens the chat on the mentioned message, so the newest one clears the chat badge — it linked to ?chat=open, and the chat opened on the "new" divider whenever the unread messages didn't fit. That left the list above the bottom, so the read marker never moved and the badge stayed, while the mention itself was out of sight. It now links to ?chat=<message id>: ChatProvider opens on any ?chat= (open, or a message id passed to ChatModal as focusId), and ChatModal opens centred on that message and flashes it. When the message is within a screen of the bottom, the list stays at the bottom and reads everything, as it always has. An older mention still leaves the messages after it unread. A message outside the first page of history falls back to the divider. ChatModal's flash is now its own callback, shared with jumping to a message. 675 frontend tests pass, and tsc, eslint, deno fmt and doc:lint are clean; reproduced and checked in the app with 36 unread mentions from a second account: the newest clears the badge, an older one opens on it with the badge kept. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 08:56:59 +00:00
khannurien
4ea192e0b1 v4: an uploaded audio file plays from its cover everywhere, and a card's play badge sits in the middle of its thumbnail — a link played from its thumbnail in every view, but an uploaded file only did on a list card: a journal tile opened the dump, an audio file's page showed a round button by its waveform and never its uploaded cover, and an uploaded video's page was a black box until played. The list card's badge sat in the thumbnail's bottom-left corner, where it looked misplaced, and clicking a playing file restarted it. The new usePlayFile hook builds the player's item and the file's artwork (fileArtworkUrl) for the card, the journal tile and the dump's page, and plays, or pauses once the file is the current one. A card's badge, ▶ or ⏸, is now centred, as is the ↗ on a link out; over a placeholder it takes the place of the glyph. An audio file without a cover shows a placeholder tinted from its file name, not a bare 🎵. In the journal, audio counts as having art (hasThumbnail), so it is an image tile, its glyph raised above the title, and a file tile plays and pauses the file. On the dump's page, the cover (96px, the card's badge on it) plays the file beside the shared Waveform, which replaces FilePreview's copy of it; a video takes the server's still as its poster. 674 frontend tests pass, and tsc, eslint, deno fmt, doc:lint and i18n:compile --strict are clean; checked in screenshots in Smooth light at 1280 and 390px, Brutalist dark on the journal at 390px and Kanagawa dark at 390px, playing. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-08 22:33:33 +00:00
khannurien
2c031b0102 v4: the header drops the username link — on wide screens it stood beside the avatar, which leads to the same profile, so only Playlists stays there. The avatar's menu still opens with @username first. 670 frontend tests pass, and tsc, eslint, deno fmt and i18n:compile --strict are clean; checked in a screenshot in Smooth dark at 1400px. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-08 22:23:46 +00:00
khannurien
886216749e v4: the avatar menu shows at every width and holds Log out, so a profile keeps only Invite; the profile's dump cards lay out again — your own profile had four buttons in a row, Invite someone, Settings, Manage and Log out, Invite taller than the rest, and on wide screens it was the only way to reach /settings or /manage. The avatar menu now leads the header's icon row at every width, after the name and Playlists text links on wide screens, and ends with Log out under a separator, muted until hovered. The profile keeps Invite someone alone. The menu's button outweighs .app-header-nav button and the themes' button rules (Brutalist and GeoCities list it among their flat buttons), so it reads as one of the menu's links. Since b9ff9c4, a card under 472px is laid out by the narrow-card container query, but the profile columns' dense-card rules still placed the thumb, body and vote themselves, more specifically: the thumb lost its row span, so the title fell under it and every card was a tall, mostly empty box. Those rules now only make the card denser (a smaller thumb, a 2-line title, smaller type) and leave placement to the container query. 670 frontend tests pass, and tsc, eslint, deno fmt, doc:lint and i18n:compile --strict are clean; checked in screenshots in Smooth dark at 1400 and 390px, Brutalist light at 390px, GeoCities light and Kanagawa dark at 1400px, in French. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-08 22:22:15 +00:00
khannurien
10c9cf4269 v4: a playlist's cover is a banner over its header, and the edit form picks it in the same shape — the cover was a 72px square beside the title, which left the title and description a narrow column under a mostly empty band: on a phone, in Kanagawa's wide mono, a long title broke into six lines. The cover now spans the header's width above the title, cropped to 3:1 and no taller than 14rem, inside the header's padding so outlined themes keep their ring around it; the title, buttons, description and meta take the whole width under it. The edit form's Cover picker takes the same shape (.playlist-cover-banner, shared with .playlist-detail-img), and with no cover is a dashed 3:1 placeholder. ImagePicker's size is now optional, so a class can size it, and its radius takes any CSS length, here --radius-lg; a picker sized by aspect-ratio sets min-height: 0, or its image would hold it at its own height. 669 frontend tests pass, and tsc, eslint, deno fmt and doc:lint are clean; checked in screenshots in Kanagawa dark (French, 385px), Brutalist at 390px with and without a cover, and Smooth at 1100px, the page and the form. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-08 22:10:06 +00:00
khannurien
5acdf79c8d v4: the playlist and dump edit forms end in one bar of actions, a panel of its own — the playlist editor drew rules between its fields and its actions, and on a phone its Delete, Cancel and Save wrapped into a staircase, then stacked (a0c8b1d), neither of which held together. Both forms are now built as DumpEdit was: surface panels 3px apart (.form-panel, from .dump-edit-panel), the last a bar (.form-panel--bar, from .dump-edit-footer) holding Delete on the left and Cancel and Save on the right, compact (btn-sm), on one line. The new DeleteButton says "Delete" and names what goes to assistive tech ("Delete playlist", "Delete dump"); in a bar too narrow for the three labels in every face and language (a container query, under 18rem), it shows 🗑 instead. While edited, the playlist's header drops its own card for the form's panels. The phone stacking rule is gone. An auto-sized editor also re-fits when its width changes with its text unchanged: opening the form makes the page tall enough for a scrollbar, which narrowed the description after it was measured, so it overflowed; and it hides its overflow below its max-height, where a scrollbar could only come of rounding, and drops the resize grip, which the next keystroke would undo. 669 frontend tests pass, and tsc, eslint, deno fmt, doc:lint and i18n:compile --strict are clean; checked in screenshots in Kanagawa dark (French, 385 and 360px), Smooth light at 1000px and DumpEdit in Smooth at 390px. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-08 22:05:43 +00:00
khannurien
b9281da9c9 v4: the avatar menu opens from the avatar's left edge — since 7531be3 the avatar leads the header's icon row, but its menu still hung from its right edge, so on a phone it ran off the left of the screen. It now hangs from the left edge and opens over the icons. Checked in screenshots in Smooth at 390px and Brutalist at 320px, in French, and Kanagawa at 1100px. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-08 21:58:15 +00:00
khannurien
7531be300e v4: the header's controls are line icons, and the avatar leads them — 🔍, ✕, 💬 and 🔔 were four emoji in four palettes beside the avatar and the "+", none of them the theme's. They are now stroked icons (NavIcons.tsx, which the search bar's submit and ChatFab share) in the text colour, their bodies washed with the accent; the wash deepens on hover, and fills while something is unread, with a small dot in place of the red count (the count stays in the button's label and the tab's title, and ChatFab keeps its own). Themes set the line through --nav-icon-stroke, -linecap, -linejoin and -tint-strength: Brutalist heavy and square-cut, Times hairline in ink (its grayscale filter no longer needed on them), GeoCities fat and soaked. The search toggle was a ghost button with its own hover; it now shares the chat button's and the bell's rule, background and borders in every theme. The avatar sat among the icons, its hover box larger than theirs and 1.5px higher (a block .user-menu left a line-box gap under the inline-flex trigger): it now leads the row, where the name and Playlists links stand on wide screens, so the icons run unbroken to the "+"; it is 32px, and hover rings it with the accent instead of a box. 669 frontend tests pass, and tsc, eslint and deno fmt are clean; checked in screenshots in all six styles, light and dark, at 390px, at rest and unread, and Smooth and Brutalist at 1280px. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-08 21:56:19 +00:00
khannurien
a0c8b1d509 v4: auto-sized text editors no longer scroll, and on a phone an action row with a destructive button stacks — an autoResize editor set its height to scrollHeight, which leaves out the borders that a border-box height counts, so the text always overflowed by a couple of pixels: a playlist's description, a .form textarea since cf89c18, showed a scrollbar that ran under the emoji button. The height now adds the borders. On a phone, Delete, Cancel and Save didn't fit one line in French or in a wide face, and wrapped into a staircase: Delete alone on the left, Cancel and Save under it on the right. A .form-actions row with a .btn-danger now stacks under 512px, with Cancel and Save across the top (Save taking the room left) and the destructive button full width under them; the playlist editor and DumpEdit's footer both take it. 669 frontend tests pass, and tsc and deno fmt are clean; checked in screenshots in Kanagawa (French, 385px, with a long description) and DumpEdit's footer in Smooth at 390px. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-08 21:51:38 +00:00
khannurien
cf89c18eb9 v4: a playlist's edit form is a .form like the dump's, and its dumps keep the header's edges — the editor laid the cover beside its fields with display: contents, which on a phone left them a squeezed column, and drew its own inputs: the title and the slug both in bold large type, the description in another size, only the slug labelled, and a segmented control styled apart. It now stacks the shared fields, each under its label (Cover, Title with its counter, Link, Description, the visibility toggle), over the action row, as DumpEdit does, and takes the header's whole card. The dumps' drag handle and remove button took width from the cards, which sat inset from the header on both sides; they now sit in the page's side padding (--page-gutter, which .page-content now names), so the cards line up with the header at any width. Undo shows as a small btn-primary over the card it would bring back, instead of a third control beside it. CountedInput, VisibilitySegmented and the .playlist-edit-* rules had no other users and are gone. 669 frontend tests pass, and tsc, eslint, deno fmt, doc:lint and i18n:compile --strict are clean; checked in screenshots in all six styles at 390px, Kanagawa and Smooth at desktop widths too, Undo in Smooth and GeoCities. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-08 21:48:12 +00:00
khannurien
45f39a959b v4: /settings and /manage take after /notifications — the three pages the user menu and the bell lead to share one frame: a 680px column (PageShell's new narrow), a title with its icon (PageHeader: ⚙️, 🛠️, 🔔), then groups under a small capitalized label ruled to the column's end (PageGroup, the notifications' day label), their content on surface cards (.page-card, the notification items' surface). Settings drew big section headings over controls laid on the bare page, the full width wide, and Manage the same around the category manager; each section is now a card whose setting rows are ruled apart and keep their labels lined up (a subgrid of the card's columns). The notifications' column, centered by auto margins in a flex column, shrank to its content (380px with short ones); it fills its width now. Its own header, group and bell rules gave way to the shared ones, and Times, Brutalist and GeoCities dress .page-card as they dress the notification items. In the narrower column, Kanagawa's monospace no longer fit the six styles on one row, and French labels leave even less room: a segmented control of more than four options now makes two even rows at any width, as it did on phones. 670 frontend tests pass, and tsc, eslint, deno lint, deno fmt and doc:lint are clean; checked in screenshots in all six styles at 1280 and 390px, Smooth in light too, with the account's style and the notifications mocked so the shared test account was left as it was. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-08 21:40:10 +00:00
khannurien
1fda26a399 v4: /settings, /manage and /notifications title themselves alike, with a bare heading above their sections — Settings and Manage borrowed the edit form's .form-page-header, a panel rounded only on top to cap the panels under it, so on a page without them it ended in a square edge, and its title was the size of the section headings below. Notifications had a heading of its own. All three now take .page-header and .page-title: a bare h1 a size above the sections' h2, with what goes with it (the unread count) at the row's end; the edit forms keep their panel header. Times and Melange set their display type on .page-title, and the .profile-subpage-* rules, unused since the profile's pages became its tabs, are gone. (The CSS went in with d683ac0, which a concurrent session committed from the shared tree.) 21 Settings, Manage and Notifications tests pass, and deno fmt is clean; checked in screenshots in Smooth at 1280 and 390px. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-08 21:28:28 +00:00
khannurien
45e004b465 v4: the slug and rename fields lose their hints about old links — "Links to the old address keep working." under the dump and playlist link field, and "Their old name keeps leading to them…" under the moderator's rename field, described a server guarantee nobody filling the form needs to read. Only the fields' errors show under them now; UsernameField loses its hint prop. 670 frontend tests pass, tsc and i18n:compile --strict are clean. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-08 21:28:01 +00:00
khannurien
437eea0df3 v4: every inline editor ends in a btn-primary Save and a btn-secondary Cancel — comments drew their own .comment-submit-btn and .comment-cancel-btn, the profile's email its own .profile-email-btn pair, the bio a plain-text form-cancel, and the chat's edit box shrank the shared buttons with a rule of its own; they were four sizes, two orders of chrome, and the comment button's white label sat on Smooth's yellow accent. They all take the shared classes now, and the new .btn-sm is the one compact size, for the comment forms, the chat edit box and the email editor; the bio and the dump title stay full size. .btn-secondary gets a disabled state, as btn-primary has. The bespoke rules are gone, with GeoCities' bevel for the comment buttons, which btn-primary and btn-secondary already have. 670 frontend tests pass, and tsc, eslint and deno fmt are clean; the comment, reply, chat edit, email and bio editors were checked in screenshots in Smooth, Kanagawa, Brutalist and GeoCities at 1100 and 360px. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-08 21:27:26 +00:00
khannurien
d683ac0548 v4: the dump title editor's Cancel is a button, beside Save — it was the grey text link of the full forms' action rows, next to a filled Save, where every other inline editor pairs Save with a bordered Cancel; it now takes btn-secondary, the same height as btn-primary. Save and Cancel sit in .dump-title-actions and wrap together, so on a phone they go under the input as a pair instead of Cancel dropping to a line of its own. 30 Dump page tests pass, and tsc, eslint and deno fmt are clean; checked in screenshots in Smooth, Kanagawa, Brutalist and GeoCities at 1200 and 360px. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-08 21:23:28 +00:00
khannurien
b9ff9c417e v4: phone layouts hold down to 300px in every style, Kanagawa's wide monospace included, and the lint is clean — the floating chat and "+" buttons hid the end of every page (Save on the dump edit page, the last setting), since the page kept room for one button and they stack two; it keeps room for both now, read from one --fab-size. A 1fr page grid grew to its widest unbreakable content, and the edit forms' action bars didn't wrap, so at 320px the dump edit page scrolled sideways and the playlist edit's Save stuck out of its card; the grids take minmax(0, 1fr) and .form-actions wraps. The profile header's info column was a hidden horizontal scroller under a fade, which cut the username, the inviter, the email and the Log out and Invite buttons; it wraps instead (long names and emails break anywhere), and the avatar shrinks to 70% on phones. TabBar pins its leading content (the feed's category switcher) at the row's start while the tabs scroll under it, where scrolling to the active tab used to push it out of view; its edge fades show only while tabs are hidden past that edge, so the last tab is no longer dimmed once scrolled to, and a left fade marks a scrolled row. Tabs are narrower on phones, which lets Kanagawa's search tabs fit at 375px. The global player puts its track controls on a row of their own on phones, where they left the title three letters, and an audio bar narrower than 420px drops its volume slider for the waveform. The profile bio lost the margins it stacked on the page's gap. A dump card is now a container and lays out by its own width rather than the viewport's, since a playlist's is narrowed by its drag handle and remove button and titles broke every few letters: under 472px it is a grid where the vote ends the date line, as on the journal's cards, instead of hanging under the thumb, with a spacer row keeping it off the title; under 300px the thumb takes the top row and the text the full width. The lint errors that predated this are fixed: a useless fragment and two async callbacks without await in tests (they now await advanceTimersByTimeAsync), EmailKit's renderer gets the kit as an argument instead of aliasing this, its figure regex drops the literal no-break spaces \s already matches, digest-service's highlightOf takes NotificationData narrowed per case instead of any, and eslint ignores coverage/. 254 backend and 670 frontend tests pass, and tsc, deno check, deno lint, deno fmt, eslint and doc:lint are clean; every page was checked in screenshots at 300 to 480px in Kanagawa and Smooth. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-08 21:19:43 +00:00
khannurien
58c95dc4f3 v4: the notifications page restores its list and scroll on Back, and shows what it missed — it now saves its loaded pages and scroll position as it is left and restores them on Back, through useFeedCache, useScrollSave and useScrollRestore like the feeds and the profile's tabs; the saved list keeps the ones the server confirmed read as read, so coming back doesn't highlight them again. The page never refetched after a reconnect, so notifications sent while the socket was down never showed, and the next read-all marked them read unseen; a reconnect, and every restored list (after a reload the socket hasn't said yet whether anything is new), now fetch the first page again, and what is new goes on top with the socket's arrivals, those alone sorted newest first so the server's order stays. Marking read now follows the unread notifications shown, each sent once, instead of each socket message, and a list with none unread sends nothing. The arrivals shift the server's offset pages, so a next page could repeat the last one's tail: each notification shows once. The next page loads as the end scrolls into view, with FeedFooter, as every other page list does, and the loading and error states go through ListState. The "N new" pill was a number before a separately translated "new", which French made "3 nouveau"; it is one plural message, "3 nouvelles". Dates past a week and the tooltip use the app's locale rather than the browser's. The mention, comment and like icons had no colored circle; they have one now, and the decorative emoji are hidden from screen readers. 670 frontend tests pass (new: loading on scroll, the save on leaving, the restore with what came meanwhile, the reconnect backfill), and tsc, deno fmt, deno lint, doc:lint and i18n:compile --strict are clean; the restore was checked in the running app at phone width. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-08 19:54:31 +00:00
khannurien
0a16d3c96c v4: users can't rename themselves, /settings lays out again, and its selectors fit a phone — PATCH /api/me refuses a username with a 400 and changes nothing else in that request; a name changes only through PATCH /api/users/:key/username, which needs user:manage, from /@name/manage. The old name still stays an alias for good. renameUser lost its 30-day limit and its exempt flag, and RENAME_LIMITS holds the slug limit alone. /settings shows the username as plain text, and the rename hint left the catalogs. The settings grid sized its control column to its widest content, the rename hint, which stretched every control under it; that column now takes the remaining width and each control keeps its own size. A segmented control wraps instead of overflowing; in a settings row on a phone it takes the full width, and one of more than 4 options (the styles) lays out in two even rows. .btn-border turned red on hover, so Settings, Manage, Retry and Back looked like Log out; it is now neutral (Smooth's accent is red too), and only Log out takes the new btn-border--danger, muted until hovered red. A <button> and a <Link> .btn-border are now the same height. API.md and the module docs say who renames; the backend tests rename through the moderator route and check a self-rename is refused, and the malformed-name check moved to the manage tab's test. 254 backend and 667 frontend tests pass, and tsc, deno check and doc:lint are clean; Smooth, Brutalist, GeoCities and Kanagawa were checked in screenshots at desktop and phone widths. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-07 06:50:43 +00:00
khannurien
4468438ba7 v4: every page and API URL follows one of two rules, written down in API.md — pages are a hierarchy where @name is a user and any other root word a category or an app page, tabs and views are plain segments and a default view is the bare path; API resources are /api/<plural>/<key>, a user's key being an id or @name, the caller's things under /api/me, media a sub-resource, lookups by another key a query filter. ~ is gone: the feed is /, /hot, /new, /journal, /follow and /music/hot, a profile /@bob with its overview and the tabs dumps, upvoted, playlists (her own only), follow (users, and /follow/playlists), invitees and manage (role and name, for user:manage), which replace the separate UserDumps, UserUpvoted and UserPlaylists pages; /settings holds the account (now with the username) and /manage the category manager, both linked from the user menu and one's profile. Dumps and playlists live under their owner, /@bob/dumps/<slug> and /@bob/playlists/<slug>, named by slug alone: migration 0021 re-slugs them to the bare slugify(title), unique per owner and kind (the oldest keeps it, clashes get -2), renames the categories whose slug v4 reserves (hot, follow, settings…), and rewrites this site's v3 links in dump notes, comments, chat, and playlist and profile descriptions. A slug changes only when edited, through the edit forms' new link field, which proposes slugify(title) as the title changes; a username changes from /settings or, for user:manage, from /@name/manage. Every slug or name left behind stays an alias of its item or user for good (dump_slug_aliases, playlist_slug_aliases, user_aliases), so old links redirect without chaining and nobody else can take one; slugs change 3 times a day per item, names once in 30 days, moderators exempt. An alias, or a name in another case, answers the item with its current path: the pages put it in their URL (useCanonicalPath, without fetching again), and the OG middleware answers a 301. shared/paths.ts builds and parses every page path, for the client's links, the emails' (digest-service lost its own builders and its id fallbacks, and links /follow and /settings) and the server's matchers (OG tags, self previews, backlinks, self-url); shared/slug.ts holds slugify and the format categories share. Dumps and playlists carry ownerUsername and a required slug, and notifications join their users' current names and their items' paths each time they are sent or read, so renames reach them and a deleted item's has no link. The API was renamed per the plan's table: POST /api/users and /api/sessions, /api/password-resets, /api/me (with settings, password, avatar, media-token, playlists?contains=, follow, feed?from=, notifications), /api/users/:key/{dumps,upvoted,playlists,follow/users,follow/playlists,invitees,avatar} and by path /api/users/:key/{dumps,playlists}/:slug, /api/dumps?url= and ?category=, /api/dumps/:id/{file,thumbnail,metadata}, /api/playlists/:id/cover, /api/remote/{preview,image,bandcamp,youtube} (one router), and /api/unsubscribe, the v3 POST /api/users/unsubscribe staying for the digests already sent. A stored default feed tab "followed" reads as "follow" once. dumps.slug and playlists.slug stay nullable, since SQLite can't add NOT NULL without rebuilding tables others reference; every writer sets them and unique (user_id, slug) indexes hold them. 254 backend tests (new: migration 0021 on a v3 fixture, slug aliases, swaps and rate limits, username renames, aliases and limits, reads by path and their privacy, the OG 301s, both unsubscribe paths, v4-shaped digest links with no ids, notification joins, shared paths and slugs) and 668 frontend tests (new: the profile's tabs and follow views, Settings, Manage, the link field, canonical URLs on the dump, playlist and profile pages, bare default feed paths) pass, and tsc, eslint, deno check, deno lint, deno fmt and doc:lint are clean; the migration was also run on a copy of the development database. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-06 16:08:59 +00:00
khannurien
fd0d81fe66 v4: the emails' words live in the catalogs alone and their look is read from the site's stylesheets — every email message was written as i18n._({ id, message }), so its English sat in the template and again in en.po, and api/mail/styles.ts held a copy of each theme's palette, fonts and radii that a test chased. The templates now name each message by its id only (i18n._("email.reset.title", values)) and its English is written in locales/en.po, its French in fr.po; the four labels the settings tab and the unsubscribe page share with the emails read the same ids through @lingui/react's Trans instead of restating the English. i18n:extract runs without --overwrite, which would write the id where the English is, and i18n:compile runs --strict, so a language lacking a message fails the build; it caught email.digest.hl.mentionIn, never translated into French. The new api/mail/theme-tokens.ts reads src/tokens.css and src/themes/<style>.css and resolves a token for one scheme (var(), light-dark(), color-mix(), translucent colors laid over the card), and styles.ts names tokens for every color, GeoCities' rainbow, Melange's spice and shadow, Kanagawa's syntax hues, the font stack and the corner radii, keeping only what is the email's own (type scale, title, frame, divider, avatar rounding). The copies had drifted: GeoCities' stack had lost Marker Felt and its light emails drew the dark rainbow, which now also recolors its dividers on the "auto" scheme. The pills take the site's badge color, now the --color-badge-bg token that the private badge reads too. The Dockerfile copies src/tokens.css and src/themes/ into the runtime image, which reads them at load. The catalogs were re-extracted, having been stale against the code. 240 backend tests (new: the token reader, the dark rainbow; the palette-copy test is gone) and 661 frontend tests pass, and tsc, deno check and doc:lint are clean; sendDueDigests' empty-digest test fails now and then, as it did before. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-05 18:21:45 +00:00
khannurien
7960d74c55 v4: weekly digests and monthly recaps go out by email, every email is written once as Markdown in the app's one catalog, and a review of the last two commits removes what they stated twice — users get a weekly digest (what happened to them, what the people and playlists they follow posted, the week's top dumps, the community) and a monthly recap (the month's top dumps and the community), both on by default, in their own style and scheme. digest-service sends what is due every quarter of an hour from digest_schedule (migration 0020), which holds each user's rolling 7- or 30-day period; existing users' first periods are spread over the coming week or month, a digest more than a period late covers the last full one, an empty one isn't sent, and an address like admin@localhost is skipped. Each digest carries a one-click unsubscribe (List-Unsubscribe and List-Unsubscribe-Post, POST /api/users/unsubscribe) and a link to the new /unsubscribe page, which asks before turning that cadence off, since mail scanners open every link; the token (lib/jwt) names the user and the cadence and never expires. The welcome email now says the digests are on. The emails' words moved out of a hand-written api/mail/strings.ts into the Lingui catalogs the interface uses: lingui extracts api/mail/ too, the catalogs moved to a top-level locales/ that both halves read, compile emits ES modules the server imports (api/mail/i18n.ts), messages take ICU plurals, and the labels the settings tab, the unsubscribe page and the emails share are one message each (their French had already drifted apart). Each template writes its email once as Markdown, which is the plain-text part, and EmailKit.render draws its blocks in the style (a lone link is a button, a list item starting with an image a thumbnail row, a numbered list a ranking, --- the footer), so no string is declared for an HTML and a text version; user text is escaped for Markdown, which the welcome body didn't do for usernames, whose _ read as emphasis. paint() set colors before a border-top shorthand that reset them, so every divider was black. The review also gave one home to what was written twice: the locale list (shared/locale.ts, read by lingui.config, the client, the server and its catalog loader), the base style and default scheme (shared/theme.ts), an email's look (lookOf, which digest-service, welcome and reset share), whether email is on (EMAIL_ENABLED, read by email-service and index.html), a setting's column and its default in the digests' SQL (SETTINGS_COLUMNS, DEFAULT_SETTINGS), the unsubscribe token (made once, not parsed back from a URL), AuthProvider's compare-and-store, ThemeProvider's twin style and scheme code (useThemeChoice), the test JWT encoder, and duplicate digest messages. The email palettes stay a flattened copy of the themes' tokens, now under a test that caught Times' secondary and link colors drifting. The Dockerfile copies api/mail and locales so the build extracts the emails' messages and the server finds its catalogs, and deno.json maps @lingui/core for the runtime image. 236 backend tests (new: digest building, sending, catch-up, scheduling and unsubscribing; the Markdown renderer; French from the catalog; palettes against themes; the stagger migration; unsubscribe tokens) and 661 frontend tests (new: the unsubscribe page) pass, and tsc, eslint, doc:lint, deno check, deno lint and deno fmt are clean. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-04 22:14:46 +00:00
khannurien
89014639cb v4: settings are stored on the server, starting with the style, the color scheme and the digest switches — every setting lived in the browser's localStorage, so it didn't follow an account to another browser and the server couldn't read it, which the digests need (whether to send each, and in which style). user_settings (migration 0019) holds one row per user who changed anything, the defaults otherwise, and GET/PATCH /api/users/me/settings read and change it (settings-service, model/settings, validators/settings, shared/settings). AuthProvider reads the settings into the session alongside the user, on mount and on focus, so a change made elsewhere arrives; useSettings changes them at once and puts them back if the server refuses. ThemeProvider now sits inside AuthProvider: a signed-in user's choice comes from their settings and is mirrored to localStorage for index.html's first-paint script, a visitor's stays in the browser, and an account with no choice yet takes the one this browser had, once. Until anyone picks a style, the instance's GERBEUR_DEFAULT_STYLE applies, written into index.html's default-style tag along with email-enabled (src/config/instance.ts), and a browser no longer stores the default as if it had been chosen, so changing the instance's style reaches everyone who never picked one. The settings tab gets an Email section with the weekly digest and monthly recap switches, disabled with a note when the server has no SMTP. The style and scheme names moved to shared/theme.ts in the previous commit; the default feed tab and sounds stay in the browser for now. 223 backend tests (new: settings defaults, patches, isolation, refusals and cascade; index.html's new tags) and 658 frontend tests (new: useSettings, the account's theme winning and a browser's carried over, settings read into the session, the digest switches) pass, and tsc, eslint, doc:lint, deno check, deno lint and deno fmt are clean. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-04 21:20:57 +00:00
khannurien
f20ca83f0c v4: every email is rendered by its template in api/mail/ on one layout that takes the instance's style, and sent one way — the welcome email was built and sent inside the register route and the reset link in its service, each writing Markdown, running marked, and calling a nodemailer wrapper whose cc, bcc, replyTo and send result nothing used. Now api/mail/ holds one template per email (welcome.ts, password-reset.ts) on layout.ts's EmailKit, which draws each block with inline styles in a style from styles.ts: the six site styles' palettes, copied from src/themes/, and what makes each recognisable in a mail client (Brutalist's hard border and shadow, GeoCities' ridge and rainbow rules, Times' underlined ink links, Kanagawa's colored quotes, dates and wordmark, Melange's spice bar). On the "auto" scheme the light palette is inlined and the dark one swapped in by a prefers-color-scheme block; a chosen scheme sends only its own. User-written text is escaped where it lands. strings.ts holds every email's words in English and French. GERBEUR_EMAIL_LOCALE picks the language and GERBEUR_DEFAULT_STYLE the instance's style, which the welcome and reset emails are drawn in; GERBEUR_WELCOME_EMAIL_BODY still replaces the welcome's Markdown, whose blocks now take the style too. email-service sends a rendered email with its recipient and headers, refuses to load when GERBEUR_SMTPS_URL is set without GERBEUR_FROM_EMAIL (sends used to fail one by one), and takes a test transport, through which the tests now read what was sent. The welcome send moves to user-service's sendWelcomeEmail. The style and scheme names move to shared/theme.ts, which the server now needs. README, mod.ts and .env.example describe mail/ and the new variables, and drop the "notifications, invites" email that never existed. 221 backend tests pass (new: the layout's schemes, styles and escaping, both templates, the welcome on sign-up, a reset link from the email that works), and deno check, deno lint, doc:lint, deno fmt, tsc and eslint are clean. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-04 21:11:54 +00:00
khannurien
5f0cf86fe6 v3: the dump page's "In collections" list only shows the owner's private badge — every card there said "public", which says nothing on a list of playlists holding the dump. PlaylistCard takes privateBadgeOnly, passed through PlaylistCardList, which hides the visibility badge except on a private playlist of the signed-in user's; the other lists of playlists keep both badges. 5 PlaylistCard tests pass (new: privateBadgeOnly), and tsc, eslint, doc:lint and deno fmt are clean.
All checks were successful
Build and Publish Docker Image / build-and-push (push) Successful in 53s
Details
Build and Deploy Static Documentation Site / build-and-push (push) Successful in 22s
Details 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-30 21:19:06 +00:00
khannurien
f3f07ed1e3 v3: every doc opens with the role, then the mechanism, then the details — the docs were written bottom-up: what a thing does first ("Declares the three contexts WSProvider fills…"), details that only make sense once its purpose is known next, and the reason it exists last ("…so that a feed doesn't re-render"), so a reader had nothing to hang the first sentences on. The README's rule caused it, since a module doc "opened with what the module does"; it now opens with the module's role in the app, and the value-side phrasing of a type ("Returned to Y by X") comes once the role is said, with WSContext as the before and after. WSContext.ts is rewritten that way throughout, and so are the docs elsewhere that buried their purpose in a trailing "so that" (the migrations, og, image-proxy, public-address, useFading, useFeedCache, the drafts), opened on a generic verb that hid it ("Holds", "Declares", "Loads", "Defines", "Types": the contexts and providers, the shared types, the models, the hooks), or were label fragments ("The page of a dump…", "A row of tabs."). A function whose verb is its action ("Stores session"), a props or result type, and a protocol message ("Sent by X when Y") already say their role and are left as they are. Only comments changed, and the catalogs only move their references; deno fmt, deno lint and doc:lint are clean.
All checks were successful
Build and Publish Docker Image / build-and-push (push) Successful in 49s
Details
Build and Deploy Static Documentation Site / build-and-push (push) Successful in 22s
Details 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-29 21:42:08 +00:00
khannurien
2af64721c7 v3: the invite tree spaces every row the same — a child sat 0.2rem under its parent but siblings 0.4rem apart, since each row's spacing was split into top and bottom padding and a last child dropped its bottom half. Every row now has one gap above it (--invite-tree-gap), the connectors run through it from the same variable, and the user link is a block, so a theme's line height can't push the avatar off the connector's center. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-29 21:28:43 +00:00
khannurien
07ac4bc629 v3: every paged list loads its pages one way, PlaylistDetail is split like the profile, and the comments keep the README's rules — Index, FollowedFeed, useUserDumpFeed, UserPlaylists, UserListPopover, Notifications, Search and the profile's followed users each hand-wrote "fetch the next page, append it, clear loadingMore on failure". A list the API pages is now a PagedList whose next page useLoadMore appends, and a list that loads on its own goes through usePagedList, which is useFetchState (now taking an initial value, as a restored cache, that starts loaded) plus useLoadMore. Pages are read by pageUrl with readPage (a refusal throws) or readFirstPage (a refusal is empty, as fetchFirstPage was), signed-in reads go through authFetch, and useFeedCache's saveState takes the list. Notifications keeps the socket's arrivals as their own list merged at render, as Index does with recentDumps, and marks them read in one effect. PlaylistDetail (963 lines) only puts the pieces together (146): usePlaylist holds the playlist, its live sync and its edits, PlaylistHeader, PlaylistEditForm and PlaylistDumpList are one file each, and LiveDumpCard is the vote wiring DumpCardList and the playlist both use. What it re-invented is shared: the load is useFetchState, a dump going private and back is usePositionAwareSync, deletes and edits are useDumpListSync (which also keeps comment counts, which the page didn't). A removal is sent when its fade ends, or as the page is left, so an undo sends nothing: the removal/re-add/reorder dance, the pending-undo order and the 80 ms debounce of its echoes are gone. A refused reorder goes back to the server's last order and says so, instead of reloading the page. The comments written since the documentation pass broke its rules in places: module docs as a label and a colon ("The "dumps" tab of a profile: …"), noun-phrase fragments on types, fields and constants, stale lists of callers, over-long lines; they are sentences now, and the ones the refactor made wrong (PlaylistDetail as useFading's and FollowPlaylistButton's caller, "the profiles" as usePositionAwareSync's only users) say what is true. 650 frontend and 214 backend tests pass (new: usePagedList, useFetchState's initial, pageUrl/readPage, the deferred removal, a dump returning at its place in a playlist), and tsc, eslint, doc:lint, deno check, deno lint and deno fmt are clean.
All checks were successful
Build and Publish Docker Image / build-and-push (push) Successful in 49s
Details
Build and Deploy Static Documentation Site / build-and-push (push) Successful in 22s
Details 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-29 21:19:33 +00:00
khannurien
69d9bf899b v3: the profile page is split by one rule, and what it re-invented is shared across the app — UserPublicProfile was 1,949 lines because it held its data, six tabs, seven subcomponents and hand-rolled copies of things the app already had or repeated elsewhere. The page now only picks the tab and puts the pieces together (253 lines); useProfile holds the user and the three lists with their live sync, cache and scroll; each tab and header piece is one file under components/profile/; the followed and invitees tabs load on first open through the new useFetchState, keyed by username, which keeps them across tab switches. Every piece taken out replaces its copies: DumpCardList, which reads the votes from WSContext, is every list of dump cards (the index feeds, followed, search, related dumps, UserDumps, UserUpvoted, the profile), so Index no longer threads vote props into the feeds; PlaylistCardList every list of playlist cards; FeedFooter the eight sentinel/loading-more/end blocks; ListState the loading/error/empty chains; Segmented the settings toggles, RoleSwitcher and the playlist edit form, with SegmentedField built on it; LoadMoreButton the three load-more buttons; NewDumpButton UserDumps' and the profile's; useScrollRestore four copies of the scroll restore, and the profile uses useScrollSave instead of saving its whole cache on every scroll. useFading takes an onFaded callback, which replaces PlaylistDetail's copy of its timers; useUpvotedList is the vote tracking UserUpvoted and the profile each had, and a withdrawn upvote now leaves the list and the cache once faded (it was only hidden, and came back after a tab switch). usePlaylistMemberships is the membership load and toggle AddToPlaylistModal and DumpCreateModal each had. pagedList.ts (PagedList, firstPage, restoredList, fetchFirstPage, withFirst), fetchPublicUser and dataOr are shared by the profile, UserPlaylists and useUserDumpFeed; a refused side list still shows empty, as documented, and only the user fails the page. FollowedFeed's two sections are one local hook, and its readPage is expectOk. A card is its owner's when the item is the signed-in user's, everywhere: the profile's upvoted column marked every dump as the owner's on one's own profile. No message changed; the catalogs only move their references. 643 frontend tests pass (11 new for useFetchState, useFading's onFaded and pagedList), and tsc, eslint, doc:lint, deno check and deno fmt are clean.
All checks were successful
Build and Publish Docker Image / build-and-push (push) Successful in 48s
Details
Build and Deploy Static Documentation Site / build-and-push (push) Successful in 22s
Details 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-29 20:50:49 +00:00
khannurien
80f3292c8f v3: fix the frontend's findings, and the backend pieces they needed — each by a rule the code now keeps everywhere. Votes and likes go through one useOptimisticToggles in WSProvider: clicks on one item before any ack share one baseline, which a failure or the timeout restores (a second click used to cancel the first one's rollback), only the last ack's count is applied, a votes_update or comment_likes_update waits while this tab has toggles in flight, and a welcome reconciles them against the server's sets (a vote cast offline left its count one too high). lastLikeEvent, which nothing read, is gone. Follows are refetched when the new follows_changed message, sent to the follower's sockets on every follow write, says another tab changed them; a failed load keeps the Follow buttons disabled and is retried 10 s later instead of leaving them disabled for good (success: false) or enabled over empty sets (network error); follow and unfollow resolve to whether they worked, and the button says when they didn't. CategoriesProvider follows the same rule, so a network error no longer turns every category URL into a 404. Every list is kept by one rule: useDumpListSync and usePlaylistListSync never add an item, and usePositionAwareSync remembers the place of any item that leaves a list, whatever took it out, and puts it back when its next event shows it again; with dump_created now setting lastDumpEvent, a dump going private and back returns to its place on another user's profile, which never happened before, and an updated playlist missing from a list is no longer prepended. useFeedCache keys every cache by viewer (the profile pages restored the owner's private dumps to whoever went back) and reads it again when its key changes (another profile in the same page showed the previous one's list). Every response of a list's first page or next pages goes through expectOk (Index, UserPlaylists, useUserDumpFeed and Notifications read body.data unchecked).
All checks were successful
Build and Publish Docker Image / build-and-push (push) Successful in 55s
Details
Build and Deploy Static Documentation Site / build-and-push (push) Successful in 22s
Details 
Smaller fixes: the redirects from / and /<slug> keep the navigation state, so the index hides the dump DumpEdit just deleted; Index's mainFetchDone ref, which blocked nothing, is gone; DumpEdit refuses someone else's dump up front, as the API does; the Dump page takes a handed-over dump once and clears it from the history entry, so a reload fetches; a newly upvoted dump is fetched with the viewer's token; the notifications page sends read-all once per notification; the profile's followed and invitees tabs finish loading when left mid-fetch; UserListPopover no longer closes when its own list scrolls; the vote and like counts open their lists for visitors too (a disabled button swallowed the click; they are aria-disabled now); the removal of a playlist's dump forgets it (an edit brought it back), a failed removal puts it back where it was, an undo waits for the removal in flight, and a refused undo says so; PlaylistCreateForm tells whether the dump made it into the new playlist; the cover preview's object URL is revoked; the muted icon has its cross; the bell no longer stays rung after two notifications within 700 ms; a stale invite check can't overwrite a newer one; a stored theme or color scheme is checked; the default feed tab is one store every reader follows, across tabs; the file fields enforce the size they state, derived from DUMP_MAX_FILE_SIZE_BYTES; formatBytes, friendlyFetchError and ImagePicker are translated (Intl names the units), and ImagePicker ignores keys while uploading; visited dumps and playlists keep a week of visits and are parsed once; a chime is claimed under a Web Lock, so two tabs can't both play it; a date in the future reads as "now". Dead code goes: PageShell's centerSlot, overLimit, the exports of isSafeHex and hueFromString, the comment tree's token prop and ChatContext's isChatOpen and closeChat. The French catalogue has the 11 new messages. 214 backend and 632 frontend tests pass; tsc, deno lint, ESLint and doc:lint are clean.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-29 20:08:00 +00:00
khannurien
a36a4cac6b v3: fix the backend's correctness findings — each by a rule shared across the code rather than one more special case. Link previews: readTextCapped decoded every page as UTF-8, so a Latin-1 page came out garbled unless it went through the curl fallback, which re-encoded it on its own; pages are now decoded in the charset their Content-Type or <meta> declares, and the curl fallback hands the bytes over as they are. The image proxy served and cached for a day an error answer that claimed an image type, and read a body without Content-Length whole before checking its size; it now refuses an error status and reads through the same capped reader. The icons task's Twemoji fetch gives up after 15 s instead of holding the start up, and extractLinkHref, which only its tests used, is gone. Backlinks matched a linked URL against the raw url column, so https://x.com/ missed a dump posted as https://x.com; they now compare canonical URLs, as reposts do. The four LIKE searches put the query in raw, so % and _ were wildcards and the mention search for "a_b" found "axb"; lib/like.ts escapes them. Every list query now treats a malformed row the same way, a 500 through model/rows.ts's parseRows, where ten of them silently dropped it; the playlist dumps query uses the model's column list rather than a hand-written copy, and the aliased list is derived from the plain one. The vote and like services commit through transaction() instead of their own BEGIN and ROLLBACK, which would have rolled back a caller's transaction. 
Validation: the dump validators refuse the nulls their shared types don't allow (a null title or note on creation, a null URL on edit); a playlist title of spaces is refused and titles are stored trimmed; usernames and emails are checked by the same USERNAME_RE and EMAIL_RE in shared/validation.ts on sign-up, on PATCH /api/users/me (which checked other limits) and in the sign-up form. Mentions are read by shared/mention.ts's mentionedUsernames, which skips code and URLs as Markdown shows them: https://mastodon.social/@bob or a `@bob` in code notified bob; the editor's suggestions derive their trigger from the same pattern instead of a copy. Following an unknown user is a 404 rather than a foreign-key 500, and following a private playlist a 404 rather than a 403 that confirmed it exists. `ws` and `emojibase` are reserved category slugs, since the server answers them before the app. GET /api/users/me no longer turns every error into a 500.

Dead code goes: adminOnlyMiddleware, verifyEmailTransport, listUsers, deleteUser with disconnectUser and the force_logout message only it sent, PATCH /api/notifications/:id/read (with markOneRead), isSlugAvailable, the unreachable 500 in fetchComment, and the `if (!ctx.state.user)` checks behind authMiddleware. 214 backend and 627 frontend tests pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-29 19:35:53 +00:00
khannurien
14e3ab125b v3: fix the privacy and security findings of the comment pass — private dumps' broadcasts, mentions and attachments reached people who can't see them, and a few smaller holes. A dump's every write now goes through ws-service's broadcastDumpChange(before, after), which decides the audience: a private dump's creation, edits and deletion reach its owner's sockets alone (its deletion id went to everyone before), going private is dump_deleted for everyone else and dump_updated for the owner (whose other tabs dropped it), going public dump_created for everyone else. Mentions in a private dump, a comment on one, or a private playlist's description notify no one until it goes public (mentions are deduplicated per context, so going public notifies them once), and a moderator's edit of a comment, chat message or playlist mentions in the author's name. A link preview of a private playlist is refused, as one of a private dump already was: the preview is stored in the linking dump, which others may see. Attachments record the type of the resource that links them (migration 0018 backfills it); one linked by a private dump or playlist, or a comment on a private dump, is served only with its owner's media token, which Markdown now adds to attachment URLs, and one whose resource is gone answers 404 and is purged at startup. The Bandcamp provider claimed bandcamp.com.evil.example (the pattern wasn't anchored) and now reuses bandcamp-tralbum's anchored one; the YouTube provider took any site's ?list= for a YouTube playlist and missed m., music., youtube-nocookie and /live/ links, and now shares canonical-url's rules. Sign-in does the same scrypt work for an unknown username as for a wrong password, so timing no longer tells which exist. Usernames and emails are unique regardless of case (migration 0017, which refuses to run over existing clashes and names them): "Bob" and "bob" could both sign up and a mention picked one, emails had no constraint, and a reset request in another case silently sent nothing. The Open Graph tags escape the site name and build their URLs on PUBLIC_URL and FRONTEND_URL instead of the request's origin, which a proxy may rewrite. 
Error statuses follow one rule: APIException takes a status and derives the code from it (BAD_REQUEST 400, UNAUTHORIZED 401, FORBIDDEN 403, NOT_FOUND 404, CONFLICT 409, RATE_LIMITED 429, SERVER_ERROR 500, BAD_GATEWAY 502), so a code can't contradict its status. That fixed real bugs: a wrong current password on change-password answered 401, on which authFetch signs the user out; editing or deleting someone else's comment or chat message answered 401 rather than 403; one invalid dump edit answered 422. A taken username is 409, like a taken email.

Along the way: the database is created from schema.sql by connection.ts when the file doesn't exist, so every task works on a fresh checkout (dev and serve never ran db/init.ts, which is gone); the startup purge compared ISO dates with SQLite's datetime() as text and kept anything from the current UTC day, and now compares ISO cutoffs; the seeded admin's hash comes from lib/password.ts (split out of jwt.ts, which needs the JWT secret) rather than a copy of its format; avatar, thumbnail and playlist-cover uploads go through replaceUploadedFile, which writes beside the old file and renames over it before the record announces it (a failed avatar write lost the old avatar; a thumbnail was announced before it existed); upload.ts derives its size message from MAX_IMAGE_SIZE_BYTES and stops re-exporting the upload folders; ALLOWED_IMAGE_MIMES, which duplicated detectImageMime, is gone; the SPA fallback no longer calls next() after answering; a reply must answer a comment on the same dump (a foreign or unknown parent was a 500 or a cross-dump thread). 211 backend and 628 frontend tests pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-29 15:19:07 +00:00
khannurien
0d4e344d77 v3: comments say what the code can't — the documentation pass wanted JSDoc on every export and every field of an exported interface, which filled the code with restatements ("The dump.", "Whether it is loading.", "The props of X.", "Renders the X.") and module docs that told a reader nothing ("The hot and new feeds, and the props every main feed of the index takes."); some were wrong as well as vague. Every comment in api/, shared/ and src/ is rewritten under one rule, now stated in the README's "Comments" section: every module and export still has a JSDoc, and it gives context (who uses it and where), usage (what a caller must know: a precondition, what it leaves to the caller, a side effect) or its place in the chain (where its input comes from, where its output goes); a field, parameter, private function or local gets one only when it adds to its parent's, and context about a whole type goes on the type; a type's doc is phrased from the value's side ("Returned to Y by X when Z", "Passed to Y by X"); full sentences, as short as they can be, no label-colon-list; // goes wherever the code isn't obvious and says why, and the section banners and narration go. Every claim was checked against the code, which turned up some forty comments that were wrong: DumpCard's claimed it was only used by the hot and new feeds, the WebSocket welcome claimed to be a socket's first message (presence comes first), UpdateUserRequest said null removes a description (it keeps it), deletedDumpIds leaves out that it also holds dumps made private, THEME_COLOR claimed to mirror index.html's, and the like. The routers' JSDoc now says where they are mounted and what becomes of a thrown APIException. doc-lint.ts lets a field without a doc through; a module or export without one still fails. No code changes; 204 backend and 628 frontend tests pass, and doc:lint, deno check, deno lint and tsc are clean. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-29 14:31:14 +00:00
khannurien
42e0356d21 v3: the docs follow deno doc's grain — the index page was a hand-kept list of some 250 links into deno doc's output layout (./api/main.ts/index.html), policed by mod.test.ts; deno doc has no module index of its own beyond all_symbols.html, which lists every module by path with its summary. mod.ts is now a prose overview of the three parts and their layers, linking that list once, and mod.test.ts is gone. Re-exporting everything from mod.ts, as deno doc's tutorial does for a library, was tried and doesn't fit: export * drops the 18 routers (default exports) and makes 17 names ambiguous, and deno 2.9 renders export * as namespaces with the root's module doc and nested ones empty, losing every module doc. A router's page showed one line: its endpoint list sat on the default symbol, whose summary was cut at the colon introducing it; the list is now the module doc of all 18 routers, and the router's JSDoc says so. 22 summaries ended in a colon (the routers, BandcampPlayer, migration 0016, useDumpListSync, usePlaylistListSync) and read cut short wherever deno doc shows a first paragraph alone; they end in a full stop, doc-lint.ts now fails on a colon summary, and the README states the rule. No behaviour changes; 204 backend tests pass.
All checks were successful
Build and Publish Docker Image / build-and-push (push) Successful in 50s
Details
Build and Deploy Static Documentation Site / build-and-push (push) Successful in 21s
Details 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-28 19:28:24 +00:00
khannurien
563daa720c v3: src/ is documented like api/ and shared/, and the reference covers the whole repository — the frontend had no module docs, JSDoc on some components and hooks and not on others, props typed inline in half the components and in private interfaces in the rest, // comments on declarations, and no return types on 147 exported functions, so deno doc --lint over src/ found 637 errors. The rule api/ follows is applied to every non-test module of src/: a /** … @module */ block, JSDoc on every export and every field of an exported interface, /** */ for declarations and // for notes, verbs for functions ("Renders…" for a component), parameters named in the prose, {@link} only for symbols in scope, and a page's JSDoc names its route and who may see it (App.tsx's module doc lists the route table). For deno doc to show them, every exported function states its return type: components React.JSX.Element (or | null, or a portal), memo and forwardRef components their exotic types, contexts React.Context<…>, and a hook returning several things a documented Use<Name>Result interface (useAuth, useRequiredAuth, useFading, useEmojiTrigger, useMentionAutocomplete, useDumpCardLink, usePlayRichContent; FeedCacheResult, YouTubePlaylistControls and UseApiFormReturn are renamed to match). Every component that takes props takes an exported, documented <Name>Props interface — the 64 inline prop types are lifted into one, DumpFeed gets DumpFeedProps extending MainFeedProps. React types go through the global React namespace: the four default imports of react are gone, since deno doc can't follow them. Two duplicates go too: useMentionAutocomplete's UserResult was shared/'s UserSuggestion, and JournalCard re-exported JournalShape for no one; i18n.ts no longer re-exports Lingui's i18n, which its three users now import from @lingui/core. deno task doc and doc:lint now take src/ (but its tests and the generated locales), the index page gains a Frontend section linking every module by layer and domain, mod.test.ts walks src/ as well, and the docs workflow runs on changes to src/. deno doc doesn't document npm packages, so it takes a type imported from one — react-hook-form's FieldValues in the form fields, frimousse's SkinTone — for a private one; doc:lint now goes through doc-lint.ts, which fails on every diagnostic but those, 25 of them. The README states the frontend's part of the rule. No behaviour changes; 205 backend and 628 frontend tests pass, and tsc, deno check and deno lint are clean.
All checks were successful
Build and Publish Docker Image / build-and-push (push) Successful in 50s
Details
Build and Deploy Static Documentation Site / build-and-push (push) Successful in 22s
Details 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-28 07:31:22 +00:00
khannurien
b85a71d86e v3: shared/ is documented like api/ — deno doc --lint over api/ and shared/ goes from 385 errors to none — every module of shared/ opens with a /** … @module */ block; every export and every field of an exported interface has JSDoc, each request body naming the endpoint that takes it and each response its endpoint; the // comments that described declarations (the mention pattern, the permission union and table, four socket messages, the error message's optimistic-action context) are /** */ now, and the socket protocol's heading comment, which sat on no declaration, is its module doc. The lint wanted explicit types on MENTION_RE and DUMP_MAX_FILE_SIZE_BYTES. Along the way: the chat edit is allowed to moderators too, which the new ChatEditMessage doc says; UpdateUserRequest says that PATCH /api/users/me takes only its description and email; playlist-service's reorderPlaylist @throws is rewrapped. mod.ts and the README no longer call shared/ undocumented. No behaviour changes; 205 backend tests pass. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-27 21:13:07 +00:00
khannurien
d62e4a9883 v3: the backend is documented for deno doc, one way in every file, and the docs build and publish themselves — comments were in three shapes: file headers as // lines, as an untagged /** */ (which deno doc drops, only an @module block counts) or missing; 43 exports described by a // deno doc can't see, two JSDoc blocks sitting on the wrong declaration and one duplicated; endpoints commented in 5 of 18 route files, three ways, on router.get() calls no doc tool reads. The rule, applied to every non-test module of api/, migrations included: each opens with a /** … @module */ block; every export has JSDoc, and so does every field of an exported interface; a comment describing a declaration is /** */, // only notes code; functions open with a verb, everything else with a noun phrase; parameters are named in the prose, {@link} only for symbols in scope; whatever throws to its caller says so with @throws (for an APIException, the statuses and when); a router's JSDoc lists its endpoints, method and path, who may call it, what it does. deno doc --lint goes from 528 errors to none in api/ (shared/ keeps its 385; it is next). The lint also wanted explicit types on config.ts's settings and a few return types, JsonLdResult exported, and the routers typed Router. deno task doc builds the HTML reference into docs/ from every module of api/ and shared/ but the tests, deno task doc:lint lints it; the index page is the module doc of a root mod.ts (deno.json's exports), which introduces the backend, shared/ and the frontend to come and links every module, and mod.test.ts fails when one is missing. The docs workflow builds it on every push to main that touches them and publishes it to the gh-pages branch, like publish-assistant's. Two defects found along the way are fixed: a private playlist's cover was served to anyone who had its id or slug, and now needs its owner's media token, as private dump files do (playlistImageUrl adds it for private playlists only, so public cover URLs stay the same for everyone); og.ts and static.ts each filled index.html's placeholders, and now share lib/index-html.ts, so the OG middleware picks up a rebuilt index.html instead of keeping the first it read. The README states the documentation rule. 205 backend and 628 frontend tests pass.
All checks were successful
Build and Publish Docker Image / build-and-push (push) Successful in 47s
Details
Build and deploy docs / build-and-push (push) Successful in 26s
Details 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-27 21:07:40 +00:00
khannurien
1702370447 v3: coverage tasks for both suites — deno task test:coverage runs the Deno tests of api/ and shared/ with coverage and writes the report to coverage/api, deno task test:web:coverage runs the Vitest suite with @vitest/coverage-v8 (at vitest's own 5.0.2) into coverage/web. Each prints a table and leaves an HTML report and an lcov.info. They write to separate folders because deno test --clean empties its whole output directory, which would take the frontend report with it. The frontend report counts src/ without src/test/ and the generated src/locales/; the backend report still lists api/test/helpers.ts and setup.ts, since deno.json has no coverage exclude and a separate deno coverage --exclude pass merged the raw profiles into different figures. No behaviour changes. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-27 20:30:05 +00:00
khannurien
d392be7cb8 v3: api/ gets one layout rule and keeps to it, and whatever crosses the wire is defined once, in shared/ — api/model/interfaces.ts and src/model.ts declared the same 40 types by hand and had drifted: the error envelope, email required on one side and optional on the other, Invitee/InviteTreeEntry and LinkPreview/UrlPreviewResponse for one shape each. The validation limits, the role permissions, the chat length cap, the @mention pattern and the Bandcamp and YouTube player responses were copied too, behind "keep in sync" comments. shared/ now holds them once, one file per domain (api, category, chat, comment, dump, follow, mention, notification, permissions, player, playlist, search, user, validation, ws), and imports nothing from either half. The server sends Dates; the client reads Wire<T>, the same type with every Date an ISO string, and turns it back with src/utils/shared/deserialize.ts. APIErrorCode becomes a const object, since the frontend's erasableSyntaxOnly rejects an enum. The shared types showed two real gaps: the avatar upload and GET /api/users/me return the full self user and the frontend read them as public users, and a stored session may predate media tokens, so the client keeps it as its own Session type. The backend rule: each folder holds one kind of module, each file is named after its domain. routes/ parse and answer; services/ hold the logic and every SQL query (lib/auth.ts's token check moves to user-service); remote/ is the only place that fetches from other sites — the guarded fetch and the HTML scraping out of rich-content-service, public-address, the Bandcamp and YouTube resolvers, the link providers, the image proxy (fetch, checks and cache, out of routes/preview.ts) and the Twemoji glyphs (out of lib/site-icons.ts). model/ holds per table a row type, its guard and its mappers; validators/ the request guards; db/connection.ts the handle and transaction(), db/startup.ts the purges and default admin that db.ts ran on import, called at the same moment. Tests mirror the source one file per module, test/<layer>/<module>.test.ts, with shared/'s under shared/test/ (deno task test runs both); the test that covered the bandcamp and soundcloud providers together is two now, and the backend can() has its own, so 202 tests. deno lint src/ is clean too: window.* becomes globalThis.*, and test bodies and act() callbacks that never awaited lose their async. The README states the rule. No behaviour changes; the catalogues only move their source references.
All checks were successful
Build and Publish Docker Image / build-and-push (push) Successful in 50s
Details 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-27 20:19:48 +00:00
khannurien
e9f6a1f23e v3: deno check src/ passes again — the tests import userEvent by name — 71 errors, all the same one: @testing-library/user-event 14.6.7 ships ESM-syntax declarations without "type": "module", so Deno reads them as CommonJS and the default import is the module namespace, on which setup() and click() don't exist. Vite resolves it the other way, which is why the suite still ran. The package exports userEvent by name as well; the 40 test files that imported it use that now. No behaviour changes; the 627 tests pass and the lint is clean. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-27 19:49:49 +00:00
khannurien
2b6e68391b v3: src/ gets one layout rule and keeps to it — each layer holds one kind of module, and every layer is split into the same domains — the previous pass sorted components into folders but left hooks, helpers and a barrel inside them, kept hooks/, utils/ and contexts/ flat, and still grouped most tests by theme. The rule: components/ holds only components (.tsx), contexts/ holds FooContext.ts + FooProvider.tsx pairs, hooks/ every hook, utils/ every non-React module, pages/ one component per route and nothing else. Each of components/, contexts/, hooks/ and utils/ is split into the same domains — auth, category, chat, comment, dump, editor, feed, form, layout, player, playlist, profile, social, theme, ws — plus shared/ for what belongs to no one domain, so the chat is components/chat, contexts/chat, hooks/chat and utils/chat. components/ui becomes components/shared to match. Out of components/: useApiForm goes to hooks/form, chat's message.ts and time.ts to utils/chat (with chatDraft and chatText, now draft.ts and text.ts), the emoji skin tone to utils/editor, the remembered volume to utils/player; player/icons.tsx is PlayerIcons.tsx. components/form/index.ts is gone: everything is imported from its own file, FormProvider straight from react-hook-form, and expectOk sits beside friendlyFetchError in utils/shared/api.ts (apiError.ts), since pages that have no form used it too. Out of pages/: the index's feed views, tab bar and presence row move to components/feed; HotFeed and NewFeed, identical but for the sort, are one DumpFeed with an order prop, which also takes over MainFeedProps from pages/index/types.ts. App is a named export like everything else. Tests mirror the tree one file per module — test/<layer>/<domain>/<Module>.test.tsx — so the themed files (hooks/misc, data, editor, sync; utils/misc, text, storage, remote, journal; contexts/providers; pages/auth and UserSubpages; layout/Fabs) are split by subject, the setup each piece needs kept and the rest dropped; the floating-button test becomes one test per button, 627 tests now. The README states the rule. No behaviour changes; the catalogues only move their source references. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-27 19:44:14 +00:00
khannurien
07d0344be7 v3: src/components sorted into feature folders, the four largest files split, and the component tests mirror the new layout — 48 components sat in one flat folder, ChatModal ran to 1301 lines, DumpCreateModal to 983, and the tests were grouped by theme, so there was no telling which component had none. Components now live in category/, chat/, comment/, dump/ (with dump/create/ for the new-dump wizard), editor/, form/, layout/ (header, page shell, floating buttons), player/, playlist/, profile/, social/ (vote, like, follow, the who-voted popover) and ui/; FeedTabBar and PresenceRow, used by the index page alone, move to pages/index/. ChatModal gives up ChatMessageItem, PendingItem, ChatEditBox and ReplyRef, the message limits and text helpers (chat/message.ts), the day and clock labels (chat/time.ts) and the draft storage (utils/chatDraft.ts, next to dumpDraft.ts). DumpCreateModal gives up StepIndicator, LocalFilePreview, TitleField and DuplicateNotice; CommentThread gives up CommentForm and CommentNode; TextEditor gives up the emoji picker popup and its skin-tone storage; MediaPlayer gives up its icons (which FilePreview imported from it), Waveform, SeekBar and the remembered volume. NewPlaylistForm, a button that opens a modal holding the form, is NewPlaylistButton now, and Thumbnail, ThumbnailPlaceholder, FilePreview and RichContentCard use named exports like everything else. The tests are split one file per component under src/test/components/<folder>/ (and src/test/pages/index/), the same 625 of them. No behaviour changes; the catalogues only move their source references. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-27 19:30:28 +00:00
khannurien
de5abeceb2 v3: a performance pass over both halves — indexed queries, streamed files, cached previews, and feeds that stop re-rendering on every tick — a review turned up whole-file reads, unindexed scans and render storms; each fix comes with a test. Database: migration 0016 adds the indexes the hot queries lacked — dumps by created_at, (user_id, created_at) and slug, votes by (user_id, created_at), playlists by slug, users by lower(username) for @mentions, and the child side of every cascading foreign key (comment replies, a user's comments, follows both ways, chat messages, invites, reset tokens); the feeds, dump and playlist pages, upvotes and follower fan-out all plan as index searches now, with no temp-table sort. SQLite runs in WAL with synchronous=NORMAL and a busy timeout, and a transaction() helper wraps the multi-row writes (a dump and its categories, category sets, backlinks, attachment links, playlist reorders, follower notifications) so they commit once instead of once a row. Lists no longer run a COUNT(*) per page: they fetch limit + 1 rows and hasMore comes from the extra one; total stays only on search, the one place it's shown. Files: /api/files read the whole upload (up to 50 MB) for every range request a player makes while seeking; api/lib/serve-file.ts streams the requested bytes from disk, with an ETag and 304s, and serves avatars, attachments, covers and thumbnails too. File and thumbnail URLs carry the file's version (size and last change), so the current one is cached for a year and anything else revalidates — a same-size replacement used to stay stale, and a video's still was cached immutable forever under a URL that never changed. Feeds showed image dumps at full size: /api/thumbnails now makes a 960px preview with ffmpeg (GIFs and unreadable images go out as they are), one run per output however many cards ask at once, and ffmpeg is probed once per process instead of spawned per request. Uploads stream to disk; replacing a file writes beside it and renames over, a hard link keeping the old one for rollback instead of holding both in memory; generated stills and previews go when the file is replaced or the dump deleted, and a custom thumbnail with its dump. Outbound: fetchWithTimeout's deadline now covers the body — a server that answered at once and then dripped could hold a create open indefinitely — and pages are read to 4 MB at most; metadata is cached five minutes so the create form's preview and the post that follows share one fetch (refresh-metadata still goes to the page); proxied images sit in a 32 MB LRU. Broadcasts serialize once rather than per socket, and index.html is kept in memory until the file changes. Frontend: currentTime lived in PlayerContext, so every card re-rendered — and re-parsed its Markdown — four times a second while anything played; the playhead has its own PlayerTimeContext now. The socket context is split three ways (feed, chat via useWSChat, presence via useWSPresence), so typing, presence and chat messages leave the feeds alone. DumpCard, JournalCard and Markdown are memoized; card images use the server preview and load lazily; the feed cache is written when the page is left rather than after every pause in scrolling; the theme context value is memoized.
All checks were successful
Build and Publish Docker Image / build-and-push (push) Successful in 49s
Details 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-27 16:35:51 +00:00
khannurien
364c498a24 v3: dependencies upgraded and deno audit clean, every API response typed end to end, no explicit any left — and the site search stops handing out password hashes — deno audit reported 19 advisories (11 high); none remain. nodemailer 9 → 10.0.10 closes four of them (a quadratic address parser, two recipient-domain allow-list bypasses, resolveContent ignoring disableFileAccess/disableUrlAccess); 10's one break is Node ≥ 20, and 10.0.11 is a day too young for the minimum release age. react-router 8.0.1 → 8.4.0 closes the RSC-mode CSRF bypass. Refreshing the lockfile clears the transitive ones that came in through Vite, ESLint and Lingui: postcss, nanoid, picomatch, browserslist, brace-expansion and baseline-browser-mapping. Everything else moves to its latest too — Lingui 6.8 (its SWC plugin 6.7), React 19.3, Vite 8.3.1, react-hook-form 7.89, marked, jose, ESLint 10.11, typescript-eslint 8.70, the @types — and frimousse 0.3 → 0.4, whose declarations only add (custom emoji data resolvers, a cache). TypeScript stays on 6.0: typescript-eslint supports < 6.1, so 7.0 would stop the lint. allowScripts names @swc/core 1.16.2 and drops esbuild, which Vite 8 no longer installs.
All checks were successful
Build and Publish Docker Image / build-and-push (push) Successful in 51s
Details 
Search: GET /api/search sent each matching user's full row — password hash and email included — to anyone, signed in or not, because searchUsersGlobal returned User[] and the route passed it through. It returns profiles only now, stripped where the rows are read, and the search test fails on the old code. Types alone wouldn't have caught it (a User is structurally a PublicUser), so the strip is explicit and tested.

Typed routes. Every JSON response is declared as APIResponse<T>, the way the typed routes already did, with the missing shapes named in api/model/interfaces.ts: PublicUser (no hash, no email) and SelfUser (your own profile, with email) plus toPublicUser/toSelfUser in place of ten hand-written destructurings, AuthTokens, AuthResponse, MediaTokenResponse, InviteResponse, AttachmentResponse, SearchResults, LinkPreview, UserSuggestion and Invitee. Request bodies are `unknown` and narrowed by the existing guards, or by a new fieldOf(obj, key) where a route reads one field — no more reading off Oak's `any` or casting to Record<string, unknown>. The password-reset routes answer `{ success: true, data: null }` rather than a bare `{ success: true }`, so every success fits the envelope; the frontend only reads `success`.

Socket protocol: routes/ws.ts had its own send() taking Record<string, unknown>, so nothing checked what it sent. It takes ServerToClientMessage now, which turned up two mismatches: `pong` was missing from the union although the server answers every client ping with one, and vote_ack carried a `success: true` no type declares and no client reads. The union has PongMessage and vote_ack lost the field.

No explicit any. The YouTube playlist parse reads ytInitialData as `unknown` through a small at(node, …path) that checks each step, instead of `type Json = any`. The backend test helpers type the response body as the server's envelope: request<T>() and get/post/put/patch/del<T>() name what `data` holds, res.data throws with the server's message when the call didn't succeed and res.error when it didn't fail, and each test says which API type it expects, so a test reading a field the route doesn't send no longer compiles. Assertions that a field is absent are `assertFalse("passwordHash" in …)` rather than reading a field the type says isn't there. The socket tests' messages are ServerToClientMessage, with next()/all() keyed by type so each returns that message's shape.

No double casts in src/ either. window.YT and onYouTubeIframeAPIReady are declared on Window; the test's fake player is a class forwarding to the spies the tests inspect. WSContext, FollowContext and PlayerContext export their defaults (defaultWSValue, defaultFollowValue, defaultPlayerValue), which the tests import instead of reading React's private _currentValue. handleMentionKeyDown takes a MentionKey — the key fields it reads — so the test builds a plain object; useUserDumpFeed declares sentinelRef as the callback ref it is. In src/test/setup.ts the IntersectionObserver fake implements IntersectionObserver, and report(isIntersecting, top?) builds complete entries in place of partial ones cast in five files; the Animation fake implements Animation; the canvas getContext stub goes in through defineProperty. The XHR fake's last request is optional behind a getter that fails if none was made, the keyframes are checked with matchers, and the roleless legacy session is built by leaving the role out. One test changes meaning slightly: "leaving the view…" relied on entries with no position at all, and now leaves and re-enters at the same height, which is what it describes. The one cast left is in api/test/ws.test.ts: deno.json's "dom" lib hides Deno's WebSocket overload that takes handshake headers, and dropping that lib would break the editor's view of src/; it's commented and names the real signature.

Also: the notifications test answered a Response-typed promise with a plain object, which failed `tsc -b`; it uses the fetch stub's deferred() now. ESLint didn't read the `deno-lint-ignore no-explicit-any` comments, so `eslint .` failed on three files; those anys are gone. `deno fmt` over api/ and src/ (not the compiled src/locales/*.js); the reflow left all 332 message ids intact, and the .po files only have their source line references updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-27 16:05:29 +00:00
khannurien
c8bf0d583c v3: the tests learn to expect trouble, and seven bugs they now catch are fixed — a review of the two test-suite commits found them written mostly for the happy path; each fix below comes with the test that fails without it. Infinite scroll: a later page that failed turned loadingMore back off, which made a new IntersectionObserver, which reports a sentinel still in view at once — so an offline feed asked again as fast as the request could fail, on Search, Index, the followed feeds, user dumps and user playlists alike. useInfiniteScroll now waits 1s, 2s, 4s… up to 30s while the sentinel stays put, counted from the last load so re-renders don't restart it, and asks at once again when the sentinel leaves the view or is pushed down by new items; the fake observer in src/test can report an element in view on observe(), as a browser's does. Playlist covers: the row was updated and playlist_updated sent before the file was written, so a client fetching image?v=<new> in between cached the old cover (or a 404) under the new URL for an hour, and a failed write left image_mime set with no file behind it. The file now goes in first, written beside and renamed over, and only then does the row change. Playlist order: the server wanted every member named, but GET hides private dumps the requester can't see, so a playlist holding someone else's dump that went private — or one a moderator reorders around the owner's private dumps — could never be reordered. The requester now orders what they can see and hidden members keep their slots; the visibility rule lives in api/lib/playlist-members.ts, shared by GET, reorder and the socket. playlist_dumps_updated also sent every member's id, private ones included, to a public playlist's whole audience; each viewer now gets the ids they can see, plus a dumpCount of all members so playlist cards keep the server's count. File dumps: replacing the file and changing visibility took two requests, so a failed second one left the new file live under the old visibility; PUT /api/dumps/:id/file takes isPrivate and applies both in one UPDATE, and it now broadcasts like any edit (it sent nothing, so other viewers never saw a new title). Notifications: one that arrived while the list was loading was dropped, then marked read by the read-all after the load, so it never showed; it's kept and merged in, once. Ranges: bytes=0-0,5-6 got a 416; several ranges, another unit or an unparseable header are ignored now and the whole file sent with a 200, as RFC 9110 allows — the old test asserted the 416.
All checks were successful
Build and Publish Docker Image / build-and-push (push) Successful in 49s
Details 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-27 15:13:57 +00:00
khannurien
54e86c4366 v3: the frontend gets a test suite, and the bugs it turned up are fixed — deno task test:web runs 612 tests over src/ in about 11s: Vitest on jsdom with Testing Library, built on the app's own Vite config so the SWC/Lingui transform and the .po loader are what ship. src/test/ holds the harness: a routing fetch stub (queued once-responses, fallback defaults), a FakeWebSocket the tests open, feed and drop, jsdom polyfills, fixtures, and two render helpers — one over hand-set context values, one over the real provider tree (auth, socket, player, follows, categories, chat). It covers every util and hook, every provider (the socket protocol end to end: votes and likes with ack/timeout/error rollback, chat sends/echoes/retries/read markers/reconcile, reconnect backfill), the components (header and search history, cards, Markdown, thumbnails, media and global player, the text editor's mentions/emoji/uploads, comments, the new-dump modal, the chat), and every page.
All checks were successful
Build and Publish Docker Image / build-and-push (push) Successful in 48s
Details 
Fixed along the way. Dumps: clearing a dump's note sent `comment: undefined`, which JSON drops, so the server kept the old one; it's sent as null now. Replacing a file dropped a visibility change made in the same save (the file endpoint doesn't read it); it follows as a separate update. Renaming a dump onto a new slug kept showing the old title until reload. The thumbnail picker and delete on the edit page failed without a word, the latter as an unhandled rejection. Playlists: an edit that kept the slug waited on the socket's echo to show, and a new cover never appeared (the upload's answer was ignored, a refused one said nothing); covers were also served stale for an hour, since `max-age=3600` met an unversioned URL — setPlaylistImage now bumps updated_at and every cover URL carries `?v=` (playlistImageUrl). Removing a dump, reordering, and deleting a playlist ignored non-OK answers, leaving page and server apart; they restore, reload or report. Profile: clearing "Who am I?" sent undefined and came back on reload; your own upvoted list (and the upvoted page) was empty until the socket's welcome, and forever without one; "View all" under Following linked to a route that doesn't exist and is a Load more now. Chat: Escape on "Delete this message?" closed the whole chat — ConfirmModal takes Escape first, in the capture phase. Player: a pause or play from a media key, headset or the OS wasn't reflected (MediaPlayer follows the element's own play/pause events), and the seek bar tracked dragging with mouse events only, so on touch the thumb fought timeupdate. Feeds: a guest on /~/followed got a blank page; a server error in the followed feed read as "Could not connect". Search: a failed later page threw away the results shown. Notifications: a failed mark-read replaced a loaded list with an error. Silent failures now say so: deleting a comment, the voters/likers "Load more" (stuck on "Loading…"), and the password-reset request, which said "sent" whatever happened.

Notifications also stop hiding what was new: the bell still clears on opening the page, but the dots and the "N new" count stay for the visit instead of vanishing the moment the mark-read lands, and one arriving while the page is open is marked read too, so the bell doesn't count it.

Accessibility: Modal and ConfirmModal are dialogs (role, aria-modal, labelled/described); playlist membership rows are checkboxes that work from the keyboard; a dump's title, and the profile's email and description, can be opened for editing from the keyboard (the pencil shows on focus); the category editor's ✓ and the global player's video button have names. Player, rich-content and download labels were hard-coded English and are translated, with the new French strings. Lint: fast-refresh and render-purity rules are off for src/test/ only, where probes hand their context out on purpose.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-27 14:35:41 +00:00
khannurien
25601edaf8 v3: the backend gets a test suite, and the bugs it turned up are fixed — deno task test runs 186 tests over api/ in about 18s, against a throwaway database and uploads directory (GERBEUR_DB_PATH and GERBEUR_UPLOADS_DIR now override the defaults; api/test/setup.ts sets them before any app module loads), with requests sent through the whole middleware stack in-process via app.handle, outbound fetches and DNS stubbed, and the socket tests on a real Oak listener since an upgraded request has no response for handle() to return. It covers auth (token purposes, revocation after a password change, live roles, login rate limiting, invites claimed once even under parallel registrations, password resets), dumps end to end (create, privacy, by-url matching on the canonical key, backlinks, categories, edits, file replacement, thumbnails, refresh, delete), file serving headers and ranges, attachments, avatars, comments and likes, playlists, follows and feeds, notifications and mentions, categories, search, chat cursors and read markers, link previews and the SSRF guard on every redirect hop, the Bandcamp and YouTube resolvers, migrations (a fresh schema is left untouched; the data rewrites in 0004/0005 and 0010–0013 do what they say), CORS, OG tags, static caching, and the WebSocket protocol. Fixed along the way: an unknown category id on a new dump returned 400 but kept the dump row, and for a file dump deleted the file while keeping the row; an edit or file replacement with one was half-applied before the 400 — categories are now checked before anything is written. A suffix range (bytes=-3) was read as an end offset and got a 416 instead of the last three bytes. Reordering a playlist accepted a list naming a member twice. verifyPassword threw on a stored hash without a salt instead of answering false. And a request with no Accept-Encoding, which RFC 9110 lets a server treat as accepting anything, was sent the brotli sibling; curl and bots now get the plain file.
All checks were successful
Build and Publish Docker Image / build-and-push (push) Successful in 46s
Details 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-27 12:31:05 +00:00
khannurien
16b510764c v3: the frontend ships compressed, caches for good, and stops reaching for other people's servers — a review of the bundle's size found the server was the biggest cost. Oak's send() compressed nothing and sent every hashed chunk with max-age=0, so a first visit to the home page moved about 710 KB of JS and CSS where 200 KB would do, and every later one revalidated some thirty files. The build now writes .br and .gz next to every JS, CSS, JSON and SVG file (a precompress plugin, brotli 11), which send() already serves when the browser accepts them; /assets and /emojibase are cached for a year as immutable, index.html is no-cache, and static responses carry Vary: Accept-Encoding. A missing file under either prefix is a 404 rather than index.html with a 200, which an open tab after a deploy used to try to run as a script; the app now reloads on vite:preloadError, at most once a minute. manualChunks sent everything in node_modules to one vendor chunk, so react-hook-form and frimousse loaded with every page though only the lazy form and editor chunks use them, and its markdown rule missed hast, vfile, unified and property-information while every feed loads Markdown through DumpCard anyway; only react, react-dom, scheduler and react-router get a chunk of their own now, the rest is left to the bundler. Up-front JS goes from 133 to 96 KB gzip, the home page from 178 to 160. Fonts: index.html blocked rendering on a Google Fonts stylesheet, opening two more hosts and handing each visitor's IP to Google; they come from @fontsource now, with src/fonts.css declaring only the latin and latin-ext faces (the packages' own CSS declares every subset, which added 6 KB gzip to the stylesheet), so Vite hashes the woff2 files into /assets and a browser still only downloads its theme's font. The families are the variable builds ("Saira Variable" and so on); Fraunces keeps its opsz, SOFT and WONK axes. Emoji: frimousse fetched emojibase-data@latest from jsDelivr, unpinned and third-party; emojibase-data 17.0.0 is a dependency now, its data and messages for each lingui locale are emitted under /emojibase/17.0.0 (served from node_modules in dev), and the picker is pointed there.
All checks were successful
Build and Publish Docker Image / build-and-push (push) Successful in 45s
Details 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-27 11:44:08 +00:00
khannurien
bf28e49c34 v3: left and right step through emoji results too — with a :query picker open, only up and down went to the grid; left and right moved the caret in the textarea, so the first press of either did nothing visible. Every arrow now steps into the grid, landing one along from the automatic first match; Escape gives the caret back.
All checks were successful
Build and Publish Docker Image / build-and-push (push) Successful in 43s
Details 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-27 11:29:44 +00:00
khannurien
85d0703077 v3: the editors stop fighting the person typing — a review of every edit box turned up an Enter that sent a chat message mid-IME, an emoji picker that grabbed focus on ":D" and never went away, markdown that folded every line break, and limits that were counted but not kept. TextEditor: an Enter confirming an IME candidate no longer also sends, saves or picks. The : trigger wants two letters (Unicode ones too), not one, so emoticons, "note:" and "12:30" type through, and it keeps focus in the textarea — the picker's search follows what's typed, Enter or Tab takes the best match (a name equal to the query, then one starting with it, rather than frimousse's first in emoji order, which made ":rocket" an astronaut), the arrows step into the grid and typing there carries on in the textarea, and Escape closes just the picker, not the chat around it. It closes on a press outside, which it never did, and neither autocomplete fires inside code or a ``` fence. A ☺ button in each editor's corner opens the picker for mouse users, with a skin-tone selector that's remembered, and its search takes focus only once the picker is placed (a hidden field can't). Emoji, mentions and pasted images go in through execCommand, so Ctrl+Z takes back the insertion instead of wrecking the undo stack. Uploading an image made the textarea disabled, which dropped focus for good; it's read-only now, says "Uploading image…" under the box rather than only in an empty one's placeholder, reports a failed upload instead of swallowing it, and holds Enter until it's done. Any dropped file is caught — a non-image used to be opened by the browser in place of the page, draft and all — and a paste carrying text (a spreadsheet's cells come with a picture of themselves) pastes the text. The mention list follows the caret (a stale lookup can't close a newer one, lookups abort), wraps, scrolls its selection into view, closes on blur, and is a real listbox with aria-activedescendant. The counter shows from 80% of the limit by default rather than a standing "0 / 5000". Limits: chat Send and edit Save refused nothing over 2000 and let the server bounce it; they're disabled past it now. RichTextField's bare maxLength rule failed without a message, so an over-long "Why?" made Save do nothing and say nothing; it says so now. Inline editors answer Escape through a new onCancel (comment reply and edit, profile description, playlist description and title), the playlist editor takes Ctrl+Enter and can't be saved without a title, and saving a comment unchanged no longer stamps it edited. Markdown: a single newline is a line break, so a message written over three lines stops reading as one; links into the app route in place instead of opening a tab; headings are sized for prose instead of 2em page titles, and dropped from card teasers; GFM tables scroll in their own box and task lists lose the bullet; images load lazily. A chat message of only an emoji or three is shown large. Themes: brutalist and geocities give the new buttons no chrome, and geocities' ghost-button list, split in two by the placeholder-thumbnail rules in 1cb904d, is whole again, so its modal ✕, emoji cells and the picker's ✕ are flat as intended. 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-27 10:24:30 +00:00
764 changed files with 79601 additions and 25892 deletions
Expand all files Collapse all files

21
.devcontainer/Dockerfile
View File

@@ -1,3 +1,20 @@
FROM denoland/deno:bin-2.9.0 AS deno
FROM mcr.microsoft.com/devcontainers/typescript-node:20
FROM denoland/deno:bin-2.9.7 AS deno
FROM mcr.microsoft.com/devcontainers/typescript-node:26
COPY --from=deno /deno /usr/local/bin/deno
# The API draws thumbnails and waveforms with ffmpeg, in the workspace and in
# the instances' services alike (compose.yaml), which don't get the features.
RUN apt-get update \
&& apt-get install -y --no-install-recommends ffmpeg \
&& rm -rf /var/lib/apt/lists/*
# Playwright's Firefox and Chromium headless shell, with the system libraries
# they need, for screenshots of the dev instances (`npm:playwright@1.63.0`): a
# script importing another version looks for other builds. They live outside
# the home directory, which a rebuild would otherwise leave without them.
ENV PLAYWRIGHT_BROWSERS_PATH=/opt/ms-playwright
RUN DENO_DIR=/tmp/deno deno run --no-lock -A npm:playwright@1.63.0 install \
--with-deps firefox chromium-headless-shell \
&& chmod -R a+rX /opt/ms-playwright \
&& rm -rf /tmp/deno /var/lib/apt/lists/*
# The services share Deno's cache through a volume (compose.yaml), which Docker
# creates owned as this directory is, so it must exist and be node's.
RUN mkdir -p /home/node/.cache/deno && chown -R node:node /home/node/.cache

57
.devcontainer/compose.yaml Normal file
View File

@@ -0,0 +1,57 @@
# The dev environment: the workspace VS Code attaches to (`dev`), and one
# service per dev instance of gerbeur (dev/instances.ts), which start with it.
# Each instance service runs dev/start.ts for its own instance, which derives
# its ports, URLs, database and uploads from its entry there; a new instance is
# an entry there and a service here.
#
# Every service shares the host's network, as the workspace always did: an
# instance's URLs are localhost ones, which the browser and the other
# instances' servers must both reach.
#
# From the workspace's terminal (docker-outside-of-docker):
# docker compose -p gerbeur-dev logs -f musique
# docker compose -p gerbeur-dev restart gerbeur
# docker compose -p gerbeur-dev stop gerbeur musique # then `deno task dev`
name: gerbeur-dev
x-workspace: &workspace
build:
context: .
dockerfile: Dockerfile
network_mode: host
user: node
working_dir: /workspaces/gerbeur
volumes:
- ..:/workspaces/gerbeur:cached
# Shared, so that each service doesn't download the jsr and npm packages
# again.
- deno-cache:/home/node/.cache/deno
x-instance: &instance
<<: *workspace
restart: unless-stopped
depends_on:
install:
condition: service_completed_successfully
services:
dev:
<<: *workspace
command: sleep infinity
# Installs the packages and draws the icons once, before any instance
# starts, so that the instances don't do it at the same time.
install:
<<: *workspace
command: sh -c "deno install && deno task icons public"
gerbeur:
<<: *instance
command: deno run --env-file -A dev/start.ts gerbeur
musique:
<<: *instance
command: deno run --env-file -A dev/start.ts musique
volumes:
deno-cache:

13
.devcontainer/devcontainer-lock.json
View File

@@ -1,15 +1,20 @@
{
"features": {
"ghcr.io/devcontainers-extra/features/ffmpeg-apt-get:1": {
"version": "1.0.17",
"resolved": "ghcr.io/devcontainers-extra/features/ffmpeg-apt-get@sha256:735491c8e9b2236cae727d6d4904663f6a58eb131d8c4b6b18fd0eea3d8c5b71",
"integrity": "sha256:735491c8e9b2236cae727d6d4904663f6a58eb131d8c4b6b18fd0eea3d8c5b71"
"ghcr.io/devcontainers/features/docker-outside-of-docker:1": {
"version": "1.10.1",
"resolved": "ghcr.io/devcontainers/features/docker-outside-of-docker@sha256:698f5900fac471e046c6e9d4333fec88c0e8d952c713494fa0712dce80baeca0",
"integrity": "sha256:698f5900fac471e046c6e9d4333fec88c0e8d952c713494fa0712dce80baeca0"
},
"ghcr.io/khannurien/devcontainer-features/claude-code:1": {
"version": "1.2.0",
"resolved": "ghcr.io/khannurien/devcontainer-features/claude-code@sha256:17eb69bfd15ff5206c0bff2945935a4349f7423cabee461a0ba2681d52205db9",
"integrity": "sha256:17eb69bfd15ff5206c0bff2945935a4349f7423cabee461a0ba2681d52205db9"
},
"ghcr.io/khannurien/devcontainer-features/opencode:1": {
"version": "1.1.0",
"resolved": "ghcr.io/khannurien/devcontainer-features/opencode@sha256:8df3f8c98a104ce7e089ba3e8df39e0d3890753c8dc92e03deb54527d30d3574",
"integrity": "sha256:8df3f8c98a104ce7e089ba3e8df39e0d3890753c8dc92e03deb54527d30d3574"
},
"ghcr.io/khannurien/devcontainer-features/rtk:1": {
"version": "1.1.0",
"resolved": "ghcr.io/khannurien/devcontainer-features/rtk@sha256:4a60b3a1d1b64904de3d856c61cdf7e5943e3c74d114609787dc88d9db7c788f",

31
.devcontainer/devcontainer.json
View File

@@ -1,19 +1,12 @@
{
"name": "Deno",
"build": {
"dockerfile": "Dockerfile"
},
"name": "gerbeur",
// Features to add to the dev container. More info: https://containers.dev/features.
// "features": {},
// Use 'forwardPorts' to make a list of ports inside the container available locally.
// "forwardPorts": [],
"runArgs": ["--network=host"],
// Use 'postCreateCommand' to run commands after the container is created.
// "postCreateCommand": "yarn install",
// The workspace and the dev instances of gerbeur, one service each: see
// compose.yaml. The instances start with the container.
"dockerComposeFile": "compose.yaml",
"service": "dev",
"workspaceFolder": "/workspaces/gerbeur",
"shutdownAction": "stopCompose",
// Configure tool-specific properties.
"customizations": {
@@ -26,9 +19,11 @@
},
"features": {
"ghcr.io/warrenbuckley/codespace-features/sqlite:1": {},
"ghcr.io/devcontainers-extra/features/ffmpeg-apt-get:1": {}
// The docker CLI, on the host's daemon, for the instances' logs and
// restarts (compose.yaml). Docker's own packages: the base image is Debian
// trixie, which has no moby-cli.
"ghcr.io/devcontainers/features/docker-outside-of-docker:1": {
"moby": false
}
}
// Uncomment to connect as root instead. More info: https://aka.ms/dev-containers-non-root.
// "remoteUser": "root"
}

42
.env.example
View File

@@ -48,19 +48,45 @@ GERBEUR_ALLOWED_ORIGINS=http://localhost:3000,http://127.0.0.1:3000
# Secret key used to sign JWTs. Generate with: openssl rand -hex 32
GERBEUR_JWT_SECRET=
# SMTP connection URL used by the email service (notifications, invites, etc.)
# SMTP connection URL for outgoing email: the welcome email, password reset
# links and the weekly/monthly digests. Leave empty to send no email at all.
# Example: smtps://username:password@smtp.example.com:465
GERBEUR_SMTPS_URL=
# Sender address used in outgoing emails (e.g. no-reply@example.com)
# Required when GERBEUR_SMTPS_URL is set.
# Sender of every email, as an address or "Name <address>"
# (e.g. gerbeur <no-reply@example.com>).
# Required when GERBEUR_SMTPS_URL is set: the server refuses to start without it.
GERBEUR_FROM_EMAIL=
# Markdown body for the account creation welcome email.
# Supports {{username}} and {{site_name}} placeholders.
# Defaults to a built-in template when unset.
# Use \n for line breaks in single-line .env values, or use a quoted multiline block.
GERBEUR_WELCOME_EMAIL_BODY="# Welcome to {{site_name}}!\n\nHi **{{username}}**,\n\nYour account has been created successfully. Welcome aboard!"
# Language of every email: en (default) or fr. The interface follows each
# visitor's browser, which the server never sees, so emails use the instance's.
GERBEUR_EMAIL_LOCALE=en
# The instance's style: smooth (default), brutalist, geocities, nyt, kanagawa
# or melange. Visitors and accounts see it until they pick one, and the welcome
# and password reset emails are drawn in it; a digest uses its reader's own
# style. Applied at runtime; changing it needs a restart of both the API and Vite.
GERBEUR_DEFAULT_STYLE=smooth
# How often, in minutes, the /community page's figures are recomputed.
GERBEUR_COMMUNITY_REFRESH_MINUTES=15
# Set to 0 to keep users of other gerbeur instances from signing in here, which
# also signs out those already in until it is set back, or to keep this
# instance's users from signing in to others, which signs them out there.
# GERBEUR_FEDERATION_INCOMING=1
# GERBEUR_FEDERATION_OUTGOING=1
# Set to 1 to let federated sign-in reach other instances on private addresses
# (localhost included) and over plain http, so that two instances on one
# machine can trust each other in development. Never in production.
# GERBEUR_FEDERATION_ALLOW_PRIVATE=1
# Markdown body of the welcome email sent on account creation, replacing the
# built-in one (written in GERBEUR_EMAIL_LOCALE). Supports {{username}} and
# {{site_name}}. Use \n for line breaks in single-line .env values, or a
# quoted multiline block.
# GERBEUR_WELCOME_EMAIL_BODY="# Welcome to {{site_name}}!\n\nHi **{{username}}**,\n\nYour account has been created successfully. Welcome aboard!"
# ── Frontend (Vite) ───────────────────────────────────────────────────────────
# These must be prefixed with VITE_ to be exposed to the client bundle.

46
.gitea/workflows/docs.yaml Normal file
View File

@@ -0,0 +1,46 @@
name: Build and Deploy Static Documentation Site
on:
push:
branches: [main]
paths:
- "api/**"
- "shared/**"
- "src/**"
- "mod.ts"
- "deno.json"
- "deno.lock"
- ".gitea/workflows/docs.yaml"
workflow_dispatch:
env:
# Same Deno as the Docker image (Dockerfile).
DENO_VERSION: "2.9.0"
jobs:
build-and-push:
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- uses: actions/checkout@v4
- uses: denoland/setup-deno@v2
with:
deno-version: ${{ env.DENO_VERSION }}
- name: Build docs
# `deno doc` writes relative links, so the site works under whatever
# path the gitea-pages proxy serves it from
# (https://pub.sqrt.fr/<owner>/<repo>/).
run: deno task doc
- name: Deploy docs
uses: peaceiris/actions-gh-pages@v4
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
force_orphan: true
publish_branch: gh-pages
publish_dir: ./docs

3
.gitignore vendored
View File

@@ -178,3 +178,6 @@ dist-ssr
*.njsproj
*.sln
*.sw?
# deno task doc
docs/

302
API.md Normal file
View File

@@ -0,0 +1,302 @@
# gerbeur's URLs
Every page and every API endpoint follows one of two rules. This file summarizes
them. Each router's module doc in `api/routes/` lists its own endpoints in full,
and a route change updates both. `shared/paths.ts` builds and parses the page
paths.
## The two rules
**Pages.** The path is a hierarchy. `@name` is a user, and `@name@host` a user
whose account lives on another instance. Any other word under the root is a
category or an app page. Under a user, each segment is a fixed tab name, and
under `dumps` or `playlists`, a slug. A view of a page is one more segment. A
default view is the bare path, with no redirect. A category is a segment where
the page is the category's own, a dump category's feed at the root (`/music`)
and a playlist category's under `/playlists` (`/playlists/mixes`); where it
narrows a list on another page, a profile's or a search's, it is `?category=`,
of the kind that list holds. The order of a list of playlists is `?sort=`, the
default order being the bare path, and the playlists page's list of the viewer's
own is `?mine=1`. Switching tabs, a tab's views or a list's order takes the
place of the current history entry; picking a category, or opening another page
or an editor, adds one.
**API.** A resource is `/api/<plural>/<key>[/<noun>]`. Its key is its id, except
a user's, which may also be `@name`. The `@` keeps the two apart, so no regex
sniffing is needed. A read by page path mirrors the page:
`/api/users/@bob/dumps/<slug>`. The caller's own things live under `/api/me`.
Media is a sub-resource. A lookup by another key is a query filter. Each concept
has one word, the same as in pages.
## Pages
| Path | Views | Access | Page |
| ------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------- | --------------------------- |
| `/` | `/hot`, `/new`, `/journal`, `/follow`, `/follow/playlists`; the bare `/` is the viewer's default tab, the bare `/follow` the users followed | anyone, `/follow` signed in | `Index` |
| `/<category>` | `/<category>/hot`, `/new`, `/journal`, `/follow`, `/follow/playlists` | anyone, `/follow` signed in | `Index` |
| `/@<name>` | the overview: dumps beside upvoted | anyone | `UserPublicProfile` |
| `/@<name>/dumps`, `/upvoted`, `/playlists`, `/invitees` | `?category=` narrows the dumps, the upvoted or the playlists; `?sort=` orders the playlists | anyone | `UserPublicProfile` |
| `/@<name>/follow` | `/@<name>/follow/playlists`; the bare tab lists users | anyone | `UserPublicProfile` |
| `/@<name>/manage` | | `user:manage`, on others' profiles | `UserPublicProfile` |
| `/@<name>/dumps/<slug>` | | anyone; a private dump, its poster | `Dump` |
| `/@<name>/dumps/<slug>/edit` | | its poster, `dump:moderate` | `DumpEdit` |
| `/@<name>/playlists/<slug>` | | anyone; a private playlist, its owner | `PlaylistDetail` |
| `/@<name>/playlists/<slug>/edit` | | its owner, `playlist:moderate` | `PlaylistDetail` |
| `/playlists` | `/playlists/<category>`, a playlist category's; `?sort=new`, `?sort=dumps` order either; `?mine=1` lists the viewer's own, signed in only | anyone | `Playlists` |
| `/search?q=` | `/search/users?q=`, `/search/playlists?q=`; `?category=` narrows the dumps or the playlists | anyone | `Search` |
| `/notifications` | | signed in | `Notifications` |
| `/community` | `/community/monthly`; the bare path is the week | signed in | `Community` |
| `/settings` | | signed in | `Settings` |
| `/manage` | | `category:manage`, `federation:manage` | `Manage` |
| `/login`, `/register?token=` | `/login?next=<path>` comes back to a page | signed out | `UserLogin`, `UserRegister` |
| `/authorize?client_id=…` | | signed in, else via `/login?next=` | `Authorize` |
| `/federation/callback?code=&state=` | | anyone | `FederationCallback` |
| `/reset-password` | `/reset-password?token=`, from the emailed link, sets the new password | anyone | `ResetPassword` |
| `/unsubscribe?token=` | | anyone | `Unsubscribe` |
`/?share_url=` opens the create form on a link shared from Android, `?chat=open`
opens the chat over any page, `?chat=<message id>` on that message, and
`#comment-<id>` scrolls a dump's page to a comment. A signed-in page opened
signed out replaces itself with `/login?next=<the page>`.
## Names
- **Slugs** of categories, dumps and playlists are lowercase letters and digits
in words joined by single hyphens (`shared/slug.ts`). A category's is at most
50 characters, a dump's or a playlist's at most 60.
- **A dump's or a playlist's slug** is made from its title when it is created,
unique among its owner's items of that kind: a clash gets `-2`, `-3`. It
changes only when someone edits it: the edit forms propose `slugify(newTitle)`
when the title changes. Its owner may change it 3 times a day; moderators
(`dump:moderate`, `playlist:moderate`) may edit any slug, at any time.
- **A username** is 1 to 32 letters, digits and underscores, unique in any case.
A page writes it in its stored case, and another case still resolves. Users
can't rename themselves: those with `user:manage` rename anyone, from
`/@<name>/manage`.
- **A user from another instance** is `name@host` here: their name at home and
their home's host, with its port if it has one (`alice@music.example`). Their
name is their home's to change, which leaves the old handle an alias here too.
Mentions write the whole handle. A handle with this instance's own host, as
another instance writes a user from here (`bob@gerbeur.example`), names the
local `bob`, in a path, an API key or a mention.
- **Aliases.** A slug or a name left behind stays an alias for good, which
always names the item or user itself, so redirects never chain. Nobody else
can take one. An item can take back one of its own. Deleting an item frees its
aliases.
- **Redirects.** A page reached by an alias, or by a name in another case, gets
the item from the API under its current path and replaces its URL. The server
answers the same path with a 301 to crawlers and full page loads.
- **Reserved words.** A category can't take a word the root already uses:
`hot new journal follow search login register notifications reset-password
unsubscribe settings manage community playlists authorize federation api ws
assets emojibase`
(`api/lib/reserved-slugs.ts`). Dump and playlist slugs reserve nothing, and no
slug has an `@`.
## API
Every JSON endpoint answers the envelope of `shared/api.ts`:
`{ "success": true, "data": … }`, or
`{ "success": false, "error": { "code",
"message" } }` with the matching HTTP
status. A list answers `{ "items", "hasMore" }` and takes `?page=` (from 1) and
`?limit=` (20 by default, 100 at most). "Signed in" means a session token as
`Authorization: Bearer …`. Media can't send a header, so a private file,
thumbnail, waveform, cover or attachment takes the owner's media token as
`?token=`.
An image the API serves, an avatar, a thumbnail, a cover, an attachment or a
remote image, takes `?w=` and answers a WebP copy scaled down to that width,
snapped up to one of `shared/image.ts`'s (128, 320, 640, 960), and never wider
than the image; without it, the image as it was uploaded, a dump's thumbnail
alone being scaled to the widest by default. `?f=compat` asks for the copy as a
JPEG or PNG, for a mail client that reads no WebP. An image no copy can be made
of goes out as it is. JSON answers of a kilobyte or more are compressed, in
brotli or gzip, for a client that accepts it.
### me
- `GET /api/me` — signed in. The caller's own user.
- `PATCH /api/me` — signed in. Changes the description and email.
- `GET /api/me/media-token` — signed in. A media token.
- `GET`, `PATCH /api/me/settings` — signed in. The caller's settings.
- `PUT /api/me/password` — signed in, with the current one. A new password, and
fresh tokens.
- `PUT /api/me/avatar` — signed in. A new avatar (multipart).
- `GET /api/me/playlists?contains=:dumpId` — signed in. The caller's playlists,
and whether each holds the dump.
- `GET /api/me/follow` — signed in. The ids of the users and playlists followed.
- `PUT`, `DELETE /api/me/follow/users/:id`, `/api/me/follow/playlists/:id` —
signed in. Follows or stops following.
- `GET /api/me/notifications` — signed in. The caller's notifications.
- `POST /api/me/notifications/read` — signed in. Marks them all read.
- `GET /api/me/grants` — signed in. The other instances the caller's account
signs in to.
- `DELETE /api/me/grants/:host` — signed in. Withdraws one, ending the caller's
sessions there.
An account from another instance changes its password, email, avatar and name
there: those endpoints answer it 400.
### users
- `POST /api/users` — with an invite. Signs up.
- `GET /api/users?q=` — anyone. Up to 8 users whose name starts with `q`.
- `GET /api/users/:key` — anyone. A profile.
- `GET /api/users/:key/avatar` — anyone. The avatar.
- `GET /api/users/:key/dumps`, `/upvoted`, `/playlists` — anyone; private items
to their owner only. Their lists; `?category=` narrows one to a category of
its kind, and `?sort=` orders the playlists as `GET /api/playlists` does.
- `GET /api/users/:key/dumps/:slug`, `/playlists/:slug` — anyone; private items
to their owner only. An item by its page path, by an alias too.
- `GET /api/users/:key/follow/users`, `/follow/playlists` — anyone. What they
follow.
- `GET /api/users/:key/invitees` — anyone. The tree of users they invited.
- `GET /api/users/:key/home?path=` — anyone. Redirects to a page of a remote
user's home instance, their profile there by default.
- `PATCH /api/users/:key/role`, `/username` — `user:manage`. Another user's
role, or anyone's name, theirs included.
### dumps
- `POST /api/dumps` — signed in. Posts a link (JSON) or a file (multipart).
- `GET /api/dumps` — anyone. The feed; `?category=<slug>` narrows it, and
`?url=` answers the dumps of the same link instead.
- `GET /api/dumps?follow=users|playlists` — signed in. The feed's `follow` tab:
the dumps of the users, or of the playlists, the caller follows; `?category=`
narrows it too.
- `GET /api/dumps/:id` — anyone who may see it. A dump.
- `GET /api/dumps/:id/voters`, `/related`, `/playlists` — the same. Who upvoted
it, the dumps linked to or from it, the playlists holding it.
- `PUT /api/dumps/:id` — its poster, `dump:moderate`. Title, slug, note,
visibility, categories, link.
- `POST /api/dumps/:id/metadata` — the same. Fetches a link's preview again.
- `DELETE /api/dumps/:id` — the same. Deletes it.
- `GET`, `PUT /api/dumps/:id/file` — the file; a new one, by its poster or
`dump:moderate`.
- `GET`, `PUT`, `DELETE /api/dumps/:id/thumbnail` — the thumbnail; a custom one,
set or removed by its poster or `dump:moderate`.
- `GET /api/dumps/:id/waveform` — anyone who may see it. An audio file's
waveform, `{ "peaks" }`, for the player to seek on.
- `GET`, `POST /api/dumps/:id/comments` — anyone who may see the dump; posting
signed in. Its comments.
### playlists
- `POST /api/playlists` — signed in. Creates a playlist, filed under the
playlist categories it names.
- `GET /api/playlists` — anyone. Every public playlist; `?category=<slug>`
narrows it to a playlist category, and `?sort=updated|new|dumps` orders it:
the last to take a dump first (the default), the newest, or the fullest. The
caller's own private dumps count, and date their playlists, for them alone.
- `GET /api/playlists/:id` — anyone who may see it. A playlist and its dumps.
- `PUT`, `DELETE /api/playlists/:id` — its owner, `playlist:moderate`. Title,
slug, description, visibility, categories; deletion.
- `POST`, `DELETE /api/playlists/:id/dumps/:dumpId` — the same. Adds or removes
a dump.
- `PUT /api/playlists/:id/order` — the same. Reorders its dumps.
- `GET`, `PUT /api/playlists/:id/cover` — the cover; a new one, by its owner or
`playlist:moderate`.
### comments
- `PATCH`, `DELETE /api/comments/:id` — its author, `comment:moderate`. Edits or
deletes a comment.
- `GET /api/comments/:id/likers` — anyone. Who likes it.
### categories
- `GET /api/categories` — anyone. Every category, dump and playlist ones apart:
each has a `kind`, and its slug is unique within its kind.
- `POST /api/categories`, `PUT`, `DELETE /api/categories/:id` —
`category:manage`. Changes them.
### community
- `GET /api/community?period=weekly|monthly` — signed in. The site's top dumps,
new members, new public playlists and totals over the last week (by default)
or month, as of the server's last refresh.
- `GET /api/community/feed` — signed in. The community feed: the 12 latest
things done in public, newest first (a public dump posted, a public playlist
created, a comment or an upvote on a public dump, someone joining), and the
last week's totals and most upvoted dump, as of the server's last refresh. A
dump has one comment event and one upvote event at most, its latest. Every
reader gets the same feed.
### chat
- `GET /api/chat?before=&after=` — signed in. A page of messages. The chat
itself goes over `/ws`.
- A message from a moderator or an admin (`chat:everyone`) that says `@everyone`
notifies every user who doesn't have the chat open, and comes flagged
`mentionsEveryone`. No account may be named `everyone`.
### invites
- `POST /api/invites` — signed in. A new invite.
- `GET /api/invites/:token` — anyone. Whether it still works.
### sessions, password-resets, unsubscribe
- `POST /api/sessions` — anyone. Signs in.
- `POST /api/password-resets` — anyone. Emails a reset link.
- `PUT /api/password-resets/:token` — with the link's token. A new password.
- `POST /api/unsubscribe?token=` — with a digest's token. Turns it off. Its v3
path, `POST /api/users/unsubscribe`, stays for good, since sent emails'
`List-Unsubscribe` headers name it.
### federation, oauth, .well-known
Federated sign-in (`shared/federation.ts`): a user signs in here with their
account on another gerbeur instance, their home, by OpenID Connect's
authorization code flow with PKCE, with tokens signed by each instance's Ed25519
key. With `GERBEUR_FEDERATION_INCOMING=0`, `POST /api/federation/logins` and
`/sessions` answer 403; with `GERBEUR_FEDERATION_OUTGOING=0`,
`GET /api/oauth/clients` and `POST /api/oauth/codes` do. An instance is named by
its issuer, its API's origin, whose host is the one in its users' handles. Where
`instance` is asked for, the origin of its pages works too: their
`/.well-known/openid-configuration` names the issuer, whose own metadata must
name those pages back as where its `/authorize` is.
- `GET /.well-known/openid-configuration`, `/.well-known/jwks.json` — anyone.
The instance's metadata and public keys, bare JSON: the one exception to the
`/api/` rule, at the paths the standard fixes.
- `POST /api/federation/logins` — anyone. Starts a sign-in with an account on
another instance; one neither trusted nor already home to someone here needs
an invite.
- `POST /api/federation/sessions` — anyone. Finishes it, as a sign-in.
- `POST /api/federation/events` — another instance. Its signed news of one of
its users here: a new name, email or avatar, or the end of their sessions. A
4xx tells it to give up on the event, a 429 or a 5xx to try again.
- `GET /api/federation/peers` — anyone the trusted instances,
`federation:manage` every listed one.
- `PUT /api/federation/peers`, `DELETE /api/federation/peers/:host` —
`federation:manage`. Trusts or blocks an instance, or takes it off the lists.
- `GET /api/oauth/clients?client_id=` — signed in. The instance asking to sign
the caller in.
- `POST /api/oauth/codes` — signed in. The caller's consent, and a code.
- `POST /api/oauth/tokens` — another instance, form-encoded. Trades a code for
an identity token, bare JSON.
### search
- `GET /api/search?q=` — anyone. A page of dumps, and up to 20 users and
playlists. It finds both kinds of item at once, so `?category=` narrows the
dumps and `?playlistCategory=` the playlists.
### attachments
- `POST /api/attachments` — signed in. Stores an image.
- `GET /api/attachments/:id` — anyone who may see the text holding it.
### remote
- `GET /api/remote/preview?url=` — signed in. A link's preview.
- `GET /api/remote/image?url=` — anyone. A remote image through this site.
- `GET /api/remote/bandcamp?url=` — anyone. A Bandcamp page's tracks.
- `GET /api/remote/youtube?list=` — anyone. A YouTube playlist's tracklist.
### /ws
- `GET /ws?token=` — anyone; a session token signs it in. The WebSocket of live
updates and the chat (`shared/ws.ts`).

9
Dockerfile
View File

@@ -10,6 +10,10 @@ RUN deno install
COPY index.html ./
COPY public/ ./public/
COPY src/ ./src/
COPY shared/ ./shared/
# The build extracts and compiles the catalogs, the emails' messages included.
COPY locales/ ./locales/
COPY api/mail/ ./api/mail/
# VITE_API_* are only needed when the API lives on a different host than the frontend.
ARG VITE_API_PROTOCOL
@@ -32,7 +36,12 @@ COPY deno.json deno.lock ./
RUN deno install
COPY api/ ./api/
COPY shared/ ./shared/
COPY --from=builder /app/dist/ ./dist/
COPY --from=builder /app/locales/ ./locales/
# The emails read their colors, fonts and radii from the site's stylesheets.
COPY src/tokens.css ./src/
COPY src/themes/ ./src/themes/
# Persistent data: database and uploaded/generated files must be mounted as volumes.
VOLUME ["/app/api/sql", "/app/api/uploads"]

252
README.md
View File

@@ -1,6 +1,11 @@
# gerbeur
A small invite-only social platform for sharing links and files. Users can post URLs and media (YouTube, SoundCloud, Bandcamp, images, audio, video, …), vote, comment, follow each other, build playlists, and search content. A real-time WebSocket layer handles live presence, vote counts, and notifications. The UI is localized (English and French) and ships multiple visual themes.
A small invite-only social platform for sharing links and files. Users can post
URLs and media (YouTube, SoundCloud, Bandcamp, images, audio, video, …), vote,
comment, follow each other, build playlists, and search content. A real-time
WebSocket layer handles live presence, vote counts, and notifications, and
weekly and monthly email digests sum up what happened. The UI is localized
(English and French) and ships multiple visual themes.
## Stack
@@ -26,20 +31,40 @@ cp .env.example .env
### Run
In the devcontainer, the dev instances listed in
[`dev/instances.ts`](dev/instances.ts) start with the container, each a service
of [`.devcontainer/compose.yaml`](.devcontainer/compose.yaml): `gerbeur` at
[http://localhost:3000](http://localhost:3000), `musique` at
[http://localhost:3001](http://localhost:3001). They run side by side so that
signing in from one instance to another can be tried on one machine, and from
the workspace's terminal:
```sh
deno task dev
docker compose -p gerbeur-dev logs -f musique # follow an instance
docker compose -p gerbeur-dev restart gerbeur # restart one
```
This starts both the API server (port 8000, with file watching) and the Vite dev server (port 3000) concurrently.
Outside the devcontainer, or with those services stopped, `deno task dev` starts
them all, and `deno task dev gerbeur` only those named.
Open [http://localhost:3000](http://localhost:3000). On first run a default `admin` / `admin` account is created — change the password immediately.
Each instance has an API server, restarted on change, and a Vite server, and its
output is prefixed with its name. They follow one rule
([`dev/start.ts`](dev/start.ts)): instance `<name>` keeps its database in
`api/sql/<name>.db` and its uploads in `api/uploads/<name>/`, reads `.env`, and
takes what sets it apart (ports, site name, emoji, style) from its entry. A new
instance is an entry in `dev/instances.ts` and a service in `compose.yaml`.
On first run each instance creates a default `admin` / `admin` account — change
the password immediately. To let one instance's users sign in to another, trust
it on the other's `/manage`.
### Environment variables
See [`.env.example`](.env.example) for the full list with descriptions. Key variables:
See [`.env.example`](.env.example) for the full list with descriptions. Key
variables:
| Variable | Description | Default |
| ---------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------- |
| ----------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------- |
| `GERBEUR_JWT_SECRET` | JWT signing secret — **required**, generate with `openssl rand -hex 32` | — |
| `GERBEUR_PUBLIC_URL` | Public-facing URL of the server (no trailing slash) — used for CORS, WebSocket origin, email links, OG URLs | `http://localhost:GERBEUR_PORT` |
| `GERBEUR_PORT` | Internal port Oak listens on | `8000` |
@@ -47,9 +72,15 @@ See [`.env.example`](.env.example) for the full list with descriptions. Key vari
| `GERBEUR_FRONTEND_URL` | Frontend base URL for email links and CORS; auto-added to allowed origins — the only variable needed when the frontend runs on a separate host | `GERBEUR_PUBLIC_URL` |
| `GERBEUR_ALLOWED_ORIGINS` | Comma-separated extra origins for CORS/WebSocket; `PUBLIC_URL` and `FRONTEND_URL` are always included — typically only needed in dev for the Vite server | `""` (empty) |
| `GERBEUR_SITE_NAME` | Site name used in the browser tab, app header, OG meta tags, and emails | `gerbeur` |
| `GERBEUR_SMTPS_URL` | SMTPS connection URL for outgoing email (`smtps://user:pass@host:465`) | unset |
| `GERBEUR_FROM_EMAIL` | Sender address for outgoing emails — required when `GERBEUR_SMTPS_URL` is set | unset |
| `GERBEUR_SMTPS_URL` | SMTPS connection URL for outgoing email (`smtps://user:pass@host:465`); unset sends no email | unset |
| `GERBEUR_FROM_EMAIL` | Sender of every email — required when `GERBEUR_SMTPS_URL` is set, or the server won't start | unset |
| `GERBEUR_EMAIL_LOCALE` | Language of every email: `en` or `fr` | `en` |
| `GERBEUR_DEFAULT_STYLE` | The instance's style (`smooth`, `brutalist`, `geocities`, `nyt`, `kanagawa`, `melange`): the site's until one is picked, and the welcome and reset emails' | `smooth` |
| `GERBEUR_WELCOME_EMAIL_BODY` | Markdown body for the account-creation welcome email; supports `{{username}}` and `{{site_name}}` | built-in template |
| `GERBEUR_FEDERATION_INCOMING` | `0` keeps users of other instances from signing in here, and signs out those already in | `1` |
| `GERBEUR_FEDERATION_OUTGOING` | `0` keeps this instance's users from signing in to other instances, and signs them out there | `1` |
| `GERBEUR_FEDERATION_ALLOW_PRIVATE` | `1` lets federated sign-in reach instances on private addresses and over http — for two instances on one machine, never in production | unset |
| `GERBEUR_COMMUNITY_REFRESH_MINUTES` | How often, in minutes, the `/community` page's figures are recomputed | `15` |
| `VITE_API_PROTOCOL` | API protocol baked into the frontend bundle (see [Production](#production)) | `http` |
| `VITE_API_HOSTNAME` | API hostname baked into the frontend bundle | `localhost` |
| `VITE_API_PORT` | API port baked into the frontend bundle | `8000` |
@@ -58,7 +89,10 @@ See [`.env.example`](.env.example) for the full list with descriptions. Key vari
### Docker (recommended)
The standard deployment runs API and frontend in a single container. The API server (Oak) serves the compiled frontend as static files, so both share the same origin — no `VITE_API_*` build args needed. Set `GERBEUR_PUBLIC_URL` to the externally-visible URL; it is automatically allowed for HTTP/WebSocket requests.
The standard deployment runs API and frontend in a single container. The API
server (Oak) serves the compiled frontend as static files, so both share the
same origin — no `VITE_API_*` build args needed. Set `GERBEUR_PUBLIC_URL` to the
externally-visible URL; it is automatically allowed for HTTP/WebSocket requests.
```sh
docker build -t gerbeur .
@@ -77,12 +111,17 @@ docker run -d \
```
The two volumes are required for persistence:
- `gerbeur-db` — SQLite database (`api/sql/gerbeur.db`), initialized automatically on first run
- `gerbeur-db` — SQLite database (`api/sql/gerbeur.db`), initialized
automatically on first run
- `gerbeur-uploads` — user-uploaded files (`api/uploads/`)
#### Separate API and frontend (optional)
If you need to run the API on a different host than the frontend, pass the API location as build args so it gets baked into the frontend bundle. In that setup, add the frontend origin to `GERBEUR_ALLOWED_ORIGINS` so cross-origin HTTP/WebSocket requests are accepted:
If you need to run the API on a different host than the frontend, pass the API
location as build args so it gets baked into the frontend bundle. In that setup,
add the frontend origin to `GERBEUR_ALLOWED_ORIGINS` so cross-origin
HTTP/WebSocket requests are accepted:
```sh
docker build \
@@ -94,7 +133,8 @@ docker build \
### Reverse proxy
Put a reverse proxy (nginx, Caddy, …) in front of the container to handle TLS. Forward everything to port 8000. Example Caddyfile:
Put a reverse proxy (nginx, Caddy, …) in front of the container to handle TLS.
Forward everything to port 8000. Example Caddyfile:
```
example.com {
@@ -102,38 +142,190 @@ example.com {
}
```
### Several instances
Each instance is its own deployment, with its own database, uploads, admins and
domain, but its users can sign in to the others with their account, and the
other way round. A user from `music.example` is `@alice@music.example` on the
instances they sign in to. Their password, email, avatar and name stay on their
home instance, and the others follow its changes.
Every instance is ready for it: nothing to configure, as long as
`GERBEUR_PUBLIC_URL` is the instance's https origin (no path) and the instances
can reach each other. Users and admins may name an instance by the address of
its API or of its pages: when the frontend runs on a host of its own
(`GERBEUR_FRONTEND_URL`), that host must pass `/.well-known/` on to the API, as
the Vite dev server does. A handle carries the API's host. Either way can be
turned off: `GERBEUR_FEDERATION_INCOMING=0` takes no users of other instances,
and `GERBEUR_FEDERATION_OUTGOING=0` lets no user sign in elsewhere. An admin
decides who comes in, on `/manage`:
- users of a **trusted** instance sign in directly, from a button on the login
page;
- users of a **blocked** instance can't, and those already in are signed out;
- users of **any other** instance join with an invite, from the register page,
and sign in without one afterwards.
`deno task dev` runs several instances on one machine (see [Run](#run)), with
`GERBEUR_FEDERATION_ALLOW_PRIVATE=1`, which lets them reach each other over
plain http on localhost. Never set it in production. The first, `gerbeur`, takes
no users of the others, whose users sign in to each other.
## Project structure
```
dev/ # The dev instances (instances.ts) and their launcher (start.ts), for `deno task dev`
shared/ # Imported by both halves: API and socket types, limits, permissions
test/ # Tests of shared/ (Deno): test/<module>.test.ts
locales/ # Lingui catalogues (en, fr): the interface's and the emails' messages
api/
main.ts # Entry point — Oak application, middleware, routes
config.ts # Environment variables
middleware/ # errorMiddleware, authMiddleware
routes/ # HTTP routes + WebSocket
services/ # Business logic
middleware/ # Oak middleware: errors, auth, OG tags
routes/ # One router per API resource: HTTP + WebSocket, no SQL
services/ # Business logic and every SQL query
remote/ # Everything that fetches from other sites
mail/ # One template per email, and the layout they share
providers/ # Link metadata providers (YouTube, SoundCloud, Bandcamp, …)
model/ # Row types, type guards
lib/ # JWT, pagination, slugify, upload, static helpers, …
model/ # Per table: row type, row guard, row ↔ API mappers
validators/ # Per domain: request body guards
lib/ # Helpers with no SQL and no outbound fetch: JWT, slugs, files, …
db/
connection.ts # The database handle and transaction(); creates a new database
startup.ts # Housekeeping run when the database opens
schema.sql # Database schema
init.ts # First-run database initialisation
migrate.ts # Migration runner
migrations/ # Versioned schema migrations
test/ # Backend tests (Deno): test/<layer>/<module>.test.ts
sql/
gerbeur.db # SQLite database (not committed)
uploads/ # User-uploaded files — avatars, dumps, thumbnails, … (not committed)
gerbeur.db # SQLite database (not committed); <name>.db per dev instance
uploads/ # User-uploaded files — avatars, dumps, thumbnails, … and scaled/, their copies made on request (not committed); <name>/ per dev instance
src/ # React frontend (Vite)
config/ # API base URL, validation constants, feed tabs, upload limits
pages/ # Route-level components
index/ # Feed views (hot, new, followed, journal)
components/ # Shared UI components
form/ # Reusable form controls
contexts/ # Auth, WebSocket, player, follows, theme
hooks/ # Data fetching and UI hooks
utils/ # Formatting, URL, hot-score, waveform, … helpers
locales/ # Lingui message catalogues (en, fr)
config/ # API base URL, default feed tab, player mode, instance settings
pages/ # One component per route, nothing else
components/ # React components only (.tsx)
contexts/ # FooContext.ts + FooProvider.tsx pairs
hooks/ # Every hook (useFoo.ts)
utils/ # Every non-React module: pure helpers, storage, API glue
test/ # Frontend tests (Vitest): test/<layer>/<domain>/<Module>.test.tsx
themes/ # Per-theme CSS files
model.ts # Shared frontend types
i18n.ts # Lingui runtime setup
public/ # Static assets (favicon, manifest, service worker)
```
### URLs
Every page and API path follows one of two rules, which [API.md](API.md) states,
along with every page, the slug and username rules and every endpoint. Pages are
a hierarchy under `@name` for users and plain words for categories and app pages
(`/@bob/dumps/never-gonna`, `/music/hot`), built and parsed by
`shared/paths.ts`. API resources are `/api/<plural>/<key>`, the caller's own
under `/api/me`. A route change updates its router's module doc and API.md
together.
Whatever crosses the wire is defined once, in `shared/`: the API envelope
(`shared/api.ts`), each domain's types and request bodies (`shared/dump.ts`,
`shared/user.ts`, …), the socket protocol (`shared/ws.ts`), the validation
limits and the role permissions. The server sends Dates; the client reads
`Wire<T>`, the same type with every Date an ISO string, and turns it back with
`src/utils/shared/deserialize.ts`. `shared/` imports nothing from either half.
The backend is split by layer, one kind of module per folder, and each file is
named after its domain: the dumps are `routes/dumps.ts`,
`services/dump-service.ts`, `model/dump.ts` and `validators/dump.ts`. Routes
parse and answer, services hold the logic and the SQL, `remote/` is the only
place that fetches from other sites, and `mail/` renders each email that
`services/email-service.ts` sends. Tests mirror the source one file per module,
so the tests of `services/chat-service.ts` are in
`test/services/chat-service.test.ts`.
### Comments
A comment says what the code can't. If deleting it would lose nothing, it is the
wrong comment: on an export, rewrite it; anywhere else, delete it.
**Every module and every export has a JSDoc** (`/** */`). Each gives at least
one of:
1. **Context**: who uses it and where (a page and its route, a caller, a
WebSocket message).
2. **Usage**: what a caller must know to use it correctly, such as a
precondition, an invariant, what it leaves to the caller, or a side effect.
Add an example only when the call isn't obvious.
3. **Place in the chain**: where its input comes from and where its output goes.
**A doc opens with the role, then the mechanism, then the details.** Its first
sentence says what part of the app the thing serves or what problem it solves,
so a reader who has never seen it knows why it exists. How it does that (what
feeds it, what reads it, how it's split) comes next, and the details last. Each
sentence makes sense given only the ones before it. Not "Declares the three
contexts `WSProvider` fills from the WebSocket. They are split so that a feed
doesn't re-render with every chat message", which leaves the reason for last,
but "Live updates from the server reach the pages through three contexts, split
so that a feed doesn't re-render with every chat message. `WSProvider` fills
them from its WebSocket."
Once the role is said, a type's doc is phrased from the value's side, naming the
producer and the consumer: "Returned to Y by X when Z", "Passed to Y by X". It
is never "What X returns to Y" or "What X passes to Y".
Don't pack a doc into a label followed by a colon and a list ("X: one thing,
plus some other thing"). Write sentences with a subject and a verb.
When the obvious comment is useless, write the useful one: the context is always
there. A **field** (of an interface, or a parameter) gets a doc only when it
adds something its parent's doc doesn't. Context that concerns the whole type
goes on the type, not on one of its fields. The same goes for private functions
and locals. None of these count as a doc:
- a restatement of the name or type (`/** The dump. */`, "Whether it is
loading.", "The props of X.", "Renders the X.");
- a description of what the body, the JSX or the SQL plainly does;
- a list that goes stale (every caller of a widely used component): name the
kind of caller instead.
When a comment is there, it is as short as it can be, often one sentence, and it
is written in full sentences. Labels and fragments ("Upvotes on dumps.") don't
count. A module doc is a few lines at most. It opens with the module's role in
the app, then says how it plays it, who calls it and what it hands back, and it
doesn't repeat what its exports' docs say. `@throws` stays on whatever throws to
its caller, with the statuses of an `APIException` and when each happens. A
router's module doc lists its endpoints (method, path, who may call it), since
the code states them nowhere in one place. A summary never ends in a colon: the
reference shows it alone.
**`//` comments** go wherever the code isn't obvious. They explain why, not
what: a workaround, an ordering constraint, a surprising limitation, a security
reason, an event handler that must not bubble.
**Every claim is checked against the code** (grep the callers). A wrong comment
is worse than none.
### Reference
In the frontend, every exported function states its return type, so the
reference can show it: a component returns `React.JSX.Element` (or `| null`), a
hook returning several things returns a documented `Use<Name>Result` interface.
A component that takes props takes an exported `<Name>Props` interface. React's
types are used through the global `React` namespace rather than a default import
of `react`, which `deno doc` can't follow.
`deno task doc` builds the HTML reference into `docs/` from every module of
`api/`, `shared/` and `src/` but the tests. Its index page is the module doc of
`mod.ts` at the root, which introduces the repository's three parts and their
layers in prose; `deno doc` builds the list of every module itself, grouped by
path. The `docs` workflow publishes it on every push to `main` that touches
them. `deno task doc:lint` checks the rules the linter can see, and that no
summary ends in a colon, through `doc-lint.ts`. Two kinds of diagnostic are let
through. A field without a doc passes, since a field gets one only when it adds
to its type's. `deno doc` doesn't document npm packages, so it takes a type
imported from one (react-hook-form's `FieldValues`, say) for a private one, and
those references pass too.
Each of the frontend's `components/`, `contexts/`, `hooks/` and `utils/` is
split into the same domain folders — `auth`, `category`, `chat`, `comment`,
`dump`, `editor`, `feed`, `form`, `layout`, `player`, `playlist`, `profile`,
`social`, `theme`, `ws` — plus `shared/` for what belongs to no single domain.
So the chat lives in `components/chat/`, `contexts/chat/`, `hooks/chat/` and
`utils/chat/`, and its tests in `test/<same path>`. Files are imported directly
(no `index.ts` barrels), and everything uses named exports.

279
api/config.ts
View File

@@ -1,59 +1,150 @@
export const PORT = Number(Deno.env.get("GERBEUR_PORT")) || 8000;
export const SMTPS_URL = Deno.env.get("GERBEUR_SMTPS_URL")?.trim() || "";
export const FROM_EMAIL = Deno.env.get("GERBEUR_FROM_EMAIL")?.trim() || "";
/**
* Lets a deployment configure the server through the `GERBEUR_*` environment
* variables, which it reads once at import, and holds the fixed limits the
* backend applies. The deno tasks load `.env` with `--env-file`, so changing a
* setting needs a restart.
*
* @module
*/
const DEFAULT_WELCOME_EMAIL_BODY = `# Welcome to {{site_name}}!
import { BASE_STYLE, isStyleName, type StyleName } from "../shared/theme.ts";
import { isLocale, type Locale, SOURCE_LOCALE } from "../shared/locale.ts";
Hi **{{username}}**,
Your account has been created successfully. Welcome aboard!`;
export const WELCOME_EMAIL_BODY =
Deno.env.get("GERBEUR_WELCOME_EMAIL_BODY")?.trim() ||
DEFAULT_WELCOME_EMAIL_BODY;
export const JWT_SECRET = Deno.env.get("GERBEUR_JWT_SECRET")?.trim() || "";
// GERBEUR_LISTEN_HOST controls the network interface Oak binds to.
// Defaults to 0.0.0.0 so Docker port-forwarding works out of the box.
// Set to 127.0.0.1 to restrict to loopback only.
export const LISTEN_HOST = Deno.env.get("GERBEUR_LISTEN_HOST") || "0.0.0.0";
// Public-facing URL of the server — used for CORS, WebSocket origin checks,
// email links, and OG image URLs. Behind a reverse proxy, set this to the
// externally-visible URL (e.g. https://example.com). No trailing slash.
// Defaults to http://localhost:PORT for local development.
export const PUBLIC_URL =
/** The port the server listens on (`GERBEUR_PORT`, default 8000). */
export const PORT: number = Number(Deno.env.get("GERBEUR_PORT")) || 8000;
/**
* The `smtps://` URL of the server `email-service.ts` sends mail through
* (`GERBEUR_SMTPS_URL`). When it is empty, no email is sent: sign-up sends no
* welcome, a password reset request does nothing, and there are no digests.
*/
export const SMTPS_URL: string = Deno.env.get("GERBEUR_SMTPS_URL")?.trim() ||
"";
/**
* Whether the instance sends email, which is whether {@link SMTPS_URL} is
* set. `email-service.ts` opens its transport by it, and index.html tells the
* client, whose settings show the digest switches disabled without it.
*/
export const EMAIL_ENABLED: boolean = SMTPS_URL.length > 0;
/**
* The `From` address of every email (`GERBEUR_FROM_EMAIL`), as an address or
* as `Name <address>`. `email-service.ts` refuses to load without it when
* {@link SMTPS_URL} is set.
*/
export const FROM_EMAIL: string = Deno.env.get("GERBEUR_FROM_EMAIL")?.trim() ||
"";
/**
* The Markdown of the welcome email sent on registration
* (`GERBEUR_WELCOME_EMAIL_BODY`), or null for the built-in one in the
* {@link EMAIL_LOCALE}. `api/mail/welcome.ts` replaces `{{site_name}}` and
* `{{username}}` in it.
*/
export const WELCOME_EMAIL_BODY: string | null =
Deno.env.get("GERBEUR_WELCOME_EMAIL_BODY")?.trim() || null;
/**
* The language of every email (`GERBEUR_EMAIL_LOCALE`). The interface picks
* each visitor's language in their browser, which the server never learns, so
* the emails take the instance's. A value that isn't a locale means the
* source one, English.
*/
export const EMAIL_LOCALE: Locale = (() => {
const locale = Deno.env.get("GERBEUR_EMAIL_LOCALE")?.trim();
return isLocale(locale) ? locale : SOURCE_LOCALE;
})();
/**
* The instance's style (`GERBEUR_DEFAULT_STYLE`). The site shows it until a
* visitor or their account picks one, through index.html, and the emails that
* aren't a user's own (the welcome and the password reset) are drawn in it. A
* value that isn't a style means {@link BASE_STYLE}.
*/
export const DEFAULT_STYLE: StyleName = (() => {
const style = Deno.env.get("GERBEUR_DEFAULT_STYLE")?.trim();
return isStyleName(style) ? style : BASE_STYLE;
})();
/**
* The key lib/jwt.ts signs and verifies tokens with (`GERBEUR_JWT_SECRET`).
* The server refuses to start without it.
*/
export const JWT_SECRET: string = Deno.env.get("GERBEUR_JWT_SECRET")?.trim() ||
"";
/**
* The interface the server binds to (`GERBEUR_LISTEN_HOST`). The default,
* 0.0.0.0, lets Docker forward the port; 127.0.0.1 keeps it to loopback.
*/
export const LISTEN_HOST: string = Deno.env.get("GERBEUR_LISTEN_HOST") ||
"0.0.0.0";
/**
* The URL visitors reach the server at (`GERBEUR_PUBLIC_URL`), without a
* trailing slash. Behind a reverse proxy it must be the outside URL: it is an
* allowed origin, it tells links to this site's own pages from other links,
* and it prefixes the image URLs in their previews.
*/
export const PUBLIC_URL: string =
Deno.env.get("GERBEUR_PUBLIC_URL")?.trim().replace(/\/$/, "") ||
`http://localhost:${PORT}`;
// In single-container deployments the API serves the frontend, so FRONTEND_URL
// equals PUBLIC_URL. Override with GERBEUR_FRONTEND_URL when running the
// frontend on a separate host/port (e.g. Vite dev server or a CDN origin).
export const FRONTEND_URL = Deno.env.get("GERBEUR_FRONTEND_URL")?.trim() ||
/**
* Points password reset links and the Open Graph tags' page URLs at the
* frontend when it isn't served from {@link PUBLIC_URL}, as with a separate
* Vite server, and allows it as an origin. It is `GERBEUR_FRONTEND_URL`,
* without a trailing slash.
*/
export const FRONTEND_URL: string =
Deno.env.get("GERBEUR_FRONTEND_URL")?.trim().replace(/\/$/, "") ||
PUBLIC_URL;
export const DB_PATH = "api/sql/gerbeur.db";
/**
* The SQLite database file (`GERBEUR_DB_PATH`). api/test/setup.ts points it at
* a throwaway file before anything opens the database.
*/
export const DB_PATH: string = Deno.env.get("GERBEUR_DB_PATH") ||
"api/sql/gerbeur.db";
// Upload/files
export const UPLOADS_DIR = "api/uploads";
/**
* The directory uploads are stored under (`GERBEUR_UPLOADS_DIR`). Each kind has
* its subdirectory below, created by the first upload, and each file is named
* after the id of what it belongs to.
*/
export const UPLOADS_DIR: string = Deno.env.get("GERBEUR_UPLOADS_DIR") ||
"api/uploads";
/** Holds the file of each file dump. */
export const DUMPS_DIR = `${UPLOADS_DIR}/dumps`;
/** Holds each user's avatar. */
export const AVATARS_DIR = `${UPLOADS_DIR}/avatars`;
/** Holds each playlist's cover. */
export const PLAYLIST_IMAGES_DIR = `${UPLOADS_DIR}/playlist-images`;
/**
* Holds the images pasted into chat messages, comments, dump notes and
* descriptions. The startup tasks purge those nothing links to.
*/
export const ATTACHMENTS_DIR = `${UPLOADS_DIR}/attachments`;
/**
* Holds the thumbnails a poster uploads (`<id>-custom`) and those the
* thumbnails route generates from file dumps.
*/
export const THUMBNAILS_DIR = `${UPLOADS_DIR}/thumbnails`;
/**
* Holds the scaled-down copies of the images above, which lib/ffmpeg.ts makes
* as they are first asked for, each at its upload's path with its width. It
* can be emptied at any time: the copies are made again.
*/
export const SCALED_DIR = `${UPLOADS_DIR}/scaled`;
export const MAX_IMAGE_SIZE_BYTES = 5 * 1024 * 1024; // 5 MB
export const ALLOWED_IMAGE_MIMES = new Set([
"image/jpeg",
"image/png",
"image/gif",
"image/webp",
]);
/**
* Caps the size of an image upload, in bytes, for avatars, covers, thumbnails
* and attachments. lib/upload.ts states it in megabytes in its error.
*/
export const MAX_IMAGE_SIZE_BYTES = 5 * 1024 * 1024;
export const DUMP_MAX_FILE_SIZE_BYTES = 50 * 1024 * 1024; // 50 MB
export const DUMP_ALLOWED_MIME_PREFIXES = [
/**
* The MIME type prefixes a file dump may have. dump-service also accepts the
* exact types of {@link DUMP_ALLOWED_MIME_TYPES}.
*/
export const DUMP_ALLOWED_MIME_PREFIXES: string[] = [
"text/",
"image/",
"video/",
"audio/",
];
export const DUMP_ALLOWED_MIME_TYPES = new Set([
/** The documents and archives a file dump may be, beyond the prefixes. */
export const DUMP_ALLOWED_MIME_TYPES: Set<string> = new Set([
"application/pdf",
"application/json",
"application/zip",
@@ -66,60 +157,108 @@ export const DUMP_ALLOWED_MIME_TYPES = new Set([
"application/vnd.ms-powerpoint",
]);
// Pagination
/** The page size when a request names none. */
export const PAGINATION_DEFAULT_LIMIT = 20;
/** The largest page size a request may ask for. */
export const PAGINATION_MAX_LIMIT = 100;
// Startup housekeeping
/** How long an attachment nothing links to is kept before startup purges it. */
export const ORPHANED_ATTACHMENTS_RETENTION_HOURS = 1;
/**
* How long an invite stays usable. invite-service refuses an older one, and
* the startup tasks purge it.
*/
export const UNUSED_INVITES_RETENTION_DAYS = 7;
// Validation constraints
export const VALIDATION = {
USERNAME_MIN: 1,
USERNAME_MAX: 32,
PASSWORD_MIN: 8,
PASSWORD_MAX: 128,
DUMP_TITLE_MAX: 200,
DUMP_COMMENT_MAX: 5000,
PLAYLIST_TITLE_MAX: 100,
PLAYLIST_DESCRIPTION_MAX: 2000,
COMMENT_BODY_MAX: 5000,
USER_DESCRIPTION_MAX: 2000,
CATEGORY_NAME_MAX: 50,
CATEGORY_SLUG_MAX: 50,
} as const;
/**
* How often, in minutes, the `/community` page's figures are recomputed
* (`GERBEUR_COMMUNITY_REFRESH_MINUTES`), 15 by default.
*/
export const COMMUNITY_REFRESH_MINUTES: number =
Number(Deno.env.get("GERBEUR_COMMUNITY_REFRESH_MINUTES")) || 15;
// SEO/OG
export const OG_SITE_NAME = Deno.env.get("GERBEUR_SITE_NAME") || "gerbeur";
/**
* The site's name (`GERBEUR_SITE_NAME`). It is filled into index.html, the OG
* tags, the icons' manifest and the emails.
*/
export const OG_SITE_NAME: string = Deno.env.get("GERBEUR_SITE_NAME") ||
"gerbeur";
// Emoji the favicon set + app header logo are generated/rendered from.
// Applied at runtime (server startup + index.html injection), so changing it
// only needs a restart — no rebuild.
export const SITE_EMOJI = Deno.env.get("GERBEUR_SITE_EMOJI") || "🚚";
/**
* The emoji that serves as the site's logo (`GERBEUR_SITE_EMOJI`). The icons
* script draws the favicons from it, and index.html carries it to the app's
* header, so changing it needs a restart but no rebuild.
*/
export const SITE_EMOJI: string = Deno.env.get("GERBEUR_SITE_EMOJI") || "🚚";
// Which Bandcamp playback path the frontend uses.
// "embed" — the bandcamp.com iframe (cannot autoplay)
// "native" — resolve the page's mp3 streams and play them in the app's own
// player, which autoplays and supports albums as a queue
// Anything unrecognised falls back to "embed", so a typo degrades safely.
/**
* How the frontend plays Bandcamp. `"embed"` uses Bandcamp's iframe, which
* can't autoplay. `"native"` resolves the page's streams through
* `GET /api/bandcamp/tracks` and plays them in the app's player, which
* autoplays and queues an album's tracks.
*/
export type BandcampPlayer = "embed" | "native";
/**
* How the frontend plays Bandcamp (`GERBEUR_BANDCAMP_PLAYER`), passed to it in
* index.html. Any value but `"native"` means `"embed"`, so a typo falls back to
* the player that always works.
*/
export const BANDCAMP_PLAYER: BandcampPlayer =
Deno.env.get("GERBEUR_BANDCAMP_PLAYER")?.trim() === "native"
? "native"
: "embed";
// Background color for generated icons and the manifest. Mirrors the
// hard-coded theme-color in index.html.
/**
* The background of the generated Apple touch icon, the manifest's colors, and
* the accent of previews of this site's own pages. index.html's `theme-color`
* tags don't follow it.
*/
export const THEME_COLOR = "#111827";
const rawOrigins = Deno.env.get("GERBEUR_ALLOWED_ORIGINS") ?? "";
/**
* The origins that CORS in main.ts and the WebSocket route accept. They are
* {@link PUBLIC_URL}, {@link FRONTEND_URL} and the comma-separated
* `GERBEUR_ALLOWED_ORIGINS`.
*/
export const ALLOWED_ORIGINS: string[] = Array.from(
new Set([
PUBLIC_URL,
// FRONTEND_URL is auto-included so the separate-frontend case only needs
// GERBEUR_FRONTEND_URL — no need to repeat it in GERBEUR_ALLOWED_ORIGINS.
FRONTEND_URL,
...rawOrigins.split(",").map((o) => o.trim()).filter(Boolean),
]),
);
/**
* Lets federated sign-in reach instances on private addresses, `localhost`
* among them, and over plain http (`GERBEUR_FEDERATION_ALLOW_PRIVATE=1`), so
* that two instances on one machine can trust each other in development.
* Off, `remote/gerbeur-instance.ts` reaches only public https instances, since
* an identity is only as safe as the connection it came over.
*/
export const FEDERATION_ALLOW_PRIVATE: boolean =
Deno.env.get("GERBEUR_FEDERATION_ALLOW_PRIVATE")?.trim() === "1";
/**
* Which ways federated sign-in goes, each on unless set to `0`.
*
* - `incoming` (`GERBEUR_FEDERATION_INCOMING`): users of other instances sign
* in here. Off, the login and register pages offer no other instance,
* `/manage` lists only the blocked ones, which this instance's users can't
* sign in to (and none with `outgoing` off too), `federation-service.ts`
* refuses their sign-ins, and the accounts that came from one are signed
* out (`getSessionState`) until it is on again. Their homes' events are
* still applied, so that none is lost meanwhile.
* - `outgoing` (`GERBEUR_FEDERATION_OUTGOING`): this instance's users sign in
* to others. Off, `identity-provider-service.ts` lets no instance sign them
* in, and its metadata says so, which the others check before sending
* anyone here to sign in. The settings page lists no instances, since at
* startup `main.ts` withdraws every grant they gave, which signs them out
* there.
*
* index.html tells the client both. An object, so that the tests can turn one
* off for a while.
*/
export const FEDERATION: { incoming: boolean; outgoing: boolean } = {
incoming: Deno.env.get("GERBEUR_FEDERATION_INCOMING")?.trim() !== "0",
outgoing: Deno.env.get("GERBEUR_FEDERATION_OUTGOING")?.trim() !== "0",
};

60
api/db/connection.ts Normal file
View File

@@ -0,0 +1,60 @@
/**
* Opens the database at {@link DB_PATH} when first imported, creating it from
* schema.sql if the file doesn't exist, then applies the pending migrations
* and runs the startup tasks before any importer runs. Only the services
* import it.
*
* @module
*/
import { DatabaseSync } from "node:sqlite";
import { DB_PATH } from "../config.ts";
import { runMigrations } from "./migrate.ts";
import { runStartupTasks } from "./startup.ts";
const isNew = await Deno.stat(DB_PATH).then(() => false, () => true);
/**
* Serves every query of the server, as its only connection. Its queries are
* synchronous, so each one blocks the event loop while it runs.
*/
export const db: DatabaseSync = new DatabaseSync(DB_PATH);
// schema.sql holds every migration's change, so on a new database each
// migration below finds its work done and is only recorded.
if (isNew) {
db.exec(
await Deno.readTextFile(new URL("./schema.sql", import.meta.url)),
);
}
// WAL lets readers run alongside a writer, and with synchronous = NORMAL a
// commit doesn't wait on an fsync: a power loss can lose the last commits but
// never corrupts the file. busy_timeout makes another connection to the file
// wait for the lock instead of failing. SQLite enforces foreign keys, and so
// the schema's ON DELETE cascades, only when each connection asks.
db.exec(`
PRAGMA journal_mode = WAL;
PRAGMA synchronous = NORMAL;
PRAGMA busy_timeout = 5000;
PRAGMA foreign_keys = ON;
`);
runMigrations(db);
await runStartupTasks(db);
/**
* Runs `fn` in a transaction, so its writes commit together, in one commit
* rather than one each, and a throw rolls them all back and rethrows. A call
* inside another joins the outer transaction. `fn` must be synchronous: an
* `await` would commit before the rest of it runs.
*/
export function transaction<T>(fn: () => T): T {
if (db.isTransaction) return fn();
db.exec("BEGIN;");
try {
const result = fn();
db.exec("COMMIT;");
return result;
} catch (err) {
db.exec("ROLLBACK;");
throw err;
}
}

14
api/db/init.ts
View File

@@ -1,14 +0,0 @@
import { DatabaseSync } from "node:sqlite";
import { DB_PATH } from "../config.ts";
try {
await Deno.stat(DB_PATH);
console.log("Database already exists, skipping initialization.");
} catch {
console.log("Initializing database from schema...");
const schema = Deno.readTextFileSync("api/db/schema.sql");
const db = new DatabaseSync(DB_PATH);
db.exec(schema);
db.close();
console.log("Database initialized.");
}

47
api/db/migrate.ts
View File

@@ -1,3 +1,11 @@
/**
* Brings the database up to date with the migrations under migrations/, and
* records each one it applies in `schema_migrations`. connection.ts runs it
* each time the server opens the database.
*
* @module
*/
import type { DatabaseSync } from "node:sqlite";
import { up as up0001CommentLikes } from "./migrations/0001_comment_likes.ts";
import { up as up0002DumpThumbnail } from "./migrations/0002_dump_thumbnail.ts";
@@ -14,15 +22,28 @@ import { up as up0012FixFaviconReclassification } from "./migrations/0012_fix_fa
import { up as up0013DumpUrlCanonical } from "./migrations/0013_dump_url_canonical.ts";
import { up as up0014ChatReadState } from "./migrations/0014_chat_read_state.ts";
import { up as up0015TokenRevocation } from "./migrations/0015_token_revocation.ts";
import { up as up0016PerformanceIndexes } from "./migrations/0016_performance_indexes.ts";
import { up as up0017CaseInsensitiveUsers } from "./migrations/0017_case_insensitive_users.ts";
import { up as up0018AttachmentResourceType } from "./migrations/0018_attachment_resource_type.ts";
import { up as up0019UserSettings } from "./migrations/0019_user_settings.ts";
import { up as up0020DigestSchedule } from "./migrations/0020_digest_schedule.ts";
import { up as up0021PageUrls } from "./migrations/0021_page_urls.ts";
import { up as up0022ReserveCommunity } from "./migrations/0022_reserve_community.ts";
import { up as up0023MentionEveryone } from "./migrations/0023_mention_everyone.ts";
import { up as up0024Federation } from "./migrations/0024_federation.ts";
import { up as up0025PlaylistCategories } from "./migrations/0025_playlist_categories.ts";
interface Migration {
name: string;
up: (db: DatabaseSync) => void;
}
// Append-only — never edit or reorder a migration once it has shipped.
// Each `up` must be idempotent: it also runs against fresh databases created
// straight from schema.sql, where the change it makes already exists.
/**
* Lists every migration, in the order they run. A new one goes at the end,
* along with the same change to schema.sql, and a shipped one is never edited
* or moved. A database that connection.ts made from schema.sql records none of
* them, so each `up` must find its change already made and leave it be.
*/
const MIGRATIONS: Migration[] = [
{ name: "0001_comment_likes", up: up0001CommentLikes },
{ name: "0002_dump_thumbnail", up: up0002DumpThumbnail },
@@ -42,8 +63,28 @@ const MIGRATIONS: Migration[] = [
{ name: "0013_dump_url_canonical", up: up0013DumpUrlCanonical },
{ name: "0014_chat_read_state", up: up0014ChatReadState },
{ name: "0015_token_revocation", up: up0015TokenRevocation },
{ name: "0016_performance_indexes", up: up0016PerformanceIndexes },
{ name: "0017_case_insensitive_users", up: up0017CaseInsensitiveUsers },
{
name: "0018_attachment_resource_type",
up: up0018AttachmentResourceType,
},
{ name: "0019_user_settings", up: up0019UserSettings },
{ name: "0020_digest_schedule", up: up0020DigestSchedule },
{ name: "0021_page_urls", up: up0021PageUrls },
{ name: "0022_reserve_community", up: up0022ReserveCommunity },
{ name: "0023_mention_everyone", up: up0023MentionEveryone },
{ name: "0024_federation", up: up0024Federation },
{ name: "0025_playlist_categories", up: up0025PlaylistCategories },
];
/**
* Applies each migration `db` hasn't recorded yet, in order, each in its own
* transaction.
*
* @throws {Error} When a migration fails. That one is rolled back, the later
* ones don't run, and the server doesn't start.
*/
export function runMigrations(db: DatabaseSync): void {
db.exec(`CREATE TABLE IF NOT EXISTS schema_migrations (
name TEXT PRIMARY KEY,

14
api/db/migrations/0001_comment_likes.ts
View File

@@ -1,7 +1,17 @@
/**
* Adds likes on comments, as the `comment_likes` table and a `like_count` on
* each comment that comment-like-service keeps in step, so that listing
* comments doesn't count their likes.
*
* @module
*/
import type { DatabaseSync } from "node:sqlite";
// Idempotent: safe to run against a fresh db (already created from schema.sql
// with these columns/tables) or an existing one that predates them.
/**
* Run once by `runMigrations`. Existing comments start with no likes, and a
* database made from schema.sql already has both.
*/
export function up(db: DatabaseSync): void {
const commentCols = db.prepare(`PRAGMA table_info(comments);`).all() as {
name: string;

14
api/db/migrations/0002_dump_thumbnail.ts
View File

@@ -1,7 +1,17 @@
/**
* Lets a poster upload a thumbnail that replaces the fetched one. It adds
* `dumps.custom_thumbnail_mime`, which holds the uploaded image's type; NULL
* means the dump has none.
*
* @module
*/
import type { DatabaseSync } from "node:sqlite";
// Idempotent: safe to run against a fresh db (already created from schema.sql
// with this column) or an existing one that predates it.
/**
* Run once by `runMigrations`. Existing dumps keep their fetched thumbnails,
* and a database made from schema.sql already has the column.
*/
export function up(db: DatabaseSync): void {
const dumpCols = db.prepare(`PRAGMA table_info(dumps);`).all() as {
name: string;

15
api/db/migrations/0003_dump_backlinks.ts
View File

@@ -1,7 +1,18 @@
/**
* Lets a dump page list the dumps related to it. It adds `dump_backlinks`,
* which records the links from a dump's note or comments to other dumps and
* which backlink-service fills, and an index on `dumps.url` to find the dumps
* an external link was posted as.
*
* @module
*/
import type { DatabaseSync } from "node:sqlite";
// Idempotent: safe to run against a fresh db (already created from schema.sql
// with these) or an existing one that predates them.
/**
* Run once by `runMigrations`. It doesn't scan the notes and comments already
* posted, so their links are recorded only once they are edited.
*/
export function up(db: DatabaseSync): void {
db.exec(`CREATE TABLE IF NOT EXISTS dump_backlinks (
source_type TEXT NOT NULL,

16
api/db/migrations/0004_user_roles.ts
View File

@@ -1,7 +1,16 @@
/**
* Makes room for moderators between users and admins. It adds `users.role`,
* which replaces the `is_admin` flag, and 0005 then drops the flag.
*
* @module
*/
import type { DatabaseSync } from "node:sqlite";
// Idempotent: safe to run against a fresh db (already created from schema.sql
// with the `role` column) or an existing one that predates it.
/**
* Run once by `runMigrations`. Every user becomes a `user`, except those with
* `is_admin` set, who become admins.
*/
export function up(db: DatabaseSync): void {
const userCols = db.prepare(`PRAGMA table_info(users);`).all() as {
name: string;
@@ -12,8 +21,7 @@ export function up(db: DatabaseSync): void {
);
}
// Backfill existing admins to role 'admin'. Guarded because the legacy
// is_admin column is absent on fresh databases (and dropped by 0005).
// schema.sql never had is_admin.
if (userCols.some((c) => c.name === "is_admin")) {
db.exec(
`UPDATE users SET role = 'admin' WHERE is_admin = 1 AND role != 'admin';`,

13
api/db/migrations/0005_drop_is_admin.ts
View File

@@ -1,8 +1,15 @@
/**
* Drops `users.is_admin`, now that 0004 has copied it into `users.role`.
*
* @module
*/
import type { DatabaseSync } from "node:sqlite";
// Drops the legacy is_admin column now that `role` is the single source of
// truth. Idempotent: skips when the column is already gone (fresh databases
// created from schema.sql never had it). Must run after 0004 has backfilled.
/**
* Run once by `runMigrations`, after 0004. A database made from schema.sql
* never had the column.
*/
export function up(db: DatabaseSync): void {
const userCols = db.prepare(`PRAGMA table_info(users);`).all() as {
name: string;

13
api/db/migrations/0006_categories.ts
View File

@@ -1,7 +1,16 @@
/**
* Adds categories, which admins create and posters file their dumps under, as
* the `categories` table and the `dump_categories` join table.
*
* @module
*/
import type { DatabaseSync } from "node:sqlite";
// Idempotent: safe to run against a fresh db (already created from schema.sql
// with the categories tables) or an existing one that predates them.
/**
* Run once by `runMigrations`. Existing dumps start in no category, and a
* database made from schema.sql already has both tables.
*/
export function up(db: DatabaseSync): void {
const tables = new Set(
(db.prepare(

13
api/db/migrations/0007_password_reset_tokens.ts
View File

@@ -1,7 +1,16 @@
/**
* Adds `password_reset_tokens`, which holds the single-use tokens that
* password-reset-service emails to users who forgot their password.
*
* @module
*/
import type { DatabaseSync } from "node:sqlite";
// Idempotent: safe to run against a fresh db (already created from schema.sql
// with password_reset_tokens) or an existing one that predates the table.
/**
* Run once by `runMigrations`. A database made from schema.sql already has the
* table.
*/
export function up(db: DatabaseSync): void {
const tables = new Set(
(db.prepare(

12
api/db/migrations/0008_chat_messages.ts
View File

@@ -1,7 +1,15 @@
/**
* Adds `chat_messages`, which holds the messages of the site-wide chat.
*
* @module
*/
import type { DatabaseSync } from "node:sqlite";
// Idempotent: safe to run against a fresh db (already created from schema.sql
// with this table) or an existing one that predates it.
/**
* Run once by `runMigrations`. A database made from schema.sql already has the
* table.
*/
export function up(db: DatabaseSync): void {
db.exec(`CREATE TABLE IF NOT EXISTS chat_messages (
id TEXT NOT NULL PRIMARY KEY,

20
api/db/migrations/0009_chat_reply.ts
View File

@@ -1,12 +1,18 @@
/**
* Lets a chat message reply to another, through `chat_messages.reply_to_id`.
* The column has no foreign key on purpose: when the replied-to message is
* deleted, the id stays and chat-service's join reads null, so the chat shows
* "deleted message" where the quote was.
*
* @module
*/
import type { DatabaseSync } from "node:sqlite";
// Adds an optional self-reference so a chat message can reply to another.
// Kept as a plain column (no FK): when the replied-to message is later deleted
// the id is retained but the join yields null, letting the UI show a subtle
// "deleted message" reference instead of breaking the insert.
//
// Idempotent: also runs against fresh databases created from schema.sql, where
// the column already exists.
/**
* Run once by `runMigrations`. Existing messages reply to nothing, and a
* database made from schema.sql already has the column.
*/
export function up(db: DatabaseSync): void {
const cols = db.prepare(`PRAGMA table_info(chat_messages);`).all() as {
name: string;

39
api/db/migrations/0010_youtube_embed_start.ts
View File

@@ -1,21 +1,22 @@
/**
* Makes the YouTube dumps posted with a start time, before the YouTube provider
* kept it, start there instead of at 0:00. `rich_content.embedUrl` is computed
* when a dump is posted, and it used to drop the source URL's `t` (`?t=1265`,
* `?t=1h2m3s`); this adds the `start` parameter back to the stored embed URLs.
*
* It reads the time from the stored source URL and fetches nothing. It keeps
* its own copy of the parsing, so that it migrates every database alike should
* the provider's change.
*
* @module
*/
import type { DatabaseSync } from "node:sqlite";
// Backfills the `start` parameter on stored YouTube embed URLs.
//
// `rich_content.embedUrl` is computed once, at dump creation, and previously
// dropped the timestamp carried by the source URL (`?t=1265`, `?t=1h2m3s`), so
// those videos always opened at 0:00 in the global player. The provider now
// translates `t`/`start` into the `start` param the iframe player honours; this
// migration applies the same translation to rows written before that fix.
//
// Purely local — it re-derives the param from the already-stored source URL and
// makes no network calls. Self-contained (no imports from api/services) so it
// stays reproducible even if the provider's parsing evolves later.
//
// Idempotent: rows whose embed URL already carries a `start` are skipped, so a
// fresh database built from schema.sql is simply a no-op.
/** Parse `t`/`start` — plain seconds, `90s`, or `1h2m3s` / `20m5s`. */
/**
* Returns the seconds in `url`'s `t` or `start`, which may be plain seconds,
* `90s`, `20m5s` or `1h2m3s`.
*/
function extractStartSeconds(url: string): number | null {
let raw: string | null;
try {
@@ -34,6 +35,10 @@ function extractStartSeconds(url: string): number | null {
Number(hms[3] ?? 0);
}
/**
* Run once by `runMigrations`. It leaves alone an embed URL that already has a
* `start`, and a row whose `rich_content` doesn't parse.
*/
export function up(db: DatabaseSync): void {
const rows = db.prepare(
`SELECT id, url, rich_content FROM dumps
@@ -50,7 +55,7 @@ export function up(db: DatabaseSync): void {
try {
rich = JSON.parse(row.rich_content);
} catch {
continue; // malformed payload — leave it untouched
continue;
}
if (rich.type !== "youtube" || !rich.embedUrl) continue;

50
api/db/migrations/0011_split_favicon_thumbnail.ts
View File

@@ -1,30 +1,25 @@
/**
* Stops older dumps from showing a site's icon as their thumbnail. The metadata
* fetch used to fall back on the page's icon, and then on a guessed
* `/favicon.ico`, so most thumbnails were a 16×16 icon, or a 404, that the UI
* stretched over a 128×72 box. It now stops at real artwork, and a missing
* `thumbnailUrl` means none, which the frontend draws a placeholder for. This
* gives the older rows that meaning, by moving their icons out of
* `rich_content.thumbnailUrl` into the new `faviconUrl`.
*
* It judges the stored URL alone and fetches nothing, so it doesn't fill
* `accentColor`. The frontend derives a hue from the hostname when it is
* missing, and refreshing a dump's metadata fetches the real one.
*
* @module
*/
import type { DatabaseSync } from "node:sqlite";
// Moves favicon-shaped values out of `rich_content.thumbnailUrl` into the new
// `faviconUrl` field.
//
// The extraction cascade used to end at the page's icon and then at a guessed
// `${origin}/favicon.ico`, so `thumbnailUrl` was almost never empty — it just
// held a 16×16 icon (or a 404) that the UI then cover-cropped into a 128×72
// box. The cascade now stops at real artwork, and an absent `thumbnailUrl`
// means "no artwork", which is what lets the frontend draw a placeholder.
// This migration gives rows written before that change the same meaning.
//
// Purely local — it classifies the already-stored URL and makes no network
// calls, so `accentColor` is deliberately not backfilled: the frontend derives
// a stable hue from the hostname whenever one is missing, and
// `refreshDumpMetadata` fetches the real color on demand.
//
// Idempotent: rows that already carry a `faviconUrl` are skipped, so a fresh
// database built from schema.sql is a no-op.
/**
* Whether a stored thumbnail URL is really a site icon.
*
* Deliberately loose. A false positive (a genuine cover image living under
* `/assets/icons/`) renders contained on a tinted field instead of
* cover-cropped — mildly wrong, never broken — so chasing them isn't worth the
* extra rules.
* Checks whether a stored thumbnail URL is a site icon. It is loose on
* purpose, since artwork taken for an icon still shows whole on a tinted field,
* only not cropped to fill the box. 0012 fixes the rows it got wrong.
*/
function isIconUrl(raw: string): boolean {
let pathname: string;
@@ -37,6 +32,11 @@ function isIconUrl(raw: string): boolean {
/\.(ico|svg)$/i.test(pathname);
}
/**
* Run once by `runMigrations`. It leaves alone a row that already has a
* `faviconUrl`, and one whose `rich_content` doesn't parse. 0012 undoes its
* mistakes.
*/
export function up(db: DatabaseSync): void {
const rows = db.prepare(
`SELECT id, rich_content FROM dumps
@@ -53,7 +53,7 @@ export function up(db: DatabaseSync): void {
try {
rich = JSON.parse(row.rich_content);
} catch {
continue; // malformed payload — leave it untouched
continue;
}
if (rich.faviconUrl || !rich.thumbnailUrl) continue;
if (!isIconUrl(rich.thumbnailUrl)) continue;

48
api/db/migrations/0012_fix_favicon_reclassification.ts
View File

@@ -1,23 +1,19 @@
/**
* Repairs the rows 0011 got wrong. 0011 took any path containing "favicon",
* or ending in `.svg`, for an icon, so artwork such as
* `/wp-content/uploads/2022/04/mir-logo-favicon.png` lost its `thumbnailUrl`,
* and its dump showed as a pull-quote instead of an image in the journal
* mosaic.
*
* This rule judges the file name and its folder instead. It spares
* `mir-logo-favicon.png` and catches `icon_SEARCH.png`, which 0011 missed.
*
* @module
*/
import type { DatabaseSync } from "node:sqlite";
// Repairs 0011, which was too eager about what counts as a site icon.
//
// 0011 tested for "favicon" anywhere in the path, so a legitimate `og:image`
// like mirtitles.org's `/wp-content/uploads/2022/04/mir-logo-favicon.png` was
// moved into `faviconUrl` and its `thumbnailUrl` cleared — which also demoted
// the dump from an image card to a pull-quote in the journal mosaic. A blanket
// `.svg` match had the same effect on real artwork.
//
// `isIconUrl` here looks at the *filename* rather than the whole path: an icon
// is named `favicon…`, `apple-touch-icon…` or `icon…`, lives in an `icons/`
// directory, or ends in `.ico`. That both spares `mir-logo-favicon.png` and
// catches `icon_SEARCH.png`, which 0011 missed.
//
// Applied in both directions, and deliberately narrow when reversing: a value
// is only promoted back to `thumbnailUrl` if it matches 0011's rule but not
// this one, so rows written by a normal fetch aren't disturbed.
/** 0011's rule, kept verbatim so its false positives can be identified. */
/** 0011's rule, copied as it shipped, to tell which rows it moved. */
function wasIconUrl0011(raw: string): boolean {
let pathname: string;
try {
@@ -29,7 +25,11 @@ function wasIconUrl0011(raw: string): boolean {
/\.(ico|svg)$/i.test(pathname);
}
/** Whether a URL names a site icon, judged on its filename and directory. */
/**
* Checks whether a URL is a site icon. It is one when the file ends in `.ico`,
* sits in a `favicon(s)` or `icon(s)` folder, or is named `favicon`,
* `apple-touch-icon` or `icon`, alone or followed by a digit, `-`, `_` or `.`.
*/
function isIconUrl(raw: string): boolean {
let pathname: string;
try {
@@ -45,6 +45,12 @@ function isIconUrl(raw: string): boolean {
return /^(favicon|apple-touch-icon|icon)(?=$|[-_.\d])/i.test(basename);
}
/**
* Run once by `runMigrations`. It moves a `faviconUrl` back to `thumbnailUrl`
* only when 0011's rule takes it for an icon and this one doesn't. That also
* catches the ones a later fetch stored that this rule doesn't recognise, such
* as a `logo.svg`. It then moves the icons 0011 missed.
*/
export function up(db: DatabaseSync): void {
const rows = db.prepare(
`SELECT id, rich_content FROM dumps
@@ -62,10 +68,9 @@ export function up(db: DatabaseSync): void {
try {
rich = JSON.parse(row.rich_content);
} catch {
continue; // malformed payload — leave it untouched
continue;
}
// Artwork 0011 mistook for an icon: put it back.
if (
rich.faviconUrl && !rich.thumbnailUrl &&
wasIconUrl0011(rich.faviconUrl) && !isIconUrl(rich.faviconUrl)
@@ -79,7 +84,6 @@ export function up(db: DatabaseSync): void {
continue;
}
// An icon 0011's rule didn't recognize: move it now.
if (
rich.thumbnailUrl && !rich.faviconUrl && isIconUrl(rich.thumbnailUrl)
) {

34
api/db/migrations/0013_dump_url_canonical.ts
View File

@@ -1,20 +1,18 @@
import type { DatabaseSync } from "node:sqlite";
/**
* Lets the create form warn that a link was already dumped. It adds
* `dumps.url_canonical`, the key the hint looks a URL up by, and fills it for
* every URL dump already posted.
*
* It fills the key from each stored URL and fetches nothing. Its helpers are a
* copy of lib/canonical-url.ts as it stood when this shipped: importing the
* live one would give two databases migrated at different times keys made by
* different rules. A change to lib/canonical-url.ts therefore needs a new
* migration that fills the keys again.
*
* @module
*/
// Adds `dumps.url_canonical` — the lookup key behind the "already dumped?"
// check on the create form — and backfills it for every existing URL dump.
//
// Purely local: it re-derives the key from the URL already stored on each row
// and makes no network calls.
//
// The helpers below are a deliberate frozen copy of `api/lib/canonical-url.ts`
// as it stood when this migration shipped, not an import of it. A migration
// runs exactly once per database, so importing the live version would mean two
// databases migrating at different times end up with keys computed under
// different rules. Improving the shared canonicalizer therefore calls for a new
// backfill migration rather than an edit here.
//
// Idempotent: the column is only added when missing and only rows whose key is
// still NULL are touched, so a fresh database built from schema.sql is a no-op.
import type { DatabaseSync } from "node:sqlite";
const TRACKING_PARAMS = new Set([
"fbclid",
@@ -94,6 +92,10 @@ function canonicalizeUrl(raw: string): string | null {
return `https://${host}${path}${query ? `?${query}` : ""}${hash}`;
}
/**
* Run once by `runMigrations`. It fills only the keys still NULL, and leaves
* NULL a URL that isn't http(s).
*/
export function up(db: DatabaseSync): void {
const columns = db.prepare(`PRAGMA table_info(dumps);`).all() as {
name: string;

27
api/db/migrations/0014_chat_read_state.ts
View File

@@ -1,15 +1,22 @@
/**
* Moves the chat's unread state to the server, so that it survives a reload
* and agrees across tabs and devices. `users.chat_read_at` holds the time of
* the newest message the user has seen.
*
* It also replaces the index on `created_at` with one on `(created_at, id)`,
* which the history now pages on, so that two messages from the same
* millisecond can't fall between two pages.
*
* @module
*/
import type { DatabaseSync } from "node:sqlite";
// Chat unread state lives on the server so it survives reloads and agrees
// across tabs and devices: `users.chat_read_at` is the created_at of the newest
// message the user has seen. Existing users start caught up (now) rather than
// with the whole history unread; new users fall back to their own created_at.
//
// The history cursor now pages on (created_at, id) so two messages sharing a
// millisecond can't be skipped at a page boundary; the index follows suit.
//
// Idempotent: also runs against fresh databases created from schema.sql, where
// the column and index already exist.
/**
* Run once by `runMigrations`. Existing users start with every message read,
* instead of the whole history unread. A user created later starts with a
* NULL `chat_read_at`, which chat-service reads as the user's creation time.
*/
export function up(db: DatabaseSync): void {
const cols = db.prepare(`PRAGMA table_info(users);`).all() as {
name: string;

19
api/db/migrations/0015_token_revocation.ts
View File

@@ -1,12 +1,17 @@
/**
* Lets a password change end every session opened before it. Tokens are
* stateless JWTs, so this adds `users.tokens_valid_after`, and lib/auth.ts
* refuses a token issued before that time.
*
* @module
*/
import type { DatabaseSync } from "node:sqlite";
// Changing or resetting a password must end every session opened before it.
// Tokens are stateless JWTs, so each user carries a cut-off instead:
// `users.tokens_valid_after` (ISO), and a token issued before it is refused.
// NULL — the state every existing user starts in — means no cut-off.
//
// Idempotent: also runs against fresh databases created from schema.sql, where
// the column already exists.
/**
* Run once by `runMigrations`. Existing users start with NULL, which revokes no
* token.
*/
export function up(db: DatabaseSync): void {
const cols = db.prepare(`PRAGMA table_info(users);`).all() as {
name: string;

38
api/db/migrations/0016_performance_indexes.ts Normal file
View File

@@ -0,0 +1,38 @@
/**
* Adds indexes for the queries that scanned a whole table. The feeds page
* dumps by `created_at`, and a user's dumps and upvotes by `(user_id,
* created_at)`, which replace the indexes on `user_id` alone. Dumps and
* playlists are looked up by slug. Each foreign key with ON DELETE gets an
* index on its child column, which SQLite otherwise scans on every delete of
* a parent row, and the followers of a user or playlist are read on every new
* dump. Mentions look usernames up by `lower(username)`.
*
* @module
*/
import type { DatabaseSync } from "node:sqlite";
/**
* Run once by `runMigrations`. A database made from schema.sql already has
* these indexes.
*/
export function up(db: DatabaseSync): void {
db.exec(`
DROP INDEX IF EXISTS idx_dumps_user;
DROP INDEX IF EXISTS idx_votes_user;
CREATE INDEX IF NOT EXISTS idx_dumps_created ON dumps(created_at);
CREATE INDEX IF NOT EXISTS idx_dumps_user_created ON dumps(user_id, created_at);
CREATE INDEX IF NOT EXISTS idx_dumps_slug ON dumps(slug);
CREATE INDEX IF NOT EXISTS idx_votes_user_created ON votes(user_id, created_at);
CREATE INDEX IF NOT EXISTS idx_playlists_slug ON playlists(slug);
CREATE INDEX IF NOT EXISTS idx_comments_parent ON comments(parent_id);
CREATE INDEX IF NOT EXISTS idx_comments_user ON comments(user_id);
CREATE INDEX IF NOT EXISTS idx_follows_followed_user ON follows(followed_user_id);
CREATE INDEX IF NOT EXISTS idx_follows_followed_playlist ON follows(followed_playlist_id);
CREATE INDEX IF NOT EXISTS idx_chat_messages_user ON chat_messages(user_id);
CREATE INDEX IF NOT EXISTS idx_invites_inviter ON invites(inviter_id);
CREATE INDEX IF NOT EXISTS idx_users_invited_by ON users(invited_by);
CREATE INDEX IF NOT EXISTS idx_password_reset_tokens_user ON password_reset_tokens(user_id);
CREATE INDEX IF NOT EXISTS idx_users_username_lower ON users(lower(username));
`);
}

36
api/db/migrations/0017_case_insensitive_users.ts Normal file
View File

@@ -0,0 +1,36 @@
/**
* Makes usernames and emails unique regardless of case. Usernames were unique
* only as typed, so "Bob" and "bob" could both sign up, and a mention, which
* looks names up in lower case, then reached one of them at random. Emails
* had no constraint at all: two sign-ups racing past the service's check could
* share an address, and password resets are looked up by address.
*
* @module
*/
import type { DatabaseSync } from "node:sqlite";
/**
* Run once by `runMigrations`. It refuses to run, naming the clashes, when two
* accounts already share a name or an address in some case: which one keeps
* it is for whoever runs the site to decide, by editing one of them.
*/
export function up(db: DatabaseSync): void {
for (const column of ["username", "email"]) {
const clashes = db.prepare(
`SELECT lower(${column}) AS value FROM users GROUP BY lower(${column}) HAVING COUNT(*) > 1;`,
).all() as { value: string }[];
if (clashes.length > 0) {
throw new Error(
`Several accounts share a ${column} in different cases: ${
clashes.map((c) => c.value).join(", ")
}. Change all but one of each, then restart.`,
);
}
}
db.exec(`
DROP INDEX IF EXISTS idx_users_username_lower;
CREATE UNIQUE INDEX IF NOT EXISTS idx_users_username_lower ON users(lower(username));
CREATE UNIQUE INDEX IF NOT EXISTS idx_users_email_lower ON users(lower(email));
`);
}

36
api/db/migrations/0018_attachment_resource_type.ts Normal file
View File

@@ -0,0 +1,36 @@
/**
* Serves an attachment only to those who may see what it was pasted into, and
* deletes it once that is gone. It adds `attachments.resource_type`, the kind
* of thing an attachment's `resource_id` names. Without it, an attachment
* couldn't tell whose it was: one pasted into a private dump was served to
* anyone with its URL, and one whose dump, comment or message was deleted was
* kept for good.
*
* @module
*/
import type { DatabaseSync } from "node:sqlite";
/**
* Run once by `runMigrations`. Existing bound attachments get the type of the
* table that holds their resource; one whose resource is gone gets none, and
* the startup purge deletes it. A database made from schema.sql already has
* the column.
*/
export function up(db: DatabaseSync): void {
const cols = db.prepare(`PRAGMA table_info(attachments);`).all() as {
name: string;
}[];
if (cols.some((c) => c.name === "resource_type")) return;
db.exec(`
ALTER TABLE attachments ADD COLUMN resource_type TEXT;
UPDATE attachments SET resource_type = CASE
WHEN resource_id IN (SELECT id FROM dumps) THEN 'dump'
WHEN resource_id IN (SELECT id FROM comments) THEN 'comment'
WHEN resource_id IN (SELECT id FROM playlists) THEN 'playlist'
WHEN resource_id IN (SELECT id FROM chat_messages) THEN 'chat'
WHEN resource_id IN (SELECT id FROM users) THEN 'user'
END
WHERE resource_id IS NOT NULL;
`);
}

28
api/db/migrations/0019_user_settings.ts Normal file
View File

@@ -0,0 +1,28 @@
/**
* Gives each user settings stored on the server, so that they follow the
* account across browsers and the server can read them: the digests need the
* reader's choice to receive them and their style. `user_settings` holds one
* row per user who changed anything; a user without one has the defaults.
*
* @module
*/
import type { DatabaseSync } from "node:sqlite";
/**
* Run once by `runMigrations`. Existing users get no row, so they start with
* the defaults, digests on, like a new account.
*/
export function up(db: DatabaseSync): void {
db.exec(`
CREATE TABLE IF NOT EXISTS user_settings (
user_id TEXT PRIMARY KEY,
style TEXT,
color_scheme TEXT,
digest_weekly INTEGER NOT NULL DEFAULT 1,
digest_monthly INTEGER NOT NULL DEFAULT 1,
updated_at TEXT NOT NULL,
FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE
);
`);
}

52
api/db/migrations/0020_digest_schedule.ts Normal file
View File

@@ -0,0 +1,52 @@
/**
* Lets `digest-service.ts` know when each user's next digest is due.
* `digest_schedule` holds, for each user and cadence they receive, the start
* of the period their next digest covers; it is due once a full period has
* passed.
*
* @module
*/
import type { DatabaseSync } from "node:sqlite";
const DAY_MS = 86_400_000;
// A frozen copy of digest-service's periods: a migration must keep doing
// what it did when it shipped, whatever the service later becomes.
const PERIOD_DAYS = { weekly: 7, monthly: 30 } as const;
/** Spreads `userId` over `[0, days)` days, the same way each run. */
function offsetOf(userId: string, days: number): number {
let hash = 2166136261;
for (const ch of userId) hash = Math.imul(hash ^ ch.charCodeAt(0), 16777619);
return ((hash >>> 0) % (days * 24)) * 3_600_000;
}
/**
* Run once by `runMigrations`. Every user present gets both cadences, with a
* first period that ends at some hour of the coming week or month, picked
* from their id: the digests start trickling out after the deploy instead of
* all leaving a week later. Users created later are scheduled by the service
* when it first sees them.
*/
export function up(db: DatabaseSync): void {
db.exec(`
CREATE TABLE IF NOT EXISTS digest_schedule (
user_id TEXT NOT NULL,
cadence TEXT NOT NULL,
window_start TEXT NOT NULL,
PRIMARY KEY (user_id, cadence),
FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE
);
`);
const users = db.prepare(`SELECT id FROM users;`).all() as { id: string }[];
const insert = db.prepare(
`INSERT OR IGNORE INTO digest_schedule (user_id, cadence, window_start) VALUES (?, ?, ?);`,
);
const now = Date.now();
for (const { id } of users) {
for (const [cadence, days] of Object.entries(PERIOD_DAYS)) {
const start = now - days * DAY_MS + offsetOf(id, days);
insert.run(id, cadence, new Date(start).toISOString());
}
}
}

239
api/db/migrations/0021_page_urls.ts Normal file
View File

@@ -0,0 +1,239 @@
/**
* Moves the site to v4's page URLs, where dumps and playlists live under their
* owner (`/@bob/dumps/<slug>`) and are named by an editable slug alone. A
* slug was `slugify(title)-<8 hex of the id>`, unique site-wide; it is now the
* bare `slugify(title)`, unique in its owner's space for its kind, and every
* slug an item or a user leaves behind stays an alias that redirects
* (`dump_slug_aliases`, `playlist_slug_aliases`, `user_aliases`).
*
* v3 URLs aren't redirected. Instead this rewrites the links to this site's
* pages in what users wrote: dump notes, comments, chat messages, and
* playlist and profile descriptions.
*
* SQLite can't make `slug` NOT NULL without rebuilding `dumps` and
* `playlists`, which other tables reference, so the column stays nullable
* and every writer sets it; the unique index is what the services rely on.
*
* @module
*/
import type { DatabaseSync } from "node:sqlite";
import { slugify } from "../../../shared/slug.ts";
import {
dumpPath,
feedPath,
playlistPath,
type ProfileTab,
searchPath,
userPath,
} from "../../../shared/paths.ts";
import { type SlugKind, uniqueSlug } from "../../lib/slugify.ts";
import { renameReservedCategories } from "../../lib/reserved-slugs.ts";
import { isSelfHost } from "../../lib/self-url.ts";
/** An item's new path, by its v3 slug and by its id. */
type PathsByKey = Map<string, string>;
/**
* Gives every dump or playlist of `kind` the bare slug of its title, unique
* per owner, the oldest first so it keeps the bare one. Returns each one's new
* path under both keys a v3 link may have used: its old slug and its id.
*/
function reslug(db: DatabaseSync, kind: SlugKind): PathsByKey {
const table = kind === "dump" ? "dumps" : "playlists";
const rows = db.prepare(
`SELECT t.id, t.user_id, t.title, t.slug, u.username
FROM ${table} t JOIN users u ON u.id = t.user_id
ORDER BY t.created_at, t.id;`,
).all() as {
id: string;
user_id: string;
title: string;
slug: string | null;
username: string;
}[];
// Cleared first, so that an old slug doesn't take a new one's place.
db.exec(`UPDATE ${table} SET slug = NULL;`);
const set = db.prepare(`UPDATE ${table} SET slug = ? WHERE id = ?;`);
const paths: PathsByKey = new Map();
const build = kind === "dump" ? dumpPath : playlistPath;
for (const row of rows) {
const slug = uniqueSlug(db, kind, row.user_id, slugify(row.title));
set.run(slug, row.id);
const path = build({ ownerUsername: row.username, slug });
paths.set(row.id, path);
if (row.slug) paths.set(row.slug, path);
}
return paths;
}
/** Where each v3 profile tab went: a tab, the overview, or a page of its own. */
const PROFILE_TAB_V4: Record<string, ProfileTab | "" | "/settings"> = {
dumps: "",
playlists: "playlists",
followed: "follow",
invitees: "invitees",
settings: "/settings",
manage: "manage",
};
/** The v3 full lists under a profile, which are v4 tabs. */
const PROFILE_LISTS = new Set<string>(["dumps", "upvoted", "playlists"]);
/**
* Matches a link to a v3 page, its origin being optional: an absolute one is
* this site's only when its host is, a relative one only as a Markdown link's
* target. The path stops before a query or a fragment, which stay as they are.
*/
const V3_LINK_RE =
/(https?:\/\/([^/\s)\]<>"']+))?(\/(?:dumps|playlists|users|~|search\/~)\/[^\s)\]<>"'?#]+)/g;
/**
* Returns the v4 path of the v3 page at `path`, or null when it names nothing
* known, which then stays as written: a deleted dump's link is dead either way.
*/
function v4PathOf(
path: string,
dumps: PathsByKey,
playlists: PathsByKey,
username: (name: string) => string | undefined,
): string | null {
let segments: string[];
try {
segments = path.split("/").slice(1).filter(Boolean).map(
decodeURIComponent,
);
} catch {
return null;
}
const [root, key, ...rest] = segments;
switch (root) {
case "dumps": {
const target = dumps.get(key);
if (!target) return null;
if (rest.length === 0) return target;
return rest.length === 1 && rest[0] === "edit" ? `${target}/edit` : null;
}
case "playlists":
return rest.length === 0 ? playlists.get(key) ?? null : null;
case "users": {
const name = username(key) ?? key;
if (rest.length === 0) return userPath(name);
if (rest.length === 1 && PROFILE_LISTS.has(rest[0])) {
return userPath(name, rest[0] as ProfileTab);
}
if (rest.length === 2 && rest[0] === "~" && rest[1] in PROFILE_TAB_V4) {
const tab = PROFILE_TAB_V4[rest[1]];
if (tab === "/settings") return tab;
return tab ? userPath(name, tab) : userPath(name);
}
return null;
}
case "~":
if (rest.length > 0) return null;
return key === "followed"
? feedPath(undefined, "follow")
: ["hot", "new", "journal"].includes(key)
? feedPath(undefined, key as "hot")
: null;
case "search":
// `/search/~/users`, split as "search", "~", "users".
return key === "~" && (rest[0] === "users" || rest[0] === "playlists") &&
rest.length === 1
? searchPath(rest[0])
: null;
}
return null;
}
/**
* Rewrites the links to v3 pages of this site in the text column `column` of
* `table`, in every row that has one.
*/
function rewriteLinks(
db: DatabaseSync,
table: string,
column: string,
toV4: (path: string) => string | null,
): void {
const rows = db.prepare(
`SELECT id, ${column} AS text FROM ${table} WHERE ${column} IS NOT NULL;`,
).all() as { id: string; text: string }[];
const update = db.prepare(`UPDATE ${table} SET ${column} = ? WHERE id = ?;`);
for (const { id, text } of rows) {
const rewritten = text.replace(
V3_LINK_RE,
(match, origin: string | undefined, host: string, path: string, at) => {
if (origin ? !isSelfHost(host) : text[at - 1] !== "(") return match;
const v4 = toV4(path);
return v4 ? `${origin ?? ""}${v4}` : match;
},
);
if (rewritten !== text) update.run(rewritten, id);
}
}
/**
* Run once by `runMigrations`. On a database made from schema.sql the tables
* and indexes exist and there is nothing to re-slug or rewrite.
*/
export function up(db: DatabaseSync): void {
db.exec(`
CREATE TABLE IF NOT EXISTS dump_slug_aliases (
user_id TEXT NOT NULL,
slug TEXT NOT NULL,
dump_id TEXT NOT NULL REFERENCES dumps(id) ON DELETE CASCADE,
created_at TEXT NOT NULL,
PRIMARY KEY (user_id, slug)
);
CREATE INDEX IF NOT EXISTS idx_dump_slug_aliases_dump
ON dump_slug_aliases(dump_id, created_at);
CREATE TABLE IF NOT EXISTS playlist_slug_aliases (
user_id TEXT NOT NULL,
slug TEXT NOT NULL,
playlist_id TEXT NOT NULL REFERENCES playlists(id) ON DELETE CASCADE,
created_at TEXT NOT NULL,
PRIMARY KEY (user_id, slug)
);
CREATE INDEX IF NOT EXISTS idx_playlist_slug_aliases_playlist
ON playlist_slug_aliases(playlist_id, created_at);
CREATE TABLE IF NOT EXISTS user_aliases (
username TEXT NOT NULL,
user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
created_at TEXT NOT NULL,
PRIMARY KEY (username)
);
CREATE UNIQUE INDEX IF NOT EXISTS idx_user_aliases_lower
ON user_aliases(lower(username));
CREATE INDEX IF NOT EXISTS idx_user_aliases_user
ON user_aliases(user_id, created_at);
DROP INDEX IF EXISTS idx_dumps_slug;
DROP INDEX IF EXISTS idx_playlists_slug;
`);
const dumps = reslug(db, "dump");
const playlists = reslug(db, "playlist");
db.exec(`
CREATE UNIQUE INDEX IF NOT EXISTS idx_dumps_user_slug
ON dumps(user_id, slug);
CREATE UNIQUE INDEX IF NOT EXISTS idx_playlists_user_slug
ON playlists(user_id, slug);
`);
renameReservedCategories(db);
const findUser = db.prepare(
`SELECT username FROM users WHERE lower(username) = lower(?);`,
);
const username = (name: string) =>
(findUser.get(name) as { username: string } | undefined)?.username;
const toV4 = (path: string) => v4PathOf(path, dumps, playlists, username);
for (
const [table, column] of [
["dumps", "comment"],
["comments", "body"],
["chat_messages", "body"],
["playlists", "description"],
["users", "description"],
]
) rewriteLinks(db, table, column, toV4);
}

14
api/db/migrations/0022_reserve_community.ts Normal file
View File

@@ -0,0 +1,14 @@
/**
* Reserves `community` at the URL root for the community page, renaming a
* category that had it as its slug.
*
* @module
*/
import type { DatabaseSync } from "node:sqlite";
import { renameReservedCategories } from "../../lib/reserved-slugs.ts";
/** Run once by `runMigrations`. */
export function up(db: DatabaseSync): void {
renameReservedCategories(db);
}

74
api/db/migrations/0023_mention_everyone.ts Normal file
View File

@@ -0,0 +1,74 @@
/**
* Prepares the chat's `@everyone`: it adds `chat_messages.mentions_everyone`,
* and frees the name `everyone`, which no account may have any more.
*
* @module
*/
import type { DatabaseSync } from "node:sqlite";
import { EVERYONE, mentionsEveryone } from "../../../shared/mention.ts";
import { ROLE_PERMISSIONS } from "../../../shared/permissions.ts";
/**
* Run once by `runMigrations`. The messages that a moderator or an admin
* already sent with `@everyone` get the flag, so that editing one later
* doesn't notify everyone of an old message; none is notified now. A user
* named `everyone`, in any case, is renamed to the first free `everyone_2`,
* `everyone_3`…, with no alias left behind, and an alias `everyone` is
* dropped. Both are logged.
*/
export function up(db: DatabaseSync): void {
const cols = db.prepare(`PRAGMA table_info(chat_messages);`).all() as {
name: string;
}[];
if (!cols.some((c) => c.name === "mentions_everyone")) {
db.exec(
`ALTER TABLE chat_messages ADD COLUMN mentions_everyone INTEGER NOT NULL DEFAULT 0;`,
);
}
const roles = Object.entries(ROLE_PERMISSIONS)
.filter(([, perms]) => perms.includes("chat:everyone"))
.map(([role]) => role);
const messages = db.prepare(
`SELECT m.id, m.body FROM chat_messages m JOIN users u ON u.id = m.user_id
WHERE m.mentions_everyone = 0
AND u.role IN (${roles.map(() => "?").join(", ")})
AND m.body LIKE '%@%';`,
).all(...roles) as { id: string; body: string }[];
const flag = db.prepare(
`UPDATE chat_messages SET mentions_everyone = 1 WHERE id = ?;`,
);
for (const m of messages) {
if (mentionsEveryone(m.body)) flag.run(m.id);
}
const taken = (name: string) =>
!!db.prepare(
`SELECT 1 FROM users WHERE lower(username) = lower(?)
UNION ALL SELECT 1 FROM user_aliases WHERE lower(username) = lower(?);`,
).get(name, name);
const user = db.prepare(
`SELECT id, username FROM users WHERE lower(username) = ?;`,
).get(EVERYONE) as { id: string; username: string } | undefined;
if (user) {
let n = 2;
while (taken(`${EVERYONE}_${n}`)) n++;
const renamed = `${EVERYONE}_${n}`;
db.prepare(`UPDATE users SET username = ? WHERE id = ?;`).run(
renamed,
user.id,
);
console.log(
`[migrate] user "${user.username}" is now "${renamed}": @${EVERYONE} is reserved`,
);
}
const alias = db.prepare(
`DELETE FROM user_aliases WHERE lower(username) = ? RETURNING user_id;`,
).get(EVERYONE) as { user_id: string } | undefined;
if (alias) {
console.log(
`[migrate] the former name "${EVERYONE}" of user ${alias.user_id} is dropped: @${EVERYONE} is reserved`,
);
}
}

105
api/db/migrations/0024_federation.ts Normal file
View File

@@ -0,0 +1,105 @@
/**
* Prepares federated sign-in, by which a user signs in here with their
* account on another gerbeur instance, and the other way round. It adds the
* instance's signing key, the link from a local account to the one it stands
* for elsewhere, the instances an admin trusts or blocks, the state of the
* sign-ins in progress on both sides, the grants users gave other instances,
* and the events owed to them.
*
* `/authorize` and `/federation/callback` become pages, so their first
* segments are reserved.
*
* An account from another instance has no password here: its `password_hash`
* is `''`, which no password verifies. Emails stay unique among local accounts
* only, since the same person may also have a local account, or come from two
* instances, with one address.
*
* @module
*/
import type { DatabaseSync } from "node:sqlite";
import { renameReservedCategories } from "../../lib/reserved-slugs.ts";
/**
* Run once by `runMigrations`. Nothing exists to convert, since every account
* so far is local, but `authorize` and `federation` become the pages of a
* sign-in, so a category that had one as its slug is renamed.
*/
export function up(db: DatabaseSync): void {
db.exec(`
CREATE TABLE IF NOT EXISTS federation_keys (
kid TEXT PRIMARY KEY,
private_jwk TEXT NOT NULL,
public_jwk TEXT NOT NULL,
created_at TEXT NOT NULL
);
CREATE TABLE IF NOT EXISTS user_identities (
user_id TEXT PRIMARY KEY REFERENCES users(id) ON DELETE CASCADE,
issuer TEXT NOT NULL,
subject TEXT NOT NULL,
picture TEXT,
synced_at TEXT NOT NULL
);
CREATE UNIQUE INDEX IF NOT EXISTS idx_user_identities_subject
ON user_identities(issuer, subject);
CREATE TABLE IF NOT EXISTS federation_peers (
issuer TEXT PRIMARY KEY,
status TEXT NOT NULL,
name TEXT,
created_at TEXT NOT NULL
);
CREATE TABLE IF NOT EXISTS federation_logins (
state TEXT PRIMARY KEY,
issuer TEXT NOT NULL,
code_verifier TEXT NOT NULL,
nonce TEXT NOT NULL,
invite_token TEXT,
created_at TEXT NOT NULL
);
CREATE TABLE IF NOT EXISTS oauth_codes (
code_hash TEXT PRIMARY KEY,
user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
client_id TEXT NOT NULL,
redirect_uri TEXT NOT NULL,
code_challenge TEXT NOT NULL,
nonce TEXT NOT NULL,
expires_at TEXT NOT NULL
);
CREATE TABLE IF NOT EXISTS federation_grants (
user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
client_id TEXT NOT NULL,
client_name TEXT,
granted_at TEXT NOT NULL,
last_used_at TEXT NOT NULL,
PRIMARY KEY (user_id, client_id)
);
CREATE TABLE IF NOT EXISTS federation_outbox (
id TEXT PRIMARY KEY,
client_id TEXT NOT NULL,
event TEXT NOT NULL,
attempts INTEGER NOT NULL DEFAULT 0,
next_attempt_at TEXT NOT NULL,
created_at TEXT NOT NULL
);
CREATE INDEX IF NOT EXISTS idx_federation_outbox_next
ON federation_outbox(next_attempt_at);
`);
const emailIndex = db.prepare(
`SELECT sql FROM sqlite_master WHERE name = 'idx_users_email_lower';`,
).get() as { sql: string } | undefined;
if (!emailIndex?.sql.includes("WHERE")) {
db.exec(`
DROP INDEX IF EXISTS idx_users_email_lower;
CREATE UNIQUE INDEX idx_users_email_lower ON users(lower(email))
WHERE password_hash != '';
`);
}
renameReservedCategories(db);
}

110
api/db/migrations/0025_playlist_categories.ts Normal file
View File

@@ -0,0 +1,110 @@
/**
* Gives playlists categories of their own, a set apart from the dumps': each
* category gets a `kind`, `dump` or `playlist`, its slug and name now unique
* within its kind only, and `playlist_categories` files playlists under them.
*
* `/playlists` becomes the page of every public playlist, so its first segment
* is reserved.
*
* @module
*/
import type { DatabaseSync } from "node:sqlite";
import { renameReservedCategories } from "../../lib/reserved-slugs.ts";
/**
* Run once by `runMigrations`. Every existing category becomes a dump
* category, filing the same dumps.
*
* SQLite can't change a table's UNIQUE constraints in place, so `categories`
* is rebuilt. `foreign_keys` can't be switched off inside the runner's
* transaction, and dropping `categories` would then empty `dump_categories`
* through its cascade, so both are read, dropped child first, made anew as in
* schema.sql, and refilled. A database made from schema.sql already has the
* `kind` column and is left as it is.
*/
export function up(db: DatabaseSync): void {
const columns = new Set(
(db.prepare(`PRAGMA table_info(categories);`).all() as { name: string }[])
.map((c) => c.name),
);
if (!columns.has("kind")) {
const categories = db.prepare(
`SELECT id, slug, name, position, created_at, updated_at FROM categories;`,
).all() as {
id: string;
slug: string;
name: string;
position: number;
created_at: string;
updated_at: string | null;
}[];
const links = db.prepare(
`SELECT dump_id, category_id FROM dump_categories;`,
).all() as { dump_id: string; category_id: string }[];
db.exec(`DROP TABLE dump_categories;`);
db.exec(`DROP TABLE categories;`);
db.exec(
`CREATE TABLE categories (
id TEXT PRIMARY KEY,
kind TEXT NOT NULL DEFAULT 'dump' CHECK (kind IN ('dump', 'playlist')),
slug TEXT NOT NULL,
name TEXT NOT NULL,
position INTEGER NOT NULL DEFAULT 0,
created_at TEXT NOT NULL,
updated_at TEXT,
UNIQUE (kind, slug),
UNIQUE (kind, name)
);`,
);
db.exec(
`CREATE TABLE dump_categories (
dump_id TEXT NOT NULL,
category_id TEXT NOT NULL,
PRIMARY KEY (dump_id, category_id),
FOREIGN KEY (dump_id) REFERENCES dumps(id) ON DELETE CASCADE,
FOREIGN KEY (category_id) REFERENCES categories(id) ON DELETE CASCADE
);`,
);
db.exec(
`CREATE INDEX idx_dump_categories_category ON dump_categories(category_id);`,
);
const insertCategory = db.prepare(
`INSERT INTO categories (id, kind, slug, name, position, created_at, updated_at)
VALUES (?, 'dump', ?, ?, ?, ?, ?);`,
);
for (const c of categories) {
insertCategory.run(
c.id,
c.slug,
c.name,
c.position,
c.created_at,
c.updated_at,
);
}
const insertLink = db.prepare(
`INSERT INTO dump_categories (dump_id, category_id) VALUES (?, ?);`,
);
for (const l of links) insertLink.run(l.dump_id, l.category_id);
}
db.exec(
`CREATE TABLE IF NOT EXISTS playlist_categories (
playlist_id TEXT NOT NULL,
category_id TEXT NOT NULL,
PRIMARY KEY (playlist_id, category_id),
FOREIGN KEY (playlist_id) REFERENCES playlists(id) ON DELETE CASCADE,
FOREIGN KEY (category_id) REFERENCES categories(id) ON DELETE CASCADE
);`,
);
db.exec(
`CREATE INDEX IF NOT EXISTS idx_playlist_categories_category ON playlist_categories(category_id);`,
);
renameReservedCategories(db);
}

151
api/db/schema.sql
View File

@@ -69,11 +69,14 @@ CREATE TABLE playlist_dumps (
CREATE TABLE categories (
id TEXT PRIMARY KEY,
slug TEXT NOT NULL UNIQUE,
name TEXT NOT NULL UNIQUE,
kind TEXT NOT NULL DEFAULT 'dump' CHECK (kind IN ('dump', 'playlist')),
slug TEXT NOT NULL,
name TEXT NOT NULL,
position INTEGER NOT NULL DEFAULT 0,
created_at TEXT NOT NULL,
updated_at TEXT
updated_at TEXT,
UNIQUE (kind, slug),
UNIQUE (kind, name)
);
CREATE TABLE dump_categories (
@@ -84,6 +87,14 @@ CREATE TABLE dump_categories (
FOREIGN KEY (category_id) REFERENCES categories(id) ON DELETE CASCADE
);
CREATE TABLE playlist_categories (
playlist_id TEXT NOT NULL,
category_id TEXT NOT NULL,
PRIMARY KEY (playlist_id, category_id),
FOREIGN KEY (playlist_id) REFERENCES playlists(id) ON DELETE CASCADE,
FOREIGN KEY (category_id) REFERENCES categories(id) ON DELETE CASCADE
);
CREATE TABLE comments (
id TEXT PRIMARY KEY,
dump_id TEXT NOT NULL,
@@ -119,18 +130,24 @@ CREATE TABLE dump_backlinks (
FOREIGN KEY (to_dump_id) REFERENCES dumps(id) ON DELETE CASCADE
);
CREATE INDEX idx_dumps_user ON dumps(user_id);
CREATE INDEX idx_dumps_created ON dumps(created_at);
CREATE INDEX idx_dumps_user_created ON dumps(user_id, created_at);
CREATE UNIQUE INDEX idx_dumps_user_slug ON dumps(user_id, slug);
CREATE INDEX idx_dumps_url ON dumps(url);
CREATE INDEX idx_dumps_url_canonical ON dumps(url_canonical);
CREATE INDEX idx_votes_user ON votes(user_id);
CREATE INDEX idx_votes_user_created ON votes(user_id, created_at);
CREATE INDEX idx_playlists_user ON playlists(user_id);
CREATE UNIQUE INDEX idx_playlists_user_slug ON playlists(user_id, slug);
CREATE INDEX idx_playlist_dumps_order ON playlist_dumps(playlist_id, position);
CREATE INDEX idx_playlist_dumps_dump ON playlist_dumps(dump_id);
CREATE INDEX idx_comments_dump ON comments(dump_id, created_at);
CREATE INDEX idx_comments_parent ON comments(parent_id);
CREATE INDEX idx_comments_user ON comments(user_id);
CREATE INDEX idx_comment_likes_user ON comment_likes(user_id);
CREATE INDEX idx_dump_backlinks_to ON dump_backlinks(to_dump_id);
CREATE INDEX idx_dump_backlinks_from ON dump_backlinks(from_dump_id);
CREATE INDEX idx_dump_categories_category ON dump_categories(category_id);
CREATE INDEX idx_playlist_categories_category ON playlist_categories(category_id);
CREATE TABLE follows (
id TEXT PRIMARY KEY,
@@ -157,6 +174,8 @@ CREATE UNIQUE INDEX idx_follows_playlist
WHERE followed_playlist_id IS NOT NULL;
CREATE INDEX idx_follows_follower ON follows(follower_id);
CREATE INDEX idx_follows_followed_user ON follows(followed_user_id);
CREATE INDEX idx_follows_followed_playlist ON follows(followed_playlist_id);
CREATE TABLE invites (
token TEXT PRIMARY KEY,
@@ -166,11 +185,20 @@ CREATE TABLE invites (
FOREIGN KEY (inviter_id) REFERENCES users(id) ON DELETE CASCADE
);
CREATE INDEX idx_invites_inviter ON invites(inviter_id);
CREATE INDEX idx_users_invited_by ON users(invited_by);
CREATE UNIQUE INDEX idx_users_username_lower ON users(lower(username));
-- An account from another instance has no password (''), and its email may
-- also be a local account's.
CREATE UNIQUE INDEX idx_users_email_lower ON users(lower(email))
WHERE password_hash != '';
CREATE TABLE attachments (
id TEXT PRIMARY KEY,
resource_id TEXT,
mime TEXT NOT NULL,
created_at TEXT NOT NULL
created_at TEXT NOT NULL,
resource_type TEXT
);
CREATE INDEX idx_attachments_resource ON attachments(resource_id);
@@ -183,6 +211,8 @@ CREATE TABLE password_reset_tokens (
FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE
);
CREATE INDEX idx_password_reset_tokens_user ON password_reset_tokens(user_id);
CREATE TABLE notifications (
id TEXT PRIMARY KEY,
user_id TEXT NOT NULL,
@@ -198,6 +228,51 @@ CREATE UNIQUE INDEX idx_notifications_dedup
ON notifications(user_id, source_key)
WHERE source_key IS NOT NULL;
CREATE TABLE user_settings (
user_id TEXT PRIMARY KEY,
style TEXT,
color_scheme TEXT,
digest_weekly INTEGER NOT NULL DEFAULT 1,
digest_monthly INTEGER NOT NULL DEFAULT 1,
updated_at TEXT NOT NULL,
FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE
);
CREATE TABLE digest_schedule (
user_id TEXT NOT NULL,
cadence TEXT NOT NULL,
window_start TEXT NOT NULL,
PRIMARY KEY (user_id, cadence),
FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE
);
CREATE TABLE dump_slug_aliases (
user_id TEXT NOT NULL,
slug TEXT NOT NULL,
dump_id TEXT NOT NULL REFERENCES dumps(id) ON DELETE CASCADE,
created_at TEXT NOT NULL,
PRIMARY KEY (user_id, slug)
);
CREATE INDEX idx_dump_slug_aliases_dump ON dump_slug_aliases(dump_id, created_at);
CREATE TABLE playlist_slug_aliases (
user_id TEXT NOT NULL,
slug TEXT NOT NULL,
playlist_id TEXT NOT NULL REFERENCES playlists(id) ON DELETE CASCADE,
created_at TEXT NOT NULL,
PRIMARY KEY (user_id, slug)
);
CREATE INDEX idx_playlist_slug_aliases_playlist ON playlist_slug_aliases(playlist_id, created_at);
CREATE TABLE user_aliases (
username TEXT NOT NULL,
user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
created_at TEXT NOT NULL,
PRIMARY KEY (username)
);
CREATE UNIQUE INDEX idx_user_aliases_lower ON user_aliases(lower(username));
CREATE INDEX idx_user_aliases_user ON user_aliases(user_id, created_at);
CREATE TABLE chat_messages (
id TEXT NOT NULL PRIMARY KEY,
user_id TEXT NOT NULL,
@@ -205,6 +280,70 @@ CREATE TABLE chat_messages (
created_at TEXT NOT NULL,
updated_at TEXT,
reply_to_id TEXT,
mentions_everyone INTEGER NOT NULL DEFAULT 0,
FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE
);
CREATE INDEX idx_chat_messages_created_id ON chat_messages(created_at, id);
CREATE INDEX idx_chat_messages_user ON chat_messages(user_id);
CREATE TABLE federation_keys (
kid TEXT PRIMARY KEY,
private_jwk TEXT NOT NULL,
public_jwk TEXT NOT NULL,
created_at TEXT NOT NULL
);
CREATE TABLE user_identities (
user_id TEXT PRIMARY KEY REFERENCES users(id) ON DELETE CASCADE,
issuer TEXT NOT NULL,
subject TEXT NOT NULL,
picture TEXT,
synced_at TEXT NOT NULL
);
CREATE UNIQUE INDEX idx_user_identities_subject
ON user_identities(issuer, subject);
CREATE TABLE federation_peers (
issuer TEXT PRIMARY KEY,
status TEXT NOT NULL,
name TEXT,
created_at TEXT NOT NULL
);
CREATE TABLE federation_logins (
state TEXT PRIMARY KEY,
issuer TEXT NOT NULL,
code_verifier TEXT NOT NULL,
nonce TEXT NOT NULL,
invite_token TEXT,
created_at TEXT NOT NULL
);
CREATE TABLE oauth_codes (
code_hash TEXT PRIMARY KEY,
user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
client_id TEXT NOT NULL,
redirect_uri TEXT NOT NULL,
code_challenge TEXT NOT NULL,
nonce TEXT NOT NULL,
expires_at TEXT NOT NULL
);
CREATE TABLE federation_grants (
user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
client_id TEXT NOT NULL,
client_name TEXT,
granted_at TEXT NOT NULL,
last_used_at TEXT NOT NULL,
PRIMARY KEY (user_id, client_id)
);
CREATE TABLE federation_outbox (
id TEXT PRIMARY KEY,
client_id TEXT NOT NULL,
event TEXT NOT NULL,
attempts INTEGER NOT NULL DEFAULT 0,
next_attempt_at TEXT NOT NULL,
created_at TEXT NOT NULL
);
CREATE INDEX idx_federation_outbox_next ON federation_outbox(next_attempt_at);

82
api/db/startup.ts Normal file
View File

@@ -0,0 +1,82 @@
/**
* Deletes what has expired and seeds the first admin, each time connection.ts
* opens the database.
*
* @module
*/
import type { DatabaseSync } from "node:sqlite";
import {
ATTACHMENTS_DIR,
ORPHANED_ATTACHMENTS_RETENTION_HOURS,
UNUSED_INVITES_RETENTION_DAYS,
} from "../config.ts";
import { removeScaledImages } from "../lib/ffmpeg.ts";
import { hashPassword } from "../lib/password.ts";
/**
* Returns the time `ms` ago as an ISO string, the format dates are stored in.
*/
function isoAgo(ms: number): string {
return new Date(Date.now() - ms).toISOString();
}
/**
* Runs after the migrations, each time the server starts. It deletes used or
* expired reset tokens, invites past their validity, and attachments nothing
* links to or whose resource is gone. On an instance with no users, it creates
* the user `admin` with the password `admin`, which whoever installs it must
* change.
*/
export async function runStartupTasks(db: DatabaseSync): Promise<void> {
// Dates are stored as ISO strings and compared as text, so the cutoffs are
// ISO strings too. SQLite's datetime() has a space where ISO has a `T`.
db.prepare(
`DELETE FROM password_reset_tokens WHERE expires_at < ? OR used_at IS NOT NULL;`,
).run(isoAgo(0));
db.prepare(
`DELETE FROM invites WHERE used_at IS NULL AND created_at < ?;`,
).run(isoAgo(UNUSED_INVITES_RETENTION_DAYS * 86_400_000));
// An image is uploaded as soon as it's pasted, and linked only when the text
// it's pasted into is saved, so a fresh unlinked one is left alone. One whose
// dump, comment, playlist, message or user is gone goes whatever its age.
const attachmentCutoff = isoAgo(
ORPHANED_ATTACHMENTS_RETENTION_HOURS * 3_600_000,
);
const ORPHANED = `
(resource_id IS NULL AND created_at < ?)
OR (resource_id IS NOT NULL AND NOT CASE resource_type
WHEN 'dump' THEN EXISTS (SELECT 1 FROM dumps WHERE id = resource_id)
WHEN 'comment' THEN EXISTS (SELECT 1 FROM comments WHERE id = resource_id AND deleted = 0)
WHEN 'playlist' THEN EXISTS (SELECT 1 FROM playlists WHERE id = resource_id)
WHEN 'chat' THEN EXISTS (SELECT 1 FROM chat_messages WHERE id = resource_id)
WHEN 'user' THEN EXISTS (SELECT 1 FROM users WHERE id = resource_id)
ELSE 0
END)`;
const orphanedAttachments = db.prepare(
`SELECT id FROM attachments WHERE ${ORPHANED};`,
).all(attachmentCutoff) as { id: string }[];
if (orphanedAttachments.length > 0) {
for (const { id } of orphanedAttachments) {
await Deno.remove(`${ATTACHMENTS_DIR}/${id}`).catch(() => {});
await removeScaledImages(`${ATTACHMENTS_DIR}/${id}`);
}
db.prepare(`DELETE FROM attachments WHERE ${ORPHANED};`)
.run(attachmentCutoff);
}
// Registration needs an invite, so the first user can only come from here.
const userCount = db.prepare(`SELECT COUNT(*) as count FROM users`).get() as {
count: number;
};
if (userCount.count === 0) {
db.prepare(
`INSERT INTO users (id, username, password_hash, role, created_at, email) VALUES (?, 'admin', ?, 'admin', ?, 'admin@localhost')`,
).run(crypto.randomUUID(), await hashPassword("admin"), isoAgo(0));
console.log(
"Created default admin user (username: admin, password: admin)",
);
}
}

40
api/lib/api-exception.ts Normal file
View File

@@ -0,0 +1,40 @@
/**
* Lets any layer fail a request with an HTTP status, which the error middleware
* turns into the response.
*
* @module
*/
import { APIErrorCode } from "../../shared/api.ts";
/** Passed to an {@link APIException}; any other status is a type error. */
export type APIStatus = 400 | 401 | 403 | 404 | 409 | 429 | 500 | 502;
const CODES: Record<APIStatus, APIErrorCode> = {
400: APIErrorCode.BAD_REQUEST,
401: APIErrorCode.UNAUTHORIZED,
403: APIErrorCode.FORBIDDEN,
404: APIErrorCode.NOT_FOUND,
409: APIErrorCode.CONFLICT,
429: APIErrorCode.RATE_LIMITED,
500: APIErrorCode.SERVER_ERROR,
502: APIErrorCode.BAD_GATEWAY,
};
/**
* Thrown by a route, a service or a remote fetch to answer the request with an
* error. The error middleware sends it as an `APIFailure`, message included,
* so the message must be fit for the user. Use 401 for a caller who isn't
* signed in, 403 for one who is but may not, and 404 rather than 403 when the
* caller mustn't learn that the thing exists.
*/
export class APIException extends Error {
readonly code: APIErrorCode;
readonly status: APIStatus;
constructor(status: APIStatus, message: string) {
super(message);
this.status = status;
this.code = CODES[status];
}
}

72
api/lib/auth.ts
View File

@@ -1,45 +1,44 @@
/**
* Checks session and media tokens against the live user row, not just their
* signature. The auth middleware, the WebSocket route, the media routes and the
* routes open to anonymous callers all authenticate through it.
*
* @module
*/
import type { Context } from "@oak/oak";
import { verifyJWT, verifyMediaToken } from "./jwt.ts";
import {
APIErrorCode,
APIException,
type AuthPayload,
type Role,
} from "../model/interfaces.ts";
import { db } from "../model/db.ts";
import { APIException } from "./api-exception.ts";
import type { AuthPayload } from "../model/auth.ts";
import type { Role } from "../../shared/user.ts";
import { getSessionState } from "../services/user-service.ts";
/**
* Checks a signature-verified token against the live user row: the user must
* still exist, and the token must not predate their last password change
* (`users.tokens_valid_after`). Returns the live role, or null to refuse.
*
* `iat` has one-second resolution, so a token issued in the same second as the
* change survives it; one issued after it (the fresh one the change hands back)
* always does.
* Returns the live role of a verified token's user, or null when the user is
* gone or the token predates their last password change
* (`users.tokens_valid_after`). `iat` counts whole seconds, so a token issued
* in the same second as the change survives it.
*/
function checkTokenOwner(
userId: string,
iat: number | undefined,
): { role: Role } | null {
const row = db.prepare(
`SELECT role, tokens_valid_after FROM users WHERE id = ?;`,
).get(userId) as
| { role: Role; tokens_valid_after: string | null }
| undefined;
if (!row) return null;
if (row.tokens_valid_after) {
const state = getSessionState(userId);
if (!state) return null;
if (state.tokensValidAfter) {
const cutoff = Math.floor(
new Date(row.tokens_valid_after).getTime() / 1000,
new Date(state.tokensValidAfter).getTime() / 1000,
);
if ((iat ?? 0) < cutoff) return null;
}
return { role: row.role };
return { role: state.role };
}
/**
* Verifies a session token end to end: signature, expiry, a user that still
* exists and no password change since it was issued. The role comes from the
* DB, not the token, since tokens last 7 days and roles change under them.
* Returns the payload of a valid session token, or null. Besides the signature
* and expiry, it refuses a token whose user is gone or which predates a
* password change. The returned role comes from the database, not the token,
* since a token lasts 7 days and a role can change under it.
*/
export async function authenticate(
token: string,
@@ -52,8 +51,9 @@ export async function authenticate(
}
/**
* Resolves the `?token=` on a media URL to the viewer's user id. Takes a media
* token, or a session token from a client that predates them.
* Returns the user id behind the `?token=` of a file, thumbnail or playlist
* cover URL, or null when it isn't valid. A session token is accepted too, for
* clients from before media tokens.
*/
export async function authenticateMedia(token: string): Promise<string | null> {
const media = await verifyMediaToken(token);
@@ -64,15 +64,13 @@ export async function authenticateMedia(token: string): Promise<string | null> {
}
/**
* Extracts the userId from an optional Bearer token.
* Returns the caller's user id on a route open to anonymous callers, or null
* when the request has no token. A placeholder such as `Bearer undefined`
* counts as no token. An invalid token throws instead of reading as anonymous,
* so that a stale session gets a 401 the client signs out on, rather than a
* misleading 404 on a private resource.
*
* Distinguishes *missing* credentials from *invalid* ones:
* - No `Authorization: Bearer …` header (or a placeholder value such as the
* `Bearer undefined` that the client emits when logged out) → anonymous (null).
* - A real token that fails verification (expired / bad signature / revoked) →
* throws 401, so a stale client session surfaces as an auth error it can react
* to, instead of being silently downgraded to anonymous (which yields a
* misleading 404 on private resources).
* @throws {APIException} 401 when a token is present but invalid.
*/
export async function parseOptionalAuth(ctx: Context): Promise<string | null> {
const authHeader = ctx.request.headers.get("Authorization");
@@ -83,7 +81,7 @@ export async function parseOptionalAuth(ctx: Context): Promise<string | null> {
const payload = await authenticate(token);
if (!payload) {
throw new APIException(APIErrorCode.UNAUTHORIZED, 401, "Invalid token");
throw new APIException(401, "Invalid token");
}
return payload.userId;
}

59
api/lib/canonical-url.ts
View File

@@ -1,20 +1,20 @@
/**
* Canonical form of a URL, used *only* to recognise that two links point at the
* same thing — the duplicate check the create form runs while it fetches a
* preview. It is a lookup key, never something we display or fetch: `dumps.url`
* keeps the exact string the poster submitted.
* Spots reposts and links to known dumps, by reducing a URL to a key shared by
* every link to the same page or video. `dump-service` stores it in
* `dumps.url_canonical` when a dump is posted or edited, `GET /api/dumps?url=`
* matches on it to warn the create form of a repost, and backlink-service
* compares links by it. The key is never fetched or shown, so
* it may be lossy: it forces `https`, drops `www.`, tracking parameters and
* timestamps.
*
* Because it is only ever compared against other canonical forms, it may be
* lossy in ways a real URL never could be — it forces `https`, drops `www.`,
* and throws away share/tracking parameters and timestamps, so
* `http://www.example.com/a/?utm_source=x` and `https://example.com/a` collapse
* to one key.
* Migration 0013 backfilled existing rows with a frozen copy of these rules.
* Changing them needs a new backfill migration, or old rows keep old keys.
*
* Stored in `dumps.url_canonical`. Migration 0013 carries a frozen copy of this
* logic to backfill existing rows; changing the rules here therefore needs a
* new backfill migration, or old rows keep keys computed under the old rules.
* @module
*/
import { siteHostOf } from "../../shared/url.ts";
const TRACKING_PARAMS = new Set([
"fbclid",
"gclid",
@@ -40,7 +40,7 @@ function isTrackingParam(key: string): boolean {
return k.startsWith("utm_") || TRACKING_PARAMS.has(k);
}
// Hosts already stripped of a leading "www.".
/** YouTube's hosts, without their "www.". */
const YOUTUBE_HOSTS = new Set([
"youtube.com",
"m.youtube.com",
@@ -49,12 +49,23 @@ const YOUTUBE_HOSTS = new Set([
]);
/**
* The video a YouTube URL points at, in any of the shapes people paste
* (`youtu.be/ID`, `/watch?v=ID`, `/embed/ID`, `/shorts/ID`, `/live/ID`).
* Timestamps are deliberately ignored: the same video linked at 2:30 is still
* the same video for duplicate purposes.
* Returns `u`'s host in lower case and without "www.", when it is one of
* YouTube's (youtu.be aside), and null otherwise. The YouTube preview provider
* uses it too, so both agree on what a YouTube link is.
*/
function youtubeVideoId(
export function youtubeHost(u: URL): string | null {
const host = siteHostOf(u);
return YOUTUBE_HOSTS.has(host) ? host : null;
}
/**
* Returns the id of the video a YouTube URL points at, in any of the shapes
* people paste (`youtu.be/ID`, `/watch?v=ID`, `/embed/ID`, `/shorts/ID`,
* `/live/ID`), and null for any other URL. `host` is lower case, without
* "www.". The canonical key drops the timestamp, since the same video linked
* at 2:30 is still a repost; the YouTube provider reads it separately.
*/
export function youtubeVideoId(
host: string,
pathname: string,
params: URLSearchParams,
@@ -68,6 +79,10 @@ function youtubeVideoId(
return null;
}
/**
* Returns the canonical key of `raw`, or null when it isn't an http(s) URL. A
* YouTube video or playlist becomes one fixed URL whatever shape it came in.
*/
export function canonicalizeUrl(raw: string): string | null {
let u: URL;
try {
@@ -77,7 +92,7 @@ export function canonicalizeUrl(raw: string): string | null {
}
if (u.protocol !== "http:" && u.protocol !== "https:") return null;
const host = u.hostname.toLowerCase().replace(/^www\./, "");
const host = siteHostOf(u);
if (!host) return null;
const videoId = youtubeVideoId(host, u.pathname, u.searchParams);
@@ -88,7 +103,7 @@ export function canonicalizeUrl(raw: string): string | null {
: null;
if (listId) return `https://youtube.com/playlist?list=${listId}`;
// A single trailing slash is never meaningful; "/" itself becomes "".
// Trailing slashes are dropped; "/" itself becomes "".
const path = u.pathname.replace(/\/+$/, "");
const params = [...u.searchParams.entries()]
@@ -96,8 +111,8 @@ export function canonicalizeUrl(raw: string): string | null {
.sort(([a, av], [b, bv]) => a.localeCompare(b) || av.localeCompare(bv));
const query = new URLSearchParams(params).toString();
// A bare "#section" anchor points into the same page, so it is dropped —
// but "#/route" and "#!/route" address distinct pages of a hash-routed app.
// A "#section" anchor points into the same page and is dropped, but "#/route"
// and "#!/route" are distinct pages of a hash-routed app.
const hash = /^#!?\//.test(u.hash) ? u.hash : "";
return `https://${host}${path}${query ? `?${query}` : ""}${hash}`;

91
api/lib/css-color.ts Normal file
View File

@@ -0,0 +1,91 @@
/**
* Reads the CSS colors the server meets: a dumped page's `theme-color`, which
* the generic link preview keeps as its tint (`normalizeCssColor`), and the
* theme stylesheets' tokens, which the emails resolve (`theme-tokens.ts`).
*
* @module
*/
/** An sRGB color with its alpha, each channel 0–255 and alpha 0–1. */
export interface Rgba {
r: number;
g: number;
b: number;
a: number;
}
/**
* The named colors sites actually put in `theme-color`, and `transparent`,
* which the themes write; the rest of CSS's names aren't worth carrying.
*/
const NAMED: Record<string, string> = {
transparent: "#00000000",
black: "#000000",
white: "#ffffff",
red: "#ff0000",
green: "#008000",
blue: "#0000ff",
yellow: "#ffff00",
orange: "#ffa500",
purple: "#800080",
gray: "#808080",
grey: "#808080",
silver: "#c0c0c0",
maroon: "#800000",
navy: "#000080",
teal: "#008080",
olive: "#808000",
lime: "#00ff00",
aqua: "#00ffff",
cyan: "#00ffff",
fuchsia: "#ff00ff",
magenta: "#ff00ff",
};
/** Reads a channel of `rgb()`, a number or a percentage of 255. */
function channel(text: string): number {
const n = parseFloat(text);
const value = text.endsWith("%") ? (n / 100) * 255 : n;
return Math.max(0, Math.min(255, value));
}
/** Reads the alpha of `rgba()`, a number or a percentage of 1. */
function alpha(text: string): number {
const n = parseFloat(text);
return Math.max(0, Math.min(1, text.endsWith("%") ? n / 100 : n));
}
/**
* Returns the color `text` writes as a hex (3, 4, 6 or 8 digits), an `rgb()`
* or `rgba()` of numbers or percentages, or one of the common names; undefined
* for any other form. Channels out of range are clamped.
*/
export function parseCssColor(text: string): Rgba | undefined {
const value = text.trim().toLowerCase();
const hex = /^#([0-9a-f]{3,8})$/.exec(NAMED[value] ?? value)?.[1];
if (hex) {
if (hex.length === 5 || hex.length === 7) return undefined;
const full = hex.length <= 4 ? [...hex].map((c) => c + c).join("") : hex;
const byte = (i: number) => parseInt(full.slice(i * 2, i * 2 + 2), 16);
return {
r: byte(0),
g: byte(1),
b: byte(2),
a: full.length === 8 ? byte(3) / 255 : 1,
};
}
const args = /^rgba?\(([^)]+)\)$/.exec(value)?.[1];
if (!args) return undefined;
const parts = args.split(/[\s,/]+/).filter(Boolean);
if (parts.length !== 3 && parts.length !== 4) return undefined;
const [r, g, b] = parts.slice(0, 3).map(channel);
const a = parts[3] === undefined ? 1 : alpha(parts[3]);
return [r, g, b, a].some(Number.isNaN) ? undefined : { r, g, b, a };
}
/** Writes `color` as a lowercase `#rrggbb`, its alpha dropped. */
export function hexOf({ r, g, b }: Rgba): string {
return `#${
[r, g, b].map((c) => Math.round(c).toString(16).padStart(2, "0")).join("")
}`;
}

46
api/lib/dump-media.ts Normal file
View File

@@ -0,0 +1,46 @@
/**
* Resolves the dump whose media a route under `/api/dumps/:dumpId/` serves:
* its file (`routes/files.ts`), its thumbnail (`routes/thumbnails.ts`) or its
* waveform (`routes/waveforms.ts`). Media elements and `<img>`s can't send an
* Authorization header, so these routes take the viewer's media token from
* the query string, and are cached by the file's version.
*
* @module
*/
import type { Context } from "@oak/oak";
import { APIException } from "./api-exception.ts";
import { authenticateMedia } from "./auth.ts";
import { type Dump, dumpFileVersion } from "../../shared/dump.ts";
import { UUID_RE } from "../../shared/validation.ts";
import { getDump } from "../services/dump-service.ts";
/**
* Returns dump `dumpId` as the viewer of `ctx`'s `?token=` may see it, and the
* Cache-Control its media get: kept for good under a URL naming the file's
* current version (`?v=`, as `dumpFileVersion` gives), revalidated under any
* other, and never in a shared cache for a private dump. It also keeps the
* token out of the Referer of whatever the response links to.
*
* @throws {APIException} 400 when `dumpId` isn't an id, which names files on
* disk, and 404 when the viewer can't see the dump.
*/
export async function resolveMediaDump(
ctx: Context,
dumpId: string,
): Promise<{ dump: Dump; cacheControl: string }> {
if (!UUID_RE.test(dumpId)) throw new APIException(400, "Invalid dump ID");
const token = ctx.request.url.searchParams.get("token");
const viewerId = token ? await authenticateMedia(token) : null;
const dump = getDump(dumpId, viewerId ?? undefined);
ctx.response.headers.set("Referrer-Policy", "no-referrer");
const scope = dump.isPrivate ? "private" : "public";
const current = ctx.request.url.searchParams.get("v") ===
dumpFileVersion(dump);
return {
dump,
cacheControl: current
? `${scope}, max-age=31536000, immutable`
: `${scope}, no-cache`,
};
}

512
api/lib/ffmpeg.ts Normal file
View File

@@ -0,0 +1,512 @@
/**
* Makes what the server derives from an upload with ffmpeg: a video's still,
* which the thumbnails route serves, an audio file's waveform, which the
* waveform route serves, and the scaled-down copies of any image, which
* `serveImage` (lib/upload.ts) serves for every route that answers one. Each
* is made on its first request and kept under a name this module gives, a
* dump's in `THUMBNAILS_DIR` and the scaled copies in `SCALED_DIR`, until
* dump-service replaces or deletes the upload and
* {@link removeGeneratedMedia} drops them. {@link generatedMediaSweepJob}
* takes what that leaves behind.
*
* @module
*/
import { dirname, relative } from "node:path";
import {
DUMPS_DIR,
SCALED_DIR,
THUMBNAILS_DIR,
UPLOADS_DIR,
} from "../config.ts";
import { IMAGE_WIDTHS, type ImageWidth } from "../../shared/image.ts";
import type { Job } from "./scheduler.ts";
import { type DumpWaveform, peaksOf } from "../../shared/waveform.ts";
import type { APIResponse } from "../../shared/api.ts";
/**
* Spares a process per uncached request: whether ffmpeg runs is probed once
* per process, and kept here.
*/
let ffmpegProbe: Promise<boolean> | undefined;
function ffmpegAvailable(): Promise<boolean> {
ffmpegProbe ??= (async () => {
try {
const { success } = await new Deno.Command("ffmpeg", {
args: ["-version"],
stderr: "null",
stdout: "null",
}).output();
return success;
} catch {
return false;
}
})();
return ffmpegProbe;
}
/**
* Lists the container formats ffmpeg may read a video as. The upload's type is
* only what the uploader claimed, and ffmpeg guesses the format from the bytes,
* so a "video" could be an HLS or concat playlist pointing it at local files or
* internal URLs. With this list, and `-protocol_whitelist file`, ffmpeg reads
* nothing but the file, and only as a video.
*/
const VIDEO_FORMATS =
"mov,mp4,m4a,3gp,3g2,mj2,matroska,webm,avi,flv,mpegts,mpeg,ogg,asf,h264,hevc";
/**
* Lists the formats ffmpeg may read an image as, for the reason given at
* {@link VIDEO_FORMATS}.
*/
const IMAGE_FORMATS = "jpeg_pipe,png_pipe,webp_pipe,bmp_pipe,gif";
/**
* Lists the formats ffmpeg may read an audio file as, for the reason given at
* {@link VIDEO_FORMATS}; an `.m4a` or a `.webm` is read by its container's.
*/
const AUDIO_FORMATS =
"mp3,ogg,flac,wav,aiff,aac,asf,caf,w64,wv,ape,mov,mp4,m4a,matroska,webm";
/**
* Returns where the still and the waveform generated from dump `dumpId`'s
* upload are kept.
*/
export function generatedPaths(dumpId: string): {
still: string;
waveform: string;
} {
const base = `${THUMBNAILS_DIR}/${dumpId}`;
return { still: `${base}.jpg`, waveform: `${base}.waveform.json` };
}
/**
* The formats a scaled copy comes in: WebP for a browser, and a JPEG or a PNG
* for what doesn't read it, as some mail clients don't.
*/
export type ScaledFormat = "webp" | "jpg" | "png";
const SCALED_FORMATS: readonly ScaledFormat[] = ["webp", "jpg", "png"];
/**
* Returns where the copy of the upload at `srcPath` scaled to `width` is
* kept, in `format`: under `SCALED_DIR`, at the upload's own path below
* `UPLOADS_DIR`.
*/
export function scaledImagePath(
srcPath: string,
width: ImageWidth,
format: ScaledFormat = "webp",
): string {
return `${SCALED_DIR}/${relative(UPLOADS_DIR, srcPath)}.w${width}.${format}`;
}
/**
* Deletes the scaled copies of the upload at `srcPath`, once it is deleted.
* One that is replaced needs no call: its copies are made again, being older
* than it ({@link ensureGenerated}).
*/
export async function removeScaledImages(srcPath: string): Promise<void> {
await Promise.all(
IMAGE_WIDTHS.flatMap((width) =>
SCALED_FORMATS.map((format) =>
Deno.remove(scaledImagePath(srcPath, width, format)).catch(() => {})
)
),
);
}
/**
* Deletes what was generated from dump `dumpId`'s upload, once dump-service
* has replaced or deleted it. The routes make them again from a new one.
*/
export async function removeGeneratedMedia(dumpId: string): Promise<void> {
const { still, waveform } = generatedPaths(dumpId);
await Promise.all([
...[still, waveform].map((path) => Deno.remove(path).catch(() => {})),
removeScaledImages(`${DUMPS_DIR}/${dumpId}`),
]);
}
/**
* Writes `outPath` with `write`, which gets a path beside it to fill, renamed
* over `outPath` once done, so a concurrent reader never gets half a file.
*/
async function writeAtomically(
outPath: string,
write: (tmpPath: string) => Promise<void>,
): Promise<void> {
await Deno.mkdir(dirname(outPath), { recursive: true });
const tmpPath = `${outPath}.${crypto.randomUUID()}.tmp`;
try {
await write(tmpPath);
await Deno.rename(tmpPath, outPath);
} catch (err) {
await Deno.remove(tmpPath).catch(() => {});
throw err;
}
}
/**
* Runs ffmpeg with `args`, writing to `outPath`.
*
* @throws {Error} When ffmpeg fails.
*/
function runFfmpeg(args: string[], outPath: string): Promise<void> {
return writeAtomically(outPath, async (tmpPath) => {
const { success } = await new Deno.Command("ffmpeg", {
args: ["-protocol_whitelist", "file", ...args, "-y", tmpPath],
stdout: "null",
stderr: "null",
}).output();
if (!success) throw new Error("ffmpeg failed");
});
}
/** Writes a 320-pixel-wide JPEG still of a video, one second in. */
export function videoStill(srcPath: string, outPath: string): Promise<void> {
return runFfmpeg([
"-format_whitelist",
VIDEO_FORMATS,
"-ss",
"00:00:01",
"-i",
srcPath,
"-frames:v",
"1",
"-vf",
"scale=320:-1",
"-f",
"image2",
"-c:v",
"mjpeg",
], outPath);
}
/**
* Maps a JPEG's EXIF orientation to the filters that turn its pixels upright.
* A browser applies the orientation to the upload itself; the copy carries no
* EXIF, so it is turned here, and not by ffmpeg's own autorotation, which
* only its recent versions apply to images.
*/
const ORIENTATION_FILTERS: Record<number, string> = {
2: "hflip",
3: "hflip,vflip",
4: "vflip",
5: "transpose=0",
6: "transpose=1",
7: "transpose=3",
8: "transpose=2",
};
/** What ffmpeg is told to write each {@link ScaledFormat} with. */
const ENCODERS: Record<ScaledFormat, string[]> = {
webp: ["-c:v", "libwebp", "-q:v", "80", "-loop", "0", "-f", "webp"],
jpg: ["-c:v", "mjpeg", "-q:v", "4", "-f", "image2"],
png: ["-c:v", "png", "-f", "image2"],
};
/**
* What {@link scaleImage} is told of the image it scales, which
* `scalableImage` (lib/upload.ts) reads from its first bytes.
*/
export interface ImageTraits {
/** The EXIF orientation to turn the image by, 1 when it is upright. */
orientation: number;
/**
* Whether its frames are an animation, as a GIF's are. A still may hold
* more than one image too, a phone's JPEG carrying a gain map or a depth
* map after the photo, and ffmpeg reads each as a frame.
*/
animated: boolean;
}
/**
* Writes a copy of an image at most `width` wide, never wider than the image
* itself, as a WebP unless `format` names another. WebP keeps a PNG's
* transparency and a GIF's animation, at a fraction of either's bytes.
*/
export function scaleImage(
srcPath: string,
outPath: string,
width: ImageWidth,
{ orientation, animated }: ImageTraits,
format: ScaledFormat = "webp",
): Promise<void> {
const upright = ORIENTATION_FILTERS[orientation];
// A JPEG's colour planes halve its height, which they need even.
const height = format === "jpg" ? -2 : -1;
// Only an animation keeps its frames, and only as a WebP: of anything else
// the first is the image, and a copy of them all would flash through what
// follows it.
const frames = animated && format === "webp" ? [] : ["-frames:v", "1"];
return runFfmpeg([
"-noautorotate",
"-format_whitelist",
IMAGE_FORMATS,
"-i",
srcPath,
"-vf",
`${upright ? `${upright},` : ""}scale='min(${width},iw)':${height}`,
"-an",
...frames,
...ENCODERS[format],
], outPath);
}
/**
* Returns `bytes`, an image held in memory, scaled as {@link scaleImage}
* does, or null when it can't be, because ffmpeg is missing or failed. The
* image proxy scales what it fetched with it, and keeps the result itself.
*/
export async function scaleImageBytes(
bytes: Uint8Array,
width: ImageWidth,
traits: ImageTraits,
): Promise<Uint8Array | null> {
if (!await ffmpegAvailable()) return null;
const srcPath = `${SCALED_DIR}/tmp/${crypto.randomUUID()}`;
const outPath = `${srcPath}.webp`;
try {
await Deno.mkdir(dirname(srcPath), { recursive: true });
await Deno.writeFile(srcPath, bytes);
await scaleImage(srcPath, outPath, width, traits);
return await Deno.readFile(outPath);
} catch {
return null;
} finally {
await Promise.all(
[srcPath, outPath].map((path) => Deno.remove(path).catch(() => {})),
);
}
}
/**
* The sample rate the audio is decoded at for its waveform. A bar stands for
* a second or more of a song, so 8 kHz keeps every peak worth drawing at a
* sixth of the samples.
*/
const PEAKS_SAMPLE_RATE = 8000;
/** How many samples each running peak covers before `peaksOf` buckets them. */
const PEAKS_WINDOW = 256;
/**
* Decodes the audio of `srcPath` to its waveform's peaks. ffmpeg streams the
* samples, mono at {@link PEAKS_SAMPLE_RATE}, and only the peak of each
* {@link PEAKS_WINDOW} is kept, so an hour of audio holds a few hundred
* kilobytes at most.
*
* @throws {Error} When ffmpeg fails or the file holds no audio.
*/
async function audioPeaks(srcPath: string): Promise<number[]> {
const child = new Deno.Command("ffmpeg", {
args: [
"-protocol_whitelist",
"file",
"-format_whitelist",
AUDIO_FORMATS,
"-i",
srcPath,
"-vn",
"-ac",
"1",
"-ar",
String(PEAKS_SAMPLE_RATE),
"-f",
"s16le",
"-",
],
stdin: "null",
stdout: "piped",
stderr: "null",
}).spawn();
const windows: number[] = [];
let peak = 0;
let count = 0;
// A chunk may end halfway through a sample, whose first byte waits here.
let pending: number | null = null;
const take = (sample: number) => {
peak = Math.max(peak, Math.abs(sample) / 32768);
if (++count === PEAKS_WINDOW) {
windows.push(peak);
peak = 0;
count = 0;
}
};
for await (const chunk of child.stdout) {
let i = 0;
if (pending !== null) {
take(((chunk[0] << 24) >> 16) | pending);
pending = null;
i = 1;
}
for (; i + 1 < chunk.length; i += 2) {
// Little-endian, signed: the high byte is shifted into the sign bit
// and back.
take(((chunk[i + 1] << 24) >> 16) | chunk[i]);
}
if (i < chunk.length) pending = chunk[i];
}
if (count > 0) windows.push(peak);
const { success } = await child.status;
if (!success || windows.length === 0) throw new Error("ffmpeg failed");
return peaksOf(windows);
}
/**
* Writes the waveform of the audio file `srcPath` as the waveform route's
* whole response, so the route serves it as a file, with its ETag.
*/
export function audioWaveform(srcPath: string, outPath: string): Promise<void> {
return writeAtomically(outPath, async (tmpPath) => {
const body: APIResponse<DumpWaveform> = {
success: true,
data: { peaks: await audioPeaks(srcPath) },
};
await Deno.writeTextFile(tmpPath, JSON.stringify(body));
});
}
/**
* Lets a feed full of cards asking for the same new still at once wait on a
* single ffmpeg run. It holds the generations under way, by output path.
*/
const inFlight = new Map<string, Promise<void>>();
/**
* Spares an ffmpeg run per request for an image it can't read: the
* generations that failed, by output path, with the date of the source they
* failed on. A new upload gets a new try.
*/
const failed = new Map<string, number>();
/** Returns the date `path` was last written, or null when it doesn't exist. */
async function writtenAt(path: string): Promise<number | null> {
try {
return (await Deno.stat(path)).mtime?.getTime() ?? 0;
} catch {
return null;
}
}
/**
* Makes `outPath` with `generate` unless it exists. Resolves false when it
* can't be made, because ffmpeg is missing or failed.
*
* With `srcPath`, the file it is made from, `outPath` takes that file's date,
* and is made again when the two differ: an upload replaced in place, as an
* avatar or a cover is, gets new copies without anyone deleting the old.
* Without it, the caller deletes `outPath` when its source changes.
*/
export async function ensureGenerated(
outPath: string,
generate: () => Promise<void>,
srcPath?: string,
): Promise<boolean> {
const made = await writtenAt(outPath);
let version: number | null = null;
if (srcPath) {
version = await writtenAt(srcPath);
if (version === null) return false;
if (made === version) return true;
if (failed.get(outPath) === version) return false;
} else if (made !== null) {
return true;
}
if (!await ffmpegAvailable()) return false;
let pending = inFlight.get(outPath);
if (!pending) {
pending = generate()
.then(() =>
version === null
? undefined
: Deno.utime(outPath, new Date(), new Date(version))
)
.catch((err) => {
if (version !== null) failed.set(outPath, version);
throw err;
})
.finally(() => inFlight.delete(outPath));
inFlight.set(outPath, pending);
}
return await pending.then(() => true, () => false);
}
/**
* How old a temporary file must be for the sweep to take it: one being
* written, by a generation under way, is seconds old, and one a crash left
* is never finished.
*/
const ABANDONED_AFTER_MS = 60 * 60_000;
/** Yields the path of every file under `dir`, and nothing when it is missing. */
async function* filesUnder(dir: string): AsyncGenerator<string> {
let entries: Deno.DirEntry[];
try {
entries = await Array.fromAsync(Deno.readDir(dir));
} catch {
return;
}
for (const entry of entries) {
const path = `${dir}/${entry.name}`;
if (entry.isDirectory) yield* filesUnder(path);
else if (entry.isFile) yield path;
}
}
/**
* Deletes the generated files nothing serves any more, and returns how many:
* the scaled copies of an upload that is gone, which the services' own
* removals missed or never made, as for a user's avatar; the previews of
* image dumps under their former name, `<id>.preview.jpg` or `.png`, which
* the scaled copies replaced; and the temporary files a generation left
* unfinished. Nothing else of `THUMBNAILS_DIR` is touched, the custom
* thumbnails kept there being uploads.
*/
export async function sweepGeneratedMedia(): Promise<number> {
let removed = 0;
const remove = (path: string) =>
Deno.remove(path).then(() => {
removed++;
}, () => {});
const abandoned = async (path: string) => {
const at = await writtenAt(path);
return at !== null && Date.now() - at > ABANDONED_AFTER_MS;
};
for await (const path of filesUnder(THUMBNAILS_DIR)) {
if (
/\.preview\.(jpg|png)$/.test(path) ||
(path.endsWith(".tmp") && await abandoned(path))
) await remove(path);
}
for await (const path of filesUnder(SCALED_DIR)) {
// A copy is named after its upload's path: see scaledImagePath.
const copy = relative(SCALED_DIR, path).match(
/^(.+)\.w\d+\.(?:webp|jpg|png)$/,
);
const unused = copy
? await writtenAt(`${UPLOADS_DIR}/${copy[1]}`) === null
: await abandoned(path);
if (unused) await remove(path);
}
return removed;
}
/**
* Runs {@link sweepGeneratedMedia} a minute after the server starts, which
* clears what a new version no longer uses as soon as it is deployed, then
* once a day. `main.ts` starts it.
*/
export const generatedMediaSweepJob: Job = {
name: "media-sweep",
everyMs: 24 * 60 * 60_000,
firstAfterMs: 60_000,
async run() {
const removed = await sweepGeneratedMedia();
if (removed > 0) {
console.log(`[media-sweep] removed ${removed} generated files unused`);
}
},
};

53
api/lib/index-html.ts Normal file
View File

@@ -0,0 +1,53 @@
/**
* Fills the placeholders of the frontend's `index.html` with settings read at
* runtime, so they change with a restart and no rebuild. The static and OG
* middlewares serve what it returns.
*
* @module
*/
import {
BANDCAMP_PLAYER,
DEFAULT_STYLE,
EMAIL_ENABLED,
FEDERATION,
OG_SITE_NAME,
SITE_EMOJI,
} from "../config.ts";
import { emojiToCodepoint } from "./site-icons.ts";
/**
* Versions the icon URLs, so browsers fetch new icons when the emoji changes.
*/
const ICON_VERSION = emojiToCodepoint(SITE_EMOJI);
/**
* Caches each root's filled file until its mtime changes, as a rebuild under a
* running server does, so a navigation costs a stat instead of a read.
*/
const indexHtml = new Map<string, { mtime: number; html: string }>();
/**
* Returns `${root}/index.html` with the site name, emoji, icon version,
* Bandcamp player mode, default style, and whether email and each way of
* federated sign-in are on filled in.
*
* @throws {Error} When the file can't be read.
*/
export async function loadIndexHtml(root: string): Promise<string> {
const path = `${root}/index.html`;
const mtime = (await Deno.stat(path)).mtime?.getTime() ?? 0;
const cached = indexHtml.get(root);
if (cached && cached.mtime === mtime) return cached.html;
const html = (await Deno.readTextFile(path))
.replaceAll("__SITE_NAME__", OG_SITE_NAME)
.replaceAll("__SITE_EMOJI__", SITE_EMOJI)
.replaceAll("__ICON_VERSION__", ICON_VERSION)
.replaceAll("__BANDCAMP_PLAYER__", BANDCAMP_PLAYER)
.replaceAll("__DEFAULT_STYLE__", DEFAULT_STYLE)
.replaceAll("__EMAIL_ENABLED__", EMAIL_ENABLED ? "on" : "off")
.replaceAll("__FEDERATION_INCOMING__", FEDERATION.incoming ? "on" : "off")
.replaceAll("__FEDERATION_OUTGOING__", FEDERATION.outgoing ? "on" : "off");
indexHtml.set(root, { mtime, html });
return html;
}

248
api/lib/jwt.ts
View File

@@ -1,14 +1,40 @@
import { randomBytes, scrypt, timingSafeEqual } from "node:crypto";
import { Buffer } from "node:buffer";
import { jwtVerify, SignJWT } from "@panva/jose";
/**
* Signs and verifies the server's four kinds of token (session, media,
* password reset and unsubscribe), all HS256 with the one key. Only a token's signature and
* expiry are checked here: `lib/auth.ts` checks a session or media token
* against its user.
*
* It also signs and verifies the tokens instances exchange for federated
* sign-in (identity tokens and events), EdDSA with the instance's Ed25519 key,
* whose public half other instances fetch: `instance-key-service.ts` keeps
* the key, and the federation services pass it in.
*
* Importing this module throws when `GERBEUR_JWT_SECRET` is unset.
*
* @module
*/
import {
createLocalJWKSet,
exportJWK,
generateKeyPair,
importJWK,
type JSONWebKeySet,
type JWK,
type JWTPayload,
jwtVerify,
SignJWT,
} from "@panva/jose";
import {
type AuthPayload,
isAuthPayload,
isPasswordResetPayload,
type PasswordResetPayload,
} from "../model/interfaces.ts";
} from "../model/auth.ts";
import { JWT_SECRET } from "../config.ts";
import { type DigestCadence, isDigestCadence } from "../../shared/settings.ts";
import type { AuthTokens, Role } from "../../shared/user.ts";
if (!JWT_SECRET) {
throw new Error(
@@ -17,18 +43,27 @@ if (!JWT_SECRET) {
}
const JWT_KEY = new TextEncoder().encode(JWT_SECRET);
// ── Password reset tokens ─────────────────────────────────────────────────────
/**
* Signs a password reset token for `userId`, valid for an hour. The token
* isn't single-use by itself: `requestPasswordReset` stores it, and
* `resetPassword` marks it used.
*/
export async function createPasswordResetToken(
userId: string,
): Promise<string> {
return await new SignJWT({ purpose: "password-reset", userId })
.setProtectedHeader({ alg: "HS256" })
// Two requests in the same second would otherwise sign the same token,
// which is the primary key of `password_reset_tokens`.
.setJti(crypto.randomUUID())
.setExpirationTime("1h")
.sign(JWT_KEY);
}
/**
* Returns the payload of a password reset token, or null when it is invalid,
* expired or another kind of token. Whether it was used is the caller's check.
*/
export async function verifyPasswordResetToken(
token: string,
): Promise<PasswordResetPayload | null> {
@@ -41,25 +76,48 @@ export async function verifyPasswordResetToken(
}
}
// ── Auth tokens ───────────────────────────────────────────────────────────────
/**
* Signs a session token carrying `payload`, valid for 7 days.
* {@link issueTokens} issues it on sign-up, sign-in and password change, and
* the client sends it as a bearer token.
*/
export async function createJWT(
payload: Omit<AuthPayload, "exp">,
): Promise<string> {
return await new SignJWT(payload)
.setProtectedHeader({ alg: "HS256" })
// `iat` is what a password change revokes against (users.tokens_valid_after).
// A password change revokes tokens by comparing `iat` with
// `users.tokens_valid_after`.
.setIssuedAt()
.setExpirationTime("7d")
.sign(JWT_KEY);
}
// ── Media tokens ──────────────────────────────────────────────────────────────
/**
* Returns a new session token and media token for `user`, as sign-up
* (`POST /api/users`), sign-in (`POST /api/sessions`) and a password change
* (`PUT /api/me/password`) answer them.
*/
export async function issueTokens(
user: { id: string; username: string; role: Role },
): Promise<AuthTokens> {
return {
token: await createJWT({
userId: user.id,
username: user.username,
role: user.role,
}),
mediaToken: await createMediaToken(user.id),
};
}
// Media elements can't send an Authorization header, so a private dump's file
// and thumbnail URLs carry a token in the query string, where it lands in
// access logs and browser history. That token is this one: it only reads media,
// so a leaked URL exposes the owner's private files, not their account.
/**
* Signs a media token for `userId`, valid for 7 days. Media elements can't send
* an `Authorization` header, so the URLs of private files, thumbnails and
* playlist covers carry this token as `?token=`, where it lands in logs and
* browser history. It only reads media, so a leaked URL doesn't expose the
* account.
*/
export async function createMediaToken(userId: string): Promise<string> {
return await new SignJWT({ purpose: "media", userId })
.setProtectedHeader({ alg: "HS256" })
@@ -68,6 +126,10 @@ export async function createMediaToken(userId: string): Promise<string> {
.sign(JWT_KEY);
}
/**
* Returns the user id and issue time of a media token, or null when it is
* invalid, expired or another kind of token.
*/
export async function verifyMediaToken(
token: string,
): Promise<{ userId: string; iat?: number } | null> {
@@ -82,6 +144,11 @@ export async function verifyMediaToken(
}
}
/**
* Returns the payload of a session token whose signature and expiry are valid,
* or null. Call `authenticate` (`lib/auth.ts`) instead, which also checks the
* token against its user.
*/
export async function verifyJWT(token: string): Promise<AuthPayload | null> {
try {
const { payload } = await jwtVerify(token, JWT_KEY);
@@ -97,33 +164,134 @@ export async function verifyJWT(token: string): Promise<AuthPayload | null> {
}
}
export function hashPassword(password: string): Promise<string> {
const salt = randomBytes(16).toString("hex");
return new Promise((resolve, reject) => {
scrypt(password, salt, 64, (err, derivedKey) => {
if (err) reject(err);
else resolve(`${derivedKey.toString("hex")}.${salt}`);
});
});
/**
* Signs the token of a digest's unsubscribe link, which turns off `cadence`
* for `userId` without signing in. It never expires, so the link in an old
* email still works; turning the digest back on doesn't revoke it, which is
* harmless since all it can do is turn it off again.
*/
export async function createUnsubscribeToken(
userId: string,
cadence: DigestCadence,
): Promise<string> {
return await new SignJWT({ purpose: "unsubscribe", userId, cadence })
.setProtectedHeader({ alg: "HS256" })
.sign(JWT_KEY);
}
export function verifyPassword(
password: string,
storedHash: string,
): Promise<boolean> {
const [hash, salt] = storedHash.split(".");
return new Promise((resolve, reject) => {
scrypt(password, salt, 64, (err, derivedKey) => {
if (err) reject(err);
else {
const expected = Buffer.from(hash ?? "", "hex");
resolve(
expected.length === derivedKey.length &&
timingSafeEqual(expected, derivedKey),
);
/**
* Returns the user and cadence of an unsubscribe token, or null when it is
* invalid or another kind of token.
*/
export async function verifyUnsubscribeToken(
token: string,
): Promise<{ userId: string; cadence: DigestCadence } | null> {
try {
const { payload } = await jwtVerify(token, JWT_KEY);
if (
payload.purpose !== "unsubscribe" || typeof payload.userId !== "string" ||
!isDigestCadence(payload.cadence)
) {
return null;
}
return { userId: payload.userId, cadence: payload.cadence };
} catch {
return null;
}
});
});
}
/**
* The algorithm of the tokens instances exchange: Ed25519 signatures, short
* and fast, with no parameters to get wrong.
*/
const INSTANCE_ALG = "EdDSA";
/**
* An instance's key, ready to sign, from {@link importInstanceKey}. Its
* `kid` names the public half in the instance's JWKS.
*/
export interface InstanceSigningKey {
kid: string;
privateKey: CryptoKey;
}
/**
* Returns a new Ed25519 key pair as JWKs, with a random `kid`, for
* `instance-key-service.ts` to store when the instance has none. The public
* JWK carries the `kid`, `alg` and `use` a JWKS entry needs.
*/
export async function generateInstanceKey(): Promise<
{ kid: string; privateJwk: JWK; publicJwk: JWK }
> {
const { privateKey, publicKey } = await generateKeyPair(INSTANCE_ALG, {
crv: "Ed25519",
extractable: true,
});
const kid = crypto.randomUUID();
return {
kid,
privateJwk: await exportJWK(privateKey),
publicJwk: {
...await exportJWK(publicKey),
kid,
alg: INSTANCE_ALG,
use: "sig",
},
};
}
/** Returns the stored private JWK `privateJwk` as a key to sign with. */
export async function importInstanceKey(
kid: string,
privateJwk: JWK,
): Promise<InstanceSigningKey> {
const privateKey = await importJWK(privateJwk, INSTANCE_ALG);
if (!(privateKey instanceof CryptoKey)) {
throw new Error("The instance key is not an asymmetric key");
}
return { kid, privateKey };
}
/**
* Signs `payload` with the instance's key, from `issuer` (this instance) to
* `audience` (the instance it is for), valid for `expiresIn`, with a random
* `jti`. The identity tokens of `POST /api/oauth/tokens` and the events of
* the outbox are signed with it.
*/
export async function signInstanceToken(
payload: JWTPayload,
key: InstanceSigningKey,
claims: { issuer: string; audience: string; subject: string },
expiresIn: string,
): Promise<string> {
return await new SignJWT(payload)
.setProtectedHeader({ alg: INSTANCE_ALG, kid: key.kid })
.setIssuer(claims.issuer)
.setAudience(claims.audience)
.setSubject(claims.subject)
.setIssuedAt()
.setJti(crypto.randomUUID())
.setExpirationTime(expiresIn)
.sign(key.privateKey);
}
/**
* Returns the payload of `token` once its signature checks against `jwks`,
* the keys of `issuer`, and it is addressed to `audience` and unexpired.
*
* @throws {Error} jose's error when it doesn't verify; its `code` is
* `ERR_JWKS_NO_MATCHING_KEY` when no key of `jwks` has its `kid`, which
* tells the caller to refetch the keys, the issuer having rotated them.
*/
export async function verifyInstanceToken(
token: string,
jwks: JSONWebKeySet,
claims: { issuer: string; audience: string },
): Promise<JWTPayload> {
const { payload } = await jwtVerify(token, createLocalJWKSet(jwks), {
issuer: claims.issuer,
audience: claims.audience,
algorithms: [INSTANCE_ALG],
});
return payload;
}

24
api/lib/like.ts Normal file
View File

@@ -0,0 +1,24 @@
/**
* Builds the patterns of the search queries' `LIKE`, for the dump, playlist
* and user searches. A query's `%` and `_` are matched as themselves rather
* than as wildcards: usernames hold underscores, and "a_b" must not find
* "axb".
*
* @module
*/
/**
* Returns the pattern matching text that contains `query`, or that starts
* with it when `where` is `"prefix"`. The SQL must read `LIKE ? ESCAPE '\'`,
* which the {@link LIKE} fragment spells.
*/
export function likePattern(
query: string,
where: "anywhere" | "prefix" = "anywhere",
): string {
const escaped = query.replace(/[\\%_]/g, "\\$&");
return where === "prefix" ? `${escaped}%` : `%${escaped}%`;
}
/** Spells `LIKE ? ESCAPE '\'`, for a {@link likePattern} parameter. */
export const LIKE = "LIKE ? ESCAPE '\\'";

34
api/lib/pagination.ts
View File

@@ -1,9 +1,17 @@
/**
* Pages every list route the same way. It reads the page a list route's query
* string asks for, and cuts the rows a service fetched into that page, which
* the routes send as `PaginatedData`.
*
* @module
*/
import { PAGINATION_DEFAULT_LIMIT, PAGINATION_MAX_LIMIT } from "../config.ts";
/**
* Parses page/limit query parameters with sensible defaults and bounds.
* page: clamped to [1, ∞)
* limit: clamped to [1, 100], defaults to 20
* Reads the `page` and `limit` query parameters of a list route. It never
* fails: a missing or invalid `page` reads as 1, and `limit` is kept between 1
* and {@link PAGINATION_MAX_LIMIT}.
*/
export function parsePagination(
params: URLSearchParams,
@@ -22,3 +30,23 @@ export function parsePagination(
);
return { page, limit };
}
/**
* Returned to the routes by the services' list functions, and sent as
* `PaginatedData`.
*/
export interface Page<T> {
items: T[];
hasMore: boolean;
}
/**
* Turns rows fetched with `LIMIT limit + 1` into a page. The extra row, when it
* comes back, is dropped and tells that another page follows, which spares a
* COUNT(*) over the whole list.
*/
export function pageOf<T>(rows: T[], limit: number): Page<T> {
return rows.length > limit
? { items: rows.slice(0, limit), hasMore: true }
: { items: rows, hasMore: false };
}

48
api/lib/password.ts Normal file
View File

@@ -0,0 +1,48 @@
/**
* Hashes and checks passwords, for sign-up, sign-in, password changes and the
* admin that `db/startup.ts` seeds. It reads no config, so the startup tasks
* can use it before the server's settings are checked.
*
* @module
*/
import { randomBytes, scrypt, timingSafeEqual } from "node:crypto";
import { Buffer } from "node:buffer";
/**
* Hashes `password` with scrypt and a random salt, as `hash.salt` in hex, for
* `users.password_hash`.
*/
export function hashPassword(password: string): Promise<string> {
const salt = randomBytes(16).toString("hex");
return new Promise((resolve, reject) => {
scrypt(password, salt, 64, (err, derivedKey) => {
if (err) reject(err);
else resolve(`${derivedKey.toString("hex")}.${salt}`);
});
});
}
/** Checks `password` against a hash made by {@link hashPassword}. */
export function verifyPassword(
password: string,
storedHash: string,
): Promise<boolean> {
const [hash, salt] = storedHash.split(".");
// scrypt would throw on a missing salt, and a malformed hash matches nothing.
if (!hash || !salt) return Promise.resolve(false);
return new Promise((resolve, reject) => {
scrypt(password, salt, 64, (err, derivedKey) => {
if (err) reject(err);
else {
const expected = Buffer.from(hash ?? "", "hex");
resolve(
expected.length === derivedKey.length &&
timingSafeEqual(expected, derivedKey),
);
}
});
});
}

49
api/lib/permissions.ts
View File

@@ -1,41 +1,18 @@
import { type AuthPayload, type Role } from "../model/interfaces.ts";
/**
* Checks an authenticated caller's permissions. Routes call it where the
* answer depends on more than the permission, such as "the owner, or a
* moderator"; `requirePermission` gates a whole route instead.
*
* @module
*/
// Named capabilities checked at enforcement points. Extend this union as new
// gated capabilities are added (e.g. "invite:manage").
export type Permission =
| "dump:moderate"
| "comment:moderate"
| "playlist:moderate"
| "chat:moderate"
| "user:manage"
| "category:manage";
const ALL_PERMISSIONS: readonly Permission[] = [
"dump:moderate",
"comment:moderate",
"playlist:moderate",
"chat:moderate",
"user:manage",
"category:manage",
];
// Roles are fixed in code. `admin` is a superset of every permission (current
// and future); `moderator` gets the content-moderation permissions only.
export const ROLE_PERMISSIONS: Record<Role, readonly Permission[]> = {
admin: ALL_PERMISSIONS,
moderator: [
"dump:moderate",
"comment:moderate",
"playlist:moderate",
"chat:moderate",
],
user: [],
};
export function hasPermission(role: Role, permission: Permission): boolean {
return ROLE_PERMISSIONS[role].includes(permission);
}
import { hasPermission, type Permission } from "../../shared/permissions.ts";
import type { AuthPayload } from "../model/auth.ts";
/**
* Checks whether the caller's role grants `permission`. The role is the live
* one `authenticate` put in the payload, not the token's.
*/
export function can(payload: AuthPayload, permission: Permission): boolean {
return hasPermission(payload.role, permission);
}

87
api/lib/playlist-members.ts Normal file
View File

@@ -0,0 +1,87 @@
/**
* Decides which of a playlist's dumps a viewer may see. The playlist service
* filters its listings with it, the WebSocket service filters each client's
* `playlist_dumps_updated` message the same way, and the queries that list
* playlists count their dumps by the same rule.
*
* @module
*/
import type { Playlist } from "../../shared/playlist.ts";
/**
* The fields of a playlist's dump that {@link isMemberVisibleTo} needs. A
* `Dump` has them. For its broadcasts, the playlist service reads only these.
*/
export interface PlaylistMember {
id: string;
isPrivate: boolean;
/** The dump's poster, not the playlist's owner. */
userId: string;
}
/**
* Checks whether `viewerId` may see `member` in `playlist`. A public dump is
* visible to all. A private dump is visible only to the playlist's owner, and
* only if they posted it: moderators don't see it, and neither does the owner
* once someone else's dump in their playlist goes private. A null or undefined
* `viewerId` is a signed-out viewer.
*/
export function isMemberVisibleTo(
member: PlaylistMember,
playlist: Pick<Playlist, "userId">,
viewerId: string | null | undefined,
): boolean {
return !member.isPrivate ||
(member.userId === viewerId && viewerId === playlist.userId);
}
/**
* The fields of a playlist's dump that {@link dumpFiguresFor} needs: those
* that tell who may see it, and when it was added. The playlist service reads
* them for its broadcasts.
*/
export interface DatedPlaylistMember extends PlaylistMember {
addedAt: Date;
}
/** A figure of the dumps of `playlists p` that the viewer `?1` may see. */
const visibleDumps = (figure: string) =>
`(SELECT ${figure} FROM playlist_dumps pd
INNER JOIN dumps d ON d.id = pd.dump_id
WHERE pd.playlist_id = p.id
AND (d.is_private = 0 OR (d.user_id = p.user_id AND p.user_id = ?1))
)`;
/**
* The `dump_count` and `last_dump_at` columns of a query on `playlists p`:
* how many of the playlist's dumps the viewer may see, by
* {@link isMemberVisibleTo}'s rule, and when the latest of them was added,
* null without any. Both read the viewer's id, or `""` for a signed-out one,
* as `?1`: it is the query's first parameter, and each `?` after it takes the
* next.
*/
export const VISIBLE_DUMP_FIGURES = `${visibleDumps("COUNT(*)")} as dump_count,
${visibleDumps("MAX(pd.added_at)")} as last_dump_at`;
/**
* Returns what `playlist` says of its dumps to the viewer, `members` being
* all of them: the `dumpCount` and `lastDumpAt` that
* {@link VISIBLE_DUMP_FIGURES} reads in a query. The WebSocket service puts
* them on the playlist each socket is sent.
*/
export function dumpFiguresFor(
members: DatedPlaylistMember[],
playlist: Pick<Playlist, "userId">,
viewerId: string | null | undefined,
): Required<Pick<Playlist, "dumpCount">> & Pick<Playlist, "lastDumpAt"> {
const visible = members.filter((m) =>
isMemberVisibleTo(m, playlist, viewerId)
);
return {
dumpCount: visible.length,
lastDumpAt: visible.length
? new Date(Math.max(...visible.map((m) => m.addedAt.getTime())))
: undefined,
};
}

40
api/lib/rate-limit.ts
View File

@@ -1,20 +1,35 @@
// Token buckets keyed by the caller's choice (e.g. `${kind}:${userId}`) — per
// user rather than per socket or request, so more tabs or parallel requests
// don't buy more throughput.
/**
* Limits how often something may happen, with token buckets kept in memory,
* which a restart empties. Logins, password reset requests and the chat's
* typing messages use it.
*
* @module
*/
import type { Job } from "./scheduler.ts";
interface Bucket {
tokens: number;
at: number;
}
const buckets = new Map<string, Bucket>();
/**
* Passed to {@link takeToken} by each limited action, which declares its own
* as a constant.
*/
export interface RateLimit {
/** Burst size. */
/** How many tokens a full bucket holds, and so the largest burst. */
capacity: number;
/** Tokens regained per second. */
refill: number;
}
/** Takes one token for `key`; false when the bucket is empty. */
/**
* Takes one token from `key`'s bucket, or returns false when it is empty. The
* caller picks the key, such as `typing:<userId>`, and keys by user or account
* so that more tabs or parallel requests don't buy more tokens.
*/
export function takeToken(key: string, limit: RateLimit): boolean {
const now = Date.now();
const bucket = buckets.get(key) ?? { tokens: limit.capacity, at: now };
@@ -29,11 +44,18 @@ export function takeToken(key: string, limit: RateLimit): boolean {
return true;
}
// Idle buckets have refilled by now for every limit in use; drop them so the
// map tracks only keys that have been active lately.
setInterval(() => {
/**
* Drops the buckets untouched for an hour, every minute. Every limit in use
* refills within the hour, so such a bucket is full and dropping it changes
* nothing. `main.ts` starts it.
*/
export const rateLimitSweepJob: Job = {
name: "rate-limit",
everyMs: 60_000,
run() {
const cutoff = Date.now() - 60 * 60_000;
for (const [key, bucket] of buckets) {
if (bucket.at < cutoff) buckets.delete(key);
}
}, 60_000);
},
};

79
api/lib/reserved-slugs.ts
View File

@@ -1,33 +1,78 @@
// Top-level path segments that already have meaning in the SPA router
// (see src/App.tsx) plus a few reserved for infrastructure. Category slugs
// live at the URL root (e.g. /music), so they must never collide with these.
/**
* Checks which slugs a dump category may take. A dump category's feed lives at
* the URL root (`/music`), so its slug must not shadow another top-level page.
* The category service runs this check when a dump category is created or
* renamed, and the migrations that reserve more words rename the categories
* that had one with {@link renameReservedCategories}. A playlist category's
* page is `/playlists/<slug>`, under a reserved word, so it reserves nothing.
*
* Dump and playlist slugs reserve nothing: they live under their owner's
* `dumps/` and `playlists/`, where nothing else is named by a user. A user
* can't clash with a category either, since `@` is never in a slug.
*
* @module
*/
import type { DatabaseSync } from "node:sqlite";
import { FEED_TABS } from "../../shared/paths.ts";
/**
* Lists the top-level path segments a category slug may not take: the feed
* tabs (`/hot`), the SPA's own pages (`src/App.tsx`), and `api`, `ws`,
* `assets` and `emojibase`, which the server answers before the SPA. Keep it
* in step with the route table and the build's top-level folders.
*/
export const RESERVED_SLUGS: readonly string[] = [
"~",
"dumps",
"users",
"playlists",
...FEED_TABS,
"search",
"login",
"register",
"notifications",
"reset-password",
"unsubscribe",
"settings",
"manage",
"community",
"playlists",
"authorize",
"federation",
"api",
"ws",
"assets",
"emojibase",
];
// Lowercase alphanumerics, hyphen-separated, no leading/trailing/double hyphen.
export const CATEGORY_SLUG_RE = /^[a-z0-9]+(?:-[a-z0-9]+)*$/;
export function isValidSlugFormat(slug: string): boolean {
return CATEGORY_SLUG_RE.test(slug);
}
/** Checks whether `slug` is one of the {@link RESERVED_SLUGS}. */
export function isReservedSlug(slug: string): boolean {
return RESERVED_SLUGS.includes(slug);
}
/** A slug is available when it is well-formed and not reserved. */
export function isSlugAvailable(slug: string): boolean {
return isValidSlugFormat(slug) && !isReservedSlug(slug);
/**
* Renames each category whose slug is now reserved, such as `hot`, to the
* first of `<slug>-2`, `<slug>-3`… that is free, and logs it. Migration 0021
* runs it for v4's words, 0022 for `community`, 0024 for `authorize` and
* `federation`, and 0025 for `playlists`.
*
* It reads every category: until 0025 gave categories a `kind`, each was a
* dump category, and 0025 makes no other. A later migration that reserves
* more words must leave the playlist categories alone, and limit it to
* `kind = 'dump'`, which the earlier ones can't, having no such column.
*/
export function renameReservedCategories(db: DatabaseSync): void {
const rows = db.prepare(`SELECT id, slug FROM categories;`).all() as {
id: string;
slug: string;
}[];
const taken = new Set(rows.map((r) => r.slug));
for (const { id, slug } of rows) {
if (!isReservedSlug(slug)) continue;
let n = 2;
while (taken.has(`${slug}-${n}`) || isReservedSlug(`${slug}-${n}`)) n++;
const renamed = `${slug}-${n}`;
taken.add(renamed);
db.prepare(`UPDATE categories SET slug = ? WHERE id = ?;`).run(renamed, id);
console.log(
`[migrate] category "${slug}" is now "${renamed}": its slug is reserved`,
);
}
}

46
api/lib/scheduler.ts Normal file
View File

@@ -0,0 +1,46 @@
/**
* Runs the server's recurring tasks. Each module that needs one exports a
* {@link Job}, and `main.ts` passes them all to {@link startJobs} when it runs
* directly, so nothing recurs in the tests, which call a job's `run` when
* they need it.
*
* @module
*/
/** A recurring task, exported by its module for `main.ts` to start. */
export interface Job {
/** Names it in the log, as `[digest]`. */
name: string;
/** How long from one run's start to the next. */
everyMs: number;
/** How long after the start the first run waits, `everyMs` by default. */
firstAfterMs?: number;
/** Logs what it did itself, if anything; its errors are logged here. */
run(): void | Promise<void>;
}
/**
* Starts running each of `jobs` on its own timer. A run still going when the
* next is due makes that one skip, and a run that throws or rejects is logged
* and the next one runs anyway.
*/
export function startJobs(jobs: readonly Job[]): void {
for (const job of jobs) {
let running = false;
const tick = async () => {
if (running) return;
running = true;
try {
await job.run();
} catch (err) {
console.error(`[${job.name}] run failed:`, err);
} finally {
running = false;
}
};
setTimeout(() => {
tick();
setInterval(tick, job.everyMs);
}, job.firstAfterMs ?? job.everyMs);
}
}

28
api/lib/self-url.ts
View File

@@ -1,4 +1,14 @@
/**
* Recognises links to this site's own pages. The self provider previews them
* from the database, remote/fetch.ts accepts them as dump links even on a
* private host such as localhost, and migration 0021 rewrote the v3 ones in
* stored text.
*
* @module
*/
import { FRONTEND_URL, PUBLIC_URL } from "../config.ts";
import { parsePagePath } from "../../shared/paths.ts";
const SELF_HOSTS = new Set(
[PUBLIC_URL, FRONTEND_URL].flatMap((u) => {
@@ -10,14 +20,22 @@ const SELF_HOSTS = new Set(
}),
);
const SELF_PATH_RE = /^\/(dumps|users|playlists)\/[^/]+$/;
/**
* Checks whether `host` (with its port, as `URL.host` gives it) is where this
* site is served, at its public URL or its frontend's.
*/
export function isSelfHost(host: string): boolean {
return SELF_HOSTS.has(host.toLowerCase());
}
/**
* A link to one of this site's own dump, user or playlist pages. Matched on the
* host as well as the path: `/users/bob` on some other site is someone else's
* Checks whether `url` links to one of this site's own dump, user or playlist
* pages, by `parsePagePath`; an editor doesn't count. Matched on the
* host as well as the path: `/@bob` on some other site is someone else's
* page, not ours.
*/
export function isSelfUrl(url: URL): boolean {
return SELF_HOSTS.has(url.host.toLowerCase()) &&
SELF_PATH_RE.test(url.pathname);
if (!isSelfHost(url.host)) return false;
const page = parsePagePath(url.pathname);
return !!page && !(page.kind !== "user" && page.edit);
}

168
api/lib/serve-file.ts Normal file
View File

@@ -0,0 +1,168 @@
/**
* Sends files from disk with ETags and byte ranges, streamed a chunk at a
* time. The files and thumbnails routes call it, and lib/upload.ts for the
* other uploads.
*
* @module
*/
import type { Context } from "@oak/oak";
const CHUNK_SIZE = 64 * 1024;
/**
* Streams `length` bytes of an open file, from wherever its cursor stands,
* and closes it once done or cancelled. The response never holds more than a
* chunk of the file in memory — a seek in a 50 MB track reads what it plays.
*/
function fileStream(
file: Deno.FsFile,
length: number,
): ReadableStream<Uint8Array> {
let remaining = length;
let closed = false;
const close = () => {
if (closed) return;
closed = true;
file.close();
};
return new ReadableStream({
async pull(controller) {
if (remaining <= 0) {
close();
controller.close();
return;
}
const buf = new Uint8Array(Math.min(CHUNK_SIZE, remaining));
let n: number | null;
try {
n = await file.read(buf);
} catch (err) {
close();
controller.error(err);
return;
}
if (n === null) {
close();
controller.close();
return;
}
remaining -= n;
controller.enqueue(buf.subarray(0, n));
if (remaining <= 0) {
close();
controller.close();
}
},
cancel() {
close();
},
});
}
/** Checks whether an If-None-Match header names `etag` (weak comparison). */
function matchesEtag(header: string | null, etag: string): boolean {
if (!header) return false;
if (header.trim() === "*") return true;
const strip = (tag: string) => tag.trim().replace(/^W\//, "");
return header.split(",").some((tag) => strip(tag) === strip(etag));
}
/**
* Passed to {@link serveFile} by its caller, which knows the file's type and
* how long it may be cached.
*/
export interface ServeFileOptions {
mime: string;
cacheControl: string;
/**
* Answers a single `Range: bytes=…` with a 206. The files route sets it, so
* that audio and video can seek.
*/
ranges?: boolean;
}
/**
* Sends a file from disk with an ETag, answering 304 when the client already
* has it, and single byte ranges when `ranges` is set. It returns false, having
* sent nothing, when the file doesn't exist, and leaves the 404 to the caller.
*/
export async function serveFile(
ctx: Context,
path: string,
{ mime, cacheControl, ranges = false }: ServeFileOptions,
): Promise<boolean> {
let file: Deno.FsFile;
let stat: Deno.FileInfo;
try {
file = await Deno.open(path, { read: true });
stat = await file.stat();
} catch {
return false;
}
const total = stat.size;
const mtime = stat.mtime?.getTime() ?? 0;
const etag = `W/"${total.toString(36)}-${mtime.toString(36)}"`;
const headers = ctx.response.headers;
headers.set("Content-Type", mime);
headers.set("Cache-Control", cacheControl);
headers.set("ETag", etag);
if (stat.mtime) headers.set("Last-Modified", stat.mtime.toUTCString());
if (ranges) headers.set("Accept-Ranges", "bytes");
if (matchesEtag(ctx.request.headers.get("If-None-Match"), etag)) {
file.close();
ctx.response.status = 304;
return true;
}
// Only a single byte range is served as one. Anything else — several
// ranges, another unit — is a header RFC 9110 lets us ignore, so the whole
// file goes out with a 200 rather than a 416 for ranges that were fine.
const match = ranges
? ctx.request.headers.get("Range")?.match(/^bytes=(\d*)-(\d*)$/)
: null;
if (!match) {
// Explicit: a caller may have tried another file first and left a 404.
ctx.response.status = 200;
headers.set("Content-Length", String(total));
ctx.response.body = fileStream(file, total);
return true;
}
// "bytes=a-b", "bytes=a-" (to the end) or "bytes=-n" (the last n bytes).
let start: number;
let end: number;
if (match[1]) {
start = parseInt(match[1], 10);
end = match[2] ? Math.min(parseInt(match[2], 10), total - 1) : total - 1;
} else {
const suffix = parseInt(match[2], 10);
start = Math.max(total - suffix, 0);
end = total - 1;
// "bytes=-" and "bytes=-0" name no bytes at all.
if (!(suffix > 0)) start = total;
}
if (start > end || start >= total) {
file.close();
ctx.response.status = 416;
headers.set("Content-Range", `bytes */${total}`);
return true;
}
const length = end - start + 1;
try {
await file.seek(start, Deno.SeekMode.Start);
} catch (err) {
file.close();
throw err;
}
ctx.response.status = 206;
headers.set("Content-Range", `bytes ${start}-${end}/${total}`);
headers.set("Content-Length", String(length));
ctx.response.body = fileStream(file, length);
return true;
}

102
api/lib/site-icons.ts
View File

@@ -1,23 +1,26 @@
// Generates the full favicon set + web manifest from a single emoji.
//
// Twemoji ships each emoji as a plain vector-path SVG, so it rasterizes without
// any system emoji font (which matters in the minimal Alpine runtime image).
// @resvg/resvg-wasm does the rasterization; the .ico is assembled by hand.
//
// Invoked by api/scripts/generate-icons.ts at server startup (into dist/ in
// prod, public/ in dev) — see that script for how the output dir is chosen.
import { initWasm, Resvg } from "@resvg/resvg-wasm";
/**
* Renders the site's favicons and web manifest from its emoji. The `icons`
* task (`api/scripts/generate-icons.ts`) runs it before the server starts, so
* the emoji can change without a rebuild.
*
* The glyph comes from Twemoji as plain vector paths, so rendering it needs no
* emoji font, which the Alpine runtime image lacks.
*
* @module
*/
import { initWasm, Resvg } from "@resvg/resvg-wasm";
import { fetchTwemojiSvg } from "../remote/twemoji.ts";
// Pinned Twemoji release.
const TWEMOJI_VERSION = "16.0.1";
const PNG_SIZES = [16, 32, 48, 64, 96, 128, 192, 256, 512];
const APPLE_SIZE = 180;
const ICO_SIZES = [16, 32, 48];
/**
* Mirror Twemoji's filename convention: strip variation selectors (FE0F) unless
* the emoji is a ZWJ sequence, then join codepoints with "-". Also doubles as a
* stable cache-busting version string for the icon URLs.
* Returns the name Twemoji files `emoji` under: its codepoints in hex, joined
* with "-", without variation selectors (FE0F) unless the emoji is a ZWJ
* sequence. `lib/index-html.ts` also uses it as the icon URLs' cache-busting
* version.
*/
export function emojiToCodepoint(emoji: string): string {
const cleaned = emoji.includes("‍") ? emoji : emoji.replace(/️/g, "");
@@ -27,46 +30,23 @@ export function emojiToCodepoint(emoji: string): string {
let wasmReady: Promise<unknown> | undefined;
function ensureWasm(): Promise<unknown> {
if (!wasmReady) {
const wasmUrl = new URL(import.meta.resolve("@resvg/resvg-wasm/index_bg.wasm"));
const wasmUrl = new URL(
import.meta.resolve("@resvg/resvg-wasm/index_bg.wasm"),
);
wasmReady = Deno.readFile(wasmUrl).then((bytes) => initWasm(bytes));
}
return wasmReady;
}
// Cache the fetched glyph per emoji so repeated runs don't re-hit the network.
const glyphCache = new Map<string, string>();
async function fetchGlyph(emoji: string): Promise<string> {
const cp = emojiToCodepoint(emoji);
const cached = glyphCache.get(cp);
if (cached) return cached;
const url =
`https://cdn.jsdelivr.net/gh/jdecked/twemoji@${TWEMOJI_VERSION}/assets/svg/${cp}.svg`;
let res: Response;
try {
res = await fetch(url);
} catch (cause) {
throw new Error(
`Failed to fetch Twemoji glyph for "${emoji}" (${cp}) from ${url}`,
{ cause },
);
}
if (!res.ok) {
throw new Error(
`Failed to fetch Twemoji glyph for "${emoji}" (${cp}) from ${url}: ${res.status} ${res.statusText}`,
);
}
const svg = await res.text();
glyphCache.set(cp, svg);
return svg;
}
function renderPng(svg: string, size: number, background?: string): Uint8Array {
const resvg = new Resvg(svg, { fitTo: { mode: "width", value: size }, background });
const resvg = new Resvg(svg, {
fitTo: { mode: "width", value: size },
background,
});
return resvg.render().asPng();
}
// Pack PNG payloads into a (PNG-compressed) .ico container.
/** Packs PNG images into an .ico file, which may hold PNGs as they are. */
function buildIco(images: { size: number; png: Uint8Array }[]): Uint8Array {
const HEADER = 6;
const ENTRY = 16;
@@ -118,27 +98,31 @@ function buildManifest(siteName: string, bgColor: string): string {
return JSON.stringify(manifest, null, 2) + "\n";
}
/**
* Passed to {@link generateSiteIcons} by `api/scripts/generate-icons.ts`, from
* the site's config.
*/
export interface GenerateSiteIconsOptions {
/** Emoji to render the icons from. */
emoji: string;
/** Directory to write the assets into (created if missing). */
/** Created if missing. */
outDir: string;
/** Site name written into the manifest. */
siteName: string;
/** Background color for the Apple touch icon and manifest (iOS ignores transparency). */
/** Fills the Apple touch icon, and the manifest's theme and background. */
bgColor: string;
}
/**
* Generate `favicon.svg`, `favicon.ico`, `favicon-<size>x<size>.png` (16–512),
* `apple-touch-icon.png` and `manifest.webmanifest` into `outDir`.
* Throws if the emoji glyph can't be fetched.
* Writes `favicon.svg`, `favicon.ico`, `favicon-<size>x<size>.png` (16 to 512),
* `apple-touch-icon.png` and `manifest.webmanifest` into `opts.outDir`, the
* directory the site's static files are served from.
*
* @throws {Error} When the emoji's glyph can't be fetched.
*/
export async function generateSiteIcons(
opts: GenerateSiteIconsOptions,
): Promise<void> {
const { emoji, outDir, siteName, bgColor } = opts;
const svg = await fetchGlyph(emoji);
const svg = await fetchTwemojiSvg(emoji, emojiToCodepoint(emoji));
await ensureWasm();
await Deno.mkdir(outDir, { recursive: true });
@@ -151,11 +135,17 @@ export async function generateSiteIcons(
for (const size of PNG_SIZES) {
await Deno.writeFile(out(`favicon-${size}x${size}.png`), pngs.get(size)!);
}
// Apple touch icon gets a solid background — iOS doesn't honor transparency.
await Deno.writeFile(out("apple-touch-icon.png"), renderPng(svg, APPLE_SIZE, bgColor));
// iOS draws a transparent touch icon on black.
await Deno.writeFile(
out("apple-touch-icon.png"),
renderPng(svg, APPLE_SIZE, bgColor),
);
await Deno.writeFile(
out("favicon.ico"),
buildIco(ICO_SIZES.map((size) => ({ size, png: pngs.get(size)! }))),
);
await Deno.writeTextFile(out("manifest.webmanifest"), buildManifest(siteName, bgColor));
await Deno.writeTextFile(
out("manifest.webmanifest"),
buildManifest(siteName, bgColor),
);
}

96
api/lib/slugify.ts
View File

@@ -1,18 +1,86 @@
export const UUID_RE =
/^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i;
/**
* Picks and checks the slugs of dumps and playlists, which name them in their
* page URLs under their owner: `/@bob/dumps/<slug>`, `/@bob/playlists/<slug>`.
* Each kind has its own slug space per owner, shared by the items' current
* slugs and their aliases (the slugs they had before). `services/slug-service.ts`
* changes and resolves slugs with these, and migration 0021 gave every item
* one; both pass the database, since this module doesn't open it.
*
* @module
*/
export function slugify(title: string): string {
const slug = title
.toLowerCase()
.normalize("NFD")
.replace(/[\u0300-\u036f]/g, "") // strip diacritics
.replace(/[^a-z0-9]+/g, "-") // non-alphanumeric → dash
.replace(/^-+|-+$/g, "") // trim leading/trailing dashes
.substring(0, 60);
return slug || "untitled";
import type { DatabaseSync } from "node:sqlite";
import { APIException } from "./api-exception.ts";
import { isValidSlug, SLUG_MAX } from "../../shared/slug.ts";
/** A kind of item that has a slug under its owner. */
export type SlugKind = "dump" | "playlist";
/** Where each {@link SlugKind}'s slugs live. */
export const SLUG_TABLES: Record<
SlugKind,
{ items: string; aliases: string; itemId: string }
> = {
dump: { items: "dumps", aliases: "dump_slug_aliases", itemId: "dump_id" },
playlist: {
items: "playlists",
aliases: "playlist_slug_aliases",
itemId: "playlist_id",
},
};
/**
* Returns the id of the item whose current slug or alias is `slug` in
* `userId`'s space for `kind`, or undefined when it is free.
*/
export function slugHolder(
db: DatabaseSync,
kind: SlugKind,
userId: string,
slug: string,
): string | undefined {
const { items, aliases, itemId } = SLUG_TABLES[kind];
const row = db.prepare(
`SELECT id FROM ${items} WHERE user_id = ? AND slug = ?
UNION ALL
SELECT ${itemId} FROM ${aliases} WHERE user_id = ? AND slug = ?
LIMIT 1;`,
).get(userId, slug, userId, slug) as { id: string } | undefined;
return row?.id;
}
/** Stable slug tied to the record's id — unique by construction. */
export function makeSlug(title: string, id: string): string {
return `${slugify(title)}-${id.substring(0, 8)}`;
/**
* Returns `base`, or `base-2`, `base-3`… cut to fit, whichever is free in
* `userId`'s space for `kind`. `base` must be a valid slug, as `slugify`
* makes. The services take it on creating an item, the migration on
* re-slugging one, inside the transaction that writes it.
*/
export function uniqueSlug(
db: DatabaseSync,
kind: SlugKind,
userId: string,
base: string,
): string {
for (let n = 1;; n++) {
const suffix = n === 1 ? "" : `-${n}`;
const candidate = `${
base.substring(0, SLUG_MAX - suffix.length).replace(/-+$/, "")
}${suffix}`;
if (!slugHolder(db, kind, userId, candidate)) return candidate;
}
}
/**
* Throws a 400 unless `slug` is a valid slug, lowercase words of letters and
* digits joined by hyphens, at most `max` long. slug-service checks a dump's
* or a playlist's with it, and category-service a category's, with that
* kind's own limit.
*/
export function validateSlug(slug: string, max: number = SLUG_MAX): void {
if (!isValidSlug(slug, max)) {
throw new APIException(
400,
`Slug must be lowercase letters, numbers and hyphens, at most ${max} characters`,
);
}
}

72
api/lib/static.ts
View File

@@ -1,45 +1,83 @@
import { Context, Next, send } from "@oak/oak";
import { BANDCAMP_PLAYER, OG_SITE_NAME, SITE_EMOJI } from "../config.ts";
import { emojiToCodepoint } from "./site-icons.ts";
/**
* Serves the built frontend, as its static files and, for every client-side
* route, `index.html` with its placeholders filled. main.ts installs it after
* the routers.
*
* @module
*/
const ICON_VERSION = emojiToCodepoint(SITE_EMOJI);
import { Context, Next, send } from "@oak/oak";
import { loadIndexHtml } from "./index-html.ts";
async function serveIndexHtml(
context: Context<Record<string, object>>,
root: string,
) {
const filePath = `${root}/index.html`;
const raw = await Deno.readTextFile(filePath);
const html = raw
.replaceAll("__SITE_NAME__", OG_SITE_NAME)
.replaceAll("__SITE_EMOJI__", SITE_EMOJI)
.replaceAll("__ICON_VERSION__", ICON_VERSION)
.replaceAll("__BANDCAMP_PLAYER__", BANDCAMP_PLAYER);
const html = await loadIndexHtml(root);
context.response.type = "text/html";
// It names the hashed chunks of the current build, so it is always
// revalidated.
context.response.headers.set("Cache-Control", "no-cache");
context.response.body = html;
}
export function routeStaticFilesFrom(staticPaths: string[]) {
return async (context: Context<Record<string, object>>, next: Next) => {
/**
* Prefixes the build output whose URL changes whenever its content does, which
* are Vite's hashed chunks and the emoji data under its package version (see
* vite.config.ts).
*/
const IMMUTABLE_PREFIXES = ["/assets/", "/emojibase/"];
const ONE_YEAR_MS = 365 * 24 * 60 * 60 * 1000;
/**
* Returns the middleware that serves a static file from the first of
* `staticPaths` that has it, and `index.html` of the first for any other path.
* Hashed build output is cached for a year, and a missing chunk is a 404.
* main.ts passes dist/ then public/.
*/
export function routeStaticFilesFrom(
staticPaths: string[],
): (context: Context<Record<string, object>>, next: Next) => Promise<void> {
// It is the last middleware and always answers, so it never calls `next`.
return async (context: Context<Record<string, object>>, _next: Next) => {
const pathname = context.request.url.pathname;
// Serve index.html with runtime placeholder replacement
if (pathname === "/" || pathname === "/index.html") {
await serveIndexHtml(context, staticPaths[0]);
return;
}
const immutable = IMMUTABLE_PREFIXES.some((prefix) =>
pathname.startsWith(prefix)
);
// A request without Accept-Encoding technically accepts any coding (RFC
// 9110), but in practice it's curl or a bot that can't decode brotli.
const compress = context.request.headers.has("accept-encoding");
for (const path of staticPaths) {
try {
await send(context, pathname, { root: path });
// send() picks a prebuilt .br or .gz sibling when the browser accepts
// it.
await send(context, pathname, {
root: path,
brotli: compress,
gzip: compress,
...(immutable && { maxage: ONE_YEAR_MS, immutable: true }),
});
context.response.headers.append("Vary", "Accept-Encoding");
return;
} catch {
continue;
}
}
// SPA fallback: serve index.html so client-side routes work on direct navigation
// A chunk from an older build (an open tab after a deploy) must fail as a
// 404, not load index.html as a script; the app reloads on that failure.
if (immutable) {
context.response.status = 404;
return;
}
// Any other path is a client-side route, opened directly or reloaded.
await serveIndexHtml(context, staticPaths[0]);
await next();
};
}

60
api/lib/ttl-cache.ts Normal file
View File

@@ -0,0 +1,60 @@
/**
* Keeps what the server fetched from another site for a while, by key, so a
* page or a feed asking again doesn't refetch it: Bandcamp tracklists, YouTube
* playlists, and link previews. Each holder sets how long an entry lives and
* how many are kept; past that count, the oldest written goes first.
*
* @module
*/
/**
* A map whose entries expire `ttlMs` after they are written, holding
* `maxEntries` at most.
*
* @example
* const playlists = new TtlCache<YouTubePlaylist>(6 * 3_600_000, 200);
* const hit = playlists.get(listId) ?? playlists.set(listId, await fetch());
*/
export class TtlCache<V> {
#entries = new Map<string, { value: V; at: number }>();
constructor(readonly ttlMs: number, readonly maxEntries: number) {}
/** Returns the value under `key` unless it is missing or expired. */
get(key: string): V | undefined {
const hit = this.#entries.get(key);
if (!hit) return undefined;
if (Date.now() - hit.at >= this.ttlMs) {
this.#entries.delete(key);
return undefined;
}
return hit.value;
}
/** Stores `value` under `key` from now on, and returns it. */
set(key: string, value: V): V {
this.#entries.delete(key);
// A Map keeps insertion order, so the first key is the oldest write.
if (this.#entries.size >= this.maxEntries) {
const oldest = this.#entries.keys().next().value;
if (oldest !== undefined) this.#entries.delete(oldest);
}
this.#entries.set(key, { value, at: Date.now() });
return value;
}
/**
* Removes `key`, or only when `value` is still what it holds, so a late
* failure can't remove the entry that replaced its own.
*/
delete(key: string, value?: V): void {
if (value === undefined || this.#entries.get(key)?.value === value) {
this.#entries.delete(key);
}
}
/** Empties the cache, for the tests' reset between cases. */
clear(): void {
this.#entries.clear();
}
}

275
api/lib/upload.ts
View File

@@ -1,25 +1,31 @@
/**
* Checks the images users upload and serves them back, scaled down to the
* width a request names. The avatar, playlist cover, dump thumbnail and
* attachment routes call it.
*
* @module
*/
import type { Context } from "@oak/oak";
import { APIErrorCode, APIException } from "../model/interfaces.ts";
import { APIException } from "./api-exception.ts";
import { serveFile } from "./serve-file.ts";
import { dirname } from "node:path";
import { MAX_IMAGE_SIZE_BYTES } from "../config.ts";
import { type ImageWidth, parseImageWidth } from "../../shared/image.ts";
import {
ALLOWED_IMAGE_MIMES,
ATTACHMENTS_DIR,
AVATARS_DIR,
DUMPS_DIR,
MAX_IMAGE_SIZE_BYTES,
PLAYLIST_IMAGES_DIR,
UPLOADS_DIR,
} from "../config.ts";
ensureGenerated,
type ImageTraits,
type ScaledFormat,
scaledImagePath,
scaleImage,
} from "./ffmpeg.ts";
export {
ALLOWED_IMAGE_MIMES,
ATTACHMENTS_DIR,
AVATARS_DIR,
DUMPS_DIR,
PLAYLIST_IMAGES_DIR,
UPLOADS_DIR,
};
/** Detect image MIME type from magic bytes, ignoring the browser-declared type. */
/**
* Returns the MIME type of a JPEG, PNG, GIF or WebP image from its magic
* bytes, or null for anything else. These four are the only image types
* accepted anywhere. The type the browser declared is never
* trusted, since the file is later served with the type found here.
*/
export function detectImageMime(data: Uint8Array): string | null {
if (data[0] === 0xFF && data[1] === 0xD8 && data[2] === 0xFF) {
return "image/jpeg";
@@ -40,19 +46,23 @@ export function detectImageMime(data: Uint8Array): string | null {
return null;
}
/** Validates image upload data: checks size and MIME. Returns the detected MIME type or throws APIException. */
/**
* Checks an uploaded image's size and content, and returns its MIME type,
* which the route stores to serve the image with.
*
* @throws {APIException} 400 when it exceeds {@link MAX_IMAGE_SIZE_BYTES} or
* isn't a JPEG, PNG, GIF or WebP image.
*/
export function validateImageUpload(data: Uint8Array): string {
if (data.length > MAX_IMAGE_SIZE_BYTES) {
throw new APIException(
APIErrorCode.BAD_REQUEST,
400,
"File too large (max 5 MB)",
`File too large (max ${MAX_IMAGE_SIZE_BYTES / (1024 * 1024)} MB)`,
);
}
const mime = detectImageMime(data);
if (!mime) {
throw new APIException(
APIErrorCode.BAD_REQUEST,
400,
"File content is not a recognised image (JPEG, PNG, GIF, WebP)",
);
@@ -60,28 +70,217 @@ export function validateImageUpload(data: Uint8Array): string {
return mime;
}
/**
* Returns the EXIF orientation of a JPEG, 1 to 8, from the start of its
* bytes, and 1, upright, when it states none. A phone stores a photo as its
* sensor saw it and says here how to turn it.
*/
function jpegOrientation(data: Uint8Array): number {
const view = new DataView(data.buffer, data.byteOffset, data.byteLength);
// The segments follow the two-byte SOI, each a marker and its length.
let at = 2;
while (at + 4 <= data.length && data[at] === 0xFF) {
const marker = data[at + 1];
const length = view.getUint16(at + 2);
// APP1 holding "Exif\0\0", then a TIFF header and its first directory.
if (marker === 0xE1 && view.getUint32(at + 4) === 0x45786966) {
const tiff = at + 10;
if (tiff + 8 > data.length) return 1;
const little = view.getUint16(tiff) === 0x4949;
const ifd = tiff + view.getUint32(tiff + 4, little);
if (ifd + 2 > data.length) return 1;
const entries = view.getUint16(ifd, little);
for (let i = 0; i < entries; i++) {
const entry = ifd + 2 + i * 12;
if (entry + 12 > data.length) return 1;
if (view.getUint16(entry, little) === 0x0112) {
const value = view.getUint16(entry + 8, little);
return value >= 1 && value <= 8 ? value : 1;
}
}
return 1;
}
// The image data starts: no EXIF came before it.
if (marker === 0xDA) return 1;
at += 2 + length;
}
return 1;
}
/**
* How much of a file {@link scalableImage} needs: the EXIF segment, which
* comes first in a JPEG, is 64 KB at most.
*/
const IMAGE_HEAD_BYTES = 64 * 1024 + 32;
/** Returned by {@link scalableImage}, for `scaleImage`. */
export interface ScalableImage extends ImageTraits {
/**
* The format of its copy for a reader without WebP: a JPEG, or a PNG for
* an image that may be transparent, which a JPEG would fill with black.
*/
compat: "jpg" | "png";
}
/**
* Returns how to scale the image whose first {@link IMAGE_HEAD_BYTES} are
* `head`, or null when it is to be served as it is: an animated WebP, which
* ffmpeg can't read, or anything but a JPEG, PNG, GIF, WebP or BMP.
*/
export function scalableImage(head: Uint8Array): ScalableImage | null {
const mime = detectImageMime(head);
// A file dump may be a BMP ("BM"), which no upload route here accepts.
if (!mime) {
return head[0] === 0x42 && head[1] === 0x4D
? { orientation: 1, animated: false, compat: "jpg" }
: null;
}
// "RIFF....WEBPVP8X", whose flags then say whether frames follow.
const animatedWebp = mime === "image/webp" && head[15] === 0x58 &&
(head[20] & 0x02) !== 0;
if (animatedWebp) return null;
return mime === "image/jpeg"
? { orientation: jpegOrientation(head), animated: false, compat: "jpg" }
: { orientation: 1, animated: mime === "image/gif", compat: "png" };
}
/** Reads the first {@link IMAGE_HEAD_BYTES} of the file at `path`. */
async function readHead(path: string): Promise<Uint8Array | null> {
let file: Deno.FsFile;
try {
file = await Deno.open(path, { read: true });
} catch {
return null;
}
try {
const head = new Uint8Array(IMAGE_HEAD_BYTES);
let filled = 0;
while (filled < head.length) {
const n = await file.read(head.subarray(filled));
if (n === null) break;
filled += n;
}
return head.subarray(0, filled);
} finally {
file.close();
}
}
/** The type each format of a scaled copy is served as. */
const SCALED_MIMES: Record<ScaledFormat, string> = {
webp: "image/webp",
jpg: "image/jpeg",
png: "image/png",
};
/**
* Returns the copy of the image at `path` scaled to `width`, made now unless
* it was already, as its path and type, or null when there is none to serve:
* the image can't be scaled ({@link scalableImage}), or ffmpeg is missing or
* failed on it. The copy is a WebP, or with `compat` the JPEG or PNG a reader
* without WebP takes.
*/
async function scaledCopy(
path: string,
width: ImageWidth,
compat: boolean,
): Promise<{ path: string; mime: string } | null> {
// The other format depends on the image, so it is read before the copy is
// looked for; a WebP's path is known without.
let how = compat ? await describe(path) : undefined;
if (how === null) return null;
const format: ScaledFormat = how ? how.compat : "webp";
const scaledPath = scaledImagePath(path, width, format);
const made = await ensureGenerated(scaledPath, async () => {
how ??= await describe(path);
if (!how) throw new Error("Not an image that can be scaled");
await scaleImage(path, scaledPath, width, how, format);
}, path);
return made ? { path: scaledPath, mime: SCALED_MIMES[format] } : null;
}
/** Returns how to scale the image at `path`: see {@link scalableImage}. */
async function describe(path: string): Promise<ScalableImage | null> {
const head = await readHead(path);
return head && scalableImage(head);
}
/**
* Sends the image at `path`, of type `mime`, with `cacheControl`: a WebP copy
* at most as wide as the request's `?w=` names (`shared/image.ts`), or as
* `defaultWidth` without one, and the image as it is when the request names
* no width or the copy can't be made. With `?f=compat`, the copy is a JPEG,
* or a PNG for an image that may be transparent, which the digest's emails
* ask for, since not every mail client reads WebP. Every route that answers
* one of the site's own images ends here, so each takes both. It resolves
* false, having sent nothing, when the file is missing.
*/
export async function serveImage(
ctx: Context,
path: string,
mime: string,
cacheControl: string,
defaultWidth?: ImageWidth,
): Promise<boolean> {
// The browser must take the checked type, never guess another from the
// bytes.
ctx.response.headers.set("X-Content-Type-Options", "nosniff");
const width = parseImageWidth(ctx.request.url.searchParams.get("w")) ??
defaultWidth;
const compat = ctx.request.url.searchParams.get("f") === "compat";
const scaled = width ? await scaledCopy(path, width, compat) : null;
if (
scaled &&
await serveFile(ctx, scaled.path, { mime: scaled.mime, cacheControl })
) return true;
return await serveFile(ctx, path, { mime, cacheControl });
}
/**
* Serves an uploaded image inline as `mime`, or scaled down as
* {@link serveImage} does, cached for `cacheMaxAge` seconds, and only by the
* browser when `isPrivate`. It answers 404 and resolves false when the file
* is missing. The caller checks first that the viewer may see it.
*/
export async function serveUploadedFile(
ctx: Context,
filePath: string,
mime: string,
cacheMaxAge = 3600,
isPrivate = false,
defaultWidth?: ImageWidth,
): Promise<boolean> {
let data: Uint8Array;
try {
data = await Deno.readFile(filePath);
} catch {
ctx.response.status = 404;
return false;
}
ctx.response.headers.set("Content-Type", mime);
ctx.response.headers.set("Content-Disposition", "inline");
ctx.response.headers.set("X-Content-Type-Options", "nosniff");
ctx.response.headers.set(
"Cache-Control",
`${isPrivate ? "private" : "public"}, max-age=${cacheMaxAge}`,
);
// A frontend on another origin may still show the image.
ctx.response.headers.set("Cross-Origin-Resource-Policy", "cross-origin");
ctx.response.body = data;
return true;
const served = await serveImage(
ctx,
filePath,
mime,
`${isPrivate ? "private" : "public"}, max-age=${cacheMaxAge}`,
defaultWidth,
);
if (!served) ctx.response.status = 404;
return served;
}
/**
* Writes `data` to `filePath`, replacing whatever is there only once the whole
* file is written, so a failed write leaves the old file whole. The upload
* routes call it before recording the new file, so that clients told of it
* never fetch it early.
*/
export async function replaceUploadedFile(
filePath: string,
data: Uint8Array,
): Promise<void> {
const tmpPath = `${filePath}.${crypto.randomUUID()}.tmp`;
await Deno.mkdir(dirname(filePath), { recursive: true });
try {
await Deno.writeFile(tmpPath, data);
await Deno.rename(tmpPath, filePath);
} catch (err) {
await Deno.remove(tmpPath).catch(() => {});
throw err;
}
}

626
api/mail/digest.ts Normal file
View File

@@ -0,0 +1,626 @@
/**
* Renders the weekly digest and the monthly recap, as the drafts approved
* them: the weekly one has four sections (what happened to the reader, what
* the people and playlists they follow posted, the period's top dumps, and
* the community), the monthly one only the last two. `digest-service.ts`
* gathers a {@link DigestContent} and sends what this returns, drawn in the
* reader's own style and scheme.
*
* @module
*/
import { EMAIL_LOCALE, OG_SITE_NAME } from "../config.ts";
import type { DigestCadence } from "../../shared/settings.ts";
import { i18n } from "./i18n.ts";
import { DIGEST_MESSAGES } from "../../shared/digest-messages.ts";
import {
EmailKit,
type EmailLook,
escMd,
mdLink,
type RenderedEmail,
} from "./layout.ts";
/** The kinds of activity a weekly digest counts. */
export type ActivityKind =
| "upvotes"
| "followers"
| "comments"
| "mentions"
| "likes"
| "playlistFollows";
/** What a highlight of the weekly digest says happened. */
export type HighlightKind =
| "mentionComment"
| "mentionDump"
| "mentionPlaylist"
| "mentionChat"
| "mentionEveryone"
| "comment"
| "playlistFollow"
| "userFollow"
| "commentLike";
/** A dump in a digest's lists, with its links made absolute. */
export interface DigestDump {
title: string;
url: string;
poster: string;
/** The link's host, or null for an upload. */
domain: string | null;
/** An absolute image URL, or null to show no image. */
thumbnail: string | null;
votes: number;
comments: number;
/** Posted by the reader, which the lists mark. */
mine: boolean;
/** The followed playlist it was added to, when that's why it's listed. */
addedTo?: string;
}
/** One line of the weekly digest's "For you", made from a notification. */
export interface DigestHighlight {
kind: HighlightKind;
/** Who did it. */
who: string;
/**
* What they did it on, absent for a follow of the reader or the chat. Its
* `url` is null once it is deleted, and the title then shows unlinked.
*/
target?: { title: string; url: string | null };
/** The comment or message, flattened and shortened. */
quote?: string;
at: Date;
}
/**
* Passed to {@link renderDigestEmail} by `digest-service.ts`. Each section is
* null when it has nothing to show, and is left out; the service sends
* nothing when all of them are.
*/
export interface DigestContent {
cadence: DigestCadence;
/** The period covered, from `from` to `to`. */
from: Date;
to: Date;
reader: string;
/** Weekly only. */
activity: {
counts: Partial<Record<ActivityKind, number>>;
highlights: DigestHighlight[];
/** Every notification of the period, which the link to them counts. */
total: number;
} | null;
/** Weekly only. */
network: { dumps: DigestDump[]; more: number } | null;
top: DigestDump[];
community: {
members: { username: string; invitedBy: string | null }[];
membersMore: number;
playlists: { title: string; url: string; owner: string; dumps: number }[];
playlistsMore: number;
chatMessages: number;
totals: { dumps: number; upvotes: number; comments: number };
} | null;
links: {
notifications: string;
followed: string;
chat: string;
/** The community page's view of the cadence's period. */
community: string;
settings: string;
/** The one-click page that turns this cadence off. */
unsubscribe: string;
};
}
const ACTIVITY_ORDER: ActivityKind[] = [
"upvotes",
"followers",
"comments",
"mentions",
"likes",
"playlistFollows",
];
/** Returns the label of `n` of a kind of activity, as in "12 upvotes". */
function countLabel(kind: ActivityKind, n: number): string {
switch (kind) {
case "upvotes":
return i18n._({ ...DIGEST_MESSAGES.countUpvotes, values: { n } });
case "followers":
return i18n._({
id: "email.digest.count.followers",
message: "{n, plural, one {new follower} other {new followers}}",
values: { n },
});
case "comments":
return i18n._({ ...DIGEST_MESSAGES.countComments, values: { n } });
case "mentions":
return i18n._({
id: "email.digest.count.mentions",
message: "{n, plural, one {mention} other {mentions}}",
values: { n },
});
case "likes":
return i18n._({
id: "email.digest.count.likes",
message: "{n, plural, one {comment like} other {comment likes}}",
values: { n },
});
case "playlistFollows":
return i18n._({
id: "email.digest.count.playlistFollows",
message: "{n, plural, one {playlist follow} other {playlist follows}}",
values: { n },
});
}
}
/**
* Returns the sentence of a highlight, `who` and `target` already formatted
* for the part it goes in (HTML or Markdown).
*/
function highlightSentence(
kind: HighlightKind,
who: string,
target: string,
): string {
const values = { who, target };
switch (kind) {
case "mentionComment":
return i18n._({
id: "email.digest.hl.mentionComment",
message: "{who} mentioned you in a comment on {target}",
values,
});
case "mentionDump":
case "mentionChat":
return i18n._({
id: "email.digest.hl.mentionIn",
message: "{who} mentioned you in {target}",
values,
});
case "mentionPlaylist":
return i18n._({
id: "email.digest.hl.mentionPlaylist",
message: "{who} mentioned you in the playlist {target}",
values,
});
case "mentionEveryone":
return i18n._({
id: "email.digest.hl.mentionEveryone",
message: "{who} mentioned everyone in {target}",
values,
});
case "comment":
return i18n._({
id: "email.digest.hl.comment",
message: "{who} commented on your dump {target}",
values,
});
case "playlistFollow":
return i18n._({
id: "email.digest.hl.playlistFollow",
message: "{who} followed your playlist {target}",
values,
});
case "userFollow":
return i18n._({
id: "email.digest.hl.userFollow",
message: "{who} started following you",
values,
});
case "commentLike":
return i18n._({
id: "email.digest.hl.commentLike",
message: "{who} liked your comment on {target}",
values,
});
}
}
const byLine = (username: string) =>
i18n._({
id: "email.digest.by",
message: "by @{username}",
values: { username },
});
const addedToLine = (playlist: string) =>
i18n._({
id: "email.digest.addedTo",
message: "added to {playlist}",
values: { playlist },
});
const uploadLabel = () =>
i18n._({ id: "email.digest.upload", message: "upload" });
const invitedLine = (inviter: string, reader: string) =>
inviter === reader
? i18n._(DIGEST_MESSAGES.invitedByYou)
: i18n._({ ...DIGEST_MESSAGES.invitedBy, values: { username: inviter } });
const andMoreLine = (n: number) =>
i18n._({
id: "email.digest.andMore",
message: "and {n} more",
values: { n },
});
const playlistMeta = (owner: string, n: number) =>
i18n._({
id: "email.digest.playlistMeta",
message: "by @{username} · {n, plural, one {# dump} other {# dumps}}",
values: { username: owner, n },
});
/**
* Returns the digest `content` describes, written as Markdown and drawn in
* `look`, with its subject and hidden preheader.
*/
export function renderDigestEmail(
content: DigestContent,
look: EmailLook,
): RenderedEmail {
const monthly = content.cadence === "monthly";
const nf = new Intl.NumberFormat(EMAIL_LOCALE);
const range = (year: boolean) =>
new Intl.DateTimeFormat(
EMAIL_LOCALE,
year
? { month: "short", day: "numeric", year: "numeric" }
: { month: "short", day: "numeric" },
).formatRange(content.from, content.to);
const day = (d: Date) =>
new Intl.DateTimeFormat(EMAIL_LOCALE, { weekday: "short" }).format(d);
const figure = (n: number, label: string) =>
`**${nf.format(n)}** ${escMd(label)}`;
const yours = i18n._({ id: "email.digest.yours", message: "yours" });
/** A dump as a list item: its thumbnail, title, pill and detail. */
const dumpItem = (d: DigestDump, bullet: string, image = true) =>
`${bullet} ${image && d.thumbnail ? `![](${d.thumbnail}) ` : ""}${
mdLink(d.title, d.url)
}${d.mine ? ` *${escMd(yours)}*` : ""} · ${
escMd(
`${d.addedTo ? addedToLine(d.addedTo) : byLine(d.poster)}${
d.addedTo ? "" : ` · ${d.domain ?? uploadLabel()}`
} · ${nf.format(d.votes)} ${countLabel("upvotes", d.votes)} · ${
nf.format(d.comments)
} ${countLabel("comments", d.comments)}`,
)
}`;
const md: string[] = [
`*${
escMd(
`${
monthly
? i18n._(DIGEST_MESSAGES.monthly)
: i18n._(DIGEST_MESSAGES.weekly)
} · ${range(true)}`,
)
}*`,
`# ${
escMd(
monthly
? i18n._({
id: "email.digest.monthly.title",
message: "The month on {site}",
values: { site: OG_SITE_NAME },
})
: i18n._({
id: "email.digest.weekly.title",
message: "Your week, {username}",
values: { username: content.reader },
}),
)
}`,
];
const { activity, network, community } = content;
if (activity) {
md.push(
`## ${escMd(i18n._({ id: "email.digest.forYou", message: "For you" }))}`,
ACTIVITY_ORDER.filter((k) => (activity.counts[k] ?? 0) > 0)
.map((k) =>
figure(activity.counts[k]!, countLabel(k, activity.counts[k]!))
)
.join(" · "),
activity.highlights.map((h) => {
const target = h.kind === "mentionChat" || h.kind === "mentionEveryone"
? mdLink(
i18n._({ id: "email.digest.theChat", message: "the chat" }),
content.links.chat,
)
: h.target?.url
? mdLink(h.target.title, h.target.url)
: h.target
? escMd(h.target.title)
: "";
const line = `- ${
highlightSentence(h.kind, `**@${escMd(h.who)}**`, target)
} *${escMd(day(h.at))}*`;
return h.quote ? `${line}\n > ${escMd(h.quote)}` : line;
}).join("\n"),
`${
mdLink(
i18n._({
id: "email.digest.seeAll",
message:
"{n, plural, one {See the notification} other {See all # notifications}}",
values: { n: activity.total },
}),
content.links.notifications,
)
} →`,
);
}
if (network) {
md.push(
`## ${
escMd(
i18n._({
id: "email.digest.network",
message: "From people you follow",
}),
)
}`,
network.dumps.map((d) => dumpItem(d, "-")).join("\n"),
);
if (network.more > 0) {
md.push(`${
mdLink(
i18n._({
id: "email.digest.more",
message: "{n} more on your Followed feed",
values: { n: network.more },
}),
content.links.followed,
)
} →`);
}
}
if (content.top.length > 0) {
const [first] = content.top;
md.push(
`## ${
escMd(
monthly
? i18n._(DIGEST_MESSAGES.topMonth)
: i18n._(DIGEST_MESSAGES.topWeek),
)
}`,
);
// The recap gives its first dump a wide image and the next two a thumbnail.
if (monthly && first.thumbnail) md.push(`![](${first.thumbnail})`);
md.push(
content.top.map((d, i) =>
dumpItem(d, `${i + 1}.`, monthly ? i === 1 || i === 2 : true)
).join("\n"),
);
}
if (community) {
const { dumps, upvotes, comments } = community.totals;
md.push(
`## ${escMd(i18n._(DIGEST_MESSAGES.community))}`,
);
if (monthly) {
md.push(
[
figure(
dumps,
i18n._({ ...DIGEST_MESSAGES.statDumps, values: { n: dumps } }),
),
figure(upvotes, countLabel("upvotes", upvotes)),
figure(comments, countLabel("comments", comments)),
figure(
community.chatMessages,
i18n._({
...DIGEST_MESSAGES.statChat,
values: { n: community.chatMessages },
}),
),
].join(" · "),
);
}
if (community.members.length > 0) {
md.push(
`### ${
escMd(
i18n._(DIGEST_MESSAGES.newMembers),
)
}`,
[
...community.members.map((m) =>
`- **@${escMd(m.username)}**${
m.invitedBy
? ` ${escMd(invitedLine(m.invitedBy, content.reader))}`
: ""
}`
),
...(community.membersMore > 0
? [`- ${escMd(andMoreLine(community.membersMore))}`]
: []),
].join("\n"),
);
}
if (community.playlists.length > 0) {
md.push(
`### ${escMd(i18n._(DIGEST_MESSAGES.newPlaylists))}`,
[
...community.playlists.map((p) =>
`- ${mdLink(p.title, p.url)} · ${
escMd(playlistMeta(p.owner, p.dumps))
}`
),
...(community.playlistsMore > 0
? [`- ${escMd(andMoreLine(community.playlistsMore))}`]
: []),
].join("\n"),
);
}
if (!monthly && community.chatMessages > 0) {
md.push(
`### ${
escMd(i18n._({ id: "email.digest.inChat", message: "In the chat" }))
}`,
`${
escMd(
i18n._({
id: "email.digest.chatWeek",
message:
"{n, plural, one {# message this week} other {# messages this week}}",
values: { n: community.chatMessages },
}),
)
} · ${
mdLink(
i18n._({ id: "email.digest.openChat", message: "Open the chat" }),
content.links.chat,
)
}`,
);
}
if (!monthly && dumps + upvotes + comments > 0) {
md.push(
`*${
escMd(i18n._({
id: "email.digest.totals",
message:
"{dumps, plural, one {# dump} other {# dumps}}, {upvotes, plural, one {# upvote} other {# upvotes}} and {comments, plural, one {# comment} other {# comments}}.",
values: { dumps, upvotes, comments },
}))
}*`,
);
}
md.push(`${
mdLink(
i18n._({
id: "email.digest.communityPage",
message: "See it all on the community page",
}),
content.links.community,
)
} →`);
}
md.push(
"---",
escMd(
monthly
? i18n._({
id: "email.digest.monthly.reason",
message:
"You're getting the monthly recap because it's on in your settings.",
})
: i18n._({
id: "email.digest.weekly.reason",
message:
"You're getting the weekly digest because it's on in your settings.",
}),
),
`${
mdLink(
monthly
? i18n._(DIGEST_MESSAGES.monthlyOff)
: i18n._(DIGEST_MESSAGES.weeklyOff),
content.links.unsubscribe,
)
} · ${
mdLink(
i18n._({ id: "email.settings", message: "Settings" }),
content.links.settings,
)
}`,
);
return new EmailKit(look).render({
subject: monthly
? i18n._({
id: "email.digest.monthly.subject",
message: "The month on {site}, {range}",
values: { site: OG_SITE_NAME, range: range(false) },
})
: i18n._({
id: "email.digest.weekly.subject",
message: "Your week on {site}, {range}",
values: { site: OG_SITE_NAME, range: range(false) },
}),
preheader: preheader(content),
markdown: md.join("\n\n"),
});
}
/**
* Returns the line an inbox shows beside the subject: the counts that make
* the digest worth opening, at most three of them.
*/
function preheader(content: DigestContent): string {
const parts: string[] = [];
if (content.cadence === "weekly") {
for (const k of ["upvotes", "followers", "comments"] as const) {
const n = content.activity?.counts[k] ?? 0;
if (n > 0) parts.push(`${n} ${countLabel(k, n)}`);
}
const n = (content.network?.dumps.length ?? 0) +
(content.network?.more ?? 0);
if (n > 0) {
parts.push(
i18n._({
id: "email.digest.pre.network",
message:
"{n, plural, one {# new dump from people you follow} other {# new dumps from people you follow}}",
values: { n },
}),
);
}
} else {
const members = (content.community?.members.length ?? 0) +
(content.community?.membersMore ?? 0);
const playlists = (content.community?.playlists.length ?? 0) +
(content.community?.playlistsMore ?? 0);
if (content.top.length > 0) {
parts.push(
i18n._({
id: "email.digest.pre.topMonth",
message:
"{n, plural, one {the most upvoted dump of the month} other {the # most upvoted dumps of the month}}",
values: { n: content.top.length },
}),
);
}
if (members > 0) {
parts.push(
i18n._({
id: "email.digest.pre.members",
message: "{n, plural, one {# new member} other {# new members}}",
values: { n: members },
}),
);
}
if (playlists > 0) {
parts.push(
i18n._({
id: "email.digest.pre.playlists",
message: "{n, plural, one {# new playlist} other {# new playlists}}",
values: { n: playlists },
}),
);
}
}
if (parts.length === 0 && content.top.length > 0) {
parts.push(
i18n._({
id: "email.digest.pre.topWeek",
message:
"{n, plural, one {the most upvoted dump of the week} other {the # most upvoted dumps of the week}}",
values: { n: content.top.length },
}),
);
}
if (parts.length === 0) return "";
const line = new Intl.ListFormat(EMAIL_LOCALE, { type: "conjunction" })
.format(parts.slice(0, 3));
return `${line.charAt(0).toUpperCase()}${line.slice(1)}.`;
}

36
api/mail/i18n.ts Normal file
View File

@@ -0,0 +1,36 @@
/**
* Lets the emails speak the instance's language from the same catalogs as
* the interface. A template passes each message as a descriptor holding its
* id and its English, as `i18n._({ id: "email.reset.title", message: "Reset
* your password" })`, or one of `shared/digest-messages.ts`, which a page
* says too. `lingui extract` reads them into `locales/<lang>.po` beside the
* interface's messages, and `lingui compile` turns the catalogs into the
* `locales/<lang>.mjs` imported here, failing when a language lacks one. The
* ids are named rather than hashed from the English, since the server runs
* no Lingui macro.
*
* @module
*/
import { type I18n, type Messages, setupI18n } from "@lingui/core";
import { EMAIL_LOCALE } from "../config.ts";
import { LOCALES } from "../../shared/locale.ts";
const catalogs: [string, Messages][] = await Promise.all(
LOCALES.map(async (locale) =>
[
locale,
(await import(`../../locales/${locale}.mjs`)).messages as Messages,
] as [string, Messages]
),
);
/**
* Translates the emails' messages, in {@link EMAIL_LOCALE}. Every catalog is
* loaded, so a test can activate another language.
*/
export const i18n: I18n = setupI18n({
locale: EMAIL_LOCALE,
messages: Object.fromEntries(catalogs),
});

822
api/mail/layout.ts Normal file
View File

@@ -0,0 +1,822 @@
/**
* Every email the site sends shares one layout: the site's emoji and name,
* then a card holding the content, then a footer. The templates beside this
* module (welcome, password reset, digest) build their content with an
* {@link EmailKit}, which draws each block in a style from `styles.ts`, and
* `email-service.ts` sends what they return.
*
* Mail clients drop stylesheets, so every block carries inline styles. A
* reader on the "auto" scheme gets the light palette inline and the dark one
* in a `prefers-color-scheme` block that Apple's mail apps honour; elsewhere
* the light one shows.
*
* @module
*/
import { Marked, Parser, type Token, type Tokens } from "marked";
import {
DEFAULT_STYLE,
EMAIL_LOCALE,
OG_SITE_NAME,
SITE_EMOJI,
} from "../config.ts";
import {
type ColorScheme,
DEFAULT_COLOR_SCHEME,
type StyleName,
} from "../../shared/theme.ts";
import type { UserSettings } from "../../shared/settings.ts";
import { EMAIL_STYLES, type EmailPalette, type EmailStyle } from "./styles.ts";
/**
* Returned by each template to its caller, which hands it to `sendEmail`
* with a recipient.
*/
export interface RenderedEmail {
subject: string;
html: string;
/**
* The plain-text part, for clients that show no HTML. It is Markdown, so it
* reads as text and renders as the email's outline.
*/
text: string;
}
/**
* Passed to {@link EmailKit}, made by {@link lookOf}: the instance's style for
* the emails that aren't a user's own, their settings for a digest.
*/
export interface EmailLook {
style: StyleName;
scheme: ColorScheme;
}
/**
* Returns the look of a reader's settings. An unset style is the instance's,
* as on the site, and an unset scheme the system's; with no settings at all,
* as for the welcome and reset emails, both are.
*/
export function lookOf(
settings: Pick<UserSettings, "style" | "colorScheme"> | null = null,
): EmailLook {
return {
style: settings?.style ?? DEFAULT_STYLE,
scheme: settings?.colorScheme ?? DEFAULT_COLOR_SCHEME,
};
}
/**
* Names a color of an {@link EmailPalette}, as {@link EmailKit.paint} takes
* it, so that the dark block can recolor the element by the same name.
*/
export type PaletteRole = Exclude<keyof EmailPalette, "gradient">;
const ROLES: PaletteRole[] = [
"outer",
"card",
"text",
"sec",
"muted",
"accent",
"link",
"onAccent",
"rule",
"soft",
"quote",
"date",
"brand",
"border",
];
/**
* Escapes `value` for HTML text and attributes. Everything users wrote (names,
* titles, quotes) goes through it, since none of it may become markup.
*/
export function esc(value: string): string {
return value.replace(
/[&<>"']/g,
(c) =>
({ "&": "&amp;", "<": "&lt;", ">": "&gt;", '"': "&quot;", "'": "&#39;" })[
c
]!,
);
}
/**
* Escapes `value` for Markdown, so that user text in a plain-text part (a
* title with `*` or `[`, say) stays text.
*/
export function escMd(value: string): string {
return value.replace(/[\\`*_[\]<>#!|~]/g, "\\$&");
}
/** Returns a Markdown link to `url` labelled with `text`, escaped. */
export function mdLink(text: string, url: string): string {
const safeUrl = url.replace(
/[()\s]/g,
(c) => `%${c.charCodeAt(0).toString(16).toUpperCase().padStart(2, "0")}`,
);
return `[${escMd(text)}](${safeUrl})`;
}
const gradient = (colors: string[]) =>
`linear-gradient(90deg,${colors.join(",")})`;
/**
* Draws an email in one {@link EmailLook}. A template writes the email as
* Markdown and hands it to {@link EmailKit.render}, which draws each block in
* the style. The block methods return HTML: text passed to them must already
* be escaped with {@link esc}.
*/
export class EmailKit {
/** The style being drawn. */
readonly style: EmailStyle;
/** The palette inlined into the HTML. */
readonly palette: EmailPalette;
/** The palette that replaces it in dark mode, on the "auto" scheme only. */
readonly darkPalette: EmailPalette | null;
/** The font stack, for templates that set their own blocks. */
readonly font: string;
constructor(look: EmailLook) {
this.style = EMAIL_STYLES[look.style];
this.palette = look.scheme === "dark" ? this.style.dark : this.style.light;
this.darkPalette = look.scheme === DEFAULT_COLOR_SCHEME
? this.style.dark
: null;
this.font = this.style.font;
}
/** Returns `px`, scaled for the style, as a CSS length. */
fs(px: number): string {
return `${Math.round(px * this.style.scale * 2) / 2}px`;
}
/**
* Returns the `class` and `style` attributes of an element colored by
* roles: `fg` for its text, `bg` for its fill, `bd` for its border. The
* classes let the dark block recolor it; `css` is the rest of its style.
*/
paint(
roles: {
fg?: PaletteRole;
bg?: PaletteRole;
bd?: PaletteRole;
extra?: string[];
},
css = "",
): string {
const p = this.palette;
const classes: string[] = [...(roles.extra ?? [])];
let style = "";
if (roles.fg) {
classes.push(`f-${roles.fg}`);
style += `color:${p[roles.fg] ?? p.text};`;
}
if (roles.bg) {
classes.push(`b-${roles.bg}`);
style += `background-color:${p[roles.bg] ?? p.card};`;
}
if (roles.bd) {
classes.push(`d-${roles.bd}`);
style += `border-color:${p[roles.bd] ?? p.rule};`;
}
const cls = classes.length ? ` class="${classes.join(" ")}"` : "";
// The colors come last: a shorthand such as `border-top` in `css` would
// otherwise reset them.
return `${cls} style="${css}${style}"`;
}
/** Returns a link to `href` around `html`, in the link color. */
link(html: string, href: string, strong = false): string {
const s = this.style;
return `<a href="${esc(href)}"${
this.paint(
{ fg: "link" },
`text-decoration:${s.linkUnderline ? "underline" : "none"};${
strong ? `font-weight:${s.heavy};` : ""
}`,
)
}>${html}</a>`;
}
/** Returns the email's heading, in the style's title type. */
heading(html: string, marginBottom = 24): string {
const t = this.style.title;
return `<h1${
this.paint(
{ fg: t.accent ? "accent" : "text" },
`font-family:${this.font};font-size:${
this.fs(t.size)
};line-height:1.15;font-weight:${t.weight};${
t.upper ? "text-transform:uppercase;" : ""
}${
t.spacing ? `letter-spacing:${t.spacing};` : ""
}margin:0 0 ${marginBottom}px;`,
)
}>${html}</h1>`;
}
/** Returns a paragraph. `css` adds to or overrides its style. */
para(html: string, css = ""): string {
return `<p${
this.paint(
{ fg: "text" },
`font-family:${this.font};font-size:${
this.fs(15)
};line-height:1.6;margin:0 0 16px;${css}`,
)
}>${html}</p>`;
}
/** Returns a smaller paragraph in the muted color. */
note(html: string, css = ""): string {
return `<p${
this.paint(
{ fg: "muted" },
`font-family:${this.font};font-size:${
this.fs(13.5)
};line-height:1.6;margin:0 0 16px;${css}`,
)
}>${html}</p>`;
}
/** Returns a button linking to `href`. */
button(label: string, href: string): string {
const s = this.style;
const frame = s.frame === "hard"
? `border:2px solid ${this.palette.text};box-shadow:2px 2px 0 ${this.palette.text};`
: "";
return `<table role="presentation" cellpadding="0" cellspacing="0" style="margin:6px 0 22px;"><tr><td${
this.paint(
{
bg: "accent",
bd: s.frame === "hard" ? "text" : undefined,
extra: s.frame === "hard" ? ["sh-btn"] : [],
},
`border-radius:${s.radius.btn}px;${frame}`,
)
}><a href="${esc(href)}"${
this.paint(
{ fg: "onAccent" },
`display:inline-block;padding:13px 22px;font-family:${this.font};font-size:${
this.fs(13.5)
};font-weight:${
Math.max(600, s.heavy)
};letter-spacing:0.08em;text-transform:uppercase;line-height:1;text-decoration:none;`,
)
}>${label}</a></td></tr></table>`;
}
/** Returns a small uppercase pill, as on one's own dump in a list. */
pill(label: string): string {
return `<span${
this.paint(
{ fg: "sec", bg: "soft" },
`display:inline-block;margin-left:6px;padding:1px 7px;border-radius:${this.style.radius.pill}px;font-size:${
this.fs(11)
};font-weight:600;letter-spacing:0.08em;text-transform:uppercase;vertical-align:2px;`,
)
}>${label}</span>`;
}
/** Returns an image block, rounded as the style rounds images. */
image(
src: string,
alt: string,
width: number,
height: number,
href?: string,
): string {
const img = `<img src="${
esc(src)
}" width="${width}" height="${height}" alt="${
esc(alt)
}" style="display:block;max-width:100%;height:auto;border:0;border-radius:${
Math.min(this.style.radius.img, width / 8)
}px;">`;
return href ? `<a href="${esc(href)}">${img}</a>` : img;
}
/** Returns a small bold heading inside a section. */
subhead(html: string): string {
return `<div${
this.paint(
{ fg: "text" },
`font-family:${this.font};font-size:${this.fs(14)};font-weight:${
Math.max(600, this.style.heavy)
};margin:0 0 8px;`,
)
}>${html}</div>`;
}
/**
* Returns a section of the card: a divider in the style's rule (none before
* the first), an uppercase title in the accent color, and `html`.
*/
section(title: string, html: string, first = false): string {
const s = this.style;
let divider = "";
let border = "";
if (!first) {
const stops = this.palette.gradient;
if (s.rule === "gradient" && stops) {
divider =
`<div class="grad" style="height:3px;line-height:3px;font-size:0;background-color:${
stops[0]
};background-image:${gradient(stops)};margin:0 0 26px;">&nbsp;</div>`;
} else {
border = `border-top:${
s.rule === "thick" ? 2 : 1
}px solid;padding-top:26px;`;
}
}
return `<div${
this.paint(
{ bd: border ? (s.rule === "thick" ? "text" : "rule") : undefined },
`${border}padding-bottom:26px;`,
)
}>${divider}<div${
this.paint(
{ fg: "accent" },
`font-family:${this.font};font-size:${this.fs(12)};font-weight:${
Math.max(600, s.heavy)
};letter-spacing:0.12em;text-transform:uppercase;margin:0 0 16px;`,
)
}>${title}</div>${html}</div>`;
}
/**
* Returns the email written as `markdown`: the Markdown itself is the
* plain-text part, and its blocks, drawn in this kit's style, are the HTML.
* Each template writes its email once this way. The blocks map as follows.
*
* - `# Title` is the heading; an `*emphasised*` paragraph above it, the
* eyebrow. Elsewhere an emphasised paragraph is a note.
* - `## Section` starts a section, with its divider; `### Name` a subhead.
* - A link alone in a paragraph is a button, an image alone a wide image,
* and a fenced block a box for a raw link.
* - A list item whose text starts with an image, then a link, is a row
* with that thumbnail and the link as its title; what follows is its
* detail, and an emphasised word right after the link a pill. Other items
* are sentences, where emphasis marks the date. A numbered list ranks its
* rows, and a quote under an item is set as one.
* - A bold number is a figure.
* - What follows the last `---` is the footer.
*/
render(
{ subject, preheader, markdown }: {
subject: string;
preheader: string;
markdown: string;
},
): RenderedEmail {
const cut = markdown.lastIndexOf("\n---\n");
const body = cut === -1 ? markdown : markdown.slice(0, cut);
const foot = cut === -1 ? "" : markdown.slice(cut + 5);
return {
subject,
html: this.document({
subject,
preheader,
body: this.blocks(this.marked.lexer(body)),
footer: this.marked.lexer(foot)
.filter((t): t is Tokens.Paragraph => t.type === "paragraph")
.map((t) => this.inline(t.tokens, "footer")).join("<br>"),
}),
text: markdown,
};
}
/** How {@link inline} draws emphasis and links, by where they sit. */
private mode: "text" | "footer" | "row" | "sentence" = "text";
/** Draws Markdown's inline tokens with the kit's colors. */
// The kit comes in as an argument: the renderer's methods need marked's
// `this` (its parser) as well as the kit's colors.
private readonly marked: Marked = ((kit: EmailKit) => {
return new Marked({
renderer: {
link({ href, tokens }) {
const html = this.parser.parseInline(tokens);
return kit.mode === "footer"
? `<a href="${esc(href)}"${
kit.paint({ fg: "muted" }, "text-decoration:underline;")
}>${html}</a>`
: kit.link(html, href);
},
strong({ tokens }) {
const html = this.parser.parseInline(tokens);
// A figure, its digits grouped by any space: `\s` takes the no-break
// ones locales group thousands with.
const figure = /^[\d\s.,]+$/.test(html);
return `<b${
kit.paint(
{ fg: "text" },
figure
? `font-size:${kit.fs(22)};font-weight:${kit.style.heavy};`
: `font-weight:${Math.max(600, kit.style.heavy)};`,
)
}>${html}</b>`;
},
em({ tokens }) {
const html = this.parser.parseInline(tokens);
if (kit.mode === "row") return kit.pill(html);
if (kit.mode === "sentence") {
return `<span${
kit.paint(
{ fg: kit.palette.date ? "date" : "muted" },
`font-size:${kit.fs(12.5)};white-space:nowrap;${
kit.palette.date ? "font-style:italic;" : ""
}`,
)
}>${html}</span>`;
}
return `<i>${html}</i>`;
},
codespan({ text }) {
return `<code style="font-family:Menlo,Consolas,'Courier New',monospace;">${text}</code>`;
},
image({ href, text }) {
return kit.image(href, text, 104, 78);
},
},
});
})(this);
private inline(tokens: Token[], mode: EmailKit["mode"]): string {
this.mode = mode;
try {
return Parser.parseInline(tokens, this.marked.defaults);
} finally {
this.mode = "text";
}
}
/** Draws the body's blocks, gathering each `##` section with its content. */
private blocks(tokens: Token[]): string {
const out: string[] = [];
let section: { title: string; parts: string[] } | null = null;
let first = true;
let headed = false;
const push = (html: string) =>
section ? section.parts.push(html) : out.push(html);
const close = () => {
if (!section) return;
out.push(this.section(section.title, section.parts.join(""), first));
first = false;
section = null;
};
for (const token of tokens) {
switch (token.type) {
case "heading": {
const html = this.inline(token.tokens ?? [], "text");
if (token.depth === 1) {
headed = true;
push(this.heading(html));
} else if (token.depth === 2) {
close();
section = { title: html, parts: [] };
} else {
push(this.subhead(html));
}
break;
}
case "paragraph": {
const kids = (token.tokens ?? []).filter((t) =>
!(t.type === "text" && !t.raw.trim())
);
const only = kids.length === 1 ? kids[0] : null;
if (only?.type === "link") {
push(
this.button(this.inline(only.tokens ?? [], "text"), only.href),
);
} else if (only?.type === "image") {
push(
`${
this.image(only.href, only.text, 528, 264)
}<div style="height:14px;line-height:14px;font-size:0;">&nbsp;</div>`,
);
} else if (only?.type === "em") {
const html = this.inline(only.tokens ?? [], "text");
push(headed ? this.note(html) : this.eyebrow(html));
} else {
push(this.para(this.inline(token.tokens ?? [], "text")));
}
break;
}
case "list":
push(this.list(token as Tokens.List));
break;
case "blockquote":
push(this.quote(this.inline(
(token.tokens ?? []).flatMap((t) =>
"tokens" in t ? t.tokens ?? [] : []
),
"text",
)));
break;
case "code":
push(this.linkBox(esc(token.text)));
break;
case "html":
push(token.text);
break;
}
}
close();
return out.join("");
}
/** Draws a list as rows, ranked when it is numbered. */
private list(list: Tokens.List): string {
const start = typeof list.start === "number" ? list.start : 1;
return list.items.map((item, i) => {
const text = item.tokens.find((t) =>
t.type === "text" || t.type === "paragraph"
) as Tokens.Text | Tokens.Paragraph | undefined;
const quote = item.tokens.find((t) => t.type === "blockquote") as
| Tokens.Blockquote
| undefined;
let rest: Token[] = [...(text?.tokens ?? [])];
const skipSpace = () => {
while (rest[0]?.type === "text" && !rest[0].raw.trim()) {
rest = rest.slice(1);
}
};
skipSpace();
const image = rest[0]?.type === "image"
? rest.shift() as Tokens.Image
: null;
skipSpace();
const link = rest[0]?.type === "link"
? rest.shift() as Tokens.Link
: null;
let content: string;
if (link) {
const pills: Token[] = [];
skipSpace();
while (rest[0]?.type === "em") {
pills.push(rest.shift()!);
skipSpace();
}
// The detail follows the title after a " · ".
if (rest[0]?.type === "text") {
const t = rest[0] as Tokens.Text;
const trimmed = t.text.replace(/^\s*·\s*/, "");
rest[0] = { ...t, text: trimmed, raw: trimmed, tokens: undefined };
}
content = `<a href="${esc(link.href)}"${
this.paint(
{ fg: "text" },
`font-size:${
this.fs(15)
};line-height:1.35;font-weight:${this.style.heavy};text-decoration:none;`,
)
}>${this.inline(link.tokens, "text")}</a>${this.inline(pills, "row")}${
rest.length
? `<div${
this.paint(
{ fg: "muted" },
`font-size:${this.fs(13)};line-height:1.5;margin-top:3px;`,
)
}>${this.inline(rest, "text")}</div>`
: ""
}`;
} else {
content = `<div${
this.paint(
{ fg: "sec" },
`font-size:${this.fs(14.5)};line-height:1.5;`,
)
}>${this.inline(rest, "sentence")}</div>`;
}
if (quote) {
content += this.quote(this.inline(
quote.tokens.flatMap((t) => "tokens" in t ? t.tokens ?? [] : []),
"text",
));
}
const size = list.ordered ? [64, 48] : [104, 78];
return `<table role="presentation" width="100%" cellpadding="0" cellspacing="0" style="margin:0 0 ${
link || image ? 16 : 10
}px;"><tr>${
list.ordered
? `<td width="30" valign="top"${
this.paint(
{ fg: "accent" },
`font-family:${this.font};font-size:${this.fs(20)};font-weight:${
Math.max(600, this.style.heavy)
};line-height:1.2;`,
)
}>${start + i}</td>`
: ""
}${
image
? `<td width="${size[0]}" valign="top" style="padding-right:${
list.ordered ? 12 : 14
}px;">${
this.image(image.href, image.text, size[0], size[1], link?.href)
}</td>`
: ""
}<td valign="top" style="font-family:${this.font};">${content}</td></tr></table>`;
}).join("");
}
/** Returns the small uppercase line above the heading. */
private eyebrow(html: string): string {
return `<div${
this.paint(
{ fg: "muted" },
`font-family:${this.font};font-size:${
this.fs(12)
};font-weight:600;letter-spacing:0.12em;text-transform:uppercase;margin:0 0 8px;`,
)
}>${html}</div>`;
}
/** Returns a quoted comment or message. */
private quote(html: string): string {
return `<div${
this.paint(
{ fg: "quote" },
`font-family:${this.font};font-size:${
this.fs(14.5)
};line-height:1.5;font-style:italic;margin:4px 0 0;`,
)
}>“${html}”</div>`;
}
/** Returns a raw link set in a box, for clients that break buttons. */
private linkBox(html: string): string {
return `<div${
this.paint(
{ fg: "sec", bg: "soft" },
`font-family:Menlo,Consolas,'Courier New',monospace;font-size:12px;line-height:1.5;border-radius:${
Math.min(this.style.radius.img, 6)
}px;padding:10px 12px;margin:0 0 22px;word-break:break-all;`,
)
}>${html}</div>`;
}
/**
* Returns the whole HTML document: the brand, the card holding `body`, and
* the `footer`. `preheader` is the hidden first line inboxes show beside the
* subject.
*/
private document(
{ subject, preheader, body, footer }: {
subject: string;
preheader: string;
body: string;
footer: string;
},
): string {
const s = this.style;
const p = this.palette;
const R = s.radius.card;
const bar = s.bar ? p.gradient ?? null : null;
const frame = s.frame === "hard"
? `border:2px solid ${p.text};box-shadow:5px 5px 0 ${p.text};`
: s.frame === "ridge"
? `border:3px ridge ${p.border};`
: s.frame === "hairline"
? `border:1px solid ${p.rule};`
: s.frame === "soft"
? `box-shadow:0 14px 34px -10px ${p.shadow};`
: "";
const frameRoles = s.frame === "hard"
? { bd: "text" as const, extra: ["sh-card"] }
: s.frame === "ridge"
? { bd: "border" as const }
: s.frame === "hairline"
? { bd: "rule" as const }
: s.frame === "soft"
? { extra: ["sh-soft"] }
: {};
const scheme = this.darkPalette
? "light dark"
: p === s.dark
? "dark"
: "light";
return `<!doctype html>
<html lang="${EMAIL_LOCALE}">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width,initial-scale=1">
<meta name="color-scheme" content="${scheme}">
<meta name="supported-color-schemes" content="${scheme}">
<title>${esc(subject)}</title>
${
s.webFont
? `<link rel="stylesheet" href="https://fonts.googleapis.com/css2?family=${s.webFont}&display=swap">`
: ""
}
<style>
body{margin:0;padding:0;}
@media (max-width:620px){.wrap{padding:20px 10px 28px!important}.card{padding:24px 20px 8px!important}}
${this.darkCss()}
</style>
</head>
<body${this.paint({ bg: "outer" }, "margin:0;padding:0;")}>
<div style="display:none;max-height:0;overflow:hidden;opacity:0;">${
esc(preheader)
}</div>
<table role="presentation" width="100%" cellpadding="0" cellspacing="0"${
this.paint({ bg: "outer" })
}>
<tr><td align="center" class="wrap" style="padding:32px 16px 40px;">
<table role="presentation" width="100%" cellpadding="0" cellspacing="0" style="max-width:600px;">
<tr><td style="padding:0 6px 16px;">
<table role="presentation" cellpadding="0" cellspacing="0"><tr>
<td style="padding-right:10px;vertical-align:middle;font-size:24px;line-height:1;">${
esc(SITE_EMOJI)
}</td>
<td${
this.paint(
{ fg: p.brand ? "brand" : "text" },
`vertical-align:middle;font-family:${this.font};font-weight:700;font-size:${
this.fs(21)
};line-height:1;letter-spacing:-0.01em;`,
)
}>${esc(OG_SITE_NAME)}</td>
</tr></table>
</td></tr>
${
bar
? `<tr><td class="grad" style="height:5px;line-height:5px;font-size:0;background-color:${
bar[0]
};background-image:${
gradient(bar)
};border-radius:${R}px ${R}px 0 0;">&nbsp;</td></tr>`
: ""
}
<tr><td${
this.paint(
{
bg: "card",
...frameRoles,
extra: ["card", ...(frameRoles.extra ?? [])],
},
`border-radius:${
bar ? `0 0 ${R}px ${R}px` : `${R}px`
};${frame}padding:34px 36px 12px;`,
)
}>
${body}
</td></tr>
<tr><td${
this.paint(
{ fg: "muted" },
`padding:22px 6px 0;font-family:${this.font};font-size:${
this.fs(12.5)
};line-height:1.6;`,
)
}>${footer}</td></tr>
</table>
</td></tr>
</table>
</body>
</html>`;
}
/**
* Returns the CSS that recolors every painted element in dark mode, or
* nothing off the "auto" scheme.
*/
private darkCss(): string {
const d = this.darkPalette;
if (!d) return "";
const rules: string[] = [];
for (const role of ROLES) {
const color = d[role];
if (!color) continue;
rules.push(
`.f-${role}{color:${color}!important}`,
`.b-${role}{background-color:${color}!important}`,
`.d-${role}{border-color:${color}!important}`,
);
}
rules.push(
`.sh-card{box-shadow:5px 5px 0 ${d.text}!important}`,
`.sh-btn{box-shadow:2px 2px 0 ${d.text}!important}`,
);
if (d.shadow) {
rules.push(
`.sh-soft{box-shadow:0 14px 34px -10px ${d.shadow}!important}`,
);
}
if (d.gradient) {
rules.push(
`.grad{background-color:${d.gradient[0]}!important;background-image:${
gradient(d.gradient)
}!important}`,
);
}
return `@media (prefers-color-scheme:dark){${rules.join("")}}`;
}
}

92
api/mail/password-reset.ts Normal file
View File

@@ -0,0 +1,92 @@
/**
* Renders the email carrying a password reset link. `requestPasswordReset`
* (`password-reset-service.ts`) sends it, in the instance's style and
* language: whoever asks isn't signed in, so their own settings don't apply.
*
* @module
*/
import { OG_SITE_NAME } from "../config.ts";
import { i18n } from "./i18n.ts";
import {
EmailKit,
escMd,
lookOf,
mdLink,
type RenderedEmail,
} from "./layout.ts";
/**
* Returns the reset email of `username`, whose button opens `resetUrl`. The
* link stays usable for `hours`, which the email states.
*/
export function renderPasswordResetEmail(
username: string,
resetUrl: string,
hours: number,
): RenderedEmail {
const markdown = [
`# ${
escMd(i18n._({ id: "email.reset.title", message: "Reset your password" }))
}`,
escMd(
i18n._({
id: "email.reset.hi",
message: "Hi {username},",
values: { username },
}),
),
escMd(i18n._({
id: "email.reset.body",
message:
"Someone asked to reset the password of your {site} account. The button below lets you choose a new one. It works once, for {hours, plural, one {# hour} other {# hours}}.",
values: { site: OG_SITE_NAME, hours },
})),
mdLink(
i18n._({ id: "email.reset.button", message: "Reset password" }),
resetUrl,
),
`*${
escMd(
i18n._({
id: "email.reset.fallback",
message:
"If the button doesn't work, paste this link into your browser:",
}),
)
}*`,
"```\n" + resetUrl + "\n```",
`*${
escMd(
i18n._({
id: "email.reset.ignore",
message:
"If you didn't ask for this, ignore this email. Your password stays the same.",
}),
)
}*`,
"---",
escMd(
i18n._({
id: "email.reset.reason",
message:
"You're getting this email because a password reset was requested for this address.",
}),
),
].join("\n\n");
return new EmailKit(lookOf()).render({
subject: i18n._({
id: "email.reset.subject",
message: "Reset your {site} password",
values: { site: OG_SITE_NAME },
}),
preheader: i18n._({
id: "email.reset.preheader",
message:
"The link works once, for {hours, plural, one {# hour} other {# hours}}.",
values: { hours },
}),
markdown,
});
}

253
api/mail/styles.ts Normal file
View File

@@ -0,0 +1,253 @@
/**
* Carries each site style into email, so a digest looks like the site its
* reader chose and the other emails like the instance's style. A mail client
* takes no stylesheet of ours, so each style is resolved here into plain
* values, which `layout.ts` inlines.
*
* What a theme defines is read from it through `theme-tokens.ts`, never
* copied: the colors, the gradient, the font and the corner radii. What is
* the email's own is stated here: how large it sets its type, how its card
* stands out, which divider it draws, and which of the theme's own tokens
* carry the touches that make it recognisable (Kanagawa's syntax hues,
* GeoCities' rainbow, Melange's spice and warm shadow).
*
* @module
*/
import { STYLE_NAMES, type StyleName } from "../../shared/theme.ts";
import { readTokens, type Scheme } from "./theme-tokens.ts";
/**
* The colors of one style in one scheme, by role, each read from a token of
* its theme. The optional ones carry a style's particular touches.
*/
export interface EmailPalette {
/** The page around the card (`--color-bg`). */
outer: string;
/** The card holding the content (`--color-surface`). */
card: string;
/** `--color-text`. */
text: string;
/** `--color-text-secondary`. */
sec: string;
/** `--color-text-muted`. */
muted: string;
/** Section titles, ranks and button fills (`--color-accent`). */
accent: string;
/** `--color-link`. */
link: string;
/** Text on an accent fill (`--color-on-accent`). */
onAccent: string;
/** Dividers and hairlines (`--color-border`). */
rule: string;
/** Pills and the reset link's box, as the site's badges (`--color-badge-bg`). */
soft: string;
/** Quoted comments and messages (`--color-text-secondary` unless set). */
quote: string;
/** Kanagawa's fujiGray dates; others use `muted`. */
date?: string;
/** Kanagawa's oniViolet wordmark; others use `text`. */
brand?: string;
/** GeoCities' ridged border. */
border?: string;
/** Melange's warm card shadow, translucent. */
shadow?: string;
/** The colors of GeoCities' rainbow or Melange's spice, left to right. */
gradient?: string[];
}
/**
* How a style draws an email. `layout.ts` builds every block from it, so
* adding a style to `shared/theme.ts` means adding its {@link Recipe} here.
*/
export interface EmailStyle {
/**
* The theme's font stack (`--font-body`). Its first family is a web font
* only Apple's mail apps load, through {@link webFont}; other clients fall
* back along the stack.
*/
font: string;
/** The Google Fonts `family` parameter of the web font, if it is one. */
webFont?: string;
/** Multiplies every font size: Times sets its serif larger, Kanagawa its monospace smaller. */
scale: number;
/**
* Corner radii, in pixels: the card's as the site's cards
* (`--radius-xl`), the images' as its thumbnails (`--radius-md`), the
* buttons' as its buttons (`--radius-lg`), the pills' as its badges
* (`--radius-sm`). The avatars' is the email's own, since the site rounds
* them with a literal 50% that some styles square.
*/
radius: {
card: number;
img: number;
btn: number;
pill: number;
avatar: number;
};
/** The heading: its size in pixels, weight, and Brutalist's uppercase and spacing. */
title: {
size: number;
weight: number;
upper?: boolean;
spacing?: string;
accent?: boolean;
};
/** The weight of titles of items, figures and names. */
heavy: number;
/** How the card stands out from the page. */
frame: "none" | "hard" | "ridge" | "hairline" | "soft";
/** The divider between sections; "gradient" draws the palette's. */
rule: "hairline" | "thick" | "gradient";
/** Draws the palette's gradient as a bar along the card's top. */
bar?: boolean;
/** Underlines links, for styles whose links are the color of the text. */
linkUnderline?: boolean;
light: EmailPalette;
dark: EmailPalette;
}
/** The roles every style reads, and the token each is read from. */
const PALETTE_TOKENS = {
outer: "--color-bg",
card: "--color-surface",
text: "--color-text",
sec: "--color-text-secondary",
muted: "--color-text-muted",
accent: "--color-accent",
link: "--color-link",
onAccent: "--color-on-accent",
rule: "--color-border",
soft: "--color-badge-bg",
quote: "--color-text-secondary",
} satisfies Partial<Record<keyof EmailPalette, string>>;
/**
* What a style states about its email, beside what its theme defines: the
* {@link EmailStyle} less the values read from the theme, the axes of its
* web font, its avatars' radius, and the tokens of its own touches.
*/
interface Recipe
extends Omit<EmailStyle, "font" | "webFont" | "radius" | "light" | "dark"> {
/** The Google Fonts axes of the theme's first family; none for a system font. */
webFontAxes?: string;
avatarRadius: number;
/** The tokens of the palette's optional roles, or of a base role it reads elsewhere. */
tokens?: Partial<Record<keyof EmailPalette, string>>;
}
const RECIPES: Record<StyleName, Recipe> = {
smooth: {
webFontAxes: "wght@400;600;700",
scale: 1,
avatarRadius: 16,
title: { size: 30, weight: 600 },
heavy: 600,
frame: "none",
rule: "hairline",
},
brutalist: {
webFontAxes: "wght@400;700",
scale: 1,
avatarRadius: 0,
title: { size: 30, weight: 900, upper: true, spacing: "0.03em" },
heavy: 700,
frame: "hard",
rule: "thick",
},
geocities: {
scale: 1,
avatarRadius: 0,
title: { size: 30, weight: 700, accent: true },
heavy: 700,
frame: "ridge",
rule: "gradient",
bar: true,
linkUnderline: true,
tokens: { border: "--color-border", gradient: "--geo-rainbow" },
},
nyt: {
webFontAxes: "ital@0;1",
scale: 1.12,
avatarRadius: 16,
title: { size: 38, weight: 400 },
heavy: 400,
frame: "hairline",
rule: "hairline",
linkUnderline: true,
},
kanagawa: {
webFontAxes: "ital,wght@0,400;0,600;1,400",
scale: 0.9,
avatarRadius: 4,
title: { size: 26, weight: 600 },
heavy: 600,
frame: "hairline",
rule: "hairline",
tokens: {
quote: "--kngw-string",
date: "--kngw-comment",
brand: "--kngw-keyword",
},
},
melange: {
webFontAxes: "wght@400;600",
scale: 1,
avatarRadius: 16,
title: { size: 32, weight: 600 },
heavy: 600,
frame: "soft",
rule: "hairline",
bar: true,
tokens: { shadow: "--shadow-color", gradient: "--mel-spice" },
},
};
/**
* Returns the palette of `style` in `scheme` from its theme's tokens. Every
* color is flattened onto the card, as a translucent one shows on the site,
* but the shadow, which falls on the page.
*/
function readPalette(
style: StyleName,
scheme: Scheme,
own: Recipe["tokens"] = {},
): EmailPalette {
const tokens = readTokens(style, scheme);
const roles = { ...PALETTE_TOKENS, ...own };
const palette = Object.fromEntries(
Object.entries(roles)
.filter(([role]) => role !== "shadow" && role !== "gradient")
.map(([role, token]) => [role, tokens.color(token, "--color-surface")]),
) as unknown as EmailPalette;
if (own.shadow) palette.shadow = tokens.color(own.shadow);
if (own.gradient) palette.gradient = tokens.stops(own.gradient);
return palette;
}
/** Returns the email look of `style`, its theme read in both schemes. */
function readStyle(style: StyleName): EmailStyle {
const { webFontAxes, avatarRadius, tokens: own, ...recipe } = RECIPES[style];
const tokens = readTokens(style, "light");
const font = tokens.font("--font-body");
const family = font.split(",")[0].replace(/'/g, "").replaceAll(" ", "+");
return {
...recipe,
font,
webFont: webFontAxes ? `${family}:${webFontAxes}` : undefined,
radius: {
card: tokens.px("--radius-xl"),
img: tokens.px("--radius-md"),
btn: tokens.px("--radius-lg"),
pill: tokens.px("--radius-sm"),
avatar: avatarRadius,
},
light: readPalette(style, "light", own),
dark: readPalette(style, "dark", own),
};
}
/** Every style's email look, keyed by its name, read once at load. */
export const EMAIL_STYLES = Object.fromEntries(
STYLE_NAMES.map((style) => [style, readStyle(style)]),
) as Record<StyleName, EmailStyle>;

276
api/mail/theme-tokens.ts Normal file
View File

@@ -0,0 +1,276 @@
/**
* Lets the emails draw with the site's own values. It reads a style's design
* tokens out of the stylesheets the client loads, `src/tokens.css` and then
* `src/themes/<style>.css`, so `styles.ts` names a token wherever it needs a
* color, a gradient, a font or a radius, and a change to a theme reaches its
* emails with nothing copied to keep in step.
*
* Mail clients know no custom properties, so a token comes back resolved for
* one scheme: its `var()`s followed, its `light-dark()` picked, its
* `color-mix()` computed. Only the rules that set tokens on the root element
* are read (`:root`, the style's `[data-style]`, and either of those with the
* scheme's `[data-color-scheme]`); a theme states its forced schemes that way
* beside each `prefers-color-scheme` block, so the media blocks are skipped.
*
* @module
*/
import { BASE_STYLE, type StyleName } from "../../shared/theme.ts";
import { hexOf, parseCssColor, type Rgba } from "../lib/css-color.ts";
/** A scheme a token is resolved for. "auto" is drawn from both. */
export type Scheme = "light" | "dark";
/**
* Returned by {@link readTokens}: the tokens of one style in one scheme, read
* as the kind of value each is.
*/
export interface ThemeTokens {
/** Returns the resolved value of `token`, or throws when it isn't set. */
value(token: string): string;
/**
* Returns the color of `token` as `#rrggbb`. A translucent one is laid over
* `over` (another token) first, as it shows on the site; without `over` it
* stays `rgba()`, as a shadow's must.
*/
color(token: string, over?: string): string;
/** Returns the colors of the gradient `token`, in order, without positions. */
stops(token: string): string[];
/**
* Returns the font stack of `token`, single-quoted for a `style` attribute.
* A family the site bundles as "X Variable" is named "X", as a mail client
* or Google Fonts knows it.
*/
font(token: string): string;
/** Returns the length of `token` in pixels. */
px(token: string): number;
}
const SOURCES = ["../../src/tokens.css", "../../src/themes/{style}.css"];
/** Returns `css` without its comments. */
function stripComments(css: string): string {
return css.replace(/\/\*[\s\S]*?\*\//g, "");
}
/** Returns the top-level rules of `css` as their selector and body. */
function topRules(css: string): { selector: string; body: string }[] {
const rules = [];
let i = 0;
while (i < css.length) {
const open = css.indexOf("{", i);
if (open < 0) break;
let depth = 1;
let j = open + 1;
while (depth && j < css.length) {
if (css[j] === "{") depth++;
else if (css[j] === "}") depth--;
j++;
}
rules.push({
selector: css.slice(i, open).trim(),
body: css.slice(open + 1, j - 1),
});
i = j;
}
return rules;
}
/**
* Splits `text` at each `sep` outside parentheses and quotes, trimming the
* parts: the arguments of a CSS function, or the declarations of a rule.
*/
function splitTop(text: string, sep: string): string[] {
const parts: string[] = [];
let depth = 0;
let quote = "";
let start = 0;
for (let i = 0; i < text.length; i++) {
const c = text[i];
if (quote) {
if (c === quote) quote = "";
} else if (c === '"' || c === "'") quote = c;
else if (c === "(") depth++;
else if (c === ")") depth--;
else if (c === sep && depth === 0) {
parts.push(text.slice(start, i).trim());
start = i + 1;
}
}
parts.push(text.slice(start).trim());
return parts.filter((part) => part !== "");
}
/**
* Replaces each call of the CSS function `name` in `value`, innermost last
* first, with what `replace` returns for its arguments.
*/
function replaceCalls(
value: string,
name: string,
replace: (args: string[]) => string,
): string {
for (;;) {
const at = value.lastIndexOf(`${name}(`);
if (at < 0) return value;
let depth = 0;
let end = at + name.length;
do {
if (value[end] === "(") depth++;
else if (value[end] === ")") depth--;
end++;
} while (depth && end < value.length);
const args = splitTop(value.slice(at + name.length + 1, end - 1), ",");
value = value.slice(0, at) + replace(args) + value.slice(end);
}
}
/** Parses a CSS color in the forms the themes write, or throws. */
function parseColor(text: string): Rgba {
const color = parseCssColor(text);
if (!color) throw new Error(`Unreadable color: ${text}`);
return color;
}
/** Writes `color` as `#rrggbb`, or as `rgba()` when it is translucent. */
function formatColor(color: Rgba): string {
if (color.a >= 1) return hexOf(color);
const [r, g, b] = [color.r, color.g, color.b].map(Math.round);
return `rgba(${r},${g},${b},${+color.a.toFixed(3)})`;
}
/** Returns `top` laid over `under`, both straight (not premultiplied) alpha. */
function over(top: Rgba, under: Rgba): Rgba {
const a = top.a + under.a * (1 - top.a);
if (a === 0) return { r: 0, g: 0, b: 0, a: 0 };
const mix = (t: number, u: number) =>
(t * top.a + u * under.a * (1 - top.a)) / a;
return {
r: mix(top.r, under.r),
g: mix(top.g, under.g),
b: mix(top.b, under.b),
a,
};
}
/**
* Computes `color-mix(in srgb, …)` from its arguments, the two colors with
* their optional percentages, as CSS does: premultiplied, and scaled down
* when the percentages sum under 100%.
*/
function colorMix(args: string[]): string {
const [space, ...colors] = args;
if (space !== "in srgb" || colors.length !== 2) {
throw new Error(`Unsupported color-mix: ${args.join(", ")}`);
}
const parsed = colors.map((arg) => {
const pct = arg.match(/\s([\d.]+)%$/);
return {
color: parseColor(pct ? arg.slice(0, pct.index) : arg),
pct: pct ? parseFloat(pct[1]) / 100 : null,
};
});
const [p1, p2] = [
parsed[0].pct ?? 1 - (parsed[1].pct ?? 0.5),
parsed[1].pct ?? 1 - (parsed[0].pct ?? 0.5),
];
const sum = p1 + p2;
const [w1, w2] = [p1 / sum, p2 / sum];
const [c1, c2] = [parsed[0].color, parsed[1].color];
const a = c1.a * w1 + c2.a * w2;
const mix = (x: number, y: number) =>
a === 0 ? 0 : (x * c1.a * w1 + y * c2.a * w2) / a;
return formatColor({
r: mix(c1.r, c2.r),
g: mix(c1.g, c2.g),
b: mix(c1.b, c2.b),
a: a * Math.min(sum, 1),
});
}
/**
* Returns the tokens of `style` in `scheme`, read from the site's stylesheets
* at the time of the call. `styles.ts` reads each style once, at load.
*/
export function readTokens(style: StyleName, scheme: Scheme): ThemeTokens {
const styleSel = style === BASE_STYLE
? `(?:\\[data-style="${style}"\\]|:not\\(\\[data-style\\]\\))`
: `\\[data-style="${style}"\\]`;
const rootRule = new RegExp(
`^(?::root)?(?:${styleSel})?(?:\\[data-color-scheme="${scheme}"\\])?$`,
);
// Later declarations win, as in the cascade: the base tokens, then the
// theme's, then its forced scheme's.
const declared = new Map<string, string>();
for (const source of SOURCES) {
const css = stripComments(Deno.readTextFileSync(
new URL(source.replace("{style}", style), import.meta.url),
));
for (const { selector, body } of topRules(css)) {
if (selector.startsWith("@")) continue;
if (!selector.split(",").some((part) => rootRule.test(part.trim()))) {
continue;
}
for (const declaration of splitTop(body, ";")) {
const colon = declaration.indexOf(":");
const name = declaration.slice(0, colon).trim();
if (!name.startsWith("--")) continue;
declared.set(
name,
declaration.slice(colon + 1).replace(/\s+/g, " ").trim(),
);
}
}
}
const resolved = new Map<string, string>();
const value = (token: string): string => {
const known = resolved.get(token);
if (known !== undefined) return known;
const raw = declared.get(token);
if (raw === undefined) throw new Error(`${style} sets no ${token}`);
let result = replaceCalls(
raw,
"var",
([name, fallback]) =>
declared.has(name) ? value(name) : fallback ?? value(name),
);
result = replaceCalls(
result,
"light-dark",
([light, dark]) => scheme === "light" ? light : dark,
);
result = replaceCalls(result, "color-mix", colorMix);
resolved.set(token, result);
return result;
};
return {
value,
color(token, under) {
const color = parseColor(value(token));
return formatColor(under ? over(color, parseColor(value(under))) : color);
},
stops(token) {
const gradient = value(token).match(/^linear-gradient\((.*)\)$/)?.[1];
if (!gradient) throw new Error(`${style} ${token} is no linear gradient`);
return splitTop(gradient, ",")
.filter((stop) => !/^(?:to\s|[\d.-]+(?:deg|turn|rad)$)/.test(stop))
.map((stop) => formatColor(parseColor(splitTop(stop, " ")[0])));
},
font(token) {
return splitTop(value(token), ",")
.map((family) => family.replace(/["']/g, "").replace(/ Variable$/, ""))
.map((family) => /^[\w-]+$/.test(family) ? family : `'${family}'`)
.join(", ");
},
px(token) {
const length = value(token).match(/^([\d.]+)(?:px)?$/)?.[1];
if (length === undefined) {
throw new Error(`${style} ${token} is no pixel length`);
}
return parseFloat(length);
},
};
}

80
api/mail/welcome.ts Normal file
View File

@@ -0,0 +1,80 @@
/**
* Renders the email that greets a new account. `sendWelcomeEmail`
* (`user-service.ts`) sends it after sign-up, in the instance's style and
* language. It opens with the instance's `GERBEUR_WELCOME_EMAIL_BODY`, or the
* built-in body, then says the digests are on, since they are by default, and
* where to turn them off.
*
* @module
*/
import { FRONTEND_URL, OG_SITE_NAME, WELCOME_EMAIL_BODY } from "../config.ts";
import { i18n } from "./i18n.ts";
import {
EmailKit,
escMd,
lookOf,
mdLink,
type RenderedEmail,
} from "./layout.ts";
/**
* Returns the welcome email of `username`. The instance's body has
* `{{username}}` and `{{site_name}}` filled in; raw HTML in it passes
* through, since it comes from whoever runs the instance, never from a user.
*/
export function renderWelcomeEmail(username: string): RenderedEmail {
// A username may hold a `_`, which Markdown would read as emphasis.
const body = WELCOME_EMAIL_BODY
? WELCOME_EMAIL_BODY
.replaceAll("{{username}}", escMd(username))
.replaceAll("{{site_name}}", escMd(OG_SITE_NAME))
: i18n._({
id: "email.welcome.body",
message:
"# Welcome to {site}!\n\nHi **{username}**,\n\nYour account has been created successfully. Welcome aboard!",
values: { site: escMd(OG_SITE_NAME), username: escMd(username) },
});
const markdown = [
body,
mdLink(
i18n._({
id: "email.welcome.button",
message: "Open {site}",
values: { site: OG_SITE_NAME },
}),
FRONTEND_URL,
),
`*${
escMd(
i18n._({
id: "email.welcome.digests",
message:
"Once a week you'll get a digest of what happened here, and once a month a recap of the most upvoted dumps. You can turn either off in your settings.",
}),
)
}*`,
"---",
escMd(
i18n._({
id: "email.welcome.reason",
message:
"You're getting this email because an account was created on {site} with this address.",
values: { site: OG_SITE_NAME },
}),
),
].join("\n\n");
return new EmailKit(lookOf()).render({
subject: i18n._({
id: "email.welcome.subject",
message: "Welcome to {site}",
values: { site: OG_SITE_NAME },
}),
preheader: i18n._({
id: "email.welcome.preheader",
message: "Your account is ready.",
}),
markdown,
});
}

194
api/main.ts
View File

@@ -1,39 +1,81 @@
/**
* Builds the Oak application and, when run directly, starts the recurring
* jobs and listens on {@link LISTEN_HOST}:{@link PORT}, having first signed
* the users out of other instances if they may no longer sign in there
* (`FEDERATION.outgoing`). A request goes
* through the error middleware, the compression of what is answered, the OG
* tags and CORS, then the routers, one
* per resource of the API (`API.md` lists them), and last the built frontend,
* which answers every path no router did.
*
* @module
*/
import { Application } from "@oak/oak";
import { oakCors } from "@tajpouria/cors";
import attachmentsRouter from "./routes/attachments.ts";
import avatarsRouter from "./routes/avatars.ts";
import categoriesRouter from "./routes/categories.ts";
import chatRouter from "./routes/chat.ts";
import communityRouter from "./routes/community.ts";
import commentsRouter from "./routes/comments.ts";
import dumpsRouter from "./routes/dumps.ts";
import filesRouter from "./routes/files.ts";
import thumbnailsRouter from "./routes/thumbnails.ts";
import usersRouter from "./routes/users.ts";
import avatarsRouter from "./routes/avatars.ts";
import wsRouter from "./routes/ws.ts";
import previewRouter from "./routes/preview.ts";
import bandcampRouter from "./routes/bandcamp.ts";
import youtubeRouter from "./routes/youtube.ts";
import playlistsRouter from "./routes/playlists.ts";
import commentsRouter from "./routes/comments.ts";
import chatRouter from "./routes/chat.ts";
import federationRouter from "./routes/federation.ts";
import followsRouter from "./routes/follows.ts";
import notificationsRouter from "./routes/notifications.ts";
import invitesRouter from "./routes/invites.ts";
import meRouter from "./routes/me.ts";
import notificationsRouter from "./routes/notifications.ts";
import oauthRouter from "./routes/oauth.ts";
import passwordResetsRouter from "./routes/password-resets.ts";
import playlistsRouter from "./routes/playlists.ts";
import remoteRouter from "./routes/remote.ts";
import searchRouter from "./routes/search.ts";
import categoriesRouter from "./routes/categories.ts";
import sessionsRouter from "./routes/sessions.ts";
import thumbnailsRouter from "./routes/thumbnails.ts";
import waveformsRouter from "./routes/waveforms.ts";
import unsubscribeRouter from "./routes/unsubscribe.ts";
import usersRouter from "./routes/users.ts";
import wellKnownRouter from "./routes/well-known.ts";
import wsRouter from "./routes/ws.ts";
import { ALLOWED_ORIGINS, LISTEN_HOST, PORT, PUBLIC_URL } from "./config.ts";
import {
ALLOWED_ORIGINS,
FEDERATION,
LISTEN_HOST,
PORT,
PUBLIC_URL,
} from "./config.ts";
import { compressMiddleware } from "./middleware/compress.ts";
import { errorMiddleware } from "./middleware/error.ts";
import { ogMiddleware } from "./middleware/og.ts";
import { routeStaticFilesFrom } from "./lib/static.ts";
import { startJobs } from "./lib/scheduler.ts";
import { rateLimitSweepJob } from "./lib/rate-limit.ts";
import { generatedMediaSweepJob } from "./lib/ffmpeg.ts";
import { digestJob } from "./services/digest-service.ts";
import { communityJob } from "./services/community-service.ts";
import { pingJob, typingExpiryJob } from "./services/ws-service.ts";
import {
outboxJob,
withdrawAllGrants,
} from "./services/identity-provider-service.ts";
const app = new Application();
/**
* The whole server, exported so the tests' helpers can send it requests
* without listening.
*/
const app: Application = new Application();
const cors = oakCors({ origin: ALLOWED_ORIGINS });
app.use(errorMiddleware);
app.use(compressMiddleware);
app.use(ogMiddleware);
// Only invoke oakCors when the origin is present and allowed.
// The library calls next() without await when origin is falsy, which
// corrupts the response in Oak's async middleware chain.
// oakCors calls next() without awaiting it when there is no origin, which
// corrupts the response in Oak's async chain. So it only sees requests from an
// allowed origin, and the others get no CORS headers.
app.use(async (ctx, next) => {
const origin = ctx.request.headers.get("origin");
if (origin && ALLOWED_ORIGINS.includes(origin)) {
@@ -41,78 +83,37 @@ app.use(async (ctx, next) => {
}
await next();
});
app.use(
dumpsRouter.routes(),
dumpsRouter.allowedMethods(),
);
app.use(
filesRouter.routes(),
filesRouter.allowedMethods(),
);
app.use(
thumbnailsRouter.routes(),
thumbnailsRouter.allowedMethods(),
);
app.use(
attachmentsRouter.routes(),
attachmentsRouter.allowedMethods(),
);
app.use(
usersRouter.routes(),
usersRouter.allowedMethods(),
);
app.use(
avatarsRouter.routes(),
avatarsRouter.allowedMethods(),
);
app.use(
wsRouter.routes(),
wsRouter.allowedMethods(),
);
app.use(
previewRouter.routes(),
previewRouter.allowedMethods(),
);
app.use(
bandcampRouter.routes(),
bandcampRouter.allowedMethods(),
);
app.use(
youtubeRouter.routes(),
youtubeRouter.allowedMethods(),
);
app.use(
playlistsRouter.routes(),
playlistsRouter.allowedMethods(),
);
app.use(
commentsRouter.routes(),
commentsRouter.allowedMethods(),
);
app.use(
chatRouter.routes(),
chatRouter.allowedMethods(),
);
app.use(
followsRouter.routes(),
followsRouter.allowedMethods(),
);
app.use(
notificationsRouter.routes(),
notificationsRouter.allowedMethods(),
);
app.use(
invitesRouter.routes(),
invitesRouter.allowedMethods(),
);
app.use(
searchRouter.routes(),
searchRouter.allowedMethods(),
);
app.use(
categoriesRouter.routes(),
categoriesRouter.allowedMethods(),
);
for (
const router of [
meRouter,
followsRouter,
notificationsRouter,
avatarsRouter,
usersRouter,
sessionsRouter,
passwordResetsRouter,
unsubscribeRouter,
dumpsRouter,
filesRouter,
thumbnailsRouter,
waveformsRouter,
commentsRouter,
playlistsRouter,
categoriesRouter,
searchRouter,
chatRouter,
communityRouter,
invitesRouter,
attachmentsRouter,
remoteRouter,
federationRouter,
oauthRouter,
wellKnownRouter,
wsRouter,
]
) {
app.use(router.routes(), router.allowedMethods());
}
app.use(routeStaticFilesFrom([
`${Deno.cwd()}/dist`,
`${Deno.cwd()}/public`,
@@ -129,6 +130,21 @@ app.addEventListener(
);
if (import.meta.main) {
if (!FEDERATION.outgoing) {
const withdrawn = await withdrawAllGrants();
if (withdrawn > 0) {
console.log(`Signed users out of other instances (${withdrawn} grants)`);
}
}
startJobs([
digestJob,
communityJob,
pingJob,
typingExpiryJob,
rateLimitSweepJob,
generatedMediaSweepJob,
outboxJob,
]);
await app.listen({ hostname: LISTEN_HOST, port: PORT });
}

79
api/middleware/auth.ts
View File

@@ -1,3 +1,11 @@
/**
* Gates routes on a session token, and on a permission after it. The routers
* put these middleware before a handler; a route open to anonymous callers
* calls `parseOptionalAuth` (`lib/auth.ts`) instead.
*
* @module
*/
import {
type Next,
type RouteParams,
@@ -5,25 +13,34 @@ import {
type State,
} from "@oak/oak";
import { authenticate } from "../lib/auth.ts";
import {
APIErrorCode,
APIException,
type AuthPayload,
} from "../model/interfaces.ts";
import { can, type Permission } from "../lib/permissions.ts";
import { APIException } from "../lib/api-exception.ts";
import type { AuthPayload } from "../model/auth.ts";
import { can } from "../lib/permissions.ts";
import type { Permission } from "../../shared/permissions.ts";
/**
* Set on `ctx.state` by {@link authMiddleware} for the handlers after it.
* `user` carries the caller's live role, not the one in their token.
*/
export interface AuthState extends State {
user: AuthPayload;
}
// A router context whose state is guaranteed to carry the authenticated user
// (populated by authMiddleware). Generic over the route path so `ctx.params`
// stays typed per-route — e.g. `AuthContext<"/:userId/role">`.
/**
* The context type of a handler behind {@link authMiddleware}. Pass the route
* path, as in `AuthContext<"/:key/role">`, to keep `ctx.params` typed.
*/
export type AuthContext<
R extends string = string,
P extends RouteParams<R> = RouteParams<R>,
> = RouterContext<R, P, AuthState>;
/**
* Requires a valid session token in the `Authorization` header, and puts its
* payload in `ctx.state.user` for the rest of the route.
*
* @throws {APIException} 401 when the token is missing or invalid.
*/
export async function authMiddleware<R extends string>(
ctx: AuthContext<R>,
next: Next,
@@ -31,20 +48,14 @@ export async function authMiddleware<R extends string>(
const authHeader = ctx.request.headers.get("Authorization");
if (!authHeader || !authHeader.startsWith("Bearer ")) {
throw new APIException(
APIErrorCode.UNAUTHORIZED,
401,
"Missing or invalid token",
);
throw new APIException(401, "Missing or invalid token");
}
const token = authHeader.substring(7);
// Also checks the user still exists and hasn't changed their password since
// the token was issued, and takes the live role from the DB.
const payload = await authenticate(token);
if (!payload) {
throw new APIException(APIErrorCode.UNAUTHORIZED, 401, "Invalid token");
throw new APIException(401, "Invalid token");
}
ctx.state.user = payload;
@@ -52,34 +63,22 @@ export async function authMiddleware<R extends string>(
await next();
}
export async function adminOnlyMiddleware<R extends string>(
ctx: AuthContext<R>,
next: Next,
): Promise<void> {
if (ctx.state.user.role !== "admin") {
throw new APIException(
APIErrorCode.UNAUTHORIZED,
403,
"Admin access required",
);
}
await next();
}
// Gates a route on a named permission. Must run after authMiddleware. Generic
// over the route path so it composes with path-typed route handlers.
export function requirePermission(permission: Permission) {
/**
* Returns a middleware that requires the caller to hold `permission`. It must
* run after {@link authMiddleware}, as in
* `router.patch(path, authMiddleware, requirePermission("user:manage"), …)`.
* The returned middleware throws an {@link APIException} (403) when the caller
* lacks the permission.
*/
export function requirePermission(
permission: Permission,
): <R extends string>(ctx: AuthContext<R>, next: Next) => Promise<void> {
return async <R extends string>(
ctx: AuthContext<R>,
next: Next,
): Promise<void> => {
if (!ctx.state.user || !can(ctx.state.user, permission)) {
throw new APIException(
APIErrorCode.UNAUTHORIZED,
403,
"Insufficient permissions",
);
throw new APIException(403, "Insufficient permissions");
}
await next();
};

108
api/middleware/compress.ts Normal file
View File

@@ -0,0 +1,108 @@
/**
* Compresses what the routes answer in JSON, and the HTML page, for the
* browsers that accept it: a page of the feed is some twenty kilobytes as it
* is, and a fifth of that compressed. main.ts installs it after the error
* middleware, around everything that sets a body. The built frontend's files
* aren't its concern: lib/static.ts sends the copies compressed at build time.
*
* @module
*/
import type { Context, Next } from "@oak/oak";
import { promisify } from "node:util";
import { brotliCompress, constants as zlib, gzip } from "node:zlib";
const brotli = promisify(brotliCompress);
const gz = promisify(gzip);
/**
* Below this, the bytes saved aren't worth the work; it is the threshold the
* build compresses the frontend's files from (vite.config.ts).
*/
const MIN_BYTES = 1024;
/**
* Brotli's quality for a response made on the spot. Past 5 it takes several
* times longer for a few percent; the frontend's files, compressed once, are
* at 11.
*/
const BROTLI_QUALITY = 5;
/**
* Returns the coding to answer with for an `Accept-Encoding` header, brotli
* before gzip, or null when the client takes neither. A coding refused with
* `q=0` isn't taken.
*/
function pickCoding(header: string | null): "br" | "gzip" | null {
if (!header) return null;
const accepted = new Set<string>();
for (const part of header.split(",")) {
const [name, ...params] = part.trim().toLowerCase().split(";");
const q = params.map((p) => p.trim().match(/^q=([\d.]+)$/)?.[1])
.find((v) => v !== undefined);
if (q === undefined || Number(q) > 0) accepted.add(name.trim());
}
return accepted.has("br") ? "br" : accepted.has("gzip") ? "gzip" : null;
}
/**
* Returns the text of a body this middleware compresses, or null for any
* other: an object or array, which Oak would send as JSON, or a string whose
* type its route has set. Files, streams and bytes are left alone, as they
* are images, media and files compressed already.
*/
function textOf(ctx: Context): string | null {
const body = ctx.response.body;
if (typeof body === "string") {
// Oak guesses an untyped string's type from the string itself, which it
// couldn't from its compressed bytes.
return ctx.response.type || ctx.response.headers.has("Content-Type")
? body
: null;
}
if (body === null || typeof body !== "object") return null;
const proto = Object.getPrototypeOf(body);
if (proto !== Object.prototype && !Array.isArray(body)) return null;
ctx.response.type = "json";
return JSON.stringify(body);
}
/**
* Sends the response's JSON or text compressed, in brotli or gzip as the
* request accepts, when it is at least {@link MIN_BYTES} long. A response
* already encoded, a `HEAD`, and one marked `no-transform` go out as they are.
*/
export async function compressMiddleware(
ctx: Context,
next: Next,
): Promise<void> {
await next();
const headers = ctx.response.headers;
if (
ctx.request.method === "HEAD" || headers.has("Content-Encoding") ||
headers.get("Cache-Control")?.includes("no-transform")
) return;
const text = textOf(ctx);
if (text === null) return;
const raw = new TextEncoder().encode(text);
if (raw.length < MIN_BYTES) return;
// A cache must not hand the compressed copy to a client that can't read it.
headers.append("Vary", "Accept-Encoding");
const coding = pickCoding(ctx.request.headers.get("Accept-Encoding"));
if (!coding) return;
const compressed = coding === "br"
? await brotli(raw, {
params: {
[zlib.BROTLI_PARAM_MODE]: zlib.BROTLI_MODE_TEXT,
[zlib.BROTLI_PARAM_QUALITY]: BROTLI_QUALITY,
[zlib.BROTLI_PARAM_SIZE_HINT]: raw.length,
},
})
: await gz(raw);
headers.set("Content-Encoding", coding);
ctx.response.body = new Uint8Array(compressed);
}

21
api/middleware/error.ts
View File

@@ -1,8 +1,25 @@
/**
* Turns the errors a request's handling throws into `APIFailure` responses.
* main.ts installs it first, so it catches what every later middleware and
* route throws.
*
* @module
*/
import { Context, Next } from "@oak/oak";
import { APIErrorCode, APIException, APIFailure } from "../model/interfaces.ts";
import { APIErrorCode, APIFailure } from "../../shared/api.ts";
import { APIException } from "../lib/api-exception.ts";
export async function errorMiddleware(ctx: Context, next: Next) {
/**
* Answers an {@link APIException} with its status, code and message. Anything
* else is logged with its stack and answered with a 500 `SERVER_ERROR` whose
* message gives nothing away.
*/
export async function errorMiddleware(
ctx: Context,
next: Next,
): Promise<void> {
try {
await next();
} catch (err) {

146
api/middleware/og.ts
View File

@@ -1,17 +1,34 @@
import { Context, Next } from "@oak/oak";
import { getDump } from "../services/dump-service.ts";
import { getUserByUsername } from "../services/user-service.ts";
import { getPlaylistById } from "../services/playlist-service.ts";
import { BANDCAMP_PLAYER, OG_SITE_NAME, SITE_EMOJI } from "../config.ts";
import { emojiToCodepoint } from "../lib/site-icons.ts";
/**
* Gives links to the site rich previews in chat apps and social sites, which
* don't run the app. It answers a link to a dump, a profile or a public
* playlist with `index.html` carrying that page's Open Graph and Twitter card
* tags, and a link by a former name with a 301 to the current one. main.ts
* installs it before the routers.
*
* @module
*/
const ICON_VERSION = emojiToCodepoint(SITE_EMOJI);
import { Context, Next } from "@oak/oak";
import { getDumpByPath } from "../services/dump-service.ts";
import { resolveUsername } from "../services/user-service.ts";
import { getPlaylistByPath } from "../services/playlist-service.ts";
import {
dumpPath,
parsePagePath,
playlistPath,
userPath,
} from "../../shared/paths.ts";
import { FRONTEND_URL, OG_SITE_NAME, PUBLIC_URL } from "../config.ts";
import { loadIndexHtml } from "../lib/index-html.ts";
interface OGMeta {
title: string;
description?: string;
imageUrl?: string;
/** True only for real artwork — a square site icon must not claim a wide card. */
/**
* Set for a dump's artwork only. It asks for the large card, which would
* stretch a square icon or avatar.
*/
imageIsWide?: boolean;
url: string;
}
@@ -30,7 +47,7 @@ function buildTags(meta: OGMeta): string {
: "summary";
const tags = [
`<title>${escapeAttr(meta.title)}</title>`,
`<meta property="og:site_name" content="${OG_SITE_NAME}" />`,
`<meta property="og:site_name" content="${escapeAttr(OG_SITE_NAME)}" />`,
`<meta property="og:type" content="website" />`,
`<meta property="og:url" content="${escapeAttr(meta.url)}" />`,
`<meta property="og:title" content="${escapeAttr(meta.title)}" />`,
@@ -58,102 +75,111 @@ function inject(html: string, meta: OGMeta): string {
.replace("</head>", ` ${buildTags(meta)}\n </head>`);
}
let cachedHtml: string | null | undefined; // undefined = not yet loaded, null = not found
async function loadIndexHtml(): Promise<string | null> {
if (cachedHtml !== undefined) return cachedHtml;
for (
const path of [`${Deno.cwd()}/dist/index.html`, `${Deno.cwd()}/index.html`]
) {
/**
* Returns the filled `index.html` of dist/, or else the source one, which is
* all there is in development. Returns null when neither can be read.
*/
async function findIndexHtml(): Promise<string | null> {
for (const root of [`${Deno.cwd()}/dist`, Deno.cwd()]) {
try {
cachedHtml = (await Deno.readTextFile(path))
.replaceAll("__SITE_NAME__", OG_SITE_NAME)
.replaceAll("__SITE_EMOJI__", SITE_EMOJI)
.replaceAll("__ICON_VERSION__", ICON_VERSION)
.replaceAll("__BANDCAMP_PLAYER__", BANDCAMP_PLAYER);
return cachedHtml;
return await loadIndexHtml(root);
} catch {
continue;
}
}
cachedHtml = null;
return null;
}
const DUMP_RE = /^\/dumps\/([^/]+)$/;
// Allow an optional `/~/<tab>` suffix so linkable profile tabs keep OG meta.
const USER_RE = /^\/users\/([^/]+)(?:\/~\/[^/]+)?$/;
const PLAYLIST_RE = /^\/playlists\/([^/]+)$/;
/** Answers `canonical` with a 301, keeping the request's query string. */
function redirect(ctx: Context, canonical: string): void {
ctx.response.status = 301;
// Revalidated each time: a slug taken back would otherwise loop through a
// redirect the browser kept.
ctx.response.headers.set("Cache-Control", "no-cache");
ctx.response.redirect(`${FRONTEND_URL}${canonical}${ctx.request.url.search}`);
}
export async function ogMiddleware(ctx: Context, next: Next) {
/**
* Serves `index.html` with the tags of the dump, user or public playlist a
* path names (`parsePagePath`), and passes the request on when it names none
* that exists or the visitor may see. A path naming one by a former name, a
* user's or a slug, or in another case, gets a 301 to its current path. The
* page's URL is on `FRONTEND_URL` and the images' on `PUBLIC_URL`, not on the
* request's origin, which a reverse proxy may have rewritten to an internal
* host or plain http.
*/
export async function ogMiddleware(
ctx: Context,
next: Next,
): Promise<unknown> {
const { pathname } = ctx.request.url;
if (pathname.startsWith("/api/") || pathname.includes(".")) {
return await next();
}
// A file's path never starts with `@`, so parsePagePath passes it on, while
// a remote user's has dots in it: `/@bob@music.example`.
if (pathname.startsWith("/api/")) return await next();
const origin = ctx.request.url.origin;
const pageUrl = `${origin}${pathname}`;
const page = parsePagePath(pathname);
if (!page || (page.kind !== "user" && page.edit)) return await next();
const origin = PUBLIC_URL;
let meta: OGMeta | null = null;
let canonical: string | null = null;
const dumpMatch = pathname.match(DUMP_RE);
const userMatch = pathname.match(USER_RE);
const playlistMatch = pathname.match(PLAYLIST_RE);
if (dumpMatch) {
try {
const dump = getDump(dumpMatch[1]);
const user = resolveUsername(page.username);
if (page.kind === "dump") {
const dump = getDumpByPath(user.id, page.slug);
canonical = dumpPath(dump);
let imageUrl: string | undefined;
if (dump.kind === "file" && dump.fileMime?.startsWith("image/")) {
imageUrl = `${origin}/api/files/${dump.id}`;
imageUrl = `${origin}/api/dumps/${dump.id}/file`;
} else if (dump.richContent?.thumbnailUrl) {
imageUrl = dump.richContent.thumbnailUrl;
}
// No artwork: fall back to our own site icon rather than the target's
// favicon, which is typically too small to survive a social card (and may
// be hotlink-protected).
// Without artwork, the site's icon: the linked site's favicon is too
// small for a card, and may refuse hotlinking.
meta = {
title: dump.title,
description: dump.comment,
imageUrl: imageUrl ?? `${origin}/apple-touch-icon.png`,
imageIsWide: !!imageUrl,
url: pageUrl,
url: `${FRONTEND_URL}${canonical}`,
};
} catch { /* not found or private — serve default */ }
} else if (userMatch) {
try {
const user = getUserByUsername(userMatch[1]);
} else if (page.kind === "user") {
canonical = [userPath(user.username), ...page.rest].join("/");
const imageUrl = user.avatarMime
? `${origin}/api/avatars/${user.id}`
? `${origin}/api/users/${user.id}/avatar`
: undefined;
meta = {
title: user.username,
description: user.description,
imageUrl: imageUrl ?? `${origin}/apple-touch-icon.png`,
url: pageUrl,
url: `${FRONTEND_URL}${canonical}`,
};
} catch { /* not found */ }
} else if (playlistMatch) {
try {
const playlist = getPlaylistById(playlistMatch[1]);
} else {
const playlist = getPlaylistByPath(user.id, page.slug);
// getPlaylistByPath doesn't check visibility, unlike getDumpByPath.
if (playlist.isPublic) {
canonical = playlistPath(playlist);
const imageUrl = playlist.imageMime
? `${origin}/api/playlists/${playlist.id}/image`
? `${origin}/api/playlists/${playlist.id}/cover`
: undefined;
meta = {
title: playlist.title,
description: playlist.description,
imageUrl: imageUrl ?? `${origin}/apple-touch-icon.png`,
url: pageUrl,
url: `${FRONTEND_URL}${canonical}`,
};
}
} catch { /* not found or private */ }
}
} catch { /* Unknown, or private. */ }
if (!meta || !canonical) return await next();
if (canonical !== pathname.replace(/\/+$/, "")) {
return redirect(ctx, canonical);
}
if (!meta) return await next();
const html = await loadIndexHtml();
const html = await findIndexHtml();
if (!html) return await next();
ctx.response.headers.set("Content-Type", "text/html; charset=utf-8");

64
api/model/auth.ts Normal file
View File

@@ -0,0 +1,64 @@
/**
* Checks what a verified token says before it is trusted. It types the payloads
* of session and password reset tokens, and `lib/jwt.ts` runs their guards when
* it verifies one.
*
* @module
*/
import { isRole, type Role } from "../../shared/user.ts";
/**
* Signed into a session token by `createJWT`, and returned by `authenticate`
* to the auth middleware as `ctx.state.user`. By then `role` is the user's live
* role, while `username` is still the one at issue time.
*/
export interface AuthPayload {
userId: string;
username: string;
role: Role;
/** Seconds since the epoch. */
exp: number;
/** Seconds since the epoch, compared with `users.tokens_valid_after`. */
iat?: number;
}
/**
* Checks whether `obj` is an {@link AuthPayload}. A media or reset token's
* payload, which has a `purpose`, never is.
*/
export function isAuthPayload(obj: unknown): obj is AuthPayload {
return !!obj &&
typeof obj === "object" &&
// A media or reset token must not pass for a session token.
!("purpose" in obj) &&
"userId" in obj && typeof obj.userId === "string" &&
"username" in obj && typeof obj.username === "string" &&
"role" in obj && isRole(obj.role) &&
"exp" in obj && typeof obj.exp === "number";
}
/**
* Signed into a reset link's token by `createPasswordResetToken`, and returned
* by `verifyPasswordResetToken` to `resetPassword`.
*/
export interface PasswordResetPayload {
purpose: "password-reset";
userId: string;
/** Seconds since the epoch. */
exp: number;
}
/**
* Checks whether `obj` is a {@link PasswordResetPayload}. It doesn't check
* `exp`, which `jwtVerify` has already enforced.
*/
export function isPasswordResetPayload(
obj: unknown,
): obj is PasswordResetPayload {
return !!obj && typeof obj === "object" &&
"purpose" in obj &&
(obj as Record<string, unknown>).purpose === "password-reset" &&
"userId" in obj &&
typeof (obj as Record<string, unknown>).userId === "string";
}

52
api/model/category.ts Normal file
View File

@@ -0,0 +1,52 @@
/**
* Turns the rows the category service reads from `categories` into the {@link
* Category} the category routes send.
*
* @module
*/
import type { SQLOutputValue } from "node:sqlite";
import {
type Category,
type CategoryKind,
isCategoryKind,
} from "../../shared/category.ts";
/** Returned by the category service's `SELECT`s on `categories`. */
export interface CategoryRow {
id: string;
kind: CategoryKind;
slug: string;
name: string;
/** Sorts the categories, lowest first. */
position: number;
created_at: string;
updated_at: string | null;
[key: string]: SQLOutputValue;
}
/** Checks whether a row `node:sqlite` returned is a {@link CategoryRow}. */
export function isCategoryRow(obj: unknown): obj is CategoryRow {
return !!obj && typeof obj === "object" &&
"id" in obj && typeof obj.id === "string" &&
"kind" in obj && isCategoryKind(obj.kind) &&
"slug" in obj && typeof obj.slug === "string" &&
"name" in obj && typeof obj.name === "string" &&
"position" in obj && typeof obj.position === "number" &&
"created_at" in obj && typeof obj.created_at === "string" &&
"updated_at" in obj &&
(typeof obj.updated_at === "string" || obj.updated_at === null);
}
/** Converts `row` to the {@link Category} the category routes send. */
export function categoryRowToApi(row: CategoryRow): Category {
return {
id: row.id,
kind: row.kind,
slug: row.slug,
name: row.name,
position: row.position,
createdAt: new Date(row.created_at),
updatedAt: row.updated_at ? new Date(row.updated_at) : undefined,
};
}

77
api/model/chat.ts Normal file
View File

@@ -0,0 +1,77 @@
/**
* Turns the rows of `chat_messages` into the {@link ChatMessage} the API sends.
* Only `services/chat-service.ts` uses it, on the rows of its one query.
*
* @module
*/
import type { SQLOutputValue } from "node:sqlite";
import type { ChatMessage } from "../../shared/chat.ts";
/**
* Returned by chat-service's query, which joins each message to its author and
* to the message it replies to. `reply_to_id` is kept when the message it
* points to is deleted, while `reply_to_author` and `reply_to_body` then read
* null.
*/
export interface ChatMessageRow {
id: string;
user_id: string;
body: string;
created_at: string;
updated_at: string | null;
author_username: string;
author_avatar_mime: string | null;
reply_to_id: string | null;
reply_to_author: string | null;
/** Cut to chat-service's `REPLY_SNIPPET_LEN` characters. */
reply_to_body: string | null;
/** 1 or 0. */
mentions_everyone: number;
[key: string]: SQLOutputValue;
}
/**
* Checks whether `obj`, a row read by chat-service, is a
* {@link ChatMessageRow}.
*/
export function isChatMessageRow(obj: unknown): obj is ChatMessageRow {
return !!obj && typeof obj === "object" &&
"id" in obj && typeof obj.id === "string" &&
"user_id" in obj && typeof obj.user_id === "string" &&
"body" in obj && typeof obj.body === "string" &&
"created_at" in obj && typeof obj.created_at === "string" &&
"updated_at" in obj &&
(typeof obj.updated_at === "string" || obj.updated_at === null) &&
"author_username" in obj && typeof obj.author_username === "string" &&
"author_avatar_mime" in obj &&
(typeof obj.author_avatar_mime === "string" ||
obj.author_avatar_mime === null) &&
"reply_to_id" in obj &&
(typeof obj.reply_to_id === "string" || obj.reply_to_id === null) &&
"reply_to_author" in obj &&
(typeof obj.reply_to_author === "string" || obj.reply_to_author === null) &&
"reply_to_body" in obj &&
(typeof obj.reply_to_body === "string" || obj.reply_to_body === null) &&
"mentions_everyone" in obj && typeof obj.mentions_everyone === "number";
}
/**
* Converts a row to the {@link ChatMessage} that chat-service returns, turning
* its timestamps into Dates and its nulls into absent fields.
*/
export function chatMessageRowToApi(row: ChatMessageRow): ChatMessage {
return {
id: row.id,
userId: row.user_id,
body: row.body,
createdAt: new Date(row.created_at),
updatedAt: row.updated_at ? new Date(row.updated_at) : undefined,
authorUsername: row.author_username,
authorAvatarMime: row.author_avatar_mime ?? undefined,
replyToId: row.reply_to_id ?? undefined,
replyToAuthor: row.reply_to_author ?? undefined,
replyToBody: row.reply_to_body ?? undefined,
mentionsEveryone: row.mentions_everyone === 1 || undefined,
};
}

75
api/model/comment.ts Normal file
View File

@@ -0,0 +1,75 @@
/**
* Turns the rows the comment service reads from `comments` into the {@link
* Comment} the comment routes send.
*
* @module
*/
import type { SQLOutputValue } from "node:sqlite";
import type { Comment } from "../../shared/comment.ts";
/**
* Returned by the comment service's `SELECT`s on `comments`, joined with
* `users` for the `author_` columns.
*/
export interface CommentRow {
id: string;
dump_id: string;
user_id: string;
/** It is null on a top-level comment. */
parent_id: string | null;
body: string;
created_at: string;
updated_at: string | null;
/**
* 1 once deleted. The body is emptied but the row stays, so its replies keep
* their place in the thread.
*/
deleted: number;
like_count: number;
author_username: string;
author_avatar_mime: string | null;
[key: string]: SQLOutputValue;
}
/** Checks whether a row `node:sqlite` returned is a {@link CommentRow}. */
export function isCommentRow(
obj: unknown,
): obj is CommentRow {
return !!obj && typeof obj === "object" &&
"id" in obj && typeof obj.id === "string" &&
"dump_id" in obj && typeof obj.dump_id === "string" &&
"user_id" in obj && typeof obj.user_id === "string" &&
"parent_id" in obj &&
(typeof obj.parent_id === "string" || obj.parent_id === null) &&
"body" in obj && typeof obj.body === "string" &&
"created_at" in obj && typeof obj.created_at === "string" &&
"updated_at" in obj &&
(typeof obj.updated_at === "string" || obj.updated_at === null) &&
"deleted" in obj && typeof obj.deleted === "number" &&
"like_count" in obj && typeof obj.like_count === "number" &&
"author_username" in obj && typeof obj.author_username === "string" &&
"author_avatar_mime" in obj &&
(typeof obj.author_avatar_mime === "string" ||
obj.author_avatar_mime === null);
}
/**
* Converts `row` to the {@link Comment} the comment routes send and the
* WebSocket broadcasts.
*/
export function commentRowToApi(row: CommentRow): Comment {
return {
id: row.id,
dumpId: row.dump_id,
userId: row.user_id,
parentId: row.parent_id ?? undefined,
body: row.body,
createdAt: new Date(row.created_at),
updatedAt: row.updated_at ? new Date(row.updated_at) : undefined,
deleted: Boolean(row.deleted),
likeCount: row.like_count,
authorUsername: row.author_username,
authorAvatarMime: row.author_avatar_mime ?? undefined,
};
}

543
api/model/db.ts
View File

@@ -1,543 +0,0 @@
import { randomBytes, scryptSync } from "node:crypto";
import { DatabaseSync, type SQLOutputValue } from "node:sqlite";
import {
type Category,
type ChatMessage,
type Comment,
Dump,
isRole,
type Notification,
type NotificationType,
type Playlist,
type RichContent,
type Role,
type User,
} from "./interfaces.ts";
import {
ATTACHMENTS_DIR,
DB_PATH,
ORPHANED_ATTACHMENTS_RETENTION_HOURS,
UNUSED_INVITES_RETENTION_DAYS,
} from "../config.ts";
import { runMigrations } from "../db/migrate.ts";
export const db = new DatabaseSync(DB_PATH);
db.exec("PRAGMA foreign_keys = ON;");
runMigrations(db);
// Purge expired/used password reset tokens on startup
db.prepare(
`DELETE FROM password_reset_tokens WHERE expires_at < datetime('now') OR used_at IS NOT NULL;`,
).run();
// Purge expired unused invites on startup
db.prepare(
`DELETE FROM invites WHERE used_at IS NULL AND created_at < datetime('now', '-${UNUSED_INVITES_RETENTION_DAYS} days');`,
).run();
// Prune orphaned attachments (uploaded but never linked to a resource) older than 1 hour
const orphanedAttachments = db.prepare(
`SELECT id FROM attachments WHERE resource_id IS NULL AND created_at < datetime('now', '-${ORPHANED_ATTACHMENTS_RETENTION_HOURS} hour');`,
).all() as { id: string }[];
if (orphanedAttachments.length > 0) {
for (const { id } of orphanedAttachments) {
await Deno.remove(`${ATTACHMENTS_DIR}/${id}`).catch(() => {});
}
db.prepare(
`DELETE FROM attachments WHERE resource_id IS NULL AND created_at < datetime('now', '-${ORPHANED_ATTACHMENTS_RETENTION_HOURS} hour');`,
).run();
}
// Create default admin user if no users exist
const userCount = db.prepare(`SELECT COUNT(*) as count FROM users`).get() as {
count: number;
};
if (userCount.count === 0) {
const salt = randomBytes(16).toString("hex");
const hash = scryptSync("admin", salt, 64).toString("hex");
const passwordHash = `${hash}.${salt}`;
db.prepare(
`INSERT INTO users (id, username, password_hash, role, created_at, email) VALUES (?, 'admin', ?, 'admin', datetime('now'), 'admin@localhost')`,
).run(crypto.randomUUID(), passwordHash);
console.log("Created default admin user (username: admin, password: admin)");
}
/**
* Database Row Types
*/
export interface DumpRow {
id: string;
kind: string;
title: string;
comment: string | null;
user_id: string;
created_at: string;
updated_at: string | null;
slug: string | null;
url: string | null;
rich_content: string | null;
file_name: string | null;
file_mime: string | null;
file_size: number | null;
vote_count: number;
comment_count: number;
is_private: number;
custom_thumbnail_mime: string | null;
[key: string]: SQLOutputValue; // Index signature
}
// Shared `dumps` column list — keep in sync with DumpRow/isDumpRow above.
// Services import these instead of repeating the column list themselves.
export const DUMP_COLUMNS =
"id, kind, title, slug, comment, user_id, created_at, updated_at, url, rich_content, file_name, file_mime, file_size, vote_count, is_private, custom_thumbnail_mime";
export const DUMP_COLUMNS_ALIASED =
"d.id, d.kind, d.title, d.slug, d.comment, d.user_id, d.created_at, d.updated_at, d.url, d.rich_content, d.file_name, d.file_mime, d.file_size, d.vote_count, d.is_private, d.custom_thumbnail_mime";
export const DUMP_SELECT_COLUMNS = `${DUMP_COLUMNS},
(SELECT COUNT(*) FROM comments WHERE dump_id = dumps.id AND deleted = 0) as comment_count`;
export const DUMP_SELECT_COLUMNS_ALIASED = `${DUMP_COLUMNS_ALIASED},
(SELECT COUNT(*) FROM comments WHERE dump_id = d.id AND deleted = 0) as comment_count`;
export interface UserRow {
id: string;
username: string;
password_hash: string;
role: Role;
created_at: string;
updated_at: string | null;
avatar_mime: string | null;
description: string | null;
invited_by: string | null;
// Present only when joined: LEFT JOIN users i ON i.id = u.invited_by
invited_by_username: string | null;
email: string;
[key: string]: SQLOutputValue; // Index signature
}
/**
* Type Guards
*/
export function isDumpRow(obj: unknown): obj is DumpRow {
return !!obj &&
typeof obj === "object" &&
"id" in obj && typeof obj.id === "string" &&
"kind" in obj && typeof obj.kind === "string" &&
"title" in obj && typeof obj.title === "string" &&
"comment" in obj &&
(typeof obj.comment === "string" || obj.comment === null) &&
"user_id" in obj && typeof obj.user_id === "string" &&
"created_at" in obj && typeof obj.created_at === "string" &&
"updated_at" in obj &&
(typeof obj.updated_at === "string" || obj.updated_at === null) &&
"slug" in obj && (typeof obj.slug === "string" || obj.slug === null) &&
"url" in obj && (typeof obj.url === "string" || obj.url === null) &&
"rich_content" in obj &&
(typeof obj.rich_content === "string" || obj.rich_content === null) &&
"file_name" in obj &&
(typeof obj.file_name === "string" || obj.file_name === null) &&
"file_mime" in obj &&
(typeof obj.file_mime === "string" || obj.file_mime === null) &&
"file_size" in obj &&
(typeof obj.file_size === "number" || obj.file_size === null) &&
"vote_count" in obj && typeof obj.vote_count === "number" &&
"comment_count" in obj && typeof obj.comment_count === "number" &&
"is_private" in obj && typeof obj.is_private === "number" &&
"custom_thumbnail_mime" in obj &&
(typeof obj.custom_thumbnail_mime === "string" ||
obj.custom_thumbnail_mime === null);
}
export function isUserRow(obj: unknown): obj is UserRow {
return !!obj &&
typeof obj === "object" &&
"id" in obj && typeof obj.id === "string" &&
"username" in obj && typeof obj.username === "string" &&
"password_hash" in obj && typeof obj.password_hash === "string" &&
"role" in obj && isRole(obj.role) &&
"created_at" in obj && typeof obj.created_at === "string" &&
"updated_at" in obj &&
(typeof obj.updated_at === "string" || obj.updated_at === null) &&
"avatar_mime" in obj &&
(typeof obj.avatar_mime === "string" || obj.avatar_mime === null) &&
"description" in obj &&
(typeof obj.description === "string" || obj.description === null) &&
"invited_by" in obj &&
(typeof obj.invited_by === "string" || obj.invited_by === null) &&
"email" in obj && typeof obj.email === "string";
}
/**
* Conversion Helpers
*/
export function dumpRowToApi(row: DumpRow): Dump {
return {
id: row.id,
kind: row.kind as "url" | "file",
title: row.title,
slug: row.slug ?? undefined,
url: row.url ?? undefined,
comment: row.comment ?? undefined,
userId: row.user_id,
createdAt: new Date(row.created_at),
updatedAt: row.updated_at ? new Date(row.updated_at) : undefined,
richContent: row.rich_content
? (JSON.parse(row.rich_content) as RichContent)
: undefined,
fileName: row.file_name ?? undefined,
fileMime: row.file_mime ?? undefined,
fileSize: row.file_size ?? undefined,
voteCount: row.vote_count,
commentCount: row.comment_count,
isPrivate: Boolean(row.is_private),
thumbnailMime: row.custom_thumbnail_mime ?? undefined,
};
}
export function dumpApiToRow(dump: Dump): DumpRow {
return {
id: dump.id,
kind: dump.kind,
title: dump.title,
slug: dump.slug ?? null,
comment: dump.comment ?? null,
user_id: dump.userId,
created_at: dump.createdAt.toISOString(),
updated_at: dump.updatedAt?.toISOString() ?? null,
url: dump.url ?? null,
rich_content: dump.richContent ? JSON.stringify(dump.richContent) : null,
file_name: dump.fileName ?? null,
file_mime: dump.fileMime ?? null,
file_size: dump.fileSize ?? null,
vote_count: dump.voteCount,
comment_count: dump.commentCount,
is_private: dump.isPrivate ? 1 : 0,
custom_thumbnail_mime: dump.thumbnailMime ?? null,
};
}
export function userRowToApi(row: UserRow): User {
return {
id: row.id,
username: row.username,
passwordHash: row.password_hash,
role: row.role,
createdAt: new Date(row.created_at),
updatedAt: row.updated_at ? new Date(row.updated_at) : undefined,
avatarMime: row.avatar_mime ?? undefined,
description: row.description ?? undefined,
invitedByUsername: typeof row.invited_by_username === "string"
? row.invited_by_username
: undefined,
email: row.email,
};
}
export function userApiToRow(user: User): UserRow {
return {
id: user.id,
username: user.username,
password_hash: user.passwordHash,
role: user.role,
created_at: user.createdAt.toISOString(),
updated_at: user.updatedAt?.toISOString() ?? null,
avatar_mime: user.avatarMime ?? null,
description: user.description ?? null,
invited_by: null,
invited_by_username: null,
email: user.email,
};
}
export interface CommentRow {
id: string;
dump_id: string;
user_id: string;
parent_id: string | null;
body: string;
created_at: string;
updated_at: string | null;
deleted: number;
like_count: number;
author_username: string;
author_avatar_mime: string | null;
[key: string]: SQLOutputValue;
}
export function isCommentRow(
obj: unknown,
): obj is CommentRow {
return !!obj && typeof obj === "object" &&
"id" in obj && typeof obj.id === "string" &&
"dump_id" in obj && typeof obj.dump_id === "string" &&
"user_id" in obj && typeof obj.user_id === "string" &&
"parent_id" in obj &&
(typeof obj.parent_id === "string" || obj.parent_id === null) &&
"body" in obj && typeof obj.body === "string" &&
"created_at" in obj && typeof obj.created_at === "string" &&
"updated_at" in obj &&
(typeof obj.updated_at === "string" || obj.updated_at === null) &&
"deleted" in obj && typeof obj.deleted === "number" &&
"like_count" in obj && typeof obj.like_count === "number" &&
"author_username" in obj && typeof obj.author_username === "string" &&
"author_avatar_mime" in obj &&
(typeof obj.author_avatar_mime === "string" ||
obj.author_avatar_mime === null);
}
export function commentRowToApi(row: CommentRow): Comment {
return {
id: row.id,
dumpId: row.dump_id,
userId: row.user_id,
parentId: row.parent_id ?? undefined,
body: row.body,
createdAt: new Date(row.created_at),
updatedAt: row.updated_at ? new Date(row.updated_at) : undefined,
deleted: Boolean(row.deleted),
likeCount: row.like_count,
authorUsername: row.author_username,
authorAvatarMime: row.author_avatar_mime ?? undefined,
};
}
export interface ChatMessageRow {
id: string;
user_id: string;
body: string;
created_at: string;
updated_at: string | null;
author_username: string;
author_avatar_mime: string | null;
// Reply columns: reply_to_id is the stored target; the author/body preview are
// LEFT-JOINed from the target and go null when there's no reply (or it was
// since deleted).
reply_to_id: string | null;
reply_to_author: string | null;
reply_to_body: string | null;
[key: string]: SQLOutputValue;
}
export function isChatMessageRow(obj: unknown): obj is ChatMessageRow {
return !!obj && typeof obj === "object" &&
"id" in obj && typeof obj.id === "string" &&
"user_id" in obj && typeof obj.user_id === "string" &&
"body" in obj && typeof obj.body === "string" &&
"created_at" in obj && typeof obj.created_at === "string" &&
"updated_at" in obj &&
(typeof obj.updated_at === "string" || obj.updated_at === null) &&
"author_username" in obj && typeof obj.author_username === "string" &&
"author_avatar_mime" in obj &&
(typeof obj.author_avatar_mime === "string" ||
obj.author_avatar_mime === null) &&
"reply_to_id" in obj &&
(typeof obj.reply_to_id === "string" || obj.reply_to_id === null) &&
"reply_to_author" in obj &&
(typeof obj.reply_to_author === "string" || obj.reply_to_author === null) &&
"reply_to_body" in obj &&
(typeof obj.reply_to_body === "string" || obj.reply_to_body === null);
}
export function chatMessageRowToApi(row: ChatMessageRow): ChatMessage {
return {
id: row.id,
userId: row.user_id,
body: row.body,
createdAt: new Date(row.created_at),
updatedAt: row.updated_at ? new Date(row.updated_at) : undefined,
authorUsername: row.author_username,
authorAvatarMime: row.author_avatar_mime ?? undefined,
replyToId: row.reply_to_id ?? undefined,
replyToAuthor: row.reply_to_author ?? undefined,
replyToBody: row.reply_to_body ?? undefined,
};
}
export interface PlaylistRow {
id: string;
user_id: string;
title: string;
slug: string | null;
description: string | null;
is_public: number;
created_at: string;
updated_at: string | null;
image_mime: string | null;
[key: string]: SQLOutputValue;
}
export function isPlaylistRow(
obj: unknown,
): obj is PlaylistRow {
return !!obj && typeof obj === "object" &&
"id" in obj && typeof obj.id === "string" &&
"user_id" in obj && typeof obj.user_id === "string" &&
"title" in obj && typeof obj.title === "string" &&
"slug" in obj && (typeof obj.slug === "string" || obj.slug === null) &&
"description" in obj &&
(typeof obj.description === "string" || obj.description === null) &&
"is_public" in obj && typeof obj.is_public === "number" &&
"created_at" in obj && typeof obj.created_at === "string" &&
"updated_at" in obj &&
(typeof obj.updated_at === "string" || obj.updated_at === null) &&
"image_mime" in obj &&
(typeof obj.image_mime === "string" || obj.image_mime === null);
}
export function playlistRowToApi(row: PlaylistRow): Playlist {
return {
id: row.id,
userId: row.user_id,
title: row.title,
slug: row.slug ?? undefined,
description: row.description ?? undefined,
isPublic: Boolean(row.is_public),
createdAt: new Date(row.created_at),
updatedAt: row.updated_at ? new Date(row.updated_at) : undefined,
imageMime: row.image_mime ?? undefined,
dumpCount: typeof row.dump_count === "number" ? row.dump_count : undefined,
ownerUsername: typeof row.owner_username === "string"
? row.owner_username
: undefined,
};
}
// ── Categories ────────────────────────────────────────────────────────────────
export interface CategoryRow {
id: string;
slug: string;
name: string;
position: number;
created_at: string;
updated_at: string | null;
[key: string]: SQLOutputValue;
}
export function isCategoryRow(obj: unknown): obj is CategoryRow {
return !!obj && typeof obj === "object" &&
"id" in obj && typeof obj.id === "string" &&
"slug" in obj && typeof obj.slug === "string" &&
"name" in obj && typeof obj.name === "string" &&
"position" in obj && typeof obj.position === "number" &&
"created_at" in obj && typeof obj.created_at === "string" &&
"updated_at" in obj &&
(typeof obj.updated_at === "string" || obj.updated_at === null);
}
export function categoryRowToApi(row: CategoryRow): Category {
return {
id: row.id,
slug: row.slug,
name: row.name,
position: row.position,
createdAt: new Date(row.created_at),
updatedAt: row.updated_at ? new Date(row.updated_at) : undefined,
};
}
export interface FollowRow {
id: string;
follower_id: string;
followed_user_id: string | null;
followed_playlist_id: string | null;
created_at: string;
[key: string]: SQLOutputValue;
}
export function isFollowRow(
obj: unknown,
): obj is FollowRow {
return !!obj && typeof obj === "object" &&
"id" in obj && typeof obj.id === "string" &&
"follower_id" in obj && typeof obj.follower_id === "string" &&
"created_at" in obj && typeof obj.created_at === "string" &&
"followed_user_id" in obj && (obj.followed_user_id === null ||
typeof obj.followed_user_id === "string") &&
"followed_playlist_id" in obj && (obj.followed_playlist_id === null ||
typeof obj.followed_playlist_id === "string");
}
// ── Notifications ─────────────────────────────────────────────────────────────
export interface NotificationRow {
id: string;
user_id: string;
type: string;
data: string;
read: number;
created_at: string;
source_key: string | null;
[key: string]: SQLOutputValue;
}
export function isNotificationRow(
obj: unknown,
): obj is NotificationRow {
return !!obj && typeof obj === "object" &&
"id" in obj && typeof obj.id === "string" &&
"user_id" in obj && typeof obj.user_id === "string" &&
"type" in obj && typeof obj.type === "string" &&
"data" in obj && typeof obj.data === "string" &&
"read" in obj && typeof obj.read === "number" &&
"created_at" in obj && typeof obj.created_at === "string" &&
"source_key" in obj &&
(typeof obj.source_key === "string" || obj.source_key === null);
}
export function notificationRowToApi(row: NotificationRow): Notification {
return {
id: row.id,
userId: row.user_id,
type: row.type as NotificationType,
data: JSON.parse(row.data),
read: Boolean(row.read),
createdAt: new Date(row.created_at),
};
}
// ── Password reset tokens ─────────────────────────────────────────────────────
export interface PasswordResetTokenRow {
token: string;
user_id: string;
expires_at: string;
used_at: string | null;
[key: string]: SQLOutputValue;
}
export function isPasswordResetTokenRow(
obj: unknown,
): obj is PasswordResetTokenRow {
return !!obj && typeof obj === "object" &&
"token" in obj && typeof obj.token === "string" &&
"user_id" in obj && typeof obj.user_id === "string" &&
"expires_at" in obj && typeof obj.expires_at === "string" &&
"used_at" in obj &&
(obj.used_at === null || typeof obj.used_at === "string");
}
// ── Invites ───────────────────────────────────────────────────────────────────
export interface InviteRow {
token: string;
inviter_id: string;
used_at: string | null;
created_at: string;
[key: string]: SQLOutputValue;
}
export function isInviteRow(
obj: unknown,
): obj is InviteRow {
return !!obj && typeof obj === "object" &&
"token" in obj && typeof obj.token === "string" &&
"inviter_id" in obj && typeof obj.inviter_id === "string" &&
"created_at" in obj && typeof obj.created_at === "string" &&
"used_at" in obj &&
(obj.used_at === null || typeof obj.used_at === "string");
}

162
api/model/dump.ts Normal file
View File

@@ -0,0 +1,162 @@
/**
* Turns the rows of `dumps` into the API's {@link Dump} the same way for every
* service that reads dumps: each selects them with the column lists here,
* checks each row with {@link isDumpRow} and converts it with {@link
* dumpRowToApi}.
*
* @module
*/
import type { SQLOutputValue } from "node:sqlite";
import type { Dump, RichContent } from "../../shared/dump.ts";
/**
* Read from the `dumps` table with {@link DUMP_SELECT_COLUMNS}, or written back
* from a {@link Dump} by {@link dumpApiToRow}. It lacks `url_canonical`, which
* only the duplicate lookup reads.
*/
export interface DumpRow {
id: string;
kind: string;
title: string;
comment: string | null;
user_id: string;
created_at: string;
updated_at: string | null;
/** Unique among its poster's dumps, and set on every row since 0021. */
slug: string;
/** The link as the poster submitted it. */
url: string | null;
/** {@link RichContent} as JSON. */
rich_content: string | null;
file_name: string | null;
file_mime: string | null;
file_size: number | null;
vote_count: number;
/** Computed by a subquery, which counts the comments that aren't deleted. */
comment_count: number;
/** The poster's current name, computed by a subquery. */
owner_username: string;
is_private: number;
custom_thumbnail_mime: string | null;
/** Lets a query select more columns, such as a join's. */
[key: string]: SQLOutputValue;
}
/**
* Lists the stored columns of {@link DumpRow}. A column added to the table must
* be added here, to {@link DumpRow} and to {@link isDumpRow}.
*/
export const DUMP_COLUMNS =
"id, kind, title, slug, comment, user_id, created_at, updated_at, url, rich_content, file_name, file_mime, file_size, vote_count, is_private, custom_thumbnail_mime";
// The same columns, for a query that aliases `dumps` as `d`.
const DUMP_COLUMNS_ALIASED = DUMP_COLUMNS.split(", ").map((c) => `d.${c}`)
.join(", ");
/**
* Selects a whole {@link DumpRow}, the stored columns and the computed
* `comment_count` and `owner_username`.
*/
export const DUMP_SELECT_COLUMNS: string = `${DUMP_COLUMNS},
(SELECT COUNT(*) FROM comments WHERE dump_id = dumps.id AND deleted = 0) as comment_count,
(SELECT username FROM users WHERE id = dumps.user_id) as owner_username`;
/**
* Selects a whole {@link DumpRow} in a query that aliases `dumps` as `d` to
* join it with other tables.
*/
export const DUMP_SELECT_COLUMNS_ALIASED: string = `${DUMP_COLUMNS_ALIASED},
(SELECT COUNT(*) FROM comments WHERE dump_id = d.id AND deleted = 0) as comment_count,
(SELECT username FROM users WHERE id = d.user_id) as owner_username`;
/**
* Checks whether `obj`, a row from a query, is a {@link DumpRow}. It checks
* each column's type, not that `kind` is `url` or `file`.
*/
export function isDumpRow(obj: unknown): obj is DumpRow {
return !!obj &&
typeof obj === "object" &&
"id" in obj && typeof obj.id === "string" &&
"kind" in obj && typeof obj.kind === "string" &&
"title" in obj && typeof obj.title === "string" &&
"comment" in obj &&
(typeof obj.comment === "string" || obj.comment === null) &&
"user_id" in obj && typeof obj.user_id === "string" &&
"created_at" in obj && typeof obj.created_at === "string" &&
"updated_at" in obj &&
(typeof obj.updated_at === "string" || obj.updated_at === null) &&
"slug" in obj && typeof obj.slug === "string" &&
"url" in obj && (typeof obj.url === "string" || obj.url === null) &&
"rich_content" in obj &&
(typeof obj.rich_content === "string" || obj.rich_content === null) &&
"file_name" in obj &&
(typeof obj.file_name === "string" || obj.file_name === null) &&
"file_mime" in obj &&
(typeof obj.file_mime === "string" || obj.file_mime === null) &&
"file_size" in obj &&
(typeof obj.file_size === "number" || obj.file_size === null) &&
"vote_count" in obj && typeof obj.vote_count === "number" &&
"comment_count" in obj && typeof obj.comment_count === "number" &&
"owner_username" in obj && typeof obj.owner_username === "string" &&
"is_private" in obj && typeof obj.is_private === "number" &&
"custom_thumbnail_mime" in obj &&
(typeof obj.custom_thumbnail_mime === "string" ||
obj.custom_thumbnail_mime === null);
}
/**
* Converts a row to the API's {@link Dump}, parsing its preview's JSON. It
* leaves `categoryIds` out: the services add them with `attachCategoryIds`.
*/
export function dumpRowToApi(row: DumpRow): Dump {
return {
id: row.id,
kind: row.kind as "url" | "file",
title: row.title,
slug: row.slug,
url: row.url ?? undefined,
comment: row.comment ?? undefined,
userId: row.user_id,
ownerUsername: row.owner_username,
createdAt: new Date(row.created_at),
updatedAt: row.updated_at ? new Date(row.updated_at) : undefined,
richContent: row.rich_content
? (JSON.parse(row.rich_content) as RichContent)
: undefined,
fileName: row.file_name ?? undefined,
fileMime: row.file_mime ?? undefined,
fileSize: row.file_size ?? undefined,
voteCount: row.vote_count,
commentCount: row.comment_count,
isPrivate: Boolean(row.is_private),
thumbnailMime: row.custom_thumbnail_mime ?? undefined,
};
}
/**
* Converts a {@link Dump} back to a row, for a service that writes an edited
* dump's columns.
*/
export function dumpApiToRow(dump: Dump): DumpRow {
return {
id: dump.id,
kind: dump.kind,
title: dump.title,
slug: dump.slug,
comment: dump.comment ?? null,
user_id: dump.userId,
created_at: dump.createdAt.toISOString(),
updated_at: dump.updatedAt?.toISOString() ?? null,
url: dump.url ?? null,
rich_content: dump.richContent ? JSON.stringify(dump.richContent) : null,
file_name: dump.fileName ?? null,
file_mime: dump.fileMime ?? null,
file_size: dump.fileSize ?? null,
vote_count: dump.voteCount,
comment_count: dump.commentCount,
owner_username: dump.ownerUsername,
is_private: dump.isPrivate ? 1 : 0,
custom_thumbnail_mime: dump.thumbnailMime ?? null,
};
}

38
api/model/follow.ts Normal file
View File

@@ -0,0 +1,38 @@
/**
* Checks the rows `follow-service.ts` reads from `follows` to answer what a
* user follows.
*
* @module
*/
import type { SQLOutputValue } from "node:sqlite";
/**
* Read by `getFollowStatus`. Exactly one of `followed_user_id` and
* `followed_playlist_id` is set, as the table's CHECK constraint requires.
*/
export interface FollowRow {
id: string;
follower_id: string;
followed_user_id: string | null;
followed_playlist_id: string | null;
created_at: string;
[key: string]: SQLOutputValue;
}
/**
* Checks whether `obj`, a row `getFollowStatus` selected, is a
* {@link FollowRow}.
*/
export function isFollowRow(
obj: unknown,
): obj is FollowRow {
return !!obj && typeof obj === "object" &&
"id" in obj && typeof obj.id === "string" &&
"follower_id" in obj && typeof obj.follower_id === "string" &&
"created_at" in obj && typeof obj.created_at === "string" &&
"followed_user_id" in obj && (obj.followed_user_id === null ||
typeof obj.followed_user_id === "string") &&
"followed_playlist_id" in obj && (obj.followed_playlist_id === null ||
typeof obj.followed_playlist_id === "string");
}

951
api/model/interfaces.ts
View File

@@ -1,951 +0,0 @@
import { VALIDATION } from "../config.ts";
/**
* Backend
*/
export interface RichContent {
type: string;
url: string;
siteName?: string;
title?: string;
description?: string;
/**
* A real preview image (og:image, a large content image, a video still).
* Never a favicon — an absent value means "this page offers no artwork",
* which is what makes the generated placeholder possible.
*/
thumbnailUrl?: string;
/** The page's own icon, used as the placeholder's glyph. */
faviconUrl?: string;
/** The page's declared brand color, normalized to `#rrggbb`. */
accentColor?: string;
videoId?: string;
embedUrl?: string;
}
export interface Dump {
id: string;
kind: "url" | "file";
title: string;
slug?: string;
comment?: string;
userId: string;
createdAt: Date;
updatedAt?: Date;
url?: string;
richContent?: RichContent;
fileName?: string;
fileMime?: string;
fileSize?: number;
voteCount: number;
commentCount: number;
isPrivate: boolean;
thumbnailMime?: string;
categoryIds?: string[];
}
/**
* Categories
*/
export interface Category {
id: string;
slug: string;
name: string;
position: number;
createdAt: Date;
updatedAt?: Date;
}
export interface CreateCategoryRequest {
slug: string;
name: string;
position?: number;
}
export interface UpdateCategoryRequest {
slug?: string;
name?: string;
position?: number;
}
export function isCreateCategoryRequest(
obj: unknown,
): obj is CreateCategoryRequest {
if (!obj || typeof obj !== "object") return false;
const o = obj as Record<string, unknown>;
if (typeof o.slug !== "string") return false;
if (typeof o.name !== "string") return false;
if ("position" in o && typeof o.position !== "number") return false;
return true;
}
export function isUpdateCategoryRequest(
obj: unknown,
): obj is UpdateCategoryRequest {
if (!obj || typeof obj !== "object") return false;
const o = obj as Record<string, unknown>;
if ("slug" in o && typeof o.slug !== "string") return false;
if ("name" in o && typeof o.name !== "string") return false;
if ("position" in o && typeof o.position !== "number") return false;
return true;
}
/**
* Authentication
*/
export type Role = "user" | "moderator" | "admin";
export const ROLES: readonly Role[] = ["user", "moderator", "admin"];
export function isRole(value: unknown): value is Role {
return typeof value === "string" &&
(ROLES as readonly string[]).includes(value);
}
export interface User {
id: string;
username: string;
passwordHash: string;
role: Role;
createdAt: Date;
updatedAt?: Date;
avatarMime?: string;
description?: string;
invitedByUsername?: string;
email: string;
}
export interface LoginUserRequest {
username: string;
password: string;
}
export interface RegisterUserRequest {
username: string;
password: string;
inviteToken: string;
email: string;
}
export interface UpdateUserRequest {
username?: string;
password?: string;
role?: Role;
description?: string | null;
email?: string;
}
export function isLoginUserRequest(obj: unknown): obj is LoginUserRequest {
return !!obj && typeof obj === "object" &&
"username" in obj && typeof obj.username === "string" &&
"password" in obj && typeof obj.password === "string";
}
export function isRegisterUserRequest(
obj: unknown,
): obj is RegisterUserRequest {
return validateRegisterUserRequest(obj) === null;
}
/** Returns a human-readable error string, or null if the request is valid. */
export function validateRegisterUserRequest(obj: unknown): string | null {
if (
!obj || typeof obj !== "object" ||
!("username" in obj) || typeof obj.username !== "string" ||
!("password" in obj) || typeof obj.password !== "string" ||
!("inviteToken" in obj) || typeof obj.inviteToken !== "string" ||
!("email" in obj) || typeof obj.email !== "string"
) return "Invalid request";
const { username, password, email } = obj as RegisterUserRequest;
if (!/^[^\s@]+@[^\s@]+\.[^\s@]+$/.test(email)) {
return "Invalid email address";
}
if (
!new RegExp(
`^[a-zA-Z0-9_]{${VALIDATION.USERNAME_MIN},${VALIDATION.USERNAME_MAX}}$`,
)
.test(username)
) {
return `Username must be ${VALIDATION.USERNAME_MIN}–${VALIDATION.USERNAME_MAX} characters and contain only letters, numbers, or underscores`;
}
if (password.length < VALIDATION.PASSWORD_MIN) {
return `Password must be at least ${VALIDATION.PASSWORD_MIN} characters`;
}
if (password.length > VALIDATION.PASSWORD_MAX) {
return `Password must be at most ${VALIDATION.PASSWORD_MAX} characters`;
}
return null;
}
export function isUpdateUserRequest(obj: unknown): obj is UpdateUserRequest {
if (!obj || typeof obj !== "object") return false;
const o = obj as Record<string, unknown>;
if ("username" in o) {
if (typeof o.username !== "string") return false;
if (!/^[a-zA-Z0-9_]{1,32}$/.test(o.username as string)) return false;
}
if ("password" in o) {
if (typeof o.password !== "string") return false;
const len = (o.password as string).length;
if (len < VALIDATION.PASSWORD_MIN || len > VALIDATION.PASSWORD_MAX) {
return false;
}
}
if ("role" in o && !isRole(o.role)) return false;
if (
"description" in o && typeof o.description !== "string" &&
o.description !== null
) return false;
if ("email" in o) {
if (typeof o.email !== "string") return false;
if (!/^[^\s@]+@[^\s@]+\.[^\s@]+$/.test(o.email as string)) return false;
}
if (
typeof o.description === "string" &&
(o.description as string).length > VALIDATION.USER_DESCRIPTION_MAX
) return false;
return true;
}
export interface AuthResponse {
token: string;
user: User;
}
export interface AuthPayload {
userId: string;
username: string;
role: Role;
exp: number;
iat?: number;
}
export function isAuthPayload(obj: unknown): obj is AuthPayload {
return !!obj &&
typeof obj === "object" &&
// Purpose-scoped tokens (password reset, media) are never session tokens.
!("purpose" in obj) &&
"userId" in obj && typeof obj.userId === "string" &&
"username" in obj && typeof obj.username === "string" &&
"role" in obj && isRole(obj.role) &&
"exp" in obj && typeof obj.exp === "number";
}
export interface PasswordResetPayload {
purpose: "password-reset";
userId: string;
exp: number;
}
export function isPasswordResetPayload(
obj: unknown,
): obj is PasswordResetPayload {
return !!obj && typeof obj === "object" &&
"purpose" in obj &&
(obj as Record<string, unknown>).purpose === "password-reset" &&
"userId" in obj &&
typeof (obj as Record<string, unknown>).userId === "string";
}
/**
* API
*/
export enum APIErrorCode {
BAD_REQUEST = "BAD_REQUEST",
NOT_FOUND = "NOT_FOUND",
SERVER_ERROR = "SERVER_ERROR",
TIMEOUT = "TIMEOUT",
UNAUTHORIZED = "UNAUTHORIZED",
VALIDATION_ERROR = "VALIDATION_ERROR",
}
export interface APIError {
code: APIErrorCode;
message: string;
}
export interface APISuccess<T> {
success: true;
data: T;
error?: never;
}
export interface APIFailure {
success: false;
data?: never;
error: APIError;
}
export type APIResponse<T> = APISuccess<T> | APIFailure;
export interface PaginatedData<T> {
items: T[];
total: number;
hasMore: boolean;
}
export class APIException extends Error {
readonly code: APIErrorCode;
readonly status: number;
constructor(code: APIErrorCode, status: number, message: string) {
super(message);
this.code = code;
this.status = status;
}
}
/**
* Comments
*/
export interface Comment {
id: string;
dumpId: string;
userId: string;
parentId?: string;
body: string;
createdAt: Date;
updatedAt?: Date;
deleted: boolean;
likeCount: number;
authorUsername: string;
authorAvatarMime?: string;
}
export interface CreateCommentRequest {
body: string;
parentId?: string;
}
/**
* Chat
*/
export interface ChatMessage {
id: string;
userId: string;
body: string;
createdAt: Date;
updatedAt?: Date;
authorUsername: string;
authorAvatarMime?: string;
/** Id of the message this one replies to, if any. Retained even when the
* target is later deleted (the preview fields below then go undefined). */
replyToId?: string;
/** Author/snippet of the replied-to message, for a subtle inline reference.
* Absent when there's no reply, or the target has since been deleted. */
replyToAuthor?: string;
replyToBody?: string;
}
/** Wire format — createdAt arrives as an ISO string over JSON. */
export type RawChatMessage = Omit<ChatMessage, "createdAt"> & {
createdAt: string;
};
export function isCreateCommentRequest(
obj: unknown,
): obj is CreateCommentRequest {
if (!obj || typeof obj !== "object") return false;
const o = obj as Record<string, unknown>;
return typeof o.body === "string" &&
(o.body as string).trim().length > 0 &&
(o.body as string).length <= VALIDATION.COMMENT_BODY_MAX &&
(!("parentId" in o) || typeof o.parentId === "string" ||
o.parentId === null);
}
export interface UpdateCommentRequest {
body: string;
}
export function isUpdateCommentRequest(
obj: unknown,
): obj is UpdateCommentRequest {
if (!obj || typeof obj !== "object") return false;
const o = obj as Record<string, unknown>;
return typeof o.body === "string" &&
(o.body as string).trim().length > 0 &&
(o.body as string).length <= VALIDATION.COMMENT_BODY_MAX;
}
/**
* Playlists
*/
export interface Playlist {
id: string;
userId: string;
title: string;
slug?: string;
description?: string;
isPublic: boolean;
createdAt: Date;
updatedAt?: Date;
imageMime?: string;
dumpCount?: number;
ownerUsername?: string;
}
export interface PlaylistWithDumps extends Playlist {
dumps: Dump[];
}
export interface PlaylistMembership {
playlist: Playlist;
hasDump: boolean;
}
export interface CreatePlaylistRequest {
title: string;
description?: string;
isPublic: boolean;
}
export interface UpdatePlaylistRequest {
title?: string;
description?: string | null;
isPublic?: boolean;
}
export interface ReorderPlaylistRequest {
dumpIds: string[];
}
export function isCreatePlaylistRequest(
obj: unknown,
): obj is CreatePlaylistRequest {
if (
!obj || typeof obj !== "object" ||
!("title" in obj) || typeof obj.title !== "string" ||
!("isPublic" in obj) || typeof obj.isPublic !== "boolean"
) return false;
const o = obj as Record<string, unknown>;
if (
(o.title as string).length === 0 ||
(o.title as string).length > VALIDATION.PLAYLIST_TITLE_MAX
) return false;
if (
"description" in o && typeof o.description !== "string" &&
o.description !== null
) return false;
if (
typeof o.description === "string" &&
(o.description as string).length > VALIDATION.PLAYLIST_DESCRIPTION_MAX
) return false;
return true;
}
export function isUpdatePlaylistRequest(
obj: unknown,
): obj is UpdatePlaylistRequest {
if (!obj || typeof obj !== "object") return false;
const o = obj as Record<string, unknown>;
if ("title" in o) {
if (typeof o.title !== "string") return false;
if (
(o.title as string).length === 0 ||
(o.title as string).length > VALIDATION.PLAYLIST_TITLE_MAX
) return false;
}
if (
"description" in o && typeof o.description !== "string" &&
o.description !== null
) return false;
if (
typeof o.description === "string" &&
(o.description as string).length > VALIDATION.PLAYLIST_DESCRIPTION_MAX
) return false;
if ("isPublic" in o && typeof o.isPublic !== "boolean") return false;
return true;
}
export function isReorderPlaylistRequest(
obj: unknown,
): obj is ReorderPlaylistRequest {
return !!obj && typeof obj === "object" &&
"dumpIds" in obj && Array.isArray(obj.dumpIds) &&
(obj.dumpIds as unknown[]).every((id) => typeof id === "string");
}
/**
* Request DTOs
*/
function isStringArray(value: unknown): value is string[] {
return Array.isArray(value) && value.every((v) => typeof v === "string");
}
export interface CreateUrlDumpRequest {
url: string;
/** Overrides the title scraped from the page — the poster edited the preview. */
title?: string;
comment?: string;
isPrivate?: boolean;
categoryIds?: string[];
}
/**
* An existing dump pointing at the same URL, as shown by the create form's
* duplicate hint. Display-only projection — see `findDumpsByUrl`.
*/
export interface DumpUrlMatch {
id: string;
slug?: string;
title: string;
username: string;
createdAt: Date;
voteCount: number;
commentCount: number;
}
export function isCreateUrlDumpRequest(
obj: unknown,
): obj is CreateUrlDumpRequest {
if (
!obj || typeof obj !== "object" ||
!("url" in obj) || typeof obj.url !== "string"
) return false;
const o = obj as Record<string, unknown>;
if (
"comment" in o && typeof o.comment !== "string" && o.comment !== null
) return false;
if (
typeof o.comment === "string" &&
(o.comment as string).length > VALIDATION.DUMP_COMMENT_MAX
) return false;
if ("title" in o && typeof o.title !== "string" && o.title !== null) {
return false;
}
if (
typeof o.title === "string" &&
(o.title as string).length > VALIDATION.DUMP_TITLE_MAX
) return false;
if ("isPrivate" in o && typeof o.isPrivate !== "boolean") return false;
if ("categoryIds" in o && !isStringArray(o.categoryIds)) return false;
return true;
}
export interface UpdateDumpRequest {
url?: string;
title?: string;
comment?: string;
isPrivate?: boolean;
categoryIds?: string[];
}
export function isUpdateDumpRequest(obj: unknown): obj is UpdateDumpRequest {
if (!obj || typeof obj !== "object") return false;
const o = obj as Record<string, unknown>;
if ("url" in o && typeof o.url !== "string" && o.url !== null) return false;
if ("title" in o) {
if (typeof o.title !== "string") return false;
const trimmed = (o.title as string).trim();
if (!trimmed || trimmed.length > VALIDATION.DUMP_TITLE_MAX) return false;
}
if (
"comment" in o && typeof o.comment !== "string" && o.comment !== null
) return false;
if (
typeof o.comment === "string" &&
(o.comment as string).length > VALIDATION.DUMP_COMMENT_MAX
) return false;
if ("isPrivate" in o && typeof o.isPrivate !== "boolean") return false;
if ("categoryIds" in o && !isStringArray(o.categoryIds)) return false;
return true;
}
/**
* WebSockets
*/
// ── Client → Server ──────────────────────────────────────────────────────────
export interface PingMessage {
type: "ping";
}
export interface PongMessage {
type: "pong";
}
export interface VoteCastMessage {
type: "vote_cast";
dumpId: string;
}
export interface VoteRemoveMessage {
type: "vote_remove";
dumpId: string;
}
export interface CommentLikeCastMessage {
type: "comment_like_cast";
commentId: string;
}
export interface CommentLikeRemoveMessage {
type: "comment_like_remove";
commentId: string;
}
export interface ChatSendMessage {
type: "chat_send";
body: string;
/** Id of the message being replied to, if this is a reply. */
replyToId?: string;
/** Client-chosen id echoed on the broadcast (or on the error), so the
* sender can match the outcome to its pending message. */
nonce?: string;
}
// Sent while the user is composing (true) and when they stop (false). The
// server rebroadcasts the live set of typers to everyone as ChatTypingUpdate.
export interface ChatTypingMessage {
type: "chat_typing";
typing: boolean;
}
// Tells the server whether this client currently has the chatbox open, so that
// mention notifications can be skipped for users who are already reading chat.
export interface ChatFocusMessage {
type: "chat_focus";
open: boolean;
}
export interface ChatEditMessage {
type: "chat_edit";
id: string;
body: string;
}
export interface ChatDeleteMessage {
type: "chat_delete";
id: string;
}
// The user has seen the chat up to the message created at `at`.
export interface ChatReadMessage {
type: "chat_read";
at: string;
}
export type ClientToServerMessage =
| PingMessage
| PongMessage
| VoteCastMessage
| VoteRemoveMessage
| CommentLikeCastMessage
| CommentLikeRemoveMessage
| ChatSendMessage
| ChatTypingMessage
| ChatFocusMessage
| ChatEditMessage
| ChatDeleteMessage
| ChatReadMessage;
// ── Server → Client ──────────────────────────────────────────────────────────
export interface OnlineUser {
userId: string;
username: string;
hasAvatar: boolean;
avatarVersion?: number;
}
export interface WelcomeMessage {
type: "welcome";
users: OnlineUser[];
myVotes: string[];
myCommentLikes: string[];
unreadNotificationCount: number;
/** The user's chat read marker and the unread count behind it (signed-in
* users only). */
chatReadAt?: string;
unreadChatCount: number;
}
export interface PresenceUpdateMessage {
type: "presence_update";
users: OnlineUser[];
}
export interface VotesUpdateMessage {
type: "votes_update";
dumpId: string;
voteCount: number;
voterId: string;
action: "cast" | "remove";
}
export interface VoteAckMessage {
type: "vote_ack";
dumpId: string;
action: "cast" | "remove";
voteCount: number;
}
export interface CommentLikesUpdateMessage {
type: "comment_likes_update";
commentId: string;
likeCount: number;
likerId: string;
action: "cast" | "remove";
}
export interface CommentLikeAckMessage {
type: "comment_like_ack";
commentId: string;
action: "cast" | "remove";
likeCount: number;
}
export interface DumpCreatedMessage {
type: "dump_created";
dump: Dump;
}
export interface DumpUpdatedMessage {
type: "dump_updated";
dump: Dump;
}
export interface DumpDeletedMessage {
type: "dump_deleted";
dumpId: string;
}
export interface PlaylistCreatedMessage {
type: "playlist_created";
playlist: Playlist;
}
export interface PlaylistUpdatedMessage {
type: "playlist_updated";
playlist: Playlist;
}
export interface PlaylistDeletedMessage {
type: "playlist_deleted";
playlistId: string;
userId: string;
}
export interface PlaylistDumpsUpdatedMessage {
type: "playlist_dumps_updated";
playlistId: string;
dumpIds: string[];
}
export interface UserUpdatedMessage {
type: "user_updated";
user: Omit<User, "passwordHash" | "email">;
}
export interface CommentCreatedMessage {
type: "comment_created";
comment: Comment;
}
export interface CommentUpdatedMessage {
type: "comment_updated";
comment: Comment;
}
export interface CommentDeletedMessage {
type: "comment_deleted";
commentId: string;
dumpId: string;
}
export interface NotificationCreatedMessage {
type: "notification_created";
notification: RawNotification;
}
export interface ErrorMessage {
type: "error";
message?: string;
// Context for optimistic-action failures so the client can revert the exact
// pending vote/like immediately instead of waiting for its ACK timeout.
dumpId?: string;
commentId?: string;
action?: "cast" | "remove";
/** Set when a chat action failed, so the client can surface it in the chat. */
scope?: "chat";
/** Echo of the failed chat_send's nonce. */
nonce?: string;
}
export interface ForceLogoutMessage {
type: "force_logout";
}
export interface ChatMessageMessage {
type: "chat_message";
message: ChatMessage;
/** The sender's nonce, so its pending copy can be retired. */
nonce?: string;
}
export interface ChatMessageUpdatedMessage {
type: "chat_message_updated";
message: ChatMessage;
}
export interface ChatMessageDeletedMessage {
type: "chat_message_deleted";
id: string;
}
// The current set of users composing a chat message. Reuses OnlineUser so the
// client can render the same avatars as the presence row.
export interface ChatTypingUpdateMessage {
type: "chat_typing_update";
users: OnlineUser[];
}
// The user's read marker moved (from any of their tabs or devices).
export interface ChatReadUpdateMessage {
type: "chat_read_update";
readAt: string;
unread: number;
}
export type ServerToClientMessage =
| PingMessage
| WelcomeMessage
| PresenceUpdateMessage
| VotesUpdateMessage
| VoteAckMessage
| CommentLikesUpdateMessage
| CommentLikeAckMessage
| DumpCreatedMessage
| DumpUpdatedMessage
| DumpDeletedMessage
| PlaylistCreatedMessage
| PlaylistUpdatedMessage
| PlaylistDeletedMessage
| PlaylistDumpsUpdatedMessage
| UserUpdatedMessage
| CommentCreatedMessage
| CommentUpdatedMessage
| CommentDeletedMessage
| NotificationCreatedMessage
| ErrorMessage
| ForceLogoutMessage
| ChatMessageMessage
| ChatMessageUpdatedMessage
| ChatMessageDeletedMessage
| ChatTypingUpdateMessage
| ChatReadUpdateMessage;
/**
* Follows
*/
export interface FollowStatus {
followedUserIds: string[];
followedPlaylistIds: string[];
}
/**
* Notifications
*/
export type NotificationType =
| "playlist_followed"
| "user_followed"
| "user_dump_posted"
| "playlist_dump_added"
| "dump_upvoted"
| "user_mentioned"
| "dump_commented"
| "comment_liked";
export interface PlaylistFollowedData {
followerId: string;
followerUsername: string;
playlistId: string;
playlistTitle: string;
}
export interface UserFollowedData {
followerId: string;
followerUsername: string;
}
export interface UserDumpPostedData {
dumperId: string;
dumperUsername: string;
dumpId: string;
dumpTitle: string;
}
export interface PlaylistDumpAddedData {
dumpId: string;
dumpTitle: string;
playlistId: string;
playlistTitle: string;
}
export interface DumpUpvotedData {
voterId: string;
voterUsername: string;
dumpId: string;
dumpTitle: string;
}
export interface UserMentionedData {
mentionerId: string;
mentionerUsername: string;
contextType: "comment" | "dump" | "playlist" | "chat";
contextId: string;
contextTitle: string;
dumpId?: string;
}
export interface DumpCommentedData {
commenterId: string;
commenterUsername: string;
commentId: string;
dumpId: string;
dumpTitle: string;
}
export interface CommentLikedData {
likerId: string;
likerUsername: string;
commentId: string;
dumpId: string;
dumpTitle: string;
}
export type NotificationData =
| PlaylistFollowedData
| UserFollowedData
| UserDumpPostedData
| PlaylistDumpAddedData
| DumpUpvotedData
| UserMentionedData
| DumpCommentedData
| CommentLikedData;
export interface Notification {
id: string;
userId: string;
type: NotificationType;
data: NotificationData;
read: boolean;
createdAt: Date;
}
/** Wire format — createdAt arrives as an ISO string over JSON. */
export type RawNotification = Omit<Notification, "createdAt"> & {
createdAt: string;
};

39
api/model/invite.ts Normal file
View File

@@ -0,0 +1,39 @@
/**
* Checks the rows `invite-service.ts` reads from `invites` to vet an invite
* before sign-up.
*
* @module
*/
import type { SQLOutputValue } from "node:sqlite";
/**
* Stored by `createInvite`, and read back by `validateInvite` when the invite
* link is opened or used to sign up.
*/
export interface InviteRow {
/** The random token the invite link carries, which is the primary key. */
token: string;
/** Becomes the new user's `invited_by`. */
inviter_id: string;
/** Set when an account is created with the invite. */
used_at: string | null;
/** Starts the 7-day window in which the invite works. */
created_at: string;
[key: string]: SQLOutputValue;
}
/**
* Checks whether `obj`, a row `validateInvite` selected, is an
* {@link InviteRow}.
*/
export function isInviteRow(
obj: unknown,
): obj is InviteRow {
return !!obj && typeof obj === "object" &&
"token" in obj && typeof obj.token === "string" &&
"inviter_id" in obj && typeof obj.inviter_id === "string" &&
"created_at" in obj && typeof obj.created_at === "string" &&
"used_at" in obj &&
(obj.used_at === null || typeof obj.used_at === "string");
}

70
api/model/notification.ts Normal file
View File

@@ -0,0 +1,70 @@
/**
* Turns the rows of `notifications`, once checked, into the {@link
* Notification} that `GET /api/me/notifications` answers.
*
* @module
*/
import type { SQLOutputValue } from "node:sqlite";
import type {
Notification,
NotificationData,
NotificationType,
} from "../../shared/notification.ts";
/**
* Stored by the triggers of `notification-service.ts`, and read back by
* `getNotificationsForUser`.
*/
export interface NotificationRow {
id: string;
user_id: string;
/** A {@link NotificationType}, not checked by the guard. */
type: string;
/** The {@link NotificationData} as JSON, not checked by the guard. */
data: string;
/** 0 or 1. */
read: number;
created_at: string;
/**
* Names the event that caused the notification. A unique index keeps a
* second row with the same `(user_id, source_key)` out; null allows
* duplicates.
*/
source_key: string | null;
[key: string]: SQLOutputValue;
}
/**
* Checks whether `obj`, a row `getNotificationsForUser` selected, is a
* {@link NotificationRow}.
*/
export function isNotificationRow(
obj: unknown,
): obj is NotificationRow {
return !!obj && typeof obj === "object" &&
"id" in obj && typeof obj.id === "string" &&
"user_id" in obj && typeof obj.user_id === "string" &&
"type" in obj && typeof obj.type === "string" &&
"data" in obj && typeof obj.data === "string" &&
"read" in obj && typeof obj.read === "number" &&
"created_at" in obj && typeof obj.created_at === "string" &&
"source_key" in obj &&
(typeof obj.source_key === "string" || obj.source_key === null);
}
/**
* Converts a row to the {@link Notification} the client gets.
*
* @throws {SyntaxError} When `data` isn't valid JSON.
*/
export function notificationRowToApi(row: NotificationRow): Notification {
return {
id: row.id,
userId: row.user_id,
type: row.type as NotificationType,
data: JSON.parse(row.data) as NotificationData,
read: Boolean(row.read),
createdAt: new Date(row.created_at),
};
}

41
api/model/password-reset.ts Normal file
View File

@@ -0,0 +1,41 @@
/**
* Checks the rows `password-reset-service.ts` reads from
* `password_reset_tokens` to make each reset link work once.
*
* @module
*/
import type { SQLOutputValue } from "node:sqlite";
/**
* Stored by `requestPasswordReset` when it emails a reset link, and read back
* by `resetPassword` when the link is used.
*/
export interface PasswordResetTokenRow {
/** The signed token itself, which the reset link carries. */
token: string;
user_id: string;
/**
* Lets `db/startup.ts` purge old rows. The token's own `exp` is what ends
* the link.
*/
expires_at: string;
/** Set when the link is used, and reset to null if the reset then fails. */
used_at: string | null;
[key: string]: SQLOutputValue;
}
/**
* Checks whether `obj`, a row `resetPassword` selected, is a
* {@link PasswordResetTokenRow}.
*/
export function isPasswordResetTokenRow(
obj: unknown,
): obj is PasswordResetTokenRow {
return !!obj && typeof obj === "object" &&
"token" in obj && typeof obj.token === "string" &&
"user_id" in obj && typeof obj.user_id === "string" &&
"expires_at" in obj && typeof obj.expires_at === "string" &&
"used_at" in obj &&
(obj.used_at === null || typeof obj.used_at === "string");
}

70
api/model/playlist.ts Normal file
View File

@@ -0,0 +1,70 @@
/**
* Turns the rows the playlist and follow services read from `playlists` into
* the {@link Playlist} the API sends.
*
* @module
*/
import type { SQLOutputValue } from "node:sqlite";
import type { Playlist } from "../../shared/playlist.ts";
/**
* Returned by the playlist and follow services' `SELECT`s on `playlists`,
* which join the owner's name as `owner_username`. A query may add the
* `dump_count` and `last_dump_at` columns (`VISIBLE_DUMP_FIGURES`), which
* {@link playlistRowToApi} carries over when present.
*/
export interface PlaylistRow {
id: string;
user_id: string;
title: string;
/** Unique among its owner's playlists, and set on every row since 0021. */
slug: string;
owner_username: string;
description: string | null;
is_public: number;
created_at: string;
updated_at: string | null;
image_mime: string | null;
[key: string]: SQLOutputValue;
}
/** Checks whether a row `node:sqlite` returned is a {@link PlaylistRow}. */
export function isPlaylistRow(
obj: unknown,
): obj is PlaylistRow {
return !!obj && typeof obj === "object" &&
"id" in obj && typeof obj.id === "string" &&
"user_id" in obj && typeof obj.user_id === "string" &&
"title" in obj && typeof obj.title === "string" &&
"slug" in obj && typeof obj.slug === "string" &&
"owner_username" in obj && typeof obj.owner_username === "string" &&
"description" in obj &&
(typeof obj.description === "string" || obj.description === null) &&
"is_public" in obj && typeof obj.is_public === "number" &&
"created_at" in obj && typeof obj.created_at === "string" &&
"updated_at" in obj &&
(typeof obj.updated_at === "string" || obj.updated_at === null) &&
"image_mime" in obj &&
(typeof obj.image_mime === "string" || obj.image_mime === null);
}
/** Converts `row` to the {@link Playlist} the playlist routes send. */
export function playlistRowToApi(row: PlaylistRow): Playlist {
return {
id: row.id,
userId: row.user_id,
title: row.title,
slug: row.slug,
description: row.description ?? undefined,
isPublic: Boolean(row.is_public),
createdAt: new Date(row.created_at),
updatedAt: row.updated_at ? new Date(row.updated_at) : undefined,
imageMime: row.image_mime ?? undefined,
dumpCount: typeof row.dump_count === "number" ? row.dump_count : undefined,
lastDumpAt: typeof row.last_dump_at === "string"
? new Date(row.last_dump_at)
: undefined,
ownerUsername: row.owner_username,
};
}

24
api/model/rows.ts Normal file
View File

@@ -0,0 +1,24 @@
/**
* Makes a damaged row fail its query with a 500 rather than silently vanish
* from a list. Every service checks the rows of its list queries with it.
*
* @module
*/
import { APIException } from "../lib/api-exception.ts";
/**
* Returns `rows` turned by `toApi`, once `isRow` has accepted each.
*
* @throws {APIException} 500 when a row doesn't pass `isRow`.
*/
export function parseRows<R, T>(
rows: unknown[],
isRow: (row: unknown) => row is R,
toApi: (row: R) => T,
): T[] {
if (!rows.every(isRow)) {
throw new APIException(500, "Malformed data in the database");
}
return rows.map(toApi);
}

66
api/model/settings.ts Normal file
View File

@@ -0,0 +1,66 @@
/**
* Checks the rows `settings-service.ts` reads from `user_settings`, and turns
* them into the {@link UserSettings} the settings endpoints answer.
*
* @module
*/
import type { SQLOutputValue } from "node:sqlite";
import type { UserSettings } from "../../shared/settings.ts";
import { isColorScheme, isStyleName } from "../../shared/theme.ts";
/**
* Stored by `updateSettings` and read back by `getSettings`. A user without a
* row has the defaults.
*/
export interface UserSettingsRow {
user_id: string;
/** A style name, or null for the instance's. */
style: string | null;
color_scheme: string | null;
/** 0 or 1. */
digest_weekly: number;
/** 0 or 1. */
digest_monthly: number;
updated_at: string;
[key: string]: SQLOutputValue;
}
/**
* Names the column of each setting, for the queries that pick users by one,
* as `digest-service.ts` does by its digest's switch.
*/
export const SETTINGS_COLUMNS: Record<keyof UserSettings, string> = {
style: "style",
colorScheme: "color_scheme",
digestWeekly: "digest_weekly",
digestMonthly: "digest_monthly",
};
/**
* Checks whether `obj`, a row `getSettings` selected, is a
* {@link UserSettingsRow}.
*/
export function isUserSettingsRow(obj: unknown): obj is UserSettingsRow {
return !!obj && typeof obj === "object" &&
"user_id" in obj && typeof obj.user_id === "string" &&
"style" in obj && (obj.style === null || typeof obj.style === "string") &&
"color_scheme" in obj &&
(obj.color_scheme === null || typeof obj.color_scheme === "string") &&
"digest_weekly" in obj && typeof obj.digest_weekly === "number" &&
"digest_monthly" in obj && typeof obj.digest_monthly === "number" &&
"updated_at" in obj && typeof obj.updated_at === "string";
}
/**
* Converts a row to the {@link UserSettings} the client gets. A style or
* scheme the code no longer knows, as after one is removed, reads as unset.
*/
export function settingsRowToApi(row: UserSettingsRow): UserSettings {
return {
style: isStyleName(row.style) ? row.style : null,
colorScheme: isColorScheme(row.color_scheme) ? row.color_scheme : null,
digestWeekly: row.digest_weekly === 1,
digestMonthly: row.digest_monthly === 1,
};
}

146
api/model/user.ts Normal file
View File

@@ -0,0 +1,146 @@
/**
* Turns the rows of `users`, once checked, into the {@link User} that
* `user-service.ts` returns. A route turns a `User` into what it may send with
* {@link toPublicUser} or {@link toSelfUser}.
*
* @module
*/
import type { SQLOutputValue } from "node:sqlite";
import {
isRole,
type PublicUser,
type Role,
type SelfUser,
} from "../../shared/user.ts";
/**
* Returned by the lookups of `user-service.ts`. It must never be sent as is:
* it carries the password hash.
*/
export interface User extends SelfUser {
/**
* From `hashPassword`, as `hash.salt`, or null for an account from another
* instance ({@link isRemoteUser}), which signs in there and never with a
* password here. The row stores that as `''`.
*/
passwordHash: string | null;
}
/**
* Checks whether `user`'s account lives on another instance, whose user signs
* in here through federation (`federation-service.ts`). Their password,
* email, avatar and name are that instance's to change, so the routes that
* change them here refuse.
*/
export function isRemoteUser(user: User): boolean {
return user.passwordHash === null;
}
/**
* Returns what anyone may see of `user`, without the hash and the email. Every
* route that sends another user's profile goes through it.
*/
export function toPublicUser(
{ passwordHash: _, email: _email, ...user }: User,
): PublicUser {
return user;
}
/**
* Returns what `user` sees of themselves, without the hash, for the routes
* that answer the caller's own account.
*/
export function toSelfUser({ passwordHash: _, ...user }: User): SelfUser {
return user;
}
/**
* Selected by `user-service.ts` and `follow-service.ts`, and converted by
* {@link userRowToApi}. It leaves out `chat_read_at` and `tokens_valid_after`,
* which only the chat's read state and `getSessionState` query.
*/
export interface UserRow {
id: string;
username: string;
/** `''` for an account from another instance, which has no password. */
password_hash: string;
role: Role;
created_at: string;
/** Bumped by any profile or avatar change. */
updated_at: string | null;
/** Null without an avatar. */
avatar_mime: string | null;
/** Markdown. */
description: string | null;
/** Null for the seeded admin, who had no inviter. */
invited_by: string | null;
/**
* Joined from the inviter's row rather than stored, and undefined when the
* query doesn't join it. The guard doesn't check it.
*/
invited_by_username: string | null;
email: string;
[key: string]: SQLOutputValue;
}
/**
* Checks whether `obj`, a row selected with the columns of {@link UserRow}, is
* one.
*/
export function isUserRow(obj: unknown): obj is UserRow {
return !!obj &&
typeof obj === "object" &&
"id" in obj && typeof obj.id === "string" &&
"username" in obj && typeof obj.username === "string" &&
"password_hash" in obj && typeof obj.password_hash === "string" &&
"role" in obj && isRole(obj.role) &&
"created_at" in obj && typeof obj.created_at === "string" &&
"updated_at" in obj &&
(typeof obj.updated_at === "string" || obj.updated_at === null) &&
"avatar_mime" in obj &&
(typeof obj.avatar_mime === "string" || obj.avatar_mime === null) &&
"description" in obj &&
(typeof obj.description === "string" || obj.description === null) &&
"invited_by" in obj &&
(typeof obj.invited_by === "string" || obj.invited_by === null) &&
"email" in obj && typeof obj.email === "string";
}
/** Converts a row to a {@link User}, turning nulls into undefined. */
export function userRowToApi(row: UserRow): User {
return {
id: row.id,
username: row.username,
passwordHash: row.password_hash || null,
role: row.role,
createdAt: new Date(row.created_at),
updatedAt: row.updated_at ? new Date(row.updated_at) : undefined,
avatarMime: row.avatar_mime ?? undefined,
description: row.description ?? undefined,
invitedByUsername: typeof row.invited_by_username === "string"
? row.invited_by_username
: undefined,
email: row.email,
};
}
/**
* Converts a {@link User} back to a row, for `updateUser`. A `User` has no
* inviter id, so `invited_by` comes out null: don't write it back.
*/
export function userApiToRow(user: User): UserRow {
return {
id: user.id,
username: user.username,
password_hash: user.passwordHash ?? "",
role: user.role,
created_at: user.createdAt.toISOString(),
updated_at: user.updatedAt?.toISOString() ?? null,
avatar_mime: user.avatarMime ?? null,
description: user.description ?? null,
invited_by: null,
invited_by_username: null,
email: user.email,
};
}

97
api/remote/bandcamp-streams.ts Normal file
View File

@@ -0,0 +1,97 @@
/**
* Resolves a Bandcamp page to its streamable tracks when the native player
* starts it, for `GET /api/remote/bandcamp`. The stream URLs are signed and
* expire 24 hours after the page is fetched, so unlike the embed URL they
* can't be stored in the dump's preview. The browser plays them from bcbits
* directly.
*
* @module
*/
import { APIException } from "../lib/api-exception.ts";
import { TtlCache } from "../lib/ttl-cache.ts";
import { fetchWithTimeout, isValidHttpUrl, readTextCapped } from "./fetch.ts";
import { BANDCAMP_HOST_RE, parseTralbum } from "./bandcamp-tralbum.ts";
import type { Tralbum } from "../../shared/player.ts";
/**
* Lives half the signatures' life, so a cached URL still has at least 12
* hours left when it is handed out.
*/
const cache = new TtlCache<Tralbum>(12 * 3_600_000, 200);
/**
* Returns `url` without its query, fragment and trailing slashes, and with its
* host in lowercase.
*/
function cacheKey(url: string): string {
try {
const u = new URL(url);
return `${u.protocol}//${u.hostname.toLowerCase()}${
u.pathname.replace(/\/+$/, "")
}`;
} catch {
return url;
}
}
/**
* Returns the tracklist of the Bandcamp page `pageUrl`, from the cache when it
* was resolved in the last 12 hours. `force` skips the cache: the player sets
* it when it re-resolves a queue whose URLs expired or are about to, which a
* cached entry could hand back.
*
* @throws {APIException} 400 when `pageUrl` isn't a public bandcamp.com URL,
* 502 when Bandcamp can't be reached or doesn't answer with a page, 404 when
* the page has nothing streamable.
*/
export async function resolveBandcamp(
pageUrl: string,
{ force = false }: { force?: boolean } = {},
): Promise<Tralbum> {
if (!isValidHttpUrl(pageUrl)) {
throw new APIException(400, "Invalid URL");
}
let hostname: string;
try {
hostname = new URL(pageUrl).hostname;
} catch {
throw new APIException(400, "Invalid URL");
}
if (!BANDCAMP_HOST_RE.test(hostname)) {
throw new APIException(400, "Not a Bandcamp URL");
}
const key = cacheKey(pageUrl);
if (!force) {
const hit = cache.get(key);
if (hit) return hit;
}
let res: Response;
try {
res = await fetchWithTimeout(pageUrl, 8000);
} catch {
throw new APIException(502, "Could not reach Bandcamp");
}
// A body left unread keeps its connection open until GC, and this endpoint
// is public: every early return cancels it.
if (!res.ok) {
await res.body?.cancel();
throw new APIException(502, `Bandcamp returned ${res.status}`);
}
if (!(res.headers.get("content-type") ?? "").startsWith("text/html")) {
await res.body?.cancel();
throw new APIException(502, "Bandcamp did not return a page");
}
const parsed = parseTralbum(await readTextCapped(res), pageUrl);
if (!parsed) {
// No data, no audio or nothing streamable yet (a preorder). The client
// plays the embed instead.
throw new APIException(404, "No streamable tracks on this page");
}
return cache.set(key, parsed);
}

92
api/remote/bandcamp-tralbum.ts Normal file
View File

@@ -0,0 +1,92 @@
/**
* Reads a release out of a Bandcamp page, for `bandcamp-streams.ts`. Every
* track and album page carries the whole release, stream URLs included, as
* HTML-escaped JSON in a `data-tralbum` attribute, so one fetch covers an
* album. The format is undocumented, so every field is treated as optional.
*
* @module
*/
import type { Tralbum, TralbumTrack } from "../../shared/player.ts";
import { decodeHtmlEntities } from "./html.ts";
/**
* Matches the hostnames `bandcamp-streams.ts` accepts: bandcamp.com and its
* subdomains, and not `bandcamp.com.example.org`. An artist on a custom domain
* doesn't match, and the player falls back to the embed.
*/
export const BANDCAMP_HOST_RE = /(?:^|\.)bandcamp\.com$/;
/** Returns the URL of the artwork `artId`; undefined when it isn't numeric. */
function artworkUrl(artId: unknown): string | undefined {
if (typeof artId !== "number" && typeof artId !== "string") return undefined;
const id = String(artId).trim();
if (!/^\d+$/.test(id)) return undefined;
// _16 is the large square art Bandcamp's own player uses.
return `https://f4.bcbits.com/img/a${id}_16.jpg`;
}
/**
* Returns the release on the Bandcamp page `html`, fetched from `sourceUrl`;
* null when the page has no release data, no audio or no streamable track.
* Stream URLs outside bcbits.com are dropped.
*/
export function parseTralbum(html: string, sourceUrl: string): Tralbum | null {
const match = html.match(/\bdata-tralbum=(["'])([\s\S]*?)\1/);
if (!match) return null;
let raw: Record<string, unknown>;
try {
raw = JSON.parse(decodeHtmlEntities(match[2]));
} catch {
return null;
}
if (!raw || typeof raw !== "object") return null;
if (raw.hasAudio === false) return null;
const rawTracks = Array.isArray(raw.trackinfo) ? raw.trackinfo : [];
const tracks: TralbumTrack[] = rawTracks.map((entry, index) => {
const t = (entry ?? {}) as Record<string, unknown>;
const file = (t.file ?? {}) as Record<string, unknown>;
const mp3 = typeof file["mp3-128"] === "string" ? file["mp3-128"] : null;
// A hostile page could put any host here, and the endpoint would hand it
// to the browser.
const streamUrl = mp3 && isBcbitsUrl(mp3) ? mp3 : null;
return {
index,
trackNum: typeof t.track_num === "number" ? t.track_num : undefined,
title: typeof t.title === "string" && t.title.trim()
? t.title
: `Track ${index + 1}`,
duration: typeof t.duration === "number" && t.duration > 0
? t.duration
: undefined,
streamUrl,
streamable: streamUrl !== null && t.streaming === 1 &&
t.unreleased_track !== true && !t.encoding_pending,
capped: t.is_capped === true,
};
});
if (!tracks.some((t) => t.streamable)) return null;
const current = (raw.current ?? {}) as Record<string, unknown>;
return {
sourceUrl,
itemType: raw.item_type === "album" ? "album" : "track",
title: typeof current.title === "string" ? current.title : undefined,
artist: typeof raw.artist === "string" ? raw.artist : undefined,
artworkUrl: artworkUrl(raw.art_id),
tracks,
resolvedAt: Date.now(),
};
}
function isBcbitsUrl(url: string): boolean {
try {
const { protocol, hostname } = new URL(url);
return protocol === "https:" && /(?:^|\.)bcbits\.com$/.test(hostname);
} catch {
return false;
}
}

332
api/remote/fetch.ts Normal file
View File

@@ -0,0 +1,332 @@
/**
* Fetches pages and images from other sites for link previews, the image proxy
* and the Bandcamp resolver, without letting a user-supplied URL reach the
* server's own network. Each hop must resolve to a public address, the whole
* exchange has a deadline, and pages are read up to a cap.
*
* @module
*/
import {
bareHostname,
isPrivateHostname,
resolvePublicAddress,
} from "./public-address.ts";
import { isSelfUrl } from "../lib/self-url.ts";
const FETCH_HEADERS = {
"User-Agent":
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36",
"Accept":
"text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8",
"Accept-Language": "fr-FR,fr;q=0.9,en-US;q=0.8,en;q=0.7",
};
const MAX_REDIRECTS = 5;
/**
* Fetches `url` with `curl --insecure`, for a site whose TLS certificate Deno
* rejects. Returns a Response carrying only the `Location` and `Content-Type`
* headers, or null when curl is missing or fails. The connection is pinned to
* `address`, the one {@link resolvePublicAddress} checked, and redirects come
* back unfollowed so that {@link fetchWithTimeout} checks each hop.
*/
async function fetchViaCurl(
url: URL,
address: string,
timeoutMs: number,
): Promise<Response | null> {
const tmpPath = await Deno.makeTempFile();
const port = url.port || (url.protocol === "https:" ? "443" : "80");
const pinned = address.includes(":") ? `[${address}]` : address;
try {
const { code, stdout } = await new Deno.Command("curl", {
args: [
"--silent",
"--insecure",
"--proto",
"=http,https",
"--resolve",
`${bareHostname(url)}:${port}:${pinned}`,
"--max-time",
String(Math.ceil(timeoutMs / 1000)),
// curl fails on a larger file rather than stopping at a cap, so this is
// looser than MAX_PAGE_BYTES, where readTextCapped still cuts.
"--max-filesize",
String(2 * MAX_PAGE_BYTES),
"--user-agent",
FETCH_HEADERS["User-Agent"],
"--header",
`Accept: ${FETCH_HEADERS["Accept"]}`,
"--header",
`Accept-Language: ${FETCH_HEADERS["Accept-Language"]}`,
"--output",
tmpPath,
"--write-out",
"%{http_code}\n%{redirect_url}\n%{content_type}",
url.href,
],
stdout: "piped",
stderr: "null",
}).output();
if (code !== 0) return null;
const [status, location, contentType = ""] = new TextDecoder()
.decode(stdout).split("\n").map((l) => l.trim());
const statusCode = Number(status);
if (!(statusCode >= 200 && statusCode <= 599)) return null;
const headers = new Headers();
if (location) headers.set("location", location);
if (contentType) headers.set("content-type", contentType);
const bytes = await Deno.readFile(tmpPath);
return new Response(bytes, { status: statusCode, headers });
} catch {
return null;
} finally {
await Deno.remove(tmpPath).catch(() => {});
}
}
/**
* The default cap of {@link readTextCapped}. The metadata previews read is in
* the `<head>`, and a page past this is a download.
*/
export const MAX_PAGE_BYTES = 4 * 1024 * 1024;
/**
* Wraps `body` so that the request's deadline `timer` is cleared only once the
* body is read to the end or cancelled. A server that answers at once and then
* drips its body is still cut off.
*/
function withDeadline(
body: ReadableStream<Uint8Array>,
timer: ReturnType<typeof setTimeout>,
): ReadableStream<Uint8Array> {
const reader = body.getReader();
return new ReadableStream({
async pull(controller) {
try {
const { done, value } = await reader.read();
if (done) {
clearTimeout(timer);
controller.close();
} else {
controller.enqueue(value);
}
} catch (err) {
clearTimeout(timer);
controller.error(err);
}
},
cancel(reason) {
clearTimeout(timer);
return reader.cancel(reason);
},
});
}
/**
* Reads the body of `res` up to `maxBytes`; the rest is cancelled, never
* downloaded. The image proxy reads through it, and {@link readTextCapped}.
*/
export async function readBytesCapped(
res: Response,
maxBytes: number,
): Promise<Uint8Array> {
if (!res.body) return new Uint8Array();
const reader = res.body.getReader();
const chunks: Uint8Array[] = [];
let size = 0;
try {
while (size < maxBytes) {
const { done, value } = await reader.read();
if (done) break;
chunks.push(value);
size += value.byteLength;
}
} finally {
await reader.cancel().catch(() => {});
}
const bytes = new Uint8Array(Math.min(size, maxBytes));
let offset = 0;
for (const chunk of chunks) {
const part = chunk.subarray(0, bytes.byteLength - offset);
bytes.set(part, offset);
offset += part.byteLength;
if (offset >= bytes.byteLength) break;
}
return bytes;
}
/**
* Returns the charset a page declares, in its `Content-Type` header or else in
* a `<meta>` near its top, as HTML allows; UTF-8 when it declares none.
*/
function declaredCharset(contentType: string, bytes: Uint8Array): string {
const fromHeader = /charset=["']?([\w-]+)/i.exec(contentType)?.[1];
if (fromHeader) return fromHeader;
// Latin-1 maps every byte to a character, so the ASCII of the tag reads
// right whatever the page's encoding.
const head = new TextDecoder("latin1").decode(bytes.subarray(0, 1024));
return /<meta[^>]+charset=["']?([\w-]+)/i.exec(head)?.[1] ?? "utf-8";
}
/**
* Reads the body of `res` as text, up to `maxBytes`, in the charset the page
* declares; the rest is cancelled, never downloaded. {@link fetchPage} and the
* Bandcamp and YouTube resolvers read pages through it.
*/
export async function readTextCapped(
res: Response,
maxBytes = MAX_PAGE_BYTES,
): Promise<string> {
const bytes = await readBytesCapped(res, maxBytes);
const charset = declaredCharset(res.headers.get("content-type") ?? "", bytes);
try {
return new TextDecoder(charset).decode(bytes);
} catch {
// TextDecoder throws on a label it doesn't know rather than guess.
return new TextDecoder().decode(bytes);
}
}
/**
* Makes one request to `url`, following no redirects, with a deadline that
* covers the body too.
*/
async function fetchOnce(
url: URL,
address: string,
timeoutMs: number,
): Promise<Response> {
const controller = new AbortController();
const timer = setTimeout(() => controller.abort(), timeoutMs);
let res: Response;
try {
res = await fetch(url, {
signal: controller.signal,
headers: FETCH_HEADERS,
redirect: "manual",
});
} catch (err) {
clearTimeout(timer);
if (!(err instanceof TypeError && err.message.includes("certificate"))) {
throw err;
}
// Deno can't be told to skip certificate checks for one request, so a
// site with an expired, self-signed or unusually signed certificate is
// fetched with curl instead.
const curlRes = await fetchViaCurl(url, address, timeoutMs);
if (curlRes) return curlRes;
throw err;
}
if (!res.body) {
clearTimeout(timer);
return res;
}
return new Response(withDeadline(res.body, timer), {
status: res.status,
statusText: res.statusText,
headers: res.headers,
});
}
/**
* Fetches `url` as {@link fetchWithTimeout} does, and also returns the URL the
* last hop answered from, which {@link fetchPage} resolves the page's links
* against.
*/
async function fetchFollowing(
url: string,
timeoutMs: number,
): Promise<{ res: Response; url: URL }> {
let current = new URL(url);
for (let hop = 0;; hop++) {
if (current.protocol !== "http:" && current.protocol !== "https:") {
throw new TypeError(`Unsupported protocol ${current.protocol}`);
}
const address = await resolvePublicAddress(current);
const res = await fetchOnce(current, address, timeoutMs);
const location = res.headers.get("location");
if (res.status < 300 || res.status >= 400 || !location) {
return { res, url: current };
}
await res.body?.cancel();
if (hop >= MAX_REDIRECTS) throw new TypeError("Too many redirects");
current = new URL(location, current);
}
}
/**
* Fetches a user-supplied URL, following up to five redirects, each of which
* must resolve to a public address too. `timeoutMs` bounds each hop, body
* included. The caller must read or cancel the body, or its connection stays
* open.
*
* @throws {TypeError} When a hop isn't http(s), there are too many redirects,
* or the request fails.
* @throws {Error} When a hop's host is private or can't be resolved.
*/
export async function fetchWithTimeout(
url: string,
timeoutMs = 5000,
): Promise<Response> {
return (await fetchFollowing(url, timeoutMs)).res;
}
/**
* Returned by {@link fetchPage} to the link preview providers.
*/
export interface FetchedPage {
/**
* The URL the page was served from, after redirects: the base its relative
* links resolve against.
*/
url: string;
/** The page's text, up to {@link MAX_PAGE_BYTES}; missing when it isn't HTML. */
html?: string;
}
/**
* Fetches a page for its link preview, through {@link fetchWithTimeout}. A
* response that isn't HTML is cancelled unread.
*
* @throws {Error} When the page answers with an error status, so that an error
* page never becomes a preview; and as {@link fetchWithTimeout} does.
*/
export async function fetchPage(url: string): Promise<FetchedPage> {
const { res, url: finalUrl } = await fetchFollowing(url, 5000);
if (!res.ok) {
await res.body?.cancel();
throw new Error(`${finalUrl} answered ${res.status}`);
}
const contentType = res.headers.get("content-type") ?? "";
if (!/^(text\/html|application\/xhtml\+xml)\b/i.test(contentType)) {
await res.body?.cancel();
return { url: finalUrl.href };
}
return { url: finalUrl.href, html: await readTextCapped(res) };
}
/**
* Checks whether `raw` is an http(s) URL on a host that isn't private, or a
* link to one of this site's own pages. Routes and services check a URL with it
* before storing, previewing or proxying it; it does no DNS lookup, which
* {@link fetchWithTimeout} does.
*/
export function isValidHttpUrl(raw: string): boolean {
try {
const u = new URL(raw);
if (u.protocol !== "http:" && u.protocol !== "https:") return false;
// This site may run on a private host (localhost in development), and the
// self provider reads its pages from the database without fetching them.
return !isPrivateHostname(u) || isSelfUrl(u);
} catch {
return false;
}
}

271
api/remote/gerbeur-instance.ts Normal file
View File

@@ -0,0 +1,271 @@
/**
* Talks to other gerbeur instances for federated sign-in: it reads their
* metadata and public keys, trades a code for an identity token, delivers
* events, and fetches a user's avatar. `federation-service.ts`,
* `identity-provider-service.ts` and `federation-peer-service.ts` reach other
* instances only through it.
*
* An identity is only as safe as the connection it came over, so unlike
* `fetch.ts` it never falls back to skipping certificate checks, and follows
* no redirect. It reaches only https instances on public addresses, unless
* `GERBEUR_FEDERATION_ALLOW_PRIVATE` lets two instances on one machine talk.
*
* @module
*/
import type { JSONWebKeySet } from "@panva/jose";
import {
type InstanceMetadata,
isInstanceMetadata,
} from "../../shared/federation.ts";
import {
FEDERATION_ALLOW_PRIVATE,
MAX_IMAGE_SIZE_BYTES,
OG_SITE_NAME,
} from "../config.ts";
import { resolvePublicAddress } from "./public-address.ts";
import { readBytesCapped, readTextCapped } from "./fetch.ts";
import { TtlCache } from "../lib/ttl-cache.ts";
/** How long a request to another instance may take, body included. */
const TIMEOUT_MS = 8000;
/** Caps a metadata document, a key set or a token response. */
const MAX_JSON_BYTES = 64 * 1024;
const USER_AGENT = `gerbeur (${OG_SITE_NAME}; federation)`;
/**
* Keeps each instance's metadata and keys for 10 minutes, so that a sign-in
* or an event doesn't fetch them each time. A key set is refetched sooner
* when a token names a key it lacks ({@link fetchKeys} with `fresh`), at most
* once a minute: anyone can send a token naming a key that doesn't exist.
*/
const metadataCache = new TtlCache<InstanceMetadata>(10 * 60_000, 500);
const keysCache = new TtlCache<JSONWebKeySet>(10 * 60_000, 500);
const keysRefetched = new TtlCache<true>(60_000, 500);
/** Keeps which issuer an instance's pages belong to ({@link resolveIssuer}). */
const issuerCache = new TtlCache<string>(10 * 60_000, 500);
/**
* Checks that `url` may be reached: https on a public address, or anything
* under `GERBEUR_FEDERATION_ALLOW_PRIVATE`.
*
* @throws {Error} When it may not.
*/
export async function assertReachable(url: URL): Promise<void> {
if (FEDERATION_ALLOW_PRIVATE) {
if (url.protocol !== "https:" && url.protocol !== "http:") {
throw new Error(`Unsupported protocol ${url.protocol}`);
}
return;
}
if (url.protocol !== "https:") {
throw new Error(`Refusing to reach ${url.origin} without https`);
}
await resolvePublicAddress(url);
}
/**
* Makes one request to another instance, after {@link assertReachable}, with
* the deadline covering the body. The caller reads or cancels the body.
*/
async function request(url: string, init: RequestInit = {}): Promise<Response> {
const target = new URL(url);
await assertReachable(target);
return await fetch(target, {
...init,
redirect: "error",
signal: AbortSignal.timeout(TIMEOUT_MS),
headers: {
"User-Agent": USER_AGENT,
"Accept": "application/json",
...init.headers,
},
});
}
/**
* Returns the JSON body of `res`, up to {@link MAX_JSON_BYTES}.
*
* @throws {Error} When the status isn't a success or the body isn't JSON.
*/
async function readJson(res: Response, what: string): Promise<unknown> {
if (!res.ok) {
await res.body?.cancel();
throw new Error(`${what} answered ${res.status}`);
}
return JSON.parse(await readTextCapped(res, MAX_JSON_BYTES));
}
/**
* Returns the metadata `origin` serves at `/.well-known/openid-configuration`,
* whichever issuer it names.
*
* @throws {Error} When it can't be reached or isn't a gerbeur instance's.
*/
async function metadataAt(origin: string): Promise<InstanceMetadata> {
const body = await readJson(
await request(`${origin}/.well-known/openid-configuration`),
origin,
);
if (!isInstanceMetadata(body)) {
throw new Error(`${origin} is not a gerbeur instance this one can trust`);
}
return body;
}
/**
* Returns the metadata of the instance at `origin`, its issuer, cached. It
* must name `origin` as its issuer and speak this protocol's version.
*
* @throws {Error} When it can't be reached or isn't a gerbeur instance.
*/
export async function fetchMetadata(origin: string): Promise<InstanceMetadata> {
const hit = metadataCache.get(origin);
if (hit) return hit;
const body = await metadataAt(origin);
if (body.issuer !== origin) {
throw new Error(`${origin} names itself ${body.issuer}`);
}
return metadataCache.set(origin, body);
}
/**
* Returns the issuer of the instance at `origin`, cached: `origin` itself, or,
* when `origin` is where the instance's pages are and its API lives at
* another, the issuer the pages' metadata names, whose own metadata must name
* those pages back as where its `/authorize` is. People know an instance by
* the address in their browser, which a sign-in form or the trust list may be
* given; a handle and the stored identities name the issuer.
*
* @throws {Error} When `origin` can't be reached, isn't a gerbeur instance,
* or names an issuer that doesn't name it back.
*/
export async function resolveIssuer(origin: string): Promise<string> {
const hit = issuerCache.get(origin);
if (hit) return hit;
if (metadataCache.get(origin)) return origin;
const body = await metadataAt(origin);
if (body.issuer === origin) {
metadataCache.set(origin, body);
return issuerCache.set(origin, origin);
}
const pages =
new URL((await fetchMetadata(body.issuer)).authorization_endpoint)
.origin;
if (pages !== origin) {
throw new Error(`${origin} names ${body.issuer}, whose pages are ${pages}`);
}
return issuerCache.set(origin, body.issuer);
}
/**
* Returns the public keys of the instance at `origin`, cached unless `fresh`
* asks to refetch them and they weren't in the last minute.
*
* @throws {Error} When they can't be fetched.
*/
export async function fetchKeys(
origin: string,
fresh = false,
): Promise<JSONWebKeySet> {
const refetch = fresh && !keysRefetched.get(origin);
const hit = refetch ? undefined : keysCache.get(origin);
if (hit) return hit;
if (refetch) keysRefetched.set(origin, true);
const { jwks_uri } = await fetchMetadata(origin);
const body = await readJson(await request(jwks_uri), `${origin}'s keys`);
if (
!body || typeof body !== "object" ||
!Array.isArray((body as { keys?: unknown }).keys)
) {
throw new Error(`${origin}'s keys are not a key set`);
}
return keysCache.set(origin, body as JSONWebKeySet);
}
/**
* Trades `code` for an identity token at the token endpoint of the instance
* at `origin`, and returns the token, unverified.
*
* @throws {Error} When the instance refuses or answers no token.
*/
export async function exchangeCode(
origin: string,
params: {
code: string;
clientId: string;
redirectUri: string;
codeVerifier: string;
},
): Promise<string> {
const { token_endpoint } = await fetchMetadata(origin);
const body = await readJson(
await request(token_endpoint, {
method: "POST",
headers: { "Content-Type": "application/x-www-form-urlencoded" },
body: new URLSearchParams({
grant_type: "authorization_code",
code: params.code,
client_id: params.clientId,
redirect_uri: params.redirectUri,
code_verifier: params.codeVerifier,
}),
}),
`${origin}'s token endpoint`,
);
const idToken = (body as { id_token?: unknown } | null)?.id_token;
if (typeof idToken !== "string") {
throw new Error(`${origin} answered no identity token`);
}
return idToken;
}
/**
* Sends the signed event `event` to the events endpoint of the instance at
* `origin`, and returns whether it took it. A 4xx other than 429 means it never
* will, which the caller treats as delivered.
*
* @throws {Error} When it can't be reached, or answers 429 or a 5xx.
*/
export async function postEvent(
origin: string,
event: string,
): Promise<boolean> {
const { gerbeur_events_endpoint } = await fetchMetadata(origin);
const res = await request(gerbeur_events_endpoint, {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ event }),
});
await res.body?.cancel();
if (res.ok) return true;
if (res.status === 429 || res.status >= 500) {
throw new Error(`${origin}'s events endpoint answered ${res.status}`);
}
return false;
}
/**
* Returns the image at `url`, a user's avatar on their home instance, up to
* the size an avatar may be.
*
* @throws {Error} When it can't be fetched.
*/
export async function fetchAvatar(url: string): Promise<Uint8Array> {
const res = await request(url, { headers: { "Accept": "image/*" } });
if (!res.ok) {
await res.body?.cancel();
throw new Error(`The avatar at ${url} answered ${res.status}`);
}
return await readBytesCapped(res, MAX_IMAGE_SIZE_BYTES + 1);
}
/** Empties the caches, for the tests' reset between cases. */
export function clearInstanceCaches(): void {
metadataCache.clear();
issuerCache.clear();
keysCache.clear();
keysRefetched.clear();
}

547
api/remote/html.ts Normal file
View File

@@ -0,0 +1,547 @@
/**
* Reads a link preview's metadata out of a page's HTML. {@link parsePage}
* scans the page once for the tags that carry it (`<meta>`, `<link>`, `<img>`,
* `<title>` and JSON-LD scripts) without building a document, and returns a
* {@link Page} that the providers' `genericPreview` picks from. Every field is
* undefined when the page doesn't say.
*
* @module
*/
import { hexOf, parseCssColor } from "../lib/css-color.ts";
/**
* The named entities decoded besides numeric ones: the XML five, and those
* page titles and descriptions actually use, Latin-1 letters and typography.
*/
const NAMED_ENTITIES: Record<string, string> = {
amp: "&",
lt: "<",
gt: ">",
quot: '"',
apos: "'",
nbsp: " ",
aelig: "æ",
AElig: "Æ",
oelig: "œ",
OElig: "Œ",
szlig: "ß",
oslash: "ø",
Oslash: "Ø",
eth: "ð",
ETH: "Ð",
thorn: "þ",
THORN: "Þ",
hellip: "…",
lsquo: "‘",
rsquo: "’",
sbquo: "‚",
ldquo: "“",
rdquo: "”",
bdquo: "„",
laquo: "«",
raquo: "»",
lsaquo: "‹",
rsaquo: "›",
ndash: "–",
mdash: "—",
bull: "•",
middot: "·",
dagger: "†",
Dagger: "‡",
prime: "′",
Prime: "″",
euro: "€",
pound: "£",
yen: "¥",
cent: "¢",
copy: "©",
reg: "®",
trade: "™",
deg: "°",
plusmn: "±",
times: "×",
divide: "÷",
micro: "µ",
para: "¶",
sect: "§",
iexcl: "¡",
iquest: "¿",
ordf: "ª",
ordm: "º",
sup2: "²",
sup3: "³",
frac14: "¼",
frac12: "½",
frac34: "¾",
shy: "­",
ensp: " ",
emsp: " ",
thinsp: " ",
zwnj: "‌",
zwj: "‍",
};
// The accented letters, `&eacute;` and the like, composed from their letter
// and accent rather than listed one by one.
for (
const [accent, mark, letters] of [
["grave", "̀", "aeiouAEIOU"],
["acute", "́", "aeiouyAEIOUY"],
["circ", "̂", "aeiouAEIOU"],
["tilde", "̃", "anoANO"],
["uml", "̈", "aeiouyAEIOUY"],
["ring", "̊", "aA"],
["cedil", "̧", "cC"],
]
) {
for (const letter of letters) {
NAMED_ENTITIES[letter + accent] = (letter + mark).normalize("NFC");
}
}
/** Returns the character of code point `n`, or undefined when there is none. */
function fromCodePoint(n: number): string | undefined {
if (n <= 0 || n > 0x10ffff || (n >= 0xd800 && n <= 0xdfff)) {
return undefined;
}
return String.fromCodePoint(n);
}
/**
* Decodes the entities in `str` in a single pass, so `&amp;quot;` reads
* `&quot;`. An entity it doesn't know, or a code point that doesn't exist, is
* left as written. Also used by `bandcamp-tralbum.ts` on the JSON a page
* carries in an attribute.
*/
export function decodeHtmlEntities(str: string): string {
return str.replace(
/&(?:#(\d{1,8})|#[xX]([0-9a-fA-F]{1,6})|([a-zA-Z][a-zA-Z0-9]{1,31}));/g,
(match, dec?: string, hex?: string, name?: string) => {
if (dec) return fromCodePoint(Number(dec)) ?? match;
if (hex) return fromCodePoint(parseInt(hex, 16)) ?? match;
return NAMED_ENTITIES[name!] ?? NAMED_ENTITIES[name!.toLowerCase()] ??
match;
},
);
}
/**
* Returns `raw` resolved against `base` as an absolute http(s) URL; undefined
* for a `data:` URI, another scheme or a URL that doesn't parse. Every URL a
* page names goes through it, so `/og.png` and `//cdn/og.png` work.
*/
export function absoluteHttpUrl(
raw: string | undefined,
base: string,
): string | undefined {
if (!raw?.trim()) return undefined;
try {
const url = new URL(raw.trim(), base);
return url.protocol === "http:" || url.protocol === "https:"
? url.href
: undefined;
} catch {
return undefined;
}
}
/** A tag's attributes by lowercase name, values decoded; the first one wins. */
type Attrs = Map<string, string>;
/**
* Matches the start of a tag the scan reads, or of a comment. Quoted values
* may hold `>`, but no tag spans a `<`, so each attempt stops at the next one
* and the scan stays linear in the page's size.
*/
const TAG_RE =
/<(meta|link|img|title|script)\b((?:[^<>"']|"[^"<]*"|'[^'<]*')*)>|<!--/gi;
/** Matches one attribute: unquoted, single- or double-quoted, or bare. */
const ATTR_RE =
/([^\s"'<>/=]+)(?:\s*=\s*(?:"([^"]*)"|'([^']*)'|([^\s"'=<>`]+)))?/g;
function parseAttrs(source: string): Attrs {
const attrs: Attrs = new Map();
for (const m of source.matchAll(ATTR_RE)) {
const name = m[1].toLowerCase();
if (attrs.has(name)) continue;
attrs.set(name, decodeHtmlEntities(m[2] ?? m[3] ?? m[4] ?? ""));
}
return attrs;
}
/** What {@link scanTags} found, in page order. */
interface Scan {
metas: Attrs[];
links: Attrs[];
imgs: Attrs[];
/** The raw text of the first `<title>`. */
title?: string;
/** The bodies of the `application/ld+json` scripts. */
jsonLd: string[];
}
/**
* Collects the tags {@link parsePage} reads in one pass. Comments and the
* bodies of scripts and the title are skipped over, so a tag written inside
* them isn't read; one left open runs to the end of the page, as in a browser.
*/
function scanTags(html: string): Scan {
const scan: Scan = { metas: [], links: [], imgs: [], jsonLd: [] };
const re = new RegExp(TAG_RE);
let m: RegExpExecArray | null;
while ((m = re.exec(html)) !== null) {
const tag = m[1]?.toLowerCase();
if (!tag) {
const end = html.indexOf("-->", re.lastIndex);
if (end < 0) break;
re.lastIndex = end + 3;
continue;
}
const attrs = parseAttrs(m[2]);
if (tag === "meta") scan.metas.push(attrs);
else if (tag === "link") scan.links.push(attrs);
else if (tag === "img") scan.imgs.push(attrs);
else {
const close = new RegExp(`</${tag}\\s*>`, "gi");
close.lastIndex = re.lastIndex;
const end = close.exec(html);
if (!end) break;
const body = html.slice(re.lastIndex, end.index);
if (tag === "title") scan.title ??= body;
else if (
attrs.get("type")?.trim().toLowerCase() === "application/ld+json"
) {
scan.jsonLd.push(body);
}
re.lastIndex = close.lastIndex;
}
}
return scan;
}
/** Returns `value` trimmed, or undefined when nothing is left. */
function nonBlank(value: string | undefined): string | undefined {
const trimmed = value?.trim();
return trimmed ? trimmed : undefined;
}
/**
* Returns the page's `theme-color`, as written: pass it to
* {@link normalizeCssColor}. Sites often ship one per color scheme with the
* dark one first, so the unscoped tag wins, then the light one, then any.
*/
function themeColorOf(metas: Attrs[]): string | undefined {
let unscoped: string | undefined;
let light: string | undefined;
let any: string | undefined;
for (const meta of metas) {
if (meta.get("name")?.toLowerCase() !== "theme-color") continue;
const content = nonBlank(meta.get("content"));
if (!content) continue;
const media = meta.get("media");
if (!media) unscoped ??= content;
else if (/light/i.test(media)) light ??= content;
any ??= content;
}
return unscoped ?? light ?? any;
}
/** Returns the lowercase tokens of a `<link>`'s `rel`. */
function relTokens(link: Attrs): string[] {
return (link.get("rel") ?? "").toLowerCase().split(/\s+/);
}
/**
* Returns the `color` of `<link rel="mask-icon">`, Safari's pinned-tab icon,
* as written. The generic preview's last resort for a brand color.
*/
function maskIconColorOf(links: Attrs[]): string | undefined {
for (const link of links) {
if (!relTokens(link).includes("mask-icon")) continue;
const color = nonBlank(link.get("color"));
if (color) return color;
}
return undefined;
}
/**
* Returns the absolute URL of the page's largest icon, among its `icon` and
* `apple-touch-icon` links, by declared `sizes`; the first one when none
* declares a size. A `mask-icon` is no candidate: it is a one-color shape for
* Safari to tint, black when drawn as is.
*
* An SVG icon wins over any size: it declares none, yet scales to any.
*/
function bestIconOf(links: Attrs[], baseUrl: string): string | undefined {
let best: { href: string; area: number } | undefined;
for (const link of links) {
const rel = relTokens(link);
if (
!rel.includes("icon") && !rel.includes("apple-touch-icon") &&
!rel.includes("apple-touch-icon-precomposed")
) continue;
const href = absoluteHttpUrl(link.get("href"), baseUrl);
if (!href) continue;
const sizes = (link.get("sizes") ?? "").toLowerCase();
const isSvg = /\.svg(\?|$)/i.test(href) ||
link.get("type")?.toLowerCase() === "image/svg+xml" ||
sizes === "any";
const area = isSvg ? Number.MAX_SAFE_INTEGER : Math.max(
0,
...[...sizes.matchAll(/(\d+)x(\d+)/g)].map(([, w, h]) => +w * +h),
);
if (!best || area > best.area) best = { href, area };
}
return best?.href;
}
/**
* Returned by the JSON-LD reading of {@link parsePage}, which the generic
* preview ranks below the page's OG and Twitter tags.
*/
export interface JsonLdResult {
/** From `name`, or else `headline`. */
title?: string;
description?: string;
/** From `image`, or else `thumbnailUrl` or `thumbnail`. */
thumbnailUrl?: string;
}
/**
* The JSON-LD types that describe the site around a page rather than the
* page itself. Their name is the site's or the company's, so a node of one of
* them is read only when no other node has a title or an image.
*/
const SITE_LD_TYPES = new Set([
"WebSite",
"Organization",
"Corporation",
"NewsMediaOrganization",
"Person",
"BreadcrumbList",
"SearchAction",
"ImageObject",
"SiteNavigationElement",
"WPHeader",
"WPFooter",
]);
function ldString(v: unknown): string | undefined {
if (typeof v === "string") return nonBlank(v);
if (Array.isArray(v) && typeof v[0] === "string") return nonBlank(v[0]);
return undefined;
}
function ldImage(v: unknown, baseUrl: string): string | undefined {
if (typeof v === "string") return absoluteHttpUrl(v, baseUrl);
if (Array.isArray(v)) return ldImage(v[0], baseUrl);
if (v && typeof v === "object") {
const o = v as Record<string, unknown>;
return ldImage(o.url ?? o.contentUrl, baseUrl);
}
return undefined;
}
/** Lists every node of `data`, through arrays and `@graph`, in order. */
function ldNodes(data: unknown): Record<string, unknown>[] {
if (Array.isArray(data)) return data.flatMap(ldNodes);
if (!data || typeof data !== "object") return [];
const o = data as Record<string, unknown>;
return o["@graph"] ? ldNodes(o["@graph"]) : [o];
}
/**
* Returns the first JSON-LD node about the page that has a title or an image,
* else the first about its site ({@link SITE_LD_TYPES}); empty when there is
* none. An image may be a URL, a list of them or an `ImageObject`. Blocks that
* aren't valid JSON are skipped.
*/
function jsonLdOf(blocks: string[], baseUrl: string): JsonLdResult {
let fallback: JsonLdResult | undefined;
for (const block of blocks) {
let data: unknown;
try {
data = JSON.parse(block);
} catch {
continue;
}
for (const node of ldNodes(data)) {
const result: JsonLdResult = {
title: ldString(node.name ?? node.headline),
description: ldString(node.description),
thumbnailUrl: ldImage(
node.image ?? node.thumbnailUrl ?? node.thumbnail,
baseUrl,
),
};
if (!result.title && !result.thumbnailUrl) continue;
const types = [node["@type"]].flat();
if (!types.some((t) => typeof t === "string" && SITE_LD_TYPES.has(t))) {
return result;
}
fallback ??= result;
}
}
return fallback ?? {};
}
/**
* Returns the source of an `<img>`: its `src`, or for a lazy-loaded image
* whose `src` is a `data:` placeholder, its `data-src` or `data-lazy-src`.
*/
function imageSource(img: Attrs, baseUrl: string): string | undefined {
const src = img.get("src")?.trim();
return src && !src.startsWith("data:")
? absoluteHttpUrl(src, baseUrl)
: absoluteHttpUrl(img.get("data-src") ?? img.get("data-lazy-src"), baseUrl);
}
/** Returns an `<img>`'s declared `name` dimension in pixels; 0 when none. */
function declaredSize(img: Attrs, name: "width" | "height"): number {
return parseInt(img.get(name) ?? "") || 0;
}
/**
* Returns the absolute URL of the first `<img>` whose declared width and height
* are both at least `minSize` pixels; an image without them doesn't count.
*/
function largeImageOf(
imgs: Attrs[],
baseUrl: string,
minSize = 200,
): string | undefined {
for (const img of imgs) {
if (
declaredSize(img, "width") < minSize ||
declaredSize(img, "height") < minSize
) continue;
const src = imageSource(img, baseUrl);
if (src) return src;
}
return undefined;
}
/** Filename words that mark an image as page furniture rather than content. */
const UI_IMAGE_KEYWORDS = new Set([
"logo",
"icon",
"sprite",
"favicon",
"avatar",
"banner",
"header",
"nav",
"menu",
"cart",
"search",
"tracking",
"pixel",
"bg",
"background",
"placeholder",
"spacer",
"blank",
"lazy",
"loading",
]);
/**
* Checks whether `src` looks like page furniture: an SVG, or a filename with
* one of the {@link UI_IMAGE_KEYWORDS} between its `-`, `_` or `.`.
*/
function isUiImage(src: string): boolean {
if (/\.svg(\?|$)/i.test(src)) return true;
const filename = src.split("?")[0].split("/").pop() ?? "";
const baseName = filename.replace(/\.[^.]+$/, "");
return baseName.toLowerCase().split(/[-_.]/).some((t) =>
UI_IMAGE_KEYWORDS.has(t)
);
}
/**
* Returns the absolute URL of the first `<img>` that looks like content rather
* than a logo, icon, tracking pixel or other page furniture: one declared
* smaller than 50 pixels either way doesn't count. The generic preview's last
* resort for a thumbnail.
*/
function firstContentImageOf(
imgs: Attrs[],
baseUrl: string,
): string | undefined {
for (const img of imgs) {
const w = declaredSize(img, "width");
const h = declaredSize(img, "height");
if ((w && w < 50) || (h && h < 50)) continue;
const src = imageSource(img, baseUrl);
if (src && !isUiImage(src)) return src;
}
return undefined;
}
/**
* Returned by {@link parsePage}: what a page says about itself, entities
* decoded, values trimmed and URLs absolute.
*/
export interface Page {
/**
* Returns the `content` of the first `<meta>` whose `property` or `name` is
* `key`, in any case: sites write `og:` tags under either attribute, and
* Twitter's too.
*/
meta(key: string): string | undefined;
/** The text of the `<title>` element. */
title?: string;
jsonLd: JsonLdResult;
/** The `theme-color`, as written: pass it to {@link normalizeCssColor}. */
themeColor?: string;
/** The `color` of the Safari pinned-tab icon, as written. */
maskIconColor?: string;
/** The largest icon the page links, which the thumbnail placeholder draws. */
bestIcon?: string;
/** The first image declared at least 200 pixels both ways. */
largeImage?: string;
/** The first image that doesn't look like page furniture. */
firstContentImage?: string;
}
/**
* Reads `html`, served from `pageUrl`, in one pass. The generic preview and
* the providers built on it read a page through it.
*/
export function parsePage(html: string, pageUrl: string): Page {
const scan = scanTags(html);
const metas = new Map<string, string>();
for (const meta of scan.metas) {
const content = nonBlank(meta.get("content"));
if (!content) continue;
for (const attr of ["property", "name"]) {
const key = meta.get(attr)?.trim().toLowerCase();
if (key && !metas.has(key)) metas.set(key, content);
}
}
return {
meta: (key) => metas.get(key.toLowerCase()),
title: nonBlank(scan.title && decodeHtmlEntities(scan.title)),
jsonLd: jsonLdOf(scan.jsonLd, pageUrl),
themeColor: themeColorOf(scan.metas),
maskIconColor: maskIconColorOf(scan.links),
bestIcon: bestIconOf(scan.links, pageUrl),
largeImage: largeImageOf(scan.imgs, pageUrl),
firstContentImage: firstContentImageOf(scan.imgs, pageUrl),
};
}
/**
* Returns `raw` as a lowercase `#rrggbb`, from a hex, `rgb()`, `rgba()` or
* common named color; undefined for any other form. The generic preview
* stores it as the preview's `accentColor`, a tint over the app's own surface,
* so alpha is dropped, and a fully transparent one is no color at all.
*/
export function normalizeCssColor(raw: string | undefined): string | undefined {
const color = raw ? parseCssColor(raw) : undefined;
return color && color.a > 0 ? hexOf(color) : undefined;
}

152
api/remote/image-proxy.ts Normal file
View File

@@ -0,0 +1,152 @@
/**
* Shows the thumbnails a browser can't load from their own site: those on plain
* http, which an https page would block, and those behind hotlink protection.
* It fetches and caches the images `GET /api/remote/image` serves, which the
* frontend routes a thumbnail through when it is on http or when loading it
* directly failed, and scales them down as the site's own are.
*
* @module
*/
import { fetchWithTimeout, readBytesCapped } from "./fetch.ts";
import type { ImageWidth } from "../../shared/image.ts";
import { scaleImageBytes } from "../lib/ffmpeg.ts";
import { scalableImage } from "../lib/upload.ts";
/**
* Returned to `GET /api/remote/image` by {@link fetchProxiedImage}, which sends
* it back under `contentType`.
*/
export interface ProxiedImage {
bytes: Uint8Array;
/** One of the raster types, without parameters. */
contentType: string;
}
const PROXYABLE_IMAGE_TYPES = new Set([
"image/jpeg",
"image/png",
"image/gif",
"image/webp",
"image/avif",
"image/bmp",
"image/x-icon",
"image/vnd.microsoft.icon",
]);
const IMAGE_CACHE_TTL_MS = 24 * 60 * 60 * 1000;
const IMAGE_CACHE_MAX_BYTES = 32 * 1024 * 1024;
/**
* Proxied images, by URL and by the width they were scaled to, since every
* viewer of a feed asks for the same thumbnails. The least recently used go
* first, and the Map's order is that of use.
*/
const imageCache = new Map<
string,
{ bytes: Uint8Array; contentType: string; at: number }
>();
let imageCacheBytes = 0;
function readImageCache(
url: string,
): { bytes: Uint8Array; contentType: string } | undefined {
const hit = imageCache.get(url);
if (!hit) return undefined;
imageCache.delete(url);
if (Date.now() - hit.at > IMAGE_CACHE_TTL_MS) {
imageCacheBytes -= hit.bytes.byteLength;
return undefined;
}
imageCache.set(url, hit); // Moves it to the end: most recently used.
return hit;
}
function writeImageCache(
url: string,
bytes: Uint8Array,
contentType: string,
): void {
const old = imageCache.get(url);
if (old) {
imageCache.delete(url);
imageCacheBytes -= old.bytes.byteLength;
}
while (
imageCacheBytes + bytes.byteLength > IMAGE_CACHE_MAX_BYTES &&
imageCache.size > 0
) {
const [oldestUrl, oldest] = imageCache.entries().next().value!;
imageCache.delete(oldestUrl);
imageCacheBytes -= oldest.bytes.byteLength;
}
imageCache.set(url, { bytes, contentType, at: Date.now() });
imageCacheBytes += bytes.byteLength;
}
/** Empties the cache, for the tests' reset between cases. */
export function clearImageCache(): void {
imageCache.clear();
imageCacheBytes = 0;
}
/**
* Returns the image at `url`, from the cache or else from its host; null when
* the host answers with an error, or with anything but a raster image of 5 MB
* at most. With `width`, it comes as a WebP copy scaled down to it, or as it
* is when no copy can be made (`scalableImage`, and ffmpeg). The route checks
* `url` with `isValidHttpUrl` first.
*
* @throws {Error} When the host can't be reached, as
* {@link fetchWithTimeout} reports it.
*/
export async function fetchProxiedImage(
url: string,
width?: ImageWidth | null,
): Promise<ProxiedImage | null> {
if (!width) return await fetchOriginal(url);
const key = `${url}#w${width}`;
const cached = readImageCache(key);
if (cached) return cached;
const original = await fetchOriginal(url);
if (!original) return null;
const how = scalableImage(original.bytes);
const bytes = how && await scaleImageBytes(original.bytes, width, how);
if (!bytes) return original;
writeImageCache(key, bytes, "image/webp");
return { bytes, contentType: "image/webp" };
}
/** Returns the image at `url` as its host sends it: see above. */
async function fetchOriginal(url: string): Promise<ProxiedImage | null> {
const cached = readImageCache(url);
if (cached) return cached;
const res = await fetchWithTimeout(url, 8000);
if (!res.ok) {
await res.body?.cancel();
return null;
}
// Raster formats only: the bytes go out under this site's origin, where an
// SVG, which can carry script, would run if opened directly.
const contentType = (res.headers.get("content-type") ?? "")
.split(";")[0].trim().toLowerCase();
if (!PROXYABLE_IMAGE_TYPES.has(contentType)) {
await res.body?.cancel();
return null;
}
const MAX_SIZE = 5 * 1024 * 1024;
const contentLength = Number(res.headers.get("content-length") ?? "0");
if (contentLength > MAX_SIZE) {
await res.body?.cancel();
return null;
}
// Without a Content-Length, the cap is found by reading one byte past it.
const bytes = await readBytesCapped(res, MAX_SIZE + 1);
if (bytes.length > MAX_SIZE) return null;
writeImageCache(url, bytes, contentType);
return { bytes, contentType };
}

64
api/remote/providers/bandcamp.ts Normal file
View File

@@ -0,0 +1,64 @@
/**
* Previews Bandcamp links from their page, for `rich-content-service.ts`.
*
* @module
*/
import type { RichContent } from "../../../shared/dump.ts";
import type { RichContentProvider } from "./provider.ts";
import { parsePage } from "../html.ts";
import { fetchPage } from "../fetch.ts";
import { BANDCAMP_HOST_RE } from "../bandcamp-tralbum.ts";
import { genericPreview } from "./generic.ts";
/**
* Returns the embed player's URL a page's `og:video` names, over https; only
* a player on bandcamp.com counts, since the frontend frames it.
*/
function embeddedPlayerUrl(raw: string | undefined): string | undefined {
if (!raw) return undefined;
try {
const url = new URL(raw);
if (
(url.protocol !== "https:" && url.protocol !== "http:") ||
url.hostname !== "bandcamp.com" ||
!url.pathname.startsWith("/EmbeddedPlayer/")
) return undefined;
url.protocol = "https:";
return url.href;
} catch {
return undefined;
}
}
/**
* Claims bandcamp.com and its subdomains, where artists have their pages, and
* takes the embed player's URL from the page's `og:video`. The native player
* fetches the page's streams itself at play time, through
* `GET /api/remote/bandcamp`.
*/
export const bandcampProvider: RichContentProvider = {
name: "bandcamp",
matches(url: string): boolean {
try {
return BANDCAMP_HOST_RE.test(new URL(url).hostname);
} catch {
return false;
}
},
async fetch(url: string): Promise<RichContent> {
const fetched = await fetchPage(url);
if (fetched.html === undefined) {
return { type: "bandcamp", siteName: "Bandcamp", url };
}
const page = parsePage(fetched.html, fetched.url);
return {
...genericPreview(page, url, fetched.url),
type: "bandcamp",
siteName: "Bandcamp",
embedUrl: embeddedPlayerUrl(page.meta("og:video")),
};
},
};

121
api/remote/providers/generic.ts Normal file
View File

@@ -0,0 +1,121 @@
/**
* Previews any page no other provider claims, for `rich-content-service.ts`,
* from whatever metadata its HTML carries. Its reading of a page,
* {@link genericPreview}, is also the base of the Bandcamp, SoundCloud and
* YouTube channel previews.
*
* @module
*/
import type { RichContent } from "../../../shared/dump.ts";
import type { RichContentProvider } from "./provider.ts";
import { siteHost } from "../../../shared/url.ts";
import {
absoluteHttpUrl,
normalizeCssColor,
type Page,
parsePage,
} from "../html.ts";
import { fetchPage } from "../fetch.ts";
/** Returns the path of `url` without its trailing slashes, `/` for the root. */
function pathOf(url: string): string {
return new URL(url).pathname.replace(/\/+$/, "") || "/";
}
/**
* Strips a trailing ` | Site`, ` – Site` or the like from a `<title>`, which
* names the page and then its site; the site is in the preview's badge.
*/
function withoutSiteSuffix(title: string, siteName: string): string {
const escaped = siteName.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
const suffix = new RegExp(`\\s+[|–—\\-·•]\\s+${escaped}\\s*$`, "i");
return title.replace(suffix, "") || title;
}
/**
* Returns the preview of the page `url` that `page` read, served from
* `finalUrl` after redirects. Each field falls back from OG tags to Twitter
* tags, JSON-LD and the markup itself. The icon and brand color feed the
* placeholder the frontend draws when there is no thumbnail.
*/
export function genericPreview(
page: Page,
url: string,
finalUrl: string,
): RichContent {
// An og:url naming the home page from another page means the OG tags are
// the site's defaults, not this page's: their title and image are skipped.
// Any other og:url is the page's canonical address, often with a slug the
// link left out, and its tags are the page's.
const ogUrl = page.meta("og:url");
let useOg = true;
try {
useOg = !ogUrl || pathOf(new URL(ogUrl, finalUrl).href) !== "/" ||
pathOf(finalUrl) === "/";
} catch { /* an og:url that doesn't parse names no other page */ }
const ogSiteName = page.meta("og:site_name");
const pageTitle = page.title && ogSiteName
? withoutSiteSuffix(page.title, ogSiteName)
: page.title;
const title = (useOg ? page.meta("og:title") : undefined) ??
page.meta("twitter:title") ??
page.jsonLd.title ??
pageTitle;
const description = page.meta("og:description") ??
page.meta("twitter:description") ??
page.jsonLd.description ??
page.meta("description");
// Never the favicon: a 16×16 icon cropped into a thumbnail looks broken,
// and the placeholder drawn instead shows the icon properly. A tag naming
// no usable URL falls through to the next.
const thumbnailUrl =
(useOg ? absoluteHttpUrl(page.meta("og:image"), finalUrl) : undefined) ??
absoluteHttpUrl(page.meta("twitter:image"), finalUrl) ??
page.jsonLd.thumbnailUrl ??
page.largeImage ??
page.firstContentImage;
// `/favicon.ico` is a guess. When it doesn't load, the placeholder shows
// the site's initial.
const faviconUrl = page.bestIcon ?? `${new URL(finalUrl).origin}/favicon.ico`;
const accentColor = normalizeCssColor(
page.themeColor ??
page.meta("msapplication-TileColor") ??
page.maskIconColor,
);
return {
type: "generic",
url,
title,
description,
thumbnailUrl,
faviconUrl,
accentColor,
siteName: ogSiteName ?? siteHost(url),
};
}
/** Matches every URL, so it comes last in the service's list. */
export const genericProvider: RichContentProvider = {
name: "generic",
matches(_url: string): boolean {
return true;
},
async fetch(url: string): Promise<RichContent> {
const fetched = await fetchPage(url);
if (fetched.html === undefined) return { type: "generic", url };
return genericPreview(
parsePage(fetched.html, fetched.url),
url,
fetched.url,
);
},
};

27
api/remote/providers/provider.ts Normal file
View File

@@ -0,0 +1,27 @@
/**
* Lets each kind of site have its own link previews. A provider implements this
* interface, and `rich-content-service.ts` asks its providers in turn and
* fetches a link's preview from the first that matches.
*
* @module
*/
import type { RichContent } from "../../../shared/dump.ts";
/**
* Implemented by each file of `remote/providers/`, and listed in priority order
* by `rich-content-service.ts`.
*/
export interface RichContentProvider {
name: string;
/** Called on a URL that `isValidHttpUrl` accepted; never throws. */
matches(url: string): boolean;
/**
* Returns the preview of `url`, which ends up in the dump's `rich_content` or
* the create form's preview. It rejects when the page can't be fetched or
* answers with an error status, and the service then falls back to a stub.
* A page that isn't HTML yields a preview without metadata. The service
* tidies what it returns, so a provider leaves text as the page wrote it.
*/
fetch(url: string): Promise<RichContent>;
}

Some files were not shown because too many files have changed in this diff Show More