Compare commits

...

4 Commits

Author SHA1 Message Date
khannurien
80f3292c8f v3: fix the frontend's findings, and the backend pieces they needed — each by a rule the code now keeps everywhere. Votes and likes go through one useOptimisticToggles in WSProvider: clicks on one item before any ack share one baseline, which a failure or the timeout restores (a second click used to cancel the first one's rollback), only the last ack's count is applied, a votes_update or comment_likes_update waits while this tab has toggles in flight, and a welcome reconciles them against the server's sets (a vote cast offline left its count one too high). lastLikeEvent, which nothing read, is gone. Follows are refetched when the new follows_changed message, sent to the follower's sockets on every follow write, says another tab changed them; a failed load keeps the Follow buttons disabled and is retried 10 s later instead of leaving them disabled for good (success: false) or enabled over empty sets (network error); follow and unfollow resolve to whether they worked, and the button says when they didn't. CategoriesProvider follows the same rule, so a network error no longer turns every category URL into a 404. Every list is kept by one rule: useDumpListSync and usePlaylistListSync never add an item, and usePositionAwareSync remembers the place of any item that leaves a list, whatever took it out, and puts it back when its next event shows it again; with dump_created now setting lastDumpEvent, a dump going private and back returns to its place on another user's profile, which never happened before, and an updated playlist missing from a list is no longer prepended. useFeedCache keys every cache by viewer (the profile pages restored the owner's private dumps to whoever went back) and reads it again when its key changes (another profile in the same page showed the previous one's list). Every response of a list's first page or next pages goes through expectOk (Index, UserPlaylists, useUserDumpFeed and Notifications read body.data unchecked).
All checks were successful
Build and Publish Docker Image / build-and-push (push) Successful in 55s
Build and Deploy Static Documentation Site / build-and-push (push) Successful in 22s
Smaller fixes: the redirects from / and /<slug> keep the navigation state, so the index hides the dump DumpEdit just deleted; Index's mainFetchDone ref, which blocked nothing, is gone; DumpEdit refuses someone else's dump up front, as the API does; the Dump page takes a handed-over dump once and clears it from the history entry, so a reload fetches; a newly upvoted dump is fetched with the viewer's token; the notifications page sends read-all once per notification; the profile's followed and invitees tabs finish loading when left mid-fetch; UserListPopover no longer closes when its own list scrolls; the vote and like counts open their lists for visitors too (a disabled button swallowed the click; they are aria-disabled now); the removal of a playlist's dump forgets it (an edit brought it back), a failed removal puts it back where it was, an undo waits for the removal in flight, and a refused undo says so; PlaylistCreateForm tells whether the dump made it into the new playlist; the cover preview's object URL is revoked; the muted icon has its cross; the bell no longer stays rung after two notifications within 700 ms; a stale invite check can't overwrite a newer one; a stored theme or color scheme is checked; the default feed tab is one store every reader follows, across tabs; the file fields enforce the size they state, derived from DUMP_MAX_FILE_SIZE_BYTES; formatBytes, friendlyFetchError and ImagePicker are translated (Intl names the units), and ImagePicker ignores keys while uploading; visited dumps and playlists keep a week of visits and are parsed once; a chime is claimed under a Web Lock, so two tabs can't both play it; a date in the future reads as "now". Dead code goes: PageShell's centerSlot, overLimit, the exports of isSafeHex and hueFromString, the comment tree's token prop and ChatContext's isChatOpen and closeChat. The French catalogue has the 11 new messages. 214 backend and 632 frontend tests pass; tsc, deno lint, ESLint and doc:lint are clean.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 20:08:00 +00:00
khannurien
a36a4cac6b v3: fix the backend's correctness findings — each by a rule shared across the code rather than one more special case. Link previews: readTextCapped decoded every page as UTF-8, so a Latin-1 page came out garbled unless it went through the curl fallback, which re-encoded it on its own; pages are now decoded in the charset their Content-Type or <meta> declares, and the curl fallback hands the bytes over as they are. The image proxy served and cached for a day an error answer that claimed an image type, and read a body without Content-Length whole before checking its size; it now refuses an error status and reads through the same capped reader. The icons task's Twemoji fetch gives up after 15 s instead of holding the start up, and extractLinkHref, which only its tests used, is gone. Backlinks matched a linked URL against the raw url column, so https://x.com/ missed a dump posted as https://x.com; they now compare canonical URLs, as reposts do. The four LIKE searches put the query in raw, so % and _ were wildcards and the mention search for "a_b" found "axb"; lib/like.ts escapes them. Every list query now treats a malformed row the same way, a 500 through model/rows.ts's parseRows, where ten of them silently dropped it; the playlist dumps query uses the model's column list rather than a hand-written copy, and the aliased list is derived from the plain one. The vote and like services commit through transaction() instead of their own BEGIN and ROLLBACK, which would have rolled back a caller's transaction.
Validation: the dump validators refuse the nulls their shared types don't allow (a null title or note on creation, a null URL on edit); a playlist title of spaces is refused and titles are stored trimmed; usernames and emails are checked by the same USERNAME_RE and EMAIL_RE in shared/validation.ts on sign-up, on PATCH /api/users/me (which checked other limits) and in the sign-up form. Mentions are read by shared/mention.ts's mentionedUsernames, which skips code and URLs as Markdown shows them: https://mastodon.social/@bob or a `@bob` in code notified bob; the editor's suggestions derive their trigger from the same pattern instead of a copy. Following an unknown user is a 404 rather than a foreign-key 500, and following a private playlist a 404 rather than a 403 that confirmed it exists. `ws` and `emojibase` are reserved category slugs, since the server answers them before the app. GET /api/users/me no longer turns every error into a 500.

Dead code goes: adminOnlyMiddleware, verifyEmailTransport, listUsers, deleteUser with disconnectUser and the force_logout message only it sent, PATCH /api/notifications/:id/read (with markOneRead), isSlugAvailable, the unreachable 500 in fetchComment, and the `if (!ctx.state.user)` checks behind authMiddleware. 214 backend and 627 frontend tests pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 19:35:53 +00:00
khannurien
14e3ab125b v3: fix the privacy and security findings of the comment pass — private dumps' broadcasts, mentions and attachments reached people who can't see them, and a few smaller holes. A dump's every write now goes through ws-service's broadcastDumpChange(before, after), which decides the audience: a private dump's creation, edits and deletion reach its owner's sockets alone (its deletion id went to everyone before), going private is dump_deleted for everyone else and dump_updated for the owner (whose other tabs dropped it), going public dump_created for everyone else. Mentions in a private dump, a comment on one, or a private playlist's description notify no one until it goes public (mentions are deduplicated per context, so going public notifies them once), and a moderator's edit of a comment, chat message or playlist mentions in the author's name. A link preview of a private playlist is refused, as one of a private dump already was: the preview is stored in the linking dump, which others may see. Attachments record the type of the resource that links them (migration 0018 backfills it); one linked by a private dump or playlist, or a comment on a private dump, is served only with its owner's media token, which Markdown now adds to attachment URLs, and one whose resource is gone answers 404 and is purged at startup. The Bandcamp provider claimed bandcamp.com.evil.example (the pattern wasn't anchored) and now reuses bandcamp-tralbum's anchored one; the YouTube provider took any site's ?list= for a YouTube playlist and missed m., music., youtube-nocookie and /live/ links, and now shares canonical-url's rules. Sign-in does the same scrypt work for an unknown username as for a wrong password, so timing no longer tells which exist. Usernames and emails are unique regardless of case (migration 0017, which refuses to run over existing clashes and names them): "Bob" and "bob" could both sign up and a mention picked one, emails had no constraint, and a reset request in another case silently sent nothing. The Open Graph tags escape the site name and build their URLs on PUBLIC_URL and FRONTEND_URL instead of the request's origin, which a proxy may rewrite.
Error statuses follow one rule: APIException takes a status and derives the code from it (BAD_REQUEST 400, UNAUTHORIZED 401, FORBIDDEN 403, NOT_FOUND 404, CONFLICT 409, RATE_LIMITED 429, SERVER_ERROR 500, BAD_GATEWAY 502), so a code can't contradict its status. That fixed real bugs: a wrong current password on change-password answered 401, on which authFetch signs the user out; editing or deleting someone else's comment or chat message answered 401 rather than 403; one invalid dump edit answered 422. A taken username is 409, like a taken email.

Along the way: the database is created from schema.sql by connection.ts when the file doesn't exist, so every task works on a fresh checkout (dev and serve never ran db/init.ts, which is gone); the startup purge compared ISO dates with SQLite's datetime() as text and kept anything from the current UTC day, and now compares ISO cutoffs; the seeded admin's hash comes from lib/password.ts (split out of jwt.ts, which needs the JWT secret) rather than a copy of its format; avatar, thumbnail and playlist-cover uploads go through replaceUploadedFile, which writes beside the old file and renames over it before the record announces it (a failed avatar write lost the old avatar; a thumbnail was announced before it existed); upload.ts derives its size message from MAX_IMAGE_SIZE_BYTES and stops re-exporting the upload folders; ALLOWED_IMAGE_MIMES, which duplicated detectImageMime, is gone; the SPA fallback no longer calls next() after answering; a reply must answer a comment on the same dump (a foreign or unknown parent was a 500 or a cross-dump thread). 211 backend and 628 frontend tests pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 15:19:07 +00:00
khannurien
0d4e344d77 v3: comments say what the code can't — the documentation pass wanted JSDoc on every export and every field of an exported interface, which filled the code with restatements ("The dump.", "Whether it is loading.", "The props of X.", "Renders the X.") and module docs that told a reader nothing ("The hot and new feeds, and the props every main feed of the index takes."); some were wrong as well as vague. Every comment in api/, shared/ and src/ is rewritten under one rule, now stated in the README's "Comments" section: every module and export still has a JSDoc, and it gives context (who uses it and where), usage (what a caller must know: a precondition, what it leaves to the caller, a side effect) or its place in the chain (where its input comes from, where its output goes); a field, parameter, private function or local gets one only when it adds to its parent's, and context about a whole type goes on the type; a type's doc is phrased from the value's side ("Returned to Y by X when Z", "Passed to Y by X"); full sentences, as short as they can be, no label-colon-list; // goes wherever the code isn't obvious and says why, and the section banners and narration go. Every claim was checked against the code, which turned up some forty comments that were wrong: DumpCard's claimed it was only used by the hot and new feeds, the WebSocket welcome claimed to be a socket's first message (presence comes first), UpdateUserRequest said null removes a description (it keeps it), deletedDumpIds leaves out that it also holds dumps made private, THEME_COLOR claimed to mirror index.html's, and the like. The routers' JSDoc now says where they are mounted and what becomes of a thrown APIException. doc-lint.ts lets a field without a doc through; a module or export without one still fails. No code changes; 204 backend and 628 frontend tests pass, and doc:lint, deno check, deno lint and tsc are clean.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 14:31:14 +00:00
347 changed files with 9218 additions and 8324 deletions

114
README.md
View File

@@ -1,6 +1,10 @@
# gerbeur # gerbeur
A small invite-only social platform for sharing links and files. Users can post URLs and media (YouTube, SoundCloud, Bandcamp, images, audio, video, …), vote, comment, follow each other, build playlists, and search content. A real-time WebSocket layer handles live presence, vote counts, and notifications. The UI is localized (English and French) and ships multiple visual themes. A small invite-only social platform for sharing links and files. Users can post
URLs and media (YouTube, SoundCloud, Bandcamp, images, audio, video, …), vote,
comment, follow each other, build playlists, and search content. A real-time
WebSocket layer handles live presence, vote counts, and notifications. The UI is
localized (English and French) and ships multiple visual themes.
## Stack ## Stack
@@ -30,13 +34,16 @@ cp .env.example .env
deno task dev deno task dev
``` ```
This starts both the API server (port 8000, with file watching) and the Vite dev server (port 3000) concurrently. This starts both the API server (port 8000, with file watching) and the Vite dev
server (port 3000) concurrently.
Open [http://localhost:3000](http://localhost:3000). On first run a default `admin` / `admin` account is created — change the password immediately. Open [http://localhost:3000](http://localhost:3000). On first run a default
`admin` / `admin` account is created — change the password immediately.
### Environment variables ### Environment variables
See [`.env.example`](.env.example) for the full list with descriptions. Key variables: See [`.env.example`](.env.example) for the full list with descriptions. Key
variables:
| Variable | Description | Default | | Variable | Description | Default |
| ---------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------- | | ---------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------- |
@@ -58,7 +65,10 @@ See [`.env.example`](.env.example) for the full list with descriptions. Key vari
### Docker (recommended) ### Docker (recommended)
The standard deployment runs API and frontend in a single container. The API server (Oak) serves the compiled frontend as static files, so both share the same origin — no `VITE_API_*` build args needed. Set `GERBEUR_PUBLIC_URL` to the externally-visible URL; it is automatically allowed for HTTP/WebSocket requests. The standard deployment runs API and frontend in a single container. The API
server (Oak) serves the compiled frontend as static files, so both share the
same origin — no `VITE_API_*` build args needed. Set `GERBEUR_PUBLIC_URL` to the
externally-visible URL; it is automatically allowed for HTTP/WebSocket requests.
```sh ```sh
docker build -t gerbeur . docker build -t gerbeur .
@@ -77,12 +87,17 @@ docker run -d \
``` ```
The two volumes are required for persistence: The two volumes are required for persistence:
- `gerbeur-db` — SQLite database (`api/sql/gerbeur.db`), initialized automatically on first run
- `gerbeur-db` — SQLite database (`api/sql/gerbeur.db`), initialized
automatically on first run
- `gerbeur-uploads` — user-uploaded files (`api/uploads/`) - `gerbeur-uploads` — user-uploaded files (`api/uploads/`)
#### Separate API and frontend (optional) #### Separate API and frontend (optional)
If you need to run the API on a different host than the frontend, pass the API location as build args so it gets baked into the frontend bundle. In that setup, add the frontend origin to `GERBEUR_ALLOWED_ORIGINS` so cross-origin HTTP/WebSocket requests are accepted: If you need to run the API on a different host than the frontend, pass the API
location as build args so it gets baked into the frontend bundle. In that setup,
add the frontend origin to `GERBEUR_ALLOWED_ORIGINS` so cross-origin
HTTP/WebSocket requests are accepted:
```sh ```sh
docker build \ docker build \
@@ -94,7 +109,8 @@ docker build \
### Reverse proxy ### Reverse proxy
Put a reverse proxy (nginx, Caddy, …) in front of the container to handle TLS. Forward everything to port 8000. Example Caddyfile: Put a reverse proxy (nginx, Caddy, …) in front of the container to handle TLS.
Forward everything to port 8000. Example Caddyfile:
``` ```
example.com { example.com {
@@ -119,10 +135,9 @@ api/
validators/ # Per domain: request body guards validators/ # Per domain: request body guards
lib/ # Helpers with no SQL and no outbound fetch: JWT, slugs, files, … lib/ # Helpers with no SQL and no outbound fetch: JWT, slugs, files, …
db/ db/
connection.ts # The database handle and transaction() connection.ts # The database handle and transaction(); creates a new database
startup.ts # Housekeeping run when the database opens startup.ts # Housekeeping run when the database opens
schema.sql # Database schema schema.sql # Database schema
init.ts # First-run database initialisation
migrate.ts # Migration runner migrate.ts # Migration runner
migrations/ # Versioned schema migrations migrations/ # Versioned schema migrations
test/ # Backend tests (Deno): test/<layer>/<module>.test.ts test/ # Backend tests (Deno): test/<layer>/<module>.test.ts
@@ -158,27 +173,65 @@ place that fetches from other sites. Tests mirror the source one file per
module, so the tests of `services/chat-service.ts` are in module, so the tests of `services/chat-service.ts` are in
`test/services/chat-service.test.ts`. `test/services/chat-service.test.ts`.
All three parts — `api/`, `shared/` and `src/` — are documented for `deno doc`, ### Comments
the same way in every file. Each module opens with a `/** … @module */` block
saying what it is for. Every export has a JSDoc comment, and so does every field A comment says what the code can't. If deleting it would lose nothing, it is the
of an exported interface; a comment that describes a declaration is always `/** wrong comment: on an export, rewrite it; anywhere else, delete it.
*/`, and `//` is only for notes on the code inside. Functions open with a verb
("Returns…", "Checks…", "Renders…" for a component), everything else with a noun **Every module and every export has a JSDoc** (`/** */`). Each gives at least
phrase. Parameters are named in the prose, in backticks. `{@link}` points at one of:
symbols the file already has in scope. A function that can throw to its caller
says so with `@throws`: for an `APIException`, the statuses and when. A router's 1. **Context**: who uses it and where (a page and its route, a caller, a
module doc lists its endpoints, one line each: method and path, who may call it, WebSocket message).
what it does; a page's JSDoc names its route and who may see it. The first 2. **Usage**: what a caller must know to use it correctly, such as a
paragraph of a doc is its summary, which the reference shows alone in lists, so precondition, an invariant, what it leaves to the caller, or a side effect.
it is a sentence that stands on its own: it never ends in a colon introducing Add an example only when the call isn't obvious.
what follows. 3. **Place in the chain**: where its input comes from and where its output goes.
A type's doc is phrased from the value's side, naming the producer and the
consumer: "Returned to Y by X when Z", "Passed to Y by X". It is never "What X
returns to Y" or "What X passes to Y".
Don't pack a doc into a label followed by a colon and a list ("X: one thing,
plus some other thing"). Write sentences with a subject and a verb.
When the obvious comment is useless, write the useful one: the context is always
there. A **field** (of an interface, or a parameter) gets a doc only when it
adds something its parent's doc doesn't. Context that concerns the whole type
goes on the type, not on one of its fields. The same goes for private functions
and locals. None of these count as a doc:
- a restatement of the name or type (`/** The dump. */`, "Whether it is
loading.", "The props of X.", "Renders the X.");
- a description of what the body, the JSX or the SQL plainly does;
- a list that goes stale (every caller of a widely used component): name the
kind of caller instead.
When a comment is there, it is as short as it can be, often one sentence, and it
is written in full sentences. Labels and fragments ("Upvotes on dumps.") don't
count. A module doc is a few lines at most. It opens with what the module does,
then says who calls it and what it hands back, and it doesn't repeat what its
exports' docs say. `@throws` stays on whatever throws to its caller, with the
statuses of an `APIException` and when each happens. A router's module doc lists
its endpoints (method, path, who may call it), since the code states them
nowhere in one place. A summary never ends in a colon: the reference shows it
alone.
**`//` comments** go wherever the code isn't obvious. They explain why, not
what: a workaround, an ordering constraint, a surprising limitation, a security
reason, an event handler that must not bubble.
**Every claim is checked against the code** (grep the callers). A wrong comment
is worse than none.
### Reference
In the frontend, every exported function states its return type, so the In the frontend, every exported function states its return type, so the
reference can show it: a component returns `React.JSX.Element` (or `| null`), a reference can show it: a component returns `React.JSX.Element` (or `| null`), a
hook returning several things returns a documented `Use<Name>Result` interface. hook returning several things returns a documented `Use<Name>Result` interface.
A component that takes props takes an exported `<Name>Props` interface, each A component that takes props takes an exported `<Name>Props` interface. React's
prop documented. React's types are used through the global `React` namespace types are used through the global `React` namespace rather than a default import
rather than a default import of `react`, which `deno doc` can't follow. of `react`, which `deno doc` can't follow.
`deno task doc` builds the HTML reference into `docs/` from every module of `deno task doc` builds the HTML reference into `docs/` from every module of
`api/`, `shared/` and `src/` but the tests (and the generated `src/locales/`). `api/`, `shared/` and `src/` but the tests (and the generated `src/locales/`).
@@ -187,9 +240,10 @@ repository's three parts and their layers in prose; `deno doc` builds the list
of every module itself, grouped by path. The `docs` workflow publishes it on of every module itself, grouped by path. The `docs` workflow publishes it on
every push to `main` that touches them. `deno task doc:lint` checks the rules every push to `main` that touches them. `deno task doc:lint` checks the rules
the linter can see, and that no summary ends in a colon, through `doc-lint.ts`. the linter can see, and that no summary ends in a colon, through `doc-lint.ts`.
`deno doc` doesn't document npm packages, so it takes a type imported from one Two kinds of diagnostic are let through. A field without a doc passes, since a
(react-hook-form's `FieldValues`, say) for a private one, and those references field gets one only when it adds to its type's. `deno doc` doesn't document npm
alone are let through. packages, so it takes a type imported from one (react-hook-form's `FieldValues`,
say) for a private one, and those references pass too.
Each of the frontend's `components/`, `contexts/`, `hooks/` and `utils/` is Each of the frontend's `components/`, `contexts/`, `hooks/` and `utils/` is
split into the same domain folders — `auth`, `category`, `chat`, `comment`, split into the same domain folders — `auth`, `category`, `chat`, `comment`,

View File

@@ -1,6 +1,7 @@
/** /**
* The server's settings: those read from `GERBEUR_*` environment variables, * Reads the server's settings from the `GERBEUR_*` environment variables, once
* with their defaults, and the fixed limits the backend applies. * at import, and holds the fixed limits the backend applies. The deno tasks
* load `.env` with `--env-file`, so changing a setting needs a restart.
* *
* @module * @module
*/ */
@@ -8,12 +9,13 @@
/** The port the server listens on (`GERBEUR_PORT`, default 8000). */ /** The port the server listens on (`GERBEUR_PORT`, default 8000). */
export const PORT: number = Number(Deno.env.get("GERBEUR_PORT")) || 8000; export const PORT: number = Number(Deno.env.get("GERBEUR_PORT")) || 8000;
/** /**
* The SMTP server mail is sent through, as an `smtps://` URL * The `smtps://` URL of the server email-service sends mail through
* (`GERBEUR_SMTPS_URL`). Empty disables email. * (`GERBEUR_SMTPS_URL`). When it is empty, the welcome and password reset
* emails are silently skipped.
*/ */
export const SMTPS_URL: string = Deno.env.get("GERBEUR_SMTPS_URL")?.trim() || export const SMTPS_URL: string = Deno.env.get("GERBEUR_SMTPS_URL")?.trim() ||
""; "";
/** The sender address of outgoing mail (`GERBEUR_FROM_EMAIL`). */ /** The `From` address of the welcome and password reset emails. */
export const FROM_EMAIL: string = Deno.env.get("GERBEUR_FROM_EMAIL")?.trim() || export const FROM_EMAIL: string = Deno.env.get("GERBEUR_FROM_EMAIL")?.trim() ||
""; "";
@@ -24,83 +26,91 @@ Hi **{{username}}**,
Your account has been created successfully. Welcome aboard!`; Your account has been created successfully. Welcome aboard!`;
/** /**
* The Markdown body of the welcome email (`GERBEUR_WELCOME_EMAIL_BODY`), with * The Markdown of the welcome email sent on registration
* `{{site_name}}` and `{{username}}` placeholders. * (`GERBEUR_WELCOME_EMAIL_BODY`). The users route replaces `{{site_name}}` and
* `{{username}}` in it and sends it as both text and HTML.
*/ */
export const WELCOME_EMAIL_BODY: string = export const WELCOME_EMAIL_BODY: string =
Deno.env.get("GERBEUR_WELCOME_EMAIL_BODY")?.trim() || Deno.env.get("GERBEUR_WELCOME_EMAIL_BODY")?.trim() ||
DEFAULT_WELCOME_EMAIL_BODY; DEFAULT_WELCOME_EMAIL_BODY;
/** The key tokens are signed with (`GERBEUR_JWT_SECRET`). Required. */ /**
* The key lib/jwt.ts signs and verifies tokens with (`GERBEUR_JWT_SECRET`).
* The server refuses to start without it.
*/
export const JWT_SECRET: string = Deno.env.get("GERBEUR_JWT_SECRET")?.trim() || export const JWT_SECRET: string = Deno.env.get("GERBEUR_JWT_SECRET")?.trim() ||
""; "";
/** /**
* The network interface the server binds to (`GERBEUR_LISTEN_HOST`). Defaults * The interface the server binds to (`GERBEUR_LISTEN_HOST`). The default,
* to 0.0.0.0 so Docker port-forwarding works out of the box; 127.0.0.1 * 0.0.0.0, lets Docker forward the port; 127.0.0.1 keeps it to loopback.
* restricts it to loopback.
*/ */
export const LISTEN_HOST: string = Deno.env.get("GERBEUR_LISTEN_HOST") || export const LISTEN_HOST: string = Deno.env.get("GERBEUR_LISTEN_HOST") ||
"0.0.0.0"; "0.0.0.0";
/** /**
* The server's public-facing URL (`GERBEUR_PUBLIC_URL`), without a trailing * The URL visitors reach the server at (`GERBEUR_PUBLIC_URL`), without a
* slash — used for CORS, WebSocket origin checks, email links, and OG image * trailing slash. Behind a reverse proxy it must be the outside URL: it is an
* URLs. Behind a reverse proxy, set it to the externally-visible URL (e.g. * allowed origin, it tells links to this site's own pages from other links,
* https://example.com). Defaults to http://localhost:PORT for local * and it prefixes the image URLs in their previews.
* development.
*/ */
export const PUBLIC_URL: string = export const PUBLIC_URL: string =
Deno.env.get("GERBEUR_PUBLIC_URL")?.trim().replace(/\/$/, "") || Deno.env.get("GERBEUR_PUBLIC_URL")?.trim().replace(/\/$/, "") ||
`http://localhost:${PORT}`; `http://localhost:${PORT}`;
/** /**
* The frontend's URL (`GERBEUR_FRONTEND_URL`). In single-container deployments * The URL the frontend is served from (`GERBEUR_FRONTEND_URL`), when it isn't
* the API serves the frontend, so it defaults to {@link PUBLIC_URL}; set it * {@link PUBLIC_URL}, as with a separate Vite server, without a trailing slash.
* when the frontend runs on a separate host or port (e.g. Vite dev server or a * It is an allowed origin, and password reset links and the Open Graph tags'
* CDN origin). * page URLs point to it.
*/ */
export const FRONTEND_URL: string = export const FRONTEND_URL: string =
Deno.env.get("GERBEUR_FRONTEND_URL")?.trim() || PUBLIC_URL; Deno.env.get("GERBEUR_FRONTEND_URL")?.trim().replace(/\/$/, "") ||
PUBLIC_URL;
/** /**
* The SQLite database file (`GERBEUR_DB_PATH`). Overridable so the test suite * The SQLite database file (`GERBEUR_DB_PATH`). api/test/setup.ts points it at
* can run against a throwaway database (see api/test/setup.ts). * a throwaway file before anything opens the database.
*/ */
export const DB_PATH: string = Deno.env.get("GERBEUR_DB_PATH") || export const DB_PATH: string = Deno.env.get("GERBEUR_DB_PATH") ||
"api/sql/gerbeur.db"; "api/sql/gerbeur.db";
// Upload/files
/** /**
* The directory uploads are stored under (`GERBEUR_UPLOADS_DIR`), one * The directory uploads are stored under (`GERBEUR_UPLOADS_DIR`). Each kind has
* subdirectory per kind. * its subdirectory below, created by the first upload, and each file is named
* after the id of what it belongs to.
*/ */
export const UPLOADS_DIR: string = Deno.env.get("GERBEUR_UPLOADS_DIR") || export const UPLOADS_DIR: string = Deno.env.get("GERBEUR_UPLOADS_DIR") ||
"api/uploads"; "api/uploads";
/** Where file dumps are stored. */ /** Holds the file of each file dump. */
export const DUMPS_DIR = `${UPLOADS_DIR}/dumps`; export const DUMPS_DIR = `${UPLOADS_DIR}/dumps`;
/** Where avatars are stored. */ /** Holds each user's avatar. */
export const AVATARS_DIR = `${UPLOADS_DIR}/avatars`; export const AVATARS_DIR = `${UPLOADS_DIR}/avatars`;
/** Where playlist cover images are stored. */ /** Holds each playlist's cover. */
export const PLAYLIST_IMAGES_DIR = `${UPLOADS_DIR}/playlist-images`; export const PLAYLIST_IMAGES_DIR = `${UPLOADS_DIR}/playlist-images`;
/** Where images pasted into chat, comments and descriptions are stored. */ /**
* Holds the images pasted into chat messages, comments, dump notes and
* descriptions. The startup tasks purge those nothing links to.
*/
export const ATTACHMENTS_DIR = `${UPLOADS_DIR}/attachments`; export const ATTACHMENTS_DIR = `${UPLOADS_DIR}/attachments`;
/** Where custom dump thumbnails are stored. */ /**
* Holds the thumbnails a poster uploads (`<id>-custom`) and those the
* thumbnails route generates from file dumps.
*/
export const THUMBNAILS_DIR = `${UPLOADS_DIR}/thumbnails`; export const THUMBNAILS_DIR = `${UPLOADS_DIR}/thumbnails`;
/** The largest image upload accepted: 5 MB. */ /**
* The largest image upload accepted, in bytes, for avatars, covers, thumbnails
* and attachments. lib/upload.ts states it in megabytes in its error.
*/
export const MAX_IMAGE_SIZE_BYTES = 5 * 1024 * 1024; export const MAX_IMAGE_SIZE_BYTES = 5 * 1024 * 1024;
/** The image types accepted for avatars, covers, thumbnails and attachments. */
export const ALLOWED_IMAGE_MIMES: Set<string> = new Set([
"image/jpeg",
"image/png",
"image/gif",
"image/webp",
]);
/** The MIME type prefixes a file dump may have. */ /**
* The MIME type prefixes a file dump may have. dump-service also accepts the
* exact types of {@link DUMP_ALLOWED_MIME_TYPES}.
*/
export const DUMP_ALLOWED_MIME_PREFIXES: string[] = [ export const DUMP_ALLOWED_MIME_PREFIXES: string[] = [
"text/", "text/",
"image/", "image/",
"video/", "video/",
"audio/", "audio/",
]; ];
/** The exact MIME types a file dump may have, beyond the prefixes. */ /** The documents and archives a file dump may be, beyond the prefixes. */
export const DUMP_ALLOWED_MIME_TYPES: Set<string> = new Set([ export const DUMP_ALLOWED_MIME_TYPES: Set<string> = new Set([
"application/pdf", "application/pdf",
"application/json", "application/json",
@@ -114,41 +124,44 @@ export const DUMP_ALLOWED_MIME_TYPES: Set<string> = new Set([
"application/vnd.ms-powerpoint", "application/vnd.ms-powerpoint",
]); ]);
// Pagination
/** The page size when a request names none. */ /** The page size when a request names none. */
export const PAGINATION_DEFAULT_LIMIT = 20; export const PAGINATION_DEFAULT_LIMIT = 20;
/** The largest page size a request may ask for. */ /** The largest page size a request may ask for. */
export const PAGINATION_MAX_LIMIT = 100; export const PAGINATION_MAX_LIMIT = 100;
// Startup housekeeping
/** How long an attachment nothing links to is kept before startup purges it. */ /** How long an attachment nothing links to is kept before startup purges it. */
export const ORPHANED_ATTACHMENTS_RETENTION_HOURS = 1; export const ORPHANED_ATTACHMENTS_RETENTION_HOURS = 1;
/** How long an unused invite is kept before startup purges it. */ /**
* How long an invite stays usable. invite-service refuses an older one, and
* the startup tasks purge it.
*/
export const UNUSED_INVITES_RETENTION_DAYS = 7; export const UNUSED_INVITES_RETENTION_DAYS = 7;
// SEO/OG /**
/** The site's name (`GERBEUR_SITE_NAME`), in page titles, OG tags and mail. */ * The site's name (`GERBEUR_SITE_NAME`). It is filled into index.html, the OG
* tags, the icons' manifest and the emails.
*/
export const OG_SITE_NAME: string = Deno.env.get("GERBEUR_SITE_NAME") || export const OG_SITE_NAME: string = Deno.env.get("GERBEUR_SITE_NAME") ||
"gerbeur"; "gerbeur";
/** /**
* The emoji the favicon set and app header logo are generated from * The emoji that serves as the site's logo (`GERBEUR_SITE_EMOJI`). The icons
* (`GERBEUR_SITE_EMOJI`). Applied at runtime (server startup and index.html * script draws the favicons from it, and index.html carries it to the app's
* injection), so changing it only needs a restart — no rebuild. * header, so changing it needs a restart but no rebuild.
*/ */
export const SITE_EMOJI: string = Deno.env.get("GERBEUR_SITE_EMOJI") || "🚚"; export const SITE_EMOJI: string = Deno.env.get("GERBEUR_SITE_EMOJI") || "🚚";
/** /**
* A Bandcamp playback path. * How the frontend plays Bandcamp. `"embed"` uses Bandcamp's iframe, which
* * can't autoplay. `"native"` resolves the page's streams through
* - `"embed"` — the bandcamp.com iframe (cannot autoplay) * `GET /api/bandcamp/tracks` and plays them in the app's player, which
* - `"native"` — resolve the page's mp3 streams and play them in the app's own * autoplays and queues an album's tracks.
* player, which autoplays and supports albums as a queue
*/ */
export type BandcampPlayer = "embed" | "native"; export type BandcampPlayer = "embed" | "native";
/** /**
* The Bandcamp playback path the frontend uses (`GERBEUR_BANDCAMP_PLAYER`). * How the frontend plays Bandcamp (`GERBEUR_BANDCAMP_PLAYER`), passed to it in
* Anything unrecognised falls back to `"embed"`, so a typo degrades safely. * index.html. Any value but `"native"` means `"embed"`, so a typo falls back to
* the player that always works.
*/ */
export const BANDCAMP_PLAYER: BandcampPlayer = export const BANDCAMP_PLAYER: BandcampPlayer =
Deno.env.get("GERBEUR_BANDCAMP_PLAYER")?.trim() === "native" Deno.env.get("GERBEUR_BANDCAMP_PLAYER")?.trim() === "native"
@@ -156,21 +169,21 @@ export const BANDCAMP_PLAYER: BandcampPlayer =
: "embed"; : "embed";
/** /**
* The background color of generated icons and the manifest. Mirrors the * The background of the generated Apple touch icon, the manifest's colors, and
* hard-coded theme-color in index.html. * the accent of previews of this site's own pages. index.html's `theme-color`
* tags don't follow it.
*/ */
export const THEME_COLOR = "#111827"; export const THEME_COLOR = "#111827";
const rawOrigins = Deno.env.get("GERBEUR_ALLOWED_ORIGINS") ?? ""; const rawOrigins = Deno.env.get("GERBEUR_ALLOWED_ORIGINS") ?? "";
/** /**
* The origins CORS and the WebSocket accept: {@link PUBLIC_URL}, * The origins that CORS in main.ts and the WebSocket route accept. They are
* {@link FRONTEND_URL} and the comma-separated `GERBEUR_ALLOWED_ORIGINS`. * {@link PUBLIC_URL}, {@link FRONTEND_URL} and the comma-separated
* `GERBEUR_ALLOWED_ORIGINS`.
*/ */
export const ALLOWED_ORIGINS: string[] = Array.from( export const ALLOWED_ORIGINS: string[] = Array.from(
new Set([ new Set([
PUBLIC_URL, PUBLIC_URL,
// FRONTEND_URL is auto-included so the separate-frontend case only needs
// GERBEUR_FRONTEND_URL — no need to repeat it in GERBEUR_ALLOWED_ORIGINS.
FRONTEND_URL, FRONTEND_URL,
...rawOrigins.split(",").map((o) => o.trim()).filter(Boolean), ...rawOrigins.split(",").map((o) => o.trim()).filter(Boolean),
]), ]),

View File

@@ -1,6 +1,8 @@
/** /**
* The database handle. Importing this module opens the database at * Opens the database at {@link DB_PATH} when first imported, creating it from
* {@link DB_PATH}, applies the pending migrations and runs the startup tasks. * schema.sql if the file doesn't exist, then applies the pending migrations
* and runs the startup tasks before any importer runs. Only the services
* import it.
* *
* @module * @module
*/ */
@@ -10,12 +12,25 @@ import { DB_PATH } from "../config.ts";
import { runMigrations } from "./migrate.ts"; import { runMigrations } from "./migrate.ts";
import { runStartupTasks } from "./startup.ts"; import { runStartupTasks } from "./startup.ts";
/** The one connection every service queries through. */ const isNew = await Deno.stat(DB_PATH).then(() => false, () => true);
/**
* The server's only connection. Its queries are synchronous, so each one
* blocks the event loop while it runs.
*/
export const db: DatabaseSync = new DatabaseSync(DB_PATH); export const db: DatabaseSync = new DatabaseSync(DB_PATH);
// schema.sql holds every migration's change, so on a new database each
// migration below finds its work done and is only recorded.
if (isNew) {
db.exec(
await Deno.readTextFile(new URL("./schema.sql", import.meta.url)),
);
}
// WAL lets readers run alongside a writer, and with synchronous = NORMAL a // WAL lets readers run alongside a writer, and with synchronous = NORMAL a
// commit no longer waits on an fsync (a crash can lose the last transactions, // commit doesn't wait on an fsync: a power loss can lose the last commits but
// never corrupt the file). busy_timeout covers the init/migration scripts and // never corrupts the file. busy_timeout makes another connection to the file
// the test harness opening the same file. // wait for the lock instead of failing. SQLite enforces foreign keys, and so
// the schema's ON DELETE cascades, only when each connection asks.
db.exec(` db.exec(`
PRAGMA journal_mode = WAL; PRAGMA journal_mode = WAL;
PRAGMA synchronous = NORMAL; PRAGMA synchronous = NORMAL;
@@ -26,9 +41,10 @@ runMigrations(db);
await runStartupTasks(db); await runStartupTasks(db);
/** /**
* Runs `fn` inside one transaction: several writes commit together (and cost * Runs `fn` in a transaction, so its writes commit together, in one commit
* one commit rather than one each), and a throw rolls them all back. Nested * rather than one each, and a throw rolls them all back and rethrows. A call
* calls join the outer transaction. * inside another joins the outer transaction. `fn` must be synchronous: an
* `await` would commit before the rest of it runs.
*/ */
export function transaction<T>(fn: () => T): T { export function transaction<T>(fn: () => T): T {
if (db.isTransaction) return fn(); if (db.isTransaction) return fn();

View File

@@ -1,21 +0,0 @@
/**
* Creates the database from schema.sql when {@link DB_PATH} doesn't exist yet;
* run once before the server starts (`deno task start`).
*
* @module
*/
import { DatabaseSync } from "node:sqlite";
import { DB_PATH } from "../config.ts";
try {
await Deno.stat(DB_PATH);
console.log("Database already exists, skipping initialization.");
} catch {
console.log("Initializing database from schema...");
const schema = Deno.readTextFileSync("api/db/schema.sql");
const db = new DatabaseSync(DB_PATH);
db.exec(schema);
db.close();
console.log("Database initialized.");
}

View File

@@ -1,6 +1,7 @@
/** /**
* Brings an existing database up to date with the migrations under * Brings the database up to date with the migrations under migrations/, and
* migrations/, recording each applied one in `schema_migrations`. * records each one it applies in `schema_migrations`. connection.ts runs it
* each time the server opens the database.
* *
* @module * @module
*/ */
@@ -22,6 +23,8 @@ import { up as up0013DumpUrlCanonical } from "./migrations/0013_dump_url_canonic
import { up as up0014ChatReadState } from "./migrations/0014_chat_read_state.ts"; import { up as up0014ChatReadState } from "./migrations/0014_chat_read_state.ts";
import { up as up0015TokenRevocation } from "./migrations/0015_token_revocation.ts"; import { up as up0015TokenRevocation } from "./migrations/0015_token_revocation.ts";
import { up as up0016PerformanceIndexes } from "./migrations/0016_performance_indexes.ts"; import { up as up0016PerformanceIndexes } from "./migrations/0016_performance_indexes.ts";
import { up as up0017CaseInsensitiveUsers } from "./migrations/0017_case_insensitive_users.ts";
import { up as up0018AttachmentResourceType } from "./migrations/0018_attachment_resource_type.ts";
interface Migration { interface Migration {
name: string; name: string;
@@ -29,10 +32,10 @@ interface Migration {
} }
/** /**
* Every migration, in order. Append-only — never edit or reorder a migration * Every migration, in the order they run. A new one goes at the end, along
* once it has shipped. Each `up` must be idempotent: it also runs against * with the same change to schema.sql, and a shipped one is never edited or
* fresh databases created straight from schema.sql, where the change it makes * moved. A database connection.ts made from schema.sql records none of them,
* already exists. * so each `up` must find its change already made and leave it be.
*/ */
const MIGRATIONS: Migration[] = [ const MIGRATIONS: Migration[] = [
{ name: "0001_comment_likes", up: up0001CommentLikes }, { name: "0001_comment_likes", up: up0001CommentLikes },
@@ -54,14 +57,19 @@ const MIGRATIONS: Migration[] = [
{ name: "0014_chat_read_state", up: up0014ChatReadState }, { name: "0014_chat_read_state", up: up0014ChatReadState },
{ name: "0015_token_revocation", up: up0015TokenRevocation }, { name: "0015_token_revocation", up: up0015TokenRevocation },
{ name: "0016_performance_indexes", up: up0016PerformanceIndexes }, { name: "0016_performance_indexes", up: up0016PerformanceIndexes },
{ name: "0017_case_insensitive_users", up: up0017CaseInsensitiveUsers },
{
name: "0018_attachment_resource_type",
up: up0018AttachmentResourceType,
},
]; ];
/** /**
* Applies each migration `db` hasn't recorded yet, each in its own * Applies each migration `db` hasn't recorded yet, in order, each in its own
* transaction. * transaction.
* *
* @throws {Error} When a migration fails; that migration is rolled back and * @throws {Error} When a migration fails. That one is rolled back, the later
* the later ones aren't run. * ones don't run, and the server doesn't start.
*/ */
export function runMigrations(db: DatabaseSync): void { export function runMigrations(db: DatabaseSync): void {
db.exec(`CREATE TABLE IF NOT EXISTS schema_migrations ( db.exec(`CREATE TABLE IF NOT EXISTS schema_migrations (

View File

@@ -1,16 +1,17 @@
/** /**
* Adds comment likes: the `comment_likes` table and the `comments.like_count` * Adds likes on comments, as the `comment_likes` table and a `like_count` on
* counter. * each comment that comment-like-service keeps in step, so that listing
* * comments doesn't count their likes.
* Idempotent: safe to run against a fresh db (already created from schema.sql
* with these columns/tables) or an existing one that predates them.
* *
* @module * @module
*/ */
import type { DatabaseSync } from "node:sqlite"; import type { DatabaseSync } from "node:sqlite";
/** Applies this migration to `db`. */ /**
* Run once by `runMigrations`. Existing comments start with no likes, and a
* database made from schema.sql already has both.
*/
export function up(db: DatabaseSync): void { export function up(db: DatabaseSync): void {
const commentCols = db.prepare(`PRAGMA table_info(comments);`).all() as { const commentCols = db.prepare(`PRAGMA table_info(comments);`).all() as {
name: string; name: string;

View File

@@ -1,15 +1,17 @@
/** /**
* Adds `dumps.custom_thumbnail_mime`, for thumbnails uploaded by the poster. * Adds `dumps.custom_thumbnail_mime`, so a poster can upload a thumbnail that
* * replaces the fetched one. It holds the uploaded image's type, and NULL means
* Idempotent: safe to run against a fresh db (already created from schema.sql * the dump has none.
* with this column) or an existing one that predates it.
* *
* @module * @module
*/ */
import type { DatabaseSync } from "node:sqlite"; import type { DatabaseSync } from "node:sqlite";
/** Applies this migration to `db`. */ /**
* Run once by `runMigrations`. Existing dumps keep their fetched thumbnails,
* and a database made from schema.sql already has the column.
*/
export function up(db: DatabaseSync): void { export function up(db: DatabaseSync): void {
const dumpCols = db.prepare(`PRAGMA table_info(dumps);`).all() as { const dumpCols = db.prepare(`PRAGMA table_info(dumps);`).all() as {
name: string; name: string;

View File

@@ -1,16 +1,18 @@
/** /**
* Adds the `dump_backlinks` table, recording which dumps, comments and * Adds `dump_backlinks`, which records the links from a dump's note or
* descriptions link to a dump, and indexes `dumps.url`. * comments to other dumps, so a dump page can list the dumps related to it.
* * backlink-service fills it, and the index on `dumps.url` lets it find the
* Idempotent: safe to run against a fresh db (already created from schema.sql * dumps an external link was posted as.
* with these) or an existing one that predates them.
* *
* @module * @module
*/ */
import type { DatabaseSync } from "node:sqlite"; import type { DatabaseSync } from "node:sqlite";
/** Applies this migration to `db`. */ /**
* Run once by `runMigrations`. It doesn't scan the notes and comments already
* posted, so their links are recorded only once they are edited.
*/
export function up(db: DatabaseSync): void { export function up(db: DatabaseSync): void {
db.exec(`CREATE TABLE IF NOT EXISTS dump_backlinks ( db.exec(`CREATE TABLE IF NOT EXISTS dump_backlinks (
source_type TEXT NOT NULL, source_type TEXT NOT NULL,

View File

@@ -1,15 +1,16 @@
/** /**
* Adds `users.role` and makes every legacy `is_admin` user an admin. * Adds `users.role`, which replaces the `is_admin` flag so that moderators can
* * exist between users and admins. 0005 then drops the flag.
* Idempotent: safe to run against a fresh db (already created from schema.sql
* with the `role` column) or an existing one that predates it.
* *
* @module * @module
*/ */
import type { DatabaseSync } from "node:sqlite"; import type { DatabaseSync } from "node:sqlite";
/** Applies this migration to `db`. */ /**
* Run once by `runMigrations`. Every user becomes a `user`, except those with
* `is_admin` set, who become admins.
*/
export function up(db: DatabaseSync): void { export function up(db: DatabaseSync): void {
const userCols = db.prepare(`PRAGMA table_info(users);`).all() as { const userCols = db.prepare(`PRAGMA table_info(users);`).all() as {
name: string; name: string;
@@ -20,8 +21,7 @@ export function up(db: DatabaseSync): void {
); );
} }
// Backfill existing admins to role 'admin'. Guarded because the legacy // schema.sql never had is_admin.
// is_admin column is absent on fresh databases (and dropped by 0005).
if (userCols.some((c) => c.name === "is_admin")) { if (userCols.some((c) => c.name === "is_admin")) {
db.exec( db.exec(
`UPDATE users SET role = 'admin' WHERE is_admin = 1 AND role != 'admin';`, `UPDATE users SET role = 'admin' WHERE is_admin = 1 AND role != 'admin';`,

View File

@@ -1,14 +1,15 @@
/** /**
* Drops the legacy is_admin column now that `role` is the single source of * Drops `users.is_admin`, now that 0004 has copied it into `users.role`.
* truth. Idempotent: skips when the column is already gone (fresh databases
* created from schema.sql never had it). Must run after 0004 has backfilled.
* *
* @module * @module
*/ */
import type { DatabaseSync } from "node:sqlite"; import type { DatabaseSync } from "node:sqlite";
/** Applies this migration to `db`. */ /**
* Run once by `runMigrations`, after 0004. A database made from schema.sql
* never had the column.
*/
export function up(db: DatabaseSync): void { export function up(db: DatabaseSync): void {
const userCols = db.prepare(`PRAGMA table_info(users);`).all() as { const userCols = db.prepare(`PRAGMA table_info(users);`).all() as {
name: string; name: string;

View File

@@ -1,16 +1,16 @@
/** /**
* Adds categories: the `categories` table and the `dump_categories` join * Adds categories, which admins create and posters file their dumps under, as
* table. * the `categories` table and the `dump_categories` join table.
*
* Idempotent: safe to run against a fresh db (already created from schema.sql
* with the categories tables) or an existing one that predates them.
* *
* @module * @module
*/ */
import type { DatabaseSync } from "node:sqlite"; import type { DatabaseSync } from "node:sqlite";
/** Applies this migration to `db`. */ /**
* Run once by `runMigrations`. Existing dumps start in no category, and a
* database made from schema.sql already has both tables.
*/
export function up(db: DatabaseSync): void { export function up(db: DatabaseSync): void {
const tables = new Set( const tables = new Set(
(db.prepare( (db.prepare(

View File

@@ -1,15 +1,16 @@
/** /**
* Adds the `password_reset_tokens` table. * Adds `password_reset_tokens`, which holds the single-use tokens that
* * password-reset-service emails to users who forgot their password.
* Idempotent: safe to run against a fresh db (already created from schema.sql
* with password_reset_tokens) or an existing one that predates the table.
* *
* @module * @module
*/ */
import type { DatabaseSync } from "node:sqlite"; import type { DatabaseSync } from "node:sqlite";
/** Applies this migration to `db`. */ /**
* Run once by `runMigrations`. A database made from schema.sql already has the
* table.
*/
export function up(db: DatabaseSync): void { export function up(db: DatabaseSync): void {
const tables = new Set( const tables = new Set(
(db.prepare( (db.prepare(

View File

@@ -1,15 +1,15 @@
/** /**
* Adds the `chat_messages` table. * Adds `chat_messages`, which holds the messages of the site-wide chat.
*
* Idempotent: safe to run against a fresh db (already created from schema.sql
* with this table) or an existing one that predates it.
* *
* @module * @module
*/ */
import type { DatabaseSync } from "node:sqlite"; import type { DatabaseSync } from "node:sqlite";
/** Applies this migration to `db`. */ /**
* Run once by `runMigrations`. A database made from schema.sql already has the
* table.
*/
export function up(db: DatabaseSync): void { export function up(db: DatabaseSync): void {
db.exec(`CREATE TABLE IF NOT EXISTS chat_messages ( db.exec(`CREATE TABLE IF NOT EXISTS chat_messages (
id TEXT NOT NULL PRIMARY KEY, id TEXT NOT NULL PRIMARY KEY,

View File

@@ -1,18 +1,18 @@
/** /**
* Adds an optional self-reference so a chat message can reply to another. * Adds `chat_messages.reply_to_id`, so a message can reply to another. It has
* Kept as a plain column (no FK): when the replied-to message is later deleted * no foreign key on purpose: when the replied-to message is deleted, the id
* the id is retained but the join yields null, letting the UI show a subtle * stays and chat-service's join reads null, so the chat shows "deleted
* "deleted message" reference instead of breaking the insert. * message" where the quote was.
*
* Idempotent: also runs against fresh databases created from schema.sql, where
* the column already exists.
* *
* @module * @module
*/ */
import type { DatabaseSync } from "node:sqlite"; import type { DatabaseSync } from "node:sqlite";
/** Applies this migration to `db`. */ /**
* Run once by `runMigrations`. Existing messages reply to nothing, and a
* database made from schema.sql already has the column.
*/
export function up(db: DatabaseSync): void { export function up(db: DatabaseSync): void {
const cols = db.prepare(`PRAGMA table_info(chat_messages);`).all() as { const cols = db.prepare(`PRAGMA table_info(chat_messages);`).all() as {
name: string; name: string;

View File

@@ -1,25 +1,22 @@
/** /**
* Backfills the `start` parameter on stored YouTube embed URLs. * Adds the `start` parameter to the YouTube embed URLs stored before the
* YouTube provider carried it over. `rich_content.embedUrl` is computed when a
* dump is posted, and used to drop the source URL's `t` (`?t=1265`,
* `?t=1h2m3s`), so those videos played from 0:00.
* *
* `rich_content.embedUrl` is computed once, at dump creation, and previously * It reads the time from the stored source URL and fetches nothing. It keeps
* dropped the timestamp carried by the source URL (`?t=1265`, `?t=1h2m3s`), so * its own copy of the parsing, so that it migrates every database alike should
* those videos always opened at 0:00 in the global player. The provider now * the provider's change.
* translates `t`/`start` into the `start` param the iframe player honours; this
* migration applies the same translation to rows written before that fix.
*
* Purely local — it re-derives the param from the already-stored source URL and
* makes no network calls. Self-contained (no imports from api/services) so it
* stays reproducible even if the provider's parsing evolves later.
*
* Idempotent: rows whose embed URL already carries a `start` are skipped, so a
* fresh database built from schema.sql is simply a no-op.
* *
* @module * @module
*/ */
import type { DatabaseSync } from "node:sqlite"; import type { DatabaseSync } from "node:sqlite";
/** Parses `t`/`start` — plain seconds, `90s`, or `1h2m3s` / `20m5s`. */ /**
* Returns the seconds in `url`'s `t` or `start`, which may be plain seconds,
* `90s`, `20m5s` or `1h2m3s`.
*/
function extractStartSeconds(url: string): number | null { function extractStartSeconds(url: string): number | null {
let raw: string | null; let raw: string | null;
try { try {
@@ -38,7 +35,10 @@ function extractStartSeconds(url: string): number | null {
Number(hms[3] ?? 0); Number(hms[3] ?? 0);
} }
/** Applies this migration to `db`. */ /**
* Run once by `runMigrations`. It leaves alone an embed URL that already has a
* `start`, and a row whose `rich_content` doesn't parse.
*/
export function up(db: DatabaseSync): void { export function up(db: DatabaseSync): void {
const rows = db.prepare( const rows = db.prepare(
`SELECT id, url, rich_content FROM dumps `SELECT id, url, rich_content FROM dumps
@@ -55,7 +55,7 @@ export function up(db: DatabaseSync): void {
try { try {
rich = JSON.parse(row.rich_content); rich = JSON.parse(row.rich_content);
} catch { } catch {
continue; // malformed payload — leave it untouched continue;
} }
if (rich.type !== "youtube" || !rich.embedUrl) continue; if (rich.type !== "youtube" || !rich.embedUrl) continue;

View File

@@ -1,21 +1,14 @@
/** /**
* Moves favicon-shaped values out of `rich_content.thumbnailUrl` into the new * Moves site icons out of `rich_content.thumbnailUrl` into the new
* `faviconUrl` field. * `faviconUrl`. The metadata fetch used to fall back on the page's icon, and
* then on a guessed `/favicon.ico`, so most thumbnails were a 16×16 icon, or a
* 404, that the UI stretched over a 128×72 box. It now stops at real artwork,
* and a missing `thumbnailUrl` means none, which the frontend draws a
* placeholder for. This gives the older rows that meaning.
* *
* The extraction cascade used to end at the page's icon and then at a guessed * It judges the stored URL alone and fetches nothing, so it doesn't fill
* `${origin}/favicon.ico`, so `thumbnailUrl` was almost never empty — it just * `accentColor`. The frontend derives a hue from the hostname when it is
* held a 16×16 icon (or a 404) that the UI then cover-cropped into a 128×72 * missing, and refreshing a dump's metadata fetches the real one.
* box. The cascade now stops at real artwork, and an absent `thumbnailUrl`
* means "no artwork", which is what lets the frontend draw a placeholder.
* This migration gives rows written before that change the same meaning.
*
* Purely local — it classifies the already-stored URL and makes no network
* calls, so `accentColor` is deliberately not backfilled: the frontend derives
* a stable hue from the hostname whenever one is missing, and
* `refreshDumpMetadata` fetches the real color on demand.
*
* Idempotent: rows that already carry a `faviconUrl` are skipped, so a fresh
* database built from schema.sql is a no-op.
* *
* @module * @module
*/ */
@@ -23,12 +16,9 @@
import type { DatabaseSync } from "node:sqlite"; import type { DatabaseSync } from "node:sqlite";
/** /**
* Checks whether a stored thumbnail URL is really a site icon. * Checks whether a stored thumbnail URL is a site icon. It is loose on
* * purpose, since artwork taken for an icon still shows whole on a tinted field,
* Deliberately loose. A false positive (a genuine cover image living under * only not cropped to fill the box. 0012 fixes the rows it got wrong.
* `/assets/icons/`) renders contained on a tinted field instead of
* cover-cropped — mildly wrong, never broken — so chasing them isn't worth the
* extra rules.
*/ */
function isIconUrl(raw: string): boolean { function isIconUrl(raw: string): boolean {
let pathname: string; let pathname: string;
@@ -41,7 +31,11 @@ function isIconUrl(raw: string): boolean {
/\.(ico|svg)$/i.test(pathname); /\.(ico|svg)$/i.test(pathname);
} }
/** Applies this migration to `db`. */ /**
* Run once by `runMigrations`. It leaves alone a row that already has a
* `faviconUrl`, and one whose `rich_content` doesn't parse. 0012 undoes its
* mistakes.
*/
export function up(db: DatabaseSync): void { export function up(db: DatabaseSync): void {
const rows = db.prepare( const rows = db.prepare(
`SELECT id, rich_content FROM dumps `SELECT id, rich_content FROM dumps
@@ -58,7 +52,7 @@ export function up(db: DatabaseSync): void {
try { try {
rich = JSON.parse(row.rich_content); rich = JSON.parse(row.rich_content);
} catch { } catch {
continue; // malformed payload — leave it untouched continue;
} }
if (rich.faviconUrl || !rich.thumbnailUrl) continue; if (rich.faviconUrl || !rich.thumbnailUrl) continue;
if (!isIconUrl(rich.thumbnailUrl)) continue; if (!isIconUrl(rich.thumbnailUrl)) continue;

View File

@@ -1,30 +1,19 @@
/** /**
* Repairs 0011, which was too eager about what counts as a site icon. * Repairs the rows 0011 got wrong. 0011 took any path containing "favicon",
* or ending in `.svg`, for an icon, so artwork such as
* `/wp-content/uploads/2022/04/mir-logo-favicon.png` lost its `thumbnailUrl`,
* and its dump showed as a pull-quote instead of an image in the journal
* mosaic.
* *
* 0011 tested for "favicon" anywhere in the path, so a legitimate `og:image` * This rule judges the file name and its folder instead. It spares
* like mirtitles.org's `/wp-content/uploads/2022/04/mir-logo-favicon.png` was * `mir-logo-favicon.png` and catches `icon_SEARCH.png`, which 0011 missed.
* moved into `faviconUrl` and its `thumbnailUrl` cleared — which also demoted
* the dump from an image card to a pull-quote in the journal mosaic. A blanket
* `.svg` match had the same effect on real artwork.
*
* `isIconUrl` here looks at the *filename* rather than the whole path: an icon
* is named `favicon…`, `apple-touch-icon…` or `icon…`, lives in an `icons/`
* directory, or ends in `.ico`. That both spares `mir-logo-favicon.png` and
* catches `icon_SEARCH.png`, which 0011 missed.
*
* Applied in both directions, and deliberately narrow when reversing: a value
* is only promoted back to `thumbnailUrl` if it matches 0011's rule but not
* this one, so rows written by a normal fetch aren't disturbed.
* *
* @module * @module
*/ */
import type { DatabaseSync } from "node:sqlite"; import type { DatabaseSync } from "node:sqlite";
/** /** 0011's rule, copied as it shipped, to tell which rows it moved. */
* Applies 0011's rule, kept verbatim so its false positives can be
* identified.
*/
function wasIconUrl0011(raw: string): boolean { function wasIconUrl0011(raw: string): boolean {
let pathname: string; let pathname: string;
try { try {
@@ -37,8 +26,9 @@ function wasIconUrl0011(raw: string): boolean {
} }
/** /**
* Checks whether a URL names a site icon, judged on its filename and * Checks whether a URL is a site icon. It is one when the file ends in `.ico`,
* directory. * sits in a `favicon(s)` or `icon(s)` folder, or is named `favicon`,
* `apple-touch-icon` or `icon`, alone or followed by a digit, `-`, `_` or `.`.
*/ */
function isIconUrl(raw: string): boolean { function isIconUrl(raw: string): boolean {
let pathname: string; let pathname: string;
@@ -55,7 +45,12 @@ function isIconUrl(raw: string): boolean {
return /^(favicon|apple-touch-icon|icon)(?=$|[-_.\d])/i.test(basename); return /^(favicon|apple-touch-icon|icon)(?=$|[-_.\d])/i.test(basename);
} }
/** Applies this migration to `db`. */ /**
* Run once by `runMigrations`. It moves a `faviconUrl` back to `thumbnailUrl`
* only when 0011's rule takes it for an icon and this one doesn't. That also
* catches the ones a later fetch stored that this rule doesn't recognise, such
* as a `logo.svg`. It then moves the icons 0011 missed.
*/
export function up(db: DatabaseSync): void { export function up(db: DatabaseSync): void {
const rows = db.prepare( const rows = db.prepare(
`SELECT id, rich_content FROM dumps `SELECT id, rich_content FROM dumps
@@ -73,10 +68,9 @@ export function up(db: DatabaseSync): void {
try { try {
rich = JSON.parse(row.rich_content); rich = JSON.parse(row.rich_content);
} catch { } catch {
continue; // malformed payload — leave it untouched continue;
} }
// Artwork 0011 mistook for an icon: put it back.
if ( if (
rich.faviconUrl && !rich.thumbnailUrl && rich.faviconUrl && !rich.thumbnailUrl &&
wasIconUrl0011(rich.faviconUrl) && !isIconUrl(rich.faviconUrl) wasIconUrl0011(rich.faviconUrl) && !isIconUrl(rich.faviconUrl)
@@ -90,7 +84,6 @@ export function up(db: DatabaseSync): void {
continue; continue;
} }
// An icon 0011's rule didn't recognize: move it now.
if ( if (
rich.thumbnailUrl && !rich.faviconUrl && isIconUrl(rich.thumbnailUrl) rich.thumbnailUrl && !rich.faviconUrl && isIconUrl(rich.thumbnailUrl)
) { ) {

View File

@@ -1,19 +1,12 @@
/** /**
* Adds `dumps.url_canonical` — the lookup key behind the "already dumped?" * Adds `dumps.url_canonical`, the key the create form's "already dumped" hint
* check on the create form — and backfills it for every existing URL dump. * looks a URL up by, and fills it for every URL dump already posted.
* *
* Purely local: it re-derives the key from the URL already stored on each row * It fills the key from each stored URL and fetches nothing. Its helpers are a
* and makes no network calls. * copy of lib/canonical-url.ts as it stood when this shipped: importing the
* * live one would give two databases migrated at different times keys made by
* The helpers below are a deliberate frozen copy of `api/lib/canonical-url.ts` * different rules. A change to lib/canonical-url.ts therefore needs a new
* as it stood when this migration shipped, not an import of it. A migration * migration that fills the keys again.
* runs exactly once per database, so importing the live version would mean two
* databases migrating at different times end up with keys computed under
* different rules. Improving the shared canonicalizer therefore calls for a new
* backfill migration rather than an edit here.
*
* Idempotent: the column is only added when missing and only rows whose key is
* still NULL are touched, so a fresh database built from schema.sql is a no-op.
* *
* @module * @module
*/ */
@@ -98,7 +91,10 @@ function canonicalizeUrl(raw: string): string | null {
return `https://${host}${path}${query ? `?${query}` : ""}${hash}`; return `https://${host}${path}${query ? `?${query}` : ""}${hash}`;
} }
/** Applies this migration to `db`. */ /**
* Run once by `runMigrations`. It fills only the keys still NULL, and leaves
* NULL a URL that isn't http(s).
*/
export function up(db: DatabaseSync): void { export function up(db: DatabaseSync): void {
const columns = db.prepare(`PRAGMA table_info(dumps);`).all() as { const columns = db.prepare(`PRAGMA table_info(dumps);`).all() as {
name: string; name: string;

View File

@@ -1,21 +1,22 @@
/** /**
* Chat unread state lives on the server so it survives reloads and agrees * Moves the chat's unread state to the server, so that it survives a reload
* across tabs and devices: `users.chat_read_at` is the created_at of the newest * and agrees across tabs and devices. `users.chat_read_at` holds the time of
* message the user has seen. Existing users start caught up (now) rather than * the newest message the user has seen.
* with the whole history unread; new users fall back to their own created_at.
* *
* The history cursor now pages on (created_at, id) so two messages sharing a * It also replaces the index on `created_at` with one on `(created_at, id)`,
* millisecond can't be skipped at a page boundary; the index follows suit. * which the history now pages on, so that two messages from the same
* * millisecond can't fall between two pages.
* Idempotent: also runs against fresh databases created from schema.sql, where
* the column and index already exist.
* *
* @module * @module
*/ */
import type { DatabaseSync } from "node:sqlite"; import type { DatabaseSync } from "node:sqlite";
/** Applies this migration to `db`. */ /**
* Run once by `runMigrations`. Existing users start with every message read,
* instead of the whole history unread. A user created later starts with a
* NULL `chat_read_at`, which chat-service reads as the user's creation time.
*/
export function up(db: DatabaseSync): void { export function up(db: DatabaseSync): void {
const cols = db.prepare(`PRAGMA table_info(users);`).all() as { const cols = db.prepare(`PRAGMA table_info(users);`).all() as {
name: string; name: string;

View File

@@ -1,18 +1,17 @@
/** /**
* Changing or resetting a password must end every session opened before it. * Adds `users.tokens_valid_after`, so that changing a password ends every
* Tokens are stateless JWTs, so each user carries a cut-off instead: * session opened before. Tokens are stateless JWTs, so lib/auth.ts refuses a
* `users.tokens_valid_after` (ISO), and a token issued before it is refused. * token issued before that time instead.
* NULL — the state every existing user starts in — means no cut-off.
*
* Idempotent: also runs against fresh databases created from schema.sql, where
* the column already exists.
* *
* @module * @module
*/ */
import type { DatabaseSync } from "node:sqlite"; import type { DatabaseSync } from "node:sqlite";
/** Applies this migration to `db`. */ /**
* Run once by `runMigrations`. Existing users start with NULL, which revokes no
* token.
*/
export function up(db: DatabaseSync): void { export function up(db: DatabaseSync): void {
const cols = db.prepare(`PRAGMA table_info(users);`).all() as { const cols = db.prepare(`PRAGMA table_info(users);`).all() as {
name: string; name: string;

View File

@@ -1,24 +1,21 @@
/** /**
* Indexes for the queries that ran as full scans. * Adds indexes for the queries that scanned a whole table. The feeds page
* * dumps by `created_at`, and a user's dumps and upvotes by `(user_id,
* - feeds page by created_at (newest first), a user's dumps and upvotes by * created_at)`, which replace the indexes on `user_id` alone. Dumps and
* (user_id, created_at) — these replace the bare user_id indexes; * playlists are looked up by slug. Each foreign key with ON DELETE gets an
* - dump and playlist pages are looked up by slug; * index on its child column, which SQLite otherwise scans on every delete of
* - the child side of every ON DELETE CASCADE / SET NULL foreign key, which * a parent row, and the followers of a user or playlist are read on every new
* SQLite otherwise scans on each parent delete (a comment's replies, a user's * dump. Mentions look usernames up by `lower(username)`.
* comments, follows, chat messages, invites, reset tokens), and the
* followers of a user or playlist, read on every new dump;
* - @mentions resolve usernames case-insensitively, by lower(username).
*
* Idempotent: also runs against fresh databases created from schema.sql, where
* these indexes already exist.
* *
* @module * @module
*/ */
import type { DatabaseSync } from "node:sqlite"; import type { DatabaseSync } from "node:sqlite";
/** Applies this migration to `db`. */ /**
* Run once by `runMigrations`. A database made from schema.sql already has
* these indexes.
*/
export function up(db: DatabaseSync): void { export function up(db: DatabaseSync): void {
db.exec(` db.exec(`
DROP INDEX IF EXISTS idx_dumps_user; DROP INDEX IF EXISTS idx_dumps_user;

View File

@@ -0,0 +1,36 @@
/**
* Makes usernames and emails unique regardless of case. Usernames were unique
* only as typed, so "Bob" and "bob" could both sign up, and a mention, which
* looks names up in lower case, then reached one of them at random. Emails
* had no constraint at all: two sign-ups racing past the service's check could
* share an address, and password resets are looked up by address.
*
* @module
*/
import type { DatabaseSync } from "node:sqlite";
/**
* Run once by `runMigrations`. It refuses to run, naming the clashes, when two
* accounts already share a name or an address in some case: which one keeps
* it is for whoever runs the site to decide, by editing one of them.
*/
export function up(db: DatabaseSync): void {
for (const column of ["username", "email"]) {
const clashes = db.prepare(
`SELECT lower(${column}) AS value FROM users GROUP BY lower(${column}) HAVING COUNT(*) > 1;`,
).all() as { value: string }[];
if (clashes.length > 0) {
throw new Error(
`Several accounts share a ${column} in different cases: ${
clashes.map((c) => c.value).join(", ")
}. Change all but one of each, then restart.`,
);
}
}
db.exec(`
DROP INDEX IF EXISTS idx_users_username_lower;
CREATE UNIQUE INDEX IF NOT EXISTS idx_users_username_lower ON users(lower(username));
CREATE UNIQUE INDEX IF NOT EXISTS idx_users_email_lower ON users(lower(email));
`);
}

View File

@@ -0,0 +1,34 @@
/**
* Adds `attachments.resource_type`, the kind of thing an attachment's
* `resource_id` names. Without it, an attachment couldn't tell whose it was:
* one pasted into a private dump was served to anyone with its URL, and one
* whose dump, comment or message was deleted was kept for good.
*
* @module
*/
import type { DatabaseSync } from "node:sqlite";
/**
* Run once by `runMigrations`. Existing bound attachments get the type of the
* table that holds their resource; one whose resource is gone gets none, and
* the startup purge deletes it. A database made from schema.sql already has
* the column.
*/
export function up(db: DatabaseSync): void {
const cols = db.prepare(`PRAGMA table_info(attachments);`).all() as {
name: string;
}[];
if (cols.some((c) => c.name === "resource_type")) return;
db.exec(`
ALTER TABLE attachments ADD COLUMN resource_type TEXT;
UPDATE attachments SET resource_type = CASE
WHEN resource_id IN (SELECT id FROM dumps) THEN 'dump'
WHEN resource_id IN (SELECT id FROM comments) THEN 'comment'
WHEN resource_id IN (SELECT id FROM playlists) THEN 'playlist'
WHEN resource_id IN (SELECT id FROM chat_messages) THEN 'chat'
WHEN resource_id IN (SELECT id FROM users) THEN 'user'
END
WHERE resource_id IS NOT NULL;
`);
}

View File

@@ -175,13 +175,15 @@ CREATE TABLE invites (
CREATE INDEX idx_invites_inviter ON invites(inviter_id); CREATE INDEX idx_invites_inviter ON invites(inviter_id);
CREATE INDEX idx_users_invited_by ON users(invited_by); CREATE INDEX idx_users_invited_by ON users(invited_by);
CREATE INDEX idx_users_username_lower ON users(lower(username)); CREATE UNIQUE INDEX idx_users_username_lower ON users(lower(username));
CREATE UNIQUE INDEX idx_users_email_lower ON users(lower(email));
CREATE TABLE attachments ( CREATE TABLE attachments (
id TEXT PRIMARY KEY, id TEXT PRIMARY KEY,
resource_id TEXT, resource_id TEXT,
mime TEXT NOT NULL, mime TEXT NOT NULL,
created_at TEXT NOT NULL created_at TEXT NOT NULL,
resource_type TEXT
); );
CREATE INDEX idx_attachments_resource ON attachments(resource_id); CREATE INDEX idx_attachments_resource ON attachments(resource_id);

View File

@@ -1,57 +1,75 @@
/** /**
* Housekeeping run when the database is opened. * Deletes what has expired and seeds the first admin, each time connection.ts
* opens the database.
* *
* @module * @module
*/ */
import { randomBytes, scryptSync } from "node:crypto";
import type { DatabaseSync } from "node:sqlite"; import type { DatabaseSync } from "node:sqlite";
import { import {
ATTACHMENTS_DIR, ATTACHMENTS_DIR,
ORPHANED_ATTACHMENTS_RETENTION_HOURS, ORPHANED_ATTACHMENTS_RETENTION_HOURS,
UNUSED_INVITES_RETENTION_DAYS, UNUSED_INVITES_RETENTION_DAYS,
} from "../config.ts"; } from "../config.ts";
import { hashPassword } from "../lib/password.ts";
/** Returns the time `ms` ago as an ISO string, the format dates are stored in. */
function isoAgo(ms: number): string {
return new Date(Date.now() - ms).toISOString();
}
/** /**
* Runs once when the database is opened, after the migrations: drops what has * Runs after the migrations, each time the server starts. It deletes used or
* expired (reset tokens, unused invites, orphaned attachments), and seeds the * expired reset tokens, invites past their validity, and attachments nothing
* default admin (admin/admin) on an empty instance. * links to or whose resource is gone. On an instance with no users, it creates the user `admin` with the
* password `admin`, which whoever installs it must change.
*/ */
export async function runStartupTasks(db: DatabaseSync): Promise<void> { export async function runStartupTasks(db: DatabaseSync): Promise<void> {
// Purge expired/used password reset tokens // Dates are stored as ISO strings and compared as text, so the cutoffs are
// ISO strings too. SQLite's datetime() has a space where ISO has a `T`.
db.prepare( db.prepare(
`DELETE FROM password_reset_tokens WHERE expires_at < datetime('now') OR used_at IS NOT NULL;`, `DELETE FROM password_reset_tokens WHERE expires_at < ? OR used_at IS NOT NULL;`,
).run(); ).run(isoAgo(0));
// Purge expired unused invites
db.prepare( db.prepare(
`DELETE FROM invites WHERE used_at IS NULL AND created_at < datetime('now', '-${UNUSED_INVITES_RETENTION_DAYS} days');`, `DELETE FROM invites WHERE used_at IS NULL AND created_at < ?;`,
).run(); ).run(isoAgo(UNUSED_INVITES_RETENTION_DAYS * 86_400_000));
// Prune orphaned attachments (uploaded but never linked to a resource) older than 1 hour // An image is uploaded as soon as it's pasted, and linked only when the text
// it's pasted into is saved, so a fresh unlinked one is left alone. One whose
// dump, comment, playlist, message or user is gone goes whatever its age.
const attachmentCutoff = isoAgo(
ORPHANED_ATTACHMENTS_RETENTION_HOURS * 3_600_000,
);
const ORPHANED = `
(resource_id IS NULL AND created_at < ?)
OR (resource_id IS NOT NULL AND NOT CASE resource_type
WHEN 'dump' THEN EXISTS (SELECT 1 FROM dumps WHERE id = resource_id)
WHEN 'comment' THEN EXISTS (SELECT 1 FROM comments WHERE id = resource_id AND deleted = 0)
WHEN 'playlist' THEN EXISTS (SELECT 1 FROM playlists WHERE id = resource_id)
WHEN 'chat' THEN EXISTS (SELECT 1 FROM chat_messages WHERE id = resource_id)
WHEN 'user' THEN EXISTS (SELECT 1 FROM users WHERE id = resource_id)
ELSE 0
END)`;
const orphanedAttachments = db.prepare( const orphanedAttachments = db.prepare(
`SELECT id FROM attachments WHERE resource_id IS NULL AND created_at < datetime('now', '-${ORPHANED_ATTACHMENTS_RETENTION_HOURS} hour');`, `SELECT id FROM attachments WHERE ${ORPHANED};`,
).all() as { id: string }[]; ).all(attachmentCutoff) as { id: string }[];
if (orphanedAttachments.length > 0) { if (orphanedAttachments.length > 0) {
for (const { id } of orphanedAttachments) { for (const { id } of orphanedAttachments) {
await Deno.remove(`${ATTACHMENTS_DIR}/${id}`).catch(() => {}); await Deno.remove(`${ATTACHMENTS_DIR}/${id}`).catch(() => {});
} }
db.prepare( db.prepare(`DELETE FROM attachments WHERE ${ORPHANED};`)
`DELETE FROM attachments WHERE resource_id IS NULL AND created_at < datetime('now', '-${ORPHANED_ATTACHMENTS_RETENTION_HOURS} hour');`, .run(attachmentCutoff);
).run();
} }
// Create default admin user if no users exist // Registration needs an invite, so the first user can only come from here.
const userCount = db.prepare(`SELECT COUNT(*) as count FROM users`).get() as { const userCount = db.prepare(`SELECT COUNT(*) as count FROM users`).get() as {
count: number; count: number;
}; };
if (userCount.count === 0) { if (userCount.count === 0) {
const salt = randomBytes(16).toString("hex");
const hash = scryptSync("admin", salt, 64).toString("hex");
const passwordHash = `${hash}.${salt}`;
db.prepare( db.prepare(
`INSERT INTO users (id, username, password_hash, role, created_at, email) VALUES (?, 'admin', ?, 'admin', datetime('now'), 'admin@localhost')`, `INSERT INTO users (id, username, password_hash, role, created_at, email) VALUES (?, 'admin', ?, 'admin', ?, 'admin@localhost')`,
).run(crypto.randomUUID(), passwordHash); ).run(crypto.randomUUID(), await hashPassword("admin"), isoAgo(0));
console.log( console.log(
"Created default admin user (username: admin, password: admin)", "Created default admin user (username: admin, password: admin)",
); );

View File

@@ -1,25 +1,40 @@
/** /**
* The exception that carries an HTTP error out of any layer below a route. * Defines the exception that carries an HTTP error from any layer up to the
* error middleware.
* *
* @module * @module
*/ */
import type { APIErrorCode } from "../../shared/api.ts"; import { APIErrorCode } from "../../shared/api.ts";
/** Passed to an {@link APIException}; any other status is a type error. */
export type APIStatus = 400 | 401 | 403 | 404 | 409 | 429 | 500 | 502;
const CODES: Record<APIStatus, APIErrorCode> = {
400: APIErrorCode.BAD_REQUEST,
401: APIErrorCode.UNAUTHORIZED,
403: APIErrorCode.FORBIDDEN,
404: APIErrorCode.NOT_FOUND,
409: APIErrorCode.CONFLICT,
429: APIErrorCode.RATE_LIMITED,
500: APIErrorCode.SERVER_ERROR,
502: APIErrorCode.BAD_GATEWAY,
};
/** /**
* An error meant for the client, thrown anywhere below a route; the error * Thrown by a route, a service or a remote fetch to answer the request with an
* middleware turns it into an `APIFailure` response. * error. The error middleware sends it as an `APIFailure`, message included,
* so the message must be fit for the user. Use 401 for a caller who isn't
* signed in, 403 for one who is but may not, and 404 rather than 403 when the
* caller mustn't learn that the thing exists.
*/ */
export class APIException extends Error { export class APIException extends Error {
/** The error code the response carries. */
readonly code: APIErrorCode; readonly code: APIErrorCode;
/** The HTTP status the response is sent with. */ readonly status: APIStatus;
readonly status: number;
/** Creates an exception answering `status` with `code` and `message`. */ constructor(status: APIStatus, message: string) {
constructor(code: APIErrorCode, status: number, message: string) {
super(message); super(message);
this.code = code;
this.status = status; this.status = status;
this.code = CODES[status];
} }
} }

View File

@@ -1,26 +1,23 @@
/** /**
* Authenticates the tokens a request carries: session tokens in the * Checks session and media tokens against the live user row, not just their
* `Authorization` header and media tokens in a media URL's query string. * signature. The auth middleware, the WebSocket route, the media routes and the
* routes open to anonymous callers all authenticate through it.
* *
* @module * @module
*/ */
import type { Context } from "@oak/oak"; import type { Context } from "@oak/oak";
import { verifyJWT, verifyMediaToken } from "./jwt.ts"; import { verifyJWT, verifyMediaToken } from "./jwt.ts";
import { APIErrorCode } from "../../shared/api.ts";
import { APIException } from "./api-exception.ts"; import { APIException } from "./api-exception.ts";
import type { AuthPayload } from "../model/auth.ts"; import type { AuthPayload } from "../model/auth.ts";
import type { Role } from "../../shared/user.ts"; import type { Role } from "../../shared/user.ts";
import { getSessionState } from "../services/user-service.ts"; import { getSessionState } from "../services/user-service.ts";
/** /**
* Checks a signature-verified token against the live user row: the user must * Returns the live role of a verified token's user, or null when the user is
* still exist, and the token must not predate their last password change * gone or the token predates their last password change
* (`users.tokens_valid_after`). Returns the live role, or null to refuse. * (`users.tokens_valid_after`). `iat` counts whole seconds, so a token issued
* * in the same second as the change survives it.
* `iat` has one-second resolution, so a token issued in the same second as the
* change survives it; one issued after it (the fresh one the change hands back)
* always does.
*/ */
function checkTokenOwner( function checkTokenOwner(
userId: string, userId: string,
@@ -38,9 +35,10 @@ function checkTokenOwner(
} }
/** /**
* Verifies a session token end to end: signature, expiry, a user that still * Returns the payload of a valid session token, or null. Besides the signature
* exists and no password change since it was issued. The role comes from the * and expiry, it refuses a token whose user is gone or which predates a
* DB, not the token, since tokens last 7 days and roles change under them. * password change. The returned role comes from the database, not the token,
* since a token lasts 7 days and a role can change under it.
*/ */
export async function authenticate( export async function authenticate(
token: string, token: string,
@@ -53,8 +51,9 @@ export async function authenticate(
} }
/** /**
* Resolves the `?token=` on a media URL to the viewer's user id. Takes a media * Returns the user id behind the `?token=` of a file, thumbnail or playlist
* token, or a session token from a client that predates them. * cover URL, or null when it isn't valid. A session token is accepted too, for
* clients from before media tokens.
*/ */
export async function authenticateMedia(token: string): Promise<string | null> { export async function authenticateMedia(token: string): Promise<string | null> {
const media = await verifyMediaToken(token); const media = await verifyMediaToken(token);
@@ -65,18 +64,13 @@ export async function authenticateMedia(token: string): Promise<string | null> {
} }
/** /**
* Returns the user id of an optional Bearer token, or null for an anonymous * Returns the caller's user id on a route open to anonymous callers, or null
* request. * when the request has no token. A placeholder such as `Bearer undefined`
* counts as no token. An invalid token throws instead of reading as anonymous,
* so that a stale session gets a 401 the client signs out on, rather than a
* misleading 404 on a private resource.
* *
* Distinguishes *missing* credentials from *invalid* ones: * @throws {APIException} 401 when a token is present but invalid.
* - No `Authorization: Bearer …` header (or a placeholder value such as the
* `Bearer undefined` that the client emits when logged out) → anonymous (null).
* - A real token that fails verification (expired / bad signature / revoked) →
* throws 401, so a stale client session surfaces as an auth error it can react
* to, instead of being silently downgraded to anonymous (which yields a
* misleading 404 on private resources).
*
* @throws {APIException} 401 when a token is present but fails verification.
*/ */
export async function parseOptionalAuth(ctx: Context): Promise<string | null> { export async function parseOptionalAuth(ctx: Context): Promise<string | null> {
const authHeader = ctx.request.headers.get("Authorization"); const authHeader = ctx.request.headers.get("Authorization");
@@ -87,7 +81,7 @@ export async function parseOptionalAuth(ctx: Context): Promise<string | null> {
const payload = await authenticate(token); const payload = await authenticate(token);
if (!payload) { if (!payload) {
throw new APIException(APIErrorCode.UNAUTHORIZED, 401, "Invalid token"); throw new APIException(401, "Invalid token");
} }
return payload.userId; return payload.userId;
} }

View File

@@ -1,18 +1,13 @@
/** /**
* The canonical form of a URL, used *only* to recognise that two links point at the * Reduces a URL to a key shared by every link to the same page or video.
* same thing — the duplicate check the create form runs while it fetches a * `dump-service` stores it in `dumps.url_canonical` when a
* preview. It is a lookup key, never something we display or fetch: `dumps.url` * dump is posted or edited, and `GET /api/dumps/by-url` matches on it
* keeps the exact string the poster submitted. * to warn the create form of a repost. The key is never fetched or shown, so it
* may be lossy: it forces `https`, drops `www.`, tracking parameters and
* timestamps.
* *
* Because it is only ever compared against other canonical forms, it may be * Migration 0013 backfilled existing rows with a frozen copy of these rules.
* lossy in ways a real URL never could be — it forces `https`, drops `www.`, * Changing them needs a new backfill migration, or old rows keep old keys.
* and throws away share/tracking parameters and timestamps, so
* `http://www.example.com/a/?utm_source=x` and `https://example.com/a` collapse
* to one key.
*
* Stored in `dumps.url_canonical`. Migration 0013 carries a frozen copy of this
* logic to backfill existing rows; changing the rules here therefore needs a
* new backfill migration, or old rows keep keys computed under the old rules.
* *
* @module * @module
*/ */
@@ -42,7 +37,7 @@ function isTrackingParam(key: string): boolean {
return k.startsWith("utm_") || TRACKING_PARAMS.has(k); return k.startsWith("utm_") || TRACKING_PARAMS.has(k);
} }
/** YouTube's hosts, already stripped of a leading "www.". */ /** YouTube's hosts, without their "www.". */
const YOUTUBE_HOSTS = new Set([ const YOUTUBE_HOSTS = new Set([
"youtube.com", "youtube.com",
"m.youtube.com", "m.youtube.com",
@@ -51,12 +46,23 @@ const YOUTUBE_HOSTS = new Set([
]); ]);
/** /**
* Returns the video a YouTube URL points at, in any of the shapes people paste * Returns `u`'s host in lower case and without "www.", when it is one of
* (`youtu.be/ID`, `/watch?v=ID`, `/embed/ID`, `/shorts/ID`, `/live/ID`). * YouTube's (youtu.be aside), and null otherwise. The YouTube preview provider
* Timestamps are deliberately ignored: the same video linked at 2:30 is still * uses it too, so both agree on what a YouTube link is.
* the same video for duplicate purposes.
*/ */
function youtubeVideoId( export function youtubeHost(u: URL): string | null {
const host = u.hostname.toLowerCase().replace(/^www\./, "");
return YOUTUBE_HOSTS.has(host) ? host : null;
}
/**
* Returns the id of the video a YouTube URL points at, in any of the shapes
* people paste (`youtu.be/ID`, `/watch?v=ID`, `/embed/ID`, `/shorts/ID`,
* `/live/ID`), and null for any other URL. `host` is lower case, without
* "www.". The canonical key drops the timestamp, since the same video linked
* at 2:30 is still a repost; the YouTube provider reads it separately.
*/
export function youtubeVideoId(
host: string, host: string,
pathname: string, pathname: string,
params: URLSearchParams, params: URLSearchParams,
@@ -71,8 +77,8 @@ function youtubeVideoId(
} }
/** /**
* Returns the canonical form of `raw` (see the module doc), or null when it * Returns the canonical key of `raw`, or null when it isn't an http(s) URL. A
* isn't an http(s) URL. * YouTube video or playlist becomes one fixed URL whatever shape it came in.
*/ */
export function canonicalizeUrl(raw: string): string | null { export function canonicalizeUrl(raw: string): string | null {
let u: URL; let u: URL;
@@ -94,7 +100,7 @@ export function canonicalizeUrl(raw: string): string | null {
: null; : null;
if (listId) return `https://youtube.com/playlist?list=${listId}`; if (listId) return `https://youtube.com/playlist?list=${listId}`;
// A single trailing slash is never meaningful; "/" itself becomes "". // Trailing slashes are dropped; "/" itself becomes "".
const path = u.pathname.replace(/\/+$/, ""); const path = u.pathname.replace(/\/+$/, "");
const params = [...u.searchParams.entries()] const params = [...u.searchParams.entries()]
@@ -102,8 +108,8 @@ export function canonicalizeUrl(raw: string): string | null {
.sort(([a, av], [b, bv]) => a.localeCompare(b) || av.localeCompare(bv)); .sort(([a, av], [b, bv]) => a.localeCompare(b) || av.localeCompare(bv));
const query = new URLSearchParams(params).toString(); const query = new URLSearchParams(params).toString();
// A bare "#section" anchor points into the same page, so it is dropped — // A "#section" anchor points into the same page and is dropped, but "#/route"
// but "#/route" and "#!/route" address distinct pages of a hash-routed app. // and "#!/route" are distinct pages of a hash-routed app.
const hash = /^#!?\//.test(u.hash) ? u.hash : ""; const hash = /^#!?\//.test(u.hash) ? u.hash : "";
return `https://${host}${path}${query ? `?${query}` : ""}${hash}`; return `https://${host}${path}${query ? `?${query}` : ""}${hash}`;

View File

@@ -1,6 +1,7 @@
/** /**
* The built frontend's `index.html`, with its runtime placeholders filled: the * Fills the placeholders of the frontend's `index.html` with settings read at
* site name, emoji, icon version and Bandcamp player mode. * runtime, so they change with a restart and no rebuild. The static and OG
* middlewares serve what it returns.
* *
* @module * @module
*/ */
@@ -8,18 +9,20 @@
import { BANDCAMP_PLAYER, OG_SITE_NAME, SITE_EMOJI } from "../config.ts"; import { BANDCAMP_PLAYER, OG_SITE_NAME, SITE_EMOJI } from "../config.ts";
import { emojiToCodepoint } from "./site-icons.ts"; import { emojiToCodepoint } from "./site-icons.ts";
/** The cache-busting version of the icon URLs. */ /**
* Versions the icon URLs, so browsers fetch new icons when the emoji changes.
*/
const ICON_VERSION = emojiToCodepoint(SITE_EMOJI); const ICON_VERSION = emojiToCodepoint(SITE_EMOJI);
/** /**
* `index.html` with its placeholders filled, by root. Kept until the file * Caches each root's filled file until its mtime changes, as a rebuild under a
* changes (a rebuild under a running server): a stat per navigation instead of * running server does, so a navigation costs a stat instead of a read.
* a read and four replaceAll.
*/ */
const indexHtml = new Map<string, { mtime: number; html: string }>(); const indexHtml = new Map<string, { mtime: number; html: string }>();
/** /**
* Returns `${root}/index.html` with its placeholders filled. * Returns `${root}/index.html` with the site name, emoji, icon version and
* Bandcamp player mode filled in.
* *
* @throws {Error} When the file can't be read. * @throws {Error} When the file can't be read.
*/ */

View File

@@ -1,14 +1,14 @@
/** /**
* Signed tokens and password hashes: session tokens, media tokens, password * Signs and verifies the server's three kinds of token (session, media and
* reset tokens, and scrypt hashing. * password reset), all HS256 with the one key. Only a token's signature and
* expiry are checked here: `lib/auth.ts` checks a session or media token
* against its user.
* *
* Importing this module throws when `GERBEUR_JWT_SECRET` is unset. * Importing this module throws when `GERBEUR_JWT_SECRET` is unset.
* *
* @module * @module
*/ */
import { randomBytes, scrypt, timingSafeEqual } from "node:crypto";
import { Buffer } from "node:buffer";
import { jwtVerify, SignJWT } from "@panva/jose"; import { jwtVerify, SignJWT } from "@panva/jose";
import { import {
@@ -26,22 +26,26 @@ if (!JWT_SECRET) {
} }
const JWT_KEY = new TextEncoder().encode(JWT_SECRET); const JWT_KEY = new TextEncoder().encode(JWT_SECRET);
// ── Password reset tokens ───────────────────────────────────────────────────── /**
* Signs a password reset token for `userId`, valid for an hour. The token
/** Signs a single-use password reset token for `userId`, valid for an hour. */ * isn't single-use by itself: `requestPasswordReset` stores it, and
* `resetPassword` marks it used.
*/
export async function createPasswordResetToken( export async function createPasswordResetToken(
userId: string, userId: string,
): Promise<string> { ): Promise<string> {
return await new SignJWT({ purpose: "password-reset", userId }) return await new SignJWT({ purpose: "password-reset", userId })
.setProtectedHeader({ alg: "HS256" }) .setProtectedHeader({ alg: "HS256" })
// Two requests in the same second would otherwise sign the same token,
// which is the primary key of `password_reset_tokens`.
.setJti(crypto.randomUUID()) .setJti(crypto.randomUUID())
.setExpirationTime("1h") .setExpirationTime("1h")
.sign(JWT_KEY); .sign(JWT_KEY);
} }
/** /**
* Verifies a password reset token and returns its payload, or null when it is * Returns the payload of a password reset token, or null when it is invalid,
* invalid, expired or not a reset token. * expired or another kind of token. Whether it was used is the caller's check.
*/ */
export async function verifyPasswordResetToken( export async function verifyPasswordResetToken(
token: string, token: string,
@@ -55,29 +59,29 @@ export async function verifyPasswordResetToken(
} }
} }
// ── Auth tokens ─────────────────────────────────────────────────────────────── /**
* Signs a session token carrying `payload`, valid for 7 days. The users routes
/** Signs a session token carrying `payload`, valid for 7 days. */ * issue it on sign-up, sign-in and password change, and the client sends it as
* a bearer token.
*/
export async function createJWT( export async function createJWT(
payload: Omit<AuthPayload, "exp">, payload: Omit<AuthPayload, "exp">,
): Promise<string> { ): Promise<string> {
return await new SignJWT(payload) return await new SignJWT(payload)
.setProtectedHeader({ alg: "HS256" }) .setProtectedHeader({ alg: "HS256" })
// `iat` is what a password change revokes against (users.tokens_valid_after). // A password change revokes tokens by comparing `iat` with
// `users.tokens_valid_after`.
.setIssuedAt() .setIssuedAt()
.setExpirationTime("7d") .setExpirationTime("7d")
.sign(JWT_KEY); .sign(JWT_KEY);
} }
// ── Media tokens ──────────────────────────────────────────────────────────────
/** /**
* Signs a media token for `userId`, valid for 7 days. * Signs a media token for `userId`, valid for 7 days. Media elements can't send
* * an `Authorization` header, so the URLs of private files, thumbnails and
* Media elements can't send an Authorization header, so a private dump's file * playlist covers carry this token as `?token=`, where it lands in logs and
* and thumbnail URLs carry a token in the query string, where it lands in * browser history. It only reads media, so a leaked URL doesn't expose the
* access logs and browser history. That token is this one: it only reads media, * account.
* so a leaked URL exposes the owner's private files, not their account.
*/ */
export async function createMediaToken(userId: string): Promise<string> { export async function createMediaToken(userId: string): Promise<string> {
return await new SignJWT({ purpose: "media", userId }) return await new SignJWT({ purpose: "media", userId })
@@ -88,8 +92,8 @@ export async function createMediaToken(userId: string): Promise<string> {
} }
/** /**
* Verifies a media token and returns its user id and issue time, or null when * Returns the user id and issue time of a media token, or null when it is
* it is invalid, expired or not a media token. * invalid, expired or another kind of token.
*/ */
export async function verifyMediaToken( export async function verifyMediaToken(
token: string, token: string,
@@ -106,8 +110,9 @@ export async function verifyMediaToken(
} }
/** /**
* Verifies a session token's signature and expiry and returns its payload, or * Returns the payload of a session token whose signature and expiry are valid,
* null. Checking it against the live user is `authenticate`'s job (lib/auth.ts). * or null. Call `authenticate` (`lib/auth.ts`) instead, which also checks the
* token against its user.
*/ */
export async function verifyJWT(token: string): Promise<AuthPayload | null> { export async function verifyJWT(token: string): Promise<AuthPayload | null> {
try { try {
@@ -123,39 +128,3 @@ export async function verifyJWT(token: string): Promise<AuthPayload | null> {
return null; return null;
} }
} }
/** Hashes `password` with scrypt and a random salt, as `hash.salt` in hex. */
export function hashPassword(password: string): Promise<string> {
const salt = randomBytes(16).toString("hex");
return new Promise((resolve, reject) => {
scrypt(password, salt, 64, (err, derivedKey) => {
if (err) reject(err);
else resolve(`${derivedKey.toString("hex")}.${salt}`);
});
});
}
/** Checks `password` against a hash made by {@link hashPassword}. */
export function verifyPassword(
password: string,
storedHash: string,
): Promise<boolean> {
const [hash, salt] = storedHash.split(".");
// A malformed stored hash can't match anything; scrypt would throw on the
// missing salt instead of answering.
if (!hash || !salt) return Promise.resolve(false);
return new Promise((resolve, reject) => {
scrypt(password, salt, 64, (err, derivedKey) => {
if (err) reject(err);
else {
const expected = Buffer.from(hash ?? "", "hex");
resolve(
expected.length === derivedKey.length &&
timingSafeEqual(expected, derivedKey),
);
}
});
});
}

24
api/lib/like.ts Normal file
View File

@@ -0,0 +1,24 @@
/**
* Builds the patterns of the search queries' `LIKE`, for the dump, playlist
* and user searches. A query's `%` and `_` are matched as themselves rather
* than as wildcards: usernames hold underscores, and "a_b" must not find
* "axb".
*
* @module
*/
/**
* Returns the pattern matching text that contains `query`, or that starts
* with it when `where` is `"prefix"`. The SQL must read `LIKE ? ESCAPE '\'`,
* which the {@link LIKE} fragment spells.
*/
export function likePattern(
query: string,
where: "anywhere" | "prefix" = "anywhere",
): string {
const escaped = query.replace(/[\\%_]/g, "\\$&");
return where === "prefix" ? `${escaped}%` : `%${escaped}%`;
}
/** Spells `LIKE ? ESCAPE '\'`, for a {@link likePattern} parameter. */
export const LIKE = "LIKE ? ESCAPE '\\'";

View File

@@ -1,6 +1,7 @@
/** /**
* Page-based pagination: reading `page` and `limit` from a query string, and * Reads the page a list route's query string asks for, and cuts the rows a
* cutting fetched rows into a page. * service fetched into that page. The routes send the page as
* `PaginatedData`.
* *
* @module * @module
*/ */
@@ -8,9 +9,9 @@
import { PAGINATION_DEFAULT_LIMIT, PAGINATION_MAX_LIMIT } from "../config.ts"; import { PAGINATION_DEFAULT_LIMIT, PAGINATION_MAX_LIMIT } from "../config.ts";
/** /**
* Reads the `page` and `limit` query parameters. `page` is at least 1 and * Reads the `page` and `limit` query parameters of a list route. It never
* defaults to 1; `limit` is clamped to [1, {@link PAGINATION_MAX_LIMIT}] and * fails: a missing or invalid `page` reads as 1, and `limit` is kept between 1
* defaults to `defaultLimit`. * and {@link PAGINATION_MAX_LIMIT}.
*/ */
export function parsePagination( export function parsePagination(
params: URLSearchParams, params: URLSearchParams,
@@ -30,18 +31,19 @@ export function parsePagination(
return { page, limit }; return { page, limit };
} }
/** One page of a list, and whether another follows. */ /**
* Returned to the routes by the services' list functions, and sent as
* `PaginatedData`.
*/
export interface Page<T> { export interface Page<T> {
/** The page's items, in order. */
items: T[]; items: T[];
/** Whether another page follows this one. */
hasMore: boolean; hasMore: boolean;
} }
/** /**
* Turns rows fetched with `LIMIT limit + 1` into a page: the extra row, if it * Turns rows fetched with `LIMIT limit + 1` into a page. The extra row, when it
* came back, is dropped and says there's a next page. No COUNT(*) over the * comes back, is dropped and tells that another page follows, which spares a
* whole list just to know that. * COUNT(*) over the whole list.
*/ */
export function pageOf<T>(rows: T[], limit: number): Page<T> { export function pageOf<T>(rows: T[], limit: number): Page<T> {
return rows.length > limit return rows.length > limit

48
api/lib/password.ts Normal file
View File

@@ -0,0 +1,48 @@
/**
* Hashes and checks passwords, for sign-up, sign-in, password changes and the
* admin that `db/startup.ts` seeds. It reads no config, so the startup tasks
* can use it before the server's settings are checked.
*
* @module
*/
import { randomBytes, scrypt, timingSafeEqual } from "node:crypto";
import { Buffer } from "node:buffer";
/**
* Hashes `password` with scrypt and a random salt, as `hash.salt` in hex, for
* `users.password_hash`.
*/
export function hashPassword(password: string): Promise<string> {
const salt = randomBytes(16).toString("hex");
return new Promise((resolve, reject) => {
scrypt(password, salt, 64, (err, derivedKey) => {
if (err) reject(err);
else resolve(`${derivedKey.toString("hex")}.${salt}`);
});
});
}
/** Checks `password` against a hash made by {@link hashPassword}. */
export function verifyPassword(
password: string,
storedHash: string,
): Promise<boolean> {
const [hash, salt] = storedHash.split(".");
// scrypt would throw on a missing salt, and a malformed hash matches nothing.
if (!hash || !salt) return Promise.resolve(false);
return new Promise((resolve, reject) => {
scrypt(password, salt, 64, (err, derivedKey) => {
if (err) reject(err);
else {
const expected = Buffer.from(hash ?? "", "hex");
resolve(
expected.length === derivedKey.length &&
timingSafeEqual(expected, derivedKey),
);
}
});
});
}

View File

@@ -1,5 +1,7 @@
/** /**
* Permission checks against an authenticated caller. * Checks an authenticated caller's permissions. Routes call it where the
* answer depends on more than the permission, such as "the owner, or a
* moderator"; `requirePermission` gates a whole route instead.
* *
* @module * @module
*/ */
@@ -7,7 +9,10 @@
import { hasPermission, type Permission } from "../../shared/permissions.ts"; import { hasPermission, type Permission } from "../../shared/permissions.ts";
import type { AuthPayload } from "../model/auth.ts"; import type { AuthPayload } from "../model/auth.ts";
/** Checks whether the caller's role grants `permission`. */ /**
* Checks whether the caller's role grants `permission`. The role is the live
* one `authenticate` put in the payload, not the token's.
*/
export function can(payload: AuthPayload, permission: Permission): boolean { export function can(payload: AuthPayload, permission: Permission): boolean {
return hasPermission(payload.role, permission); return hasPermission(payload.role, permission);
} }

View File

@@ -1,26 +1,30 @@
/** /**
* Which of a playlist's dumps a given viewer may see. * Decides which of a playlist's dumps a viewer may see. The playlist service
* filters its listings with it, and the WebSocket service filters each
* client's `playlist_dumps_updated` message the same way.
* *
* @module * @module
*/ */
import type { Playlist } from "../../shared/playlist.ts"; import type { Playlist } from "../../shared/playlist.ts";
/** A playlist's dump, with what deciding who may see it takes. */ /**
* The fields of a playlist's dump that {@link isMemberVisibleTo} needs. A
* `Dump` has them. For its broadcasts, the playlist service reads only these.
*/
export interface PlaylistMember { export interface PlaylistMember {
/** The dump's id. */
id: string; id: string;
/** Whether the dump is private. */
isPrivate: boolean; isPrivate: boolean;
/** The id of the dump's poster. */ /** The dump's poster, not the playlist's owner. */
userId: string; userId: string;
} }
/** /**
* Checks whether `viewerId` sees this member when the playlist is shown to them: a * Checks whether `viewerId` may see `member` in `playlist`. A public dump is
* public dump, or a private one of the owner's own, to the owner. Everyone * visible to all. A private dump is visible only to the playlist's owner, and
* else — moderators too — doesn't see private dumps, and neither does the * only if they posted it: moderators don't see it, and neither does the owner
* owner see someone else's dump once it has gone private. * once someone else's dump in their playlist goes private. A null or undefined
* `viewerId` is a signed-out viewer.
*/ */
export function isMemberVisibleTo( export function isMemberVisibleTo(
member: PlaylistMember, member: PlaylistMember,

View File

@@ -1,7 +1,7 @@
/** /**
* In-memory rate limiting with token buckets, keyed by the caller's choice * Limits how often something may happen, with token buckets kept in memory,
* (e.g. `${kind}:${userId}`) — per user rather than per socket or request, so * which a restart empties. Logins, password reset requests and the chat's
* more tabs or parallel requests don't buy more throughput. * typing messages use it.
* *
* @module * @module
*/ */
@@ -12,15 +12,22 @@ interface Bucket {
} }
const buckets = new Map<string, Bucket>(); const buckets = new Map<string, Bucket>();
/** A token bucket's shape. */ /**
* Passed to {@link takeToken} by each limited action, which declares its own
* as a constant.
*/
export interface RateLimit { export interface RateLimit {
/** Burst size. */ /** How many tokens a full bucket holds, and so the largest burst. */
capacity: number; capacity: number;
/** Tokens regained per second. */ /** Tokens regained per second. */
refill: number; refill: number;
} }
/** Takes one token from `key`'s bucket, or returns false when it is empty. */ /**
* Takes one token from `key`'s bucket, or returns false when it is empty. The
* caller picks the key, such as `typing:<userId>`, and keys by user or account
* so that more tabs or parallel requests don't buy more tokens.
*/
export function takeToken(key: string, limit: RateLimit): boolean { export function takeToken(key: string, limit: RateLimit): boolean {
const now = Date.now(); const now = Date.now();
const bucket = buckets.get(key) ?? { tokens: limit.capacity, at: now }; const bucket = buckets.get(key) ?? { tokens: limit.capacity, at: now };
@@ -35,8 +42,8 @@ export function takeToken(key: string, limit: RateLimit): boolean {
return true; return true;
} }
// Idle buckets have refilled by now for every limit in use; drop them so the // Every limit in use refills within the hour, so an older bucket is full and
// map tracks only keys that have been active lately. // dropping it changes nothing.
setInterval(() => { setInterval(() => {
const cutoff = Date.now() - 60 * 60_000; const cutoff = Date.now() - 60 * 60_000;
for (const [key, bucket] of buckets) { for (const [key, bucket] of buckets) {

View File

@@ -1,13 +1,17 @@
/** /**
* Which slugs a category may take. Category slugs live at the URL root (e.g. * Checks which slugs a category may take. A category's feed lives at the URL
* /music), so they must never collide with the frontend's own routes. * root (`/music`), so its slug must not shadow another top-level route. The
* category service runs these checks when a category is created or renamed.
* *
* @module * @module
*/ */
/** /**
* Top-level path segments that already have meaning in the SPA router (see * The top-level path segments a category slug may not take. They are the SPA's
* src/App.tsx), plus a few reserved for infrastructure. * own routes (`src/App.tsx`); `api`, `ws`, `assets` and `emojibase`, which the
* server answers before the SPA; and `manage`, a former route whose old links
* may still be about. Keep it in step with the route table and the build's
* top-level folders.
*/ */
export const RESERVED_SLUGS: readonly string[] = [ export const RESERVED_SLUGS: readonly string[] = [
"~", "~",
@@ -21,16 +25,18 @@ export const RESERVED_SLUGS: readonly string[] = [
"reset-password", "reset-password",
"manage", "manage",
"api", "api",
"ws",
"assets", "assets",
"emojibase",
]; ];
/** /**
* A well-formed category slug: lowercase alphanumerics, * Matches a well-formed category slug, made of lowercase letters and digits
* hyphen-separated, no leading, trailing or double hyphen. * joined by single hyphens.
*/ */
export const CATEGORY_SLUG_RE = /^[a-z0-9]+(?:-[a-z0-9]+)*$/; export const CATEGORY_SLUG_RE = /^[a-z0-9]+(?:-[a-z0-9]+)*$/;
/** Checks whether `slug` is well-formed (see {@link CATEGORY_SLUG_RE}). */ /** Checks whether `slug` matches {@link CATEGORY_SLUG_RE}. */
export function isValidSlugFormat(slug: string): boolean { export function isValidSlugFormat(slug: string): boolean {
return CATEGORY_SLUG_RE.test(slug); return CATEGORY_SLUG_RE.test(slug);
} }
@@ -39,8 +45,3 @@ export function isValidSlugFormat(slug: string): boolean {
export function isReservedSlug(slug: string): boolean { export function isReservedSlug(slug: string): boolean {
return RESERVED_SLUGS.includes(slug); return RESERVED_SLUGS.includes(slug);
} }
/** Checks whether `slug` is well-formed and not reserved. */
export function isSlugAvailable(slug: string): boolean {
return isValidSlugFormat(slug) && !isReservedSlug(slug);
}

View File

@@ -1,6 +1,7 @@
/** /**
* Recognises links to this site's own pages, which previews resolve from the * Recognises links to this site's own pages. The self provider previews them
* database instead of fetching. * from the database, and remote/fetch.ts accepts them as dump links even on a
* private host such as localhost.
* *
* @module * @module
*/ */

View File

@@ -1,6 +1,7 @@
/** /**
* Serving files from disk with ETags and byte ranges, streamed a chunk at a * Sends files from disk with ETags and byte ranges, streamed a chunk at a
* time. * time. The files and thumbnails routes call it, and lib/upload.ts for the
* other uploads.
* *
* @module * @module
*/ */
@@ -67,21 +68,24 @@ function matchesEtag(header: string | null, etag: string): boolean {
return header.split(",").some((tag) => strip(tag) === strip(etag)); return header.split(",").some((tag) => strip(tag) === strip(etag));
} }
/** How {@link serveFile} answers. */ /**
* Passed to {@link serveFile} by its caller, which knows the file's type and
* how long it may be cached.
*/
export interface ServeFileOptions { export interface ServeFileOptions {
/** The `Content-Type` sent. */
mime: string; mime: string;
/** The `Cache-Control` sent. */
cacheControl: string; cacheControl: string;
/** Answer a single `Range: bytes=…` with a 206 (media files). */ /**
* Answers a single `Range: bytes=…` with a 206. The files route sets it, so
* that audio and video can seek.
*/
ranges?: boolean; ranges?: boolean;
} }
/** /**
* Sends a file from disk with an ETag (a 304 when the client already has it) * Sends a file from disk with an ETag, answering 304 when the client already
* and, when `ranges` is set, single byte ranges. The body is streamed. * has it, and single byte ranges when `ranges` is set. It returns false, having
* * sent nothing, when the file doesn't exist, and leaves the 404 to the caller.
* Returns false, having sent nothing, when the file doesn't exist.
*/ */
export async function serveFile( export async function serveFile(
ctx: Context, ctx: Context,

View File

@@ -1,12 +1,10 @@
/** /**
* Generates the full favicon set and web manifest from a single emoji. * Renders the site's favicons and web manifest from its emoji. The `icons`
* task (`api/scripts/generate-icons.ts`) runs it before the server starts, so
* the emoji can change without a rebuild.
* *
* Twemoji ships each emoji as a plain vector-path SVG, so it rasterizes without * The glyph comes from Twemoji as plain vector paths, so rendering it needs no
* any system emoji font (which matters in the minimal Alpine runtime image). * emoji font, which the Alpine runtime image lacks.
* `@resvg/resvg-wasm` does the rasterization; the .ico is assembled by hand.
*
* Invoked by api/scripts/generate-icons.ts at server startup (into dist/ in
* prod, public/ in dev) — see that script for how the output dir is chosen.
* *
* @module * @module
*/ */
@@ -19,9 +17,10 @@ const APPLE_SIZE = 180;
const ICO_SIZES = [16, 32, 48]; const ICO_SIZES = [16, 32, 48];
/** /**
* Returns the name Twemoji files `emoji` under: its codepoints joined with * Returns the name Twemoji files `emoji` under: its codepoints in hex, joined
* "-", variation selectors (FE0F) stripped unless the emoji is a ZWJ sequence. * with "-", without variation selectors (FE0F) unless the emoji is a ZWJ
* Also doubles as a stable cache-busting version string for the icon URLs. * sequence. `lib/index-html.ts` also uses it as the icon URLs' cache-busting
* version.
*/ */
export function emojiToCodepoint(emoji: string): string { export function emojiToCodepoint(emoji: string): string {
const cleaned = emoji.includes("‍") ? emoji : emoji.replace(/️/g, ""); const cleaned = emoji.includes("‍") ? emoji : emoji.replace(/️/g, "");
@@ -47,7 +46,7 @@ function renderPng(svg: string, size: number, background?: string): Uint8Array {
return resvg.render().asPng(); return resvg.render().asPng();
} }
/** Packs PNG payloads into a (PNG-compressed) .ico container. */ /** Packs PNG images into an .ico file, which may hold PNGs as they are. */
function buildIco(images: { size: number; png: Uint8Array }[]): Uint8Array { function buildIco(images: { size: number; png: Uint8Array }[]): Uint8Array {
const HEADER = 6; const HEADER = 6;
const ENTRY = 16; const ENTRY = 16;
@@ -99,24 +98,23 @@ function buildManifest(siteName: string, bgColor: string): string {
return JSON.stringify(manifest, null, 2) + "\n"; return JSON.stringify(manifest, null, 2) + "\n";
} }
/** What {@link generateSiteIcons} renders, and where. */ /**
export interface GenerateSiteIconsOptions { * Passed to {@link generateSiteIcons} by `api/scripts/generate-icons.ts`, from
/** Emoji to render the icons from. */ * the site's config.
emoji: string;
/** Directory to write the assets into (created if missing). */
outDir: string;
/** Site name written into the manifest. */
siteName: string;
/**
* Background color for the Apple touch icon and manifest (iOS ignores
* transparency).
*/ */
export interface GenerateSiteIconsOptions {
emoji: string;
/** Created if missing. */
outDir: string;
siteName: string;
/** Fills the Apple touch icon, and the manifest's theme and background. */
bgColor: string; bgColor: string;
} }
/** /**
* Generates `favicon.svg`, `favicon.ico`, `favicon-<size>x<size>.png` (16–512), * Writes `favicon.svg`, `favicon.ico`, `favicon-<size>x<size>.png` (16 to 512),
* `apple-touch-icon.png` and `manifest.webmanifest` into `outDir`. * `apple-touch-icon.png` and `manifest.webmanifest` into `opts.outDir`, the
* directory the site's static files are served from.
* *
* @throws {Error} When the emoji's glyph can't be fetched. * @throws {Error} When the emoji's glyph can't be fetched.
*/ */
@@ -137,7 +135,7 @@ export async function generateSiteIcons(
for (const size of PNG_SIZES) { for (const size of PNG_SIZES) {
await Deno.writeFile(out(`favicon-${size}x${size}.png`), pngs.get(size)!); await Deno.writeFile(out(`favicon-${size}x${size}.png`), pngs.get(size)!);
} }
// Apple touch icon gets a solid background — iOS doesn't honor transparency. // iOS draws a transparent touch icon on black.
await Deno.writeFile( await Deno.writeFile(
out("apple-touch-icon.png"), out("apple-touch-icon.png"),
renderPng(svg, APPLE_SIZE, bgColor), renderPng(svg, APPLE_SIZE, bgColor),

View File

@@ -1,31 +1,37 @@
/** /**
* URL slugs for records that have a title. * Builds the URL slugs of dumps and playlists from their titles. The dump and
* playlist services store the slug when the record is created or retitled.
* *
* @module * @module
*/ */
/** A UUID, which a route parameter may carry instead of a slug. */ /**
* Matches a UUID. A dump or playlist route takes either an id or a slug, and
* the services test the parameter with this to know which column to look up.
*/
export const UUID_RE = export const UUID_RE =
/^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i; /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i;
/** /**
* Turns `title` into a URL-safe slug: lowercase ASCII, dash-separated, at most * Turns `title` into lowercase ASCII words joined by dashes, at most 60
* 60 characters, `untitled` when nothing is left. * characters long. Accents are dropped, and a title with no ASCII letter or
* digit gives `untitled`.
*/ */
export function slugify(title: string): string { export function slugify(title: string): string {
const slug = title const slug = title
.toLowerCase() .toLowerCase()
.normalize("NFD") .normalize("NFD")
.replace(/[\u0300-\u036f]/g, "") // strip diacritics // NFD splits an accented letter into the letter and a combining mark.
.replace(/[^a-z0-9]+/g, "-") // non-alphanumeric → dash .replace(/[\u0300-\u036f]/g, "")
.replace(/^-+|-+$/g, "") // trim leading/trailing dashes .replace(/[^a-z0-9]+/g, "-")
.replace(/^-+|-+$/g, "")
.substring(0, 60); .substring(0, 60);
return slug || "untitled"; return slug || "untitled";
} }
/** /**
* Returns the slug of a record: its title's slug followed by the start of its * Returns a record's slug, its title's {@link slugify} followed by the first 8
* `id`, so it stays unique by construction. * characters of `id`. The suffix keeps two records with the same title apart.
*/ */
export function makeSlug(title: string, id: string): string { export function makeSlug(title: string, id: string): string {
return `${slugify(title)}-${id.substring(0, 8)}`; return `${slugify(title)}-${id.substring(0, 8)}`;

View File

@@ -1,6 +1,7 @@
/** /**
* Serving the built frontend: static files, and `index.html` with its * Serves the built frontend, as its static files and, for every client-side
* placeholders filled for every client-side route. * route, `index.html` with its placeholders filled. main.ts installs it after
* the routers.
* *
* @module * @module
*/ */
@@ -8,38 +9,39 @@
import { Context, Next, send } from "@oak/oak"; import { Context, Next, send } from "@oak/oak";
import { loadIndexHtml } from "./index-html.ts"; import { loadIndexHtml } from "./index-html.ts";
/** Answers with the filled `index.html` of `root`, always revalidated. */
async function serveIndexHtml( async function serveIndexHtml(
context: Context<Record<string, object>>, context: Context<Record<string, object>>,
root: string, root: string,
) { ) {
const html = await loadIndexHtml(root); const html = await loadIndexHtml(root);
context.response.type = "text/html"; context.response.type = "text/html";
// Always revalidated: it names the hashed chunks of the current build. // It names the hashed chunks of the current build, so it is always
// revalidated.
context.response.headers.set("Cache-Control", "no-cache"); context.response.headers.set("Cache-Control", "no-cache");
context.response.body = html; context.response.body = html;
} }
/** /**
* Build output whose URL changes whenever its content does: Vite's hashed * Prefixes the build output whose URL changes whenever its content does, which
* chunks, and the emoji data under its package version (see vite.config.ts). * are Vite's hashed chunks and the emoji data under its package version (see
* vite.config.ts).
*/ */
const IMMUTABLE_PREFIXES = ["/assets/", "/emojibase/"]; const IMMUTABLE_PREFIXES = ["/assets/", "/emojibase/"];
const ONE_YEAR_MS = 365 * 24 * 60 * 60 * 1000; const ONE_YEAR_MS = 365 * 24 * 60 * 60 * 1000;
/** /**
* Returns a middleware serving static files from the first of `staticPaths` * Returns the middleware that serves a static file from the first of
* that has them, and `index.html` from the first for any other path so * `staticPaths` that has it, and `index.html` of the first for any other path.
* client-side routes load on direct navigation. Hashed build output is cached * Hashed build output is cached for a year, and a missing chunk is a 404.
* for a year, and a missing chunk is a 404. * main.ts passes dist/ then public/.
*/ */
export function routeStaticFilesFrom( export function routeStaticFilesFrom(
staticPaths: string[], staticPaths: string[],
): (context: Context<Record<string, object>>, next: Next) => Promise<void> { ): (context: Context<Record<string, object>>, next: Next) => Promise<void> {
return async (context: Context<Record<string, object>>, next: Next) => { // The last middleware: it always answers, so it never calls `next`.
return async (context: Context<Record<string, object>>, _next: Next) => {
const pathname = context.request.url.pathname; const pathname = context.request.url.pathname;
// Serve index.html with runtime placeholder replacement
if (pathname === "/" || pathname === "/index.html") { if (pathname === "/" || pathname === "/index.html") {
await serveIndexHtml(context, staticPaths[0]); await serveIndexHtml(context, staticPaths[0]);
return; return;
@@ -53,7 +55,8 @@ export function routeStaticFilesFrom(
const compress = context.request.headers.has("accept-encoding"); const compress = context.request.headers.has("accept-encoding");
for (const path of staticPaths) { for (const path of staticPaths) {
try { try {
// send() picks a prebuilt .br or .gz sibling when the browser accepts it. // send() picks a prebuilt .br or .gz sibling when the browser accepts
// it.
await send(context, pathname, { await send(context, pathname, {
root: path, root: path,
brotli: compress, brotli: compress,
@@ -74,8 +77,7 @@ export function routeStaticFilesFrom(
return; return;
} }
// SPA fallback: serve index.html so client-side routes work on direct navigation // Any other path is a client-side route, opened directly or reloaded.
await serveIndexHtml(context, staticPaths[0]); await serveIndexHtml(context, staticPaths[0]);
await next();
}; };
} }

View File

@@ -1,35 +1,21 @@
/** /**
* Checking uploaded images and serving uploaded files back. * Checks the images users upload and serves them back. The avatar, playlist
* cover, dump thumbnail and attachment routes call it.
* *
* @module * @module
*/ */
import type { Context } from "@oak/oak"; import type { Context } from "@oak/oak";
import { APIErrorCode } from "../../shared/api.ts";
import { APIException } from "./api-exception.ts"; import { APIException } from "./api-exception.ts";
import { serveFile } from "./serve-file.ts"; import { serveFile } from "./serve-file.ts";
import { import { dirname } from "node:path";
ALLOWED_IMAGE_MIMES, import { MAX_IMAGE_SIZE_BYTES } from "../config.ts";
ATTACHMENTS_DIR,
AVATARS_DIR,
DUMPS_DIR,
MAX_IMAGE_SIZE_BYTES,
PLAYLIST_IMAGES_DIR,
UPLOADS_DIR,
} from "../config.ts";
export {
ALLOWED_IMAGE_MIMES,
ATTACHMENTS_DIR,
AVATARS_DIR,
DUMPS_DIR,
PLAYLIST_IMAGES_DIR,
UPLOADS_DIR,
};
/** /**
* Returns the MIME type of a JPEG, PNG, GIF or WebP image from its magic * Returns the MIME type of a JPEG, PNG, GIF or WebP image from its magic
* bytes, ignoring the type the browser declared; null for anything else. * bytes, or null for anything else. These four are the only image types
* accepted anywhere. The type the browser declared is never
* trusted, since the file is later served with the type found here.
*/ */
export function detectImageMime(data: Uint8Array): string | null { export function detectImageMime(data: Uint8Array): string | null {
if (data[0] === 0xFF && data[1] === 0xD8 && data[2] === 0xFF) { if (data[0] === 0xFF && data[1] === 0xD8 && data[2] === 0xFF) {
@@ -52,7 +38,8 @@ export function detectImageMime(data: Uint8Array): string | null {
} }
/** /**
* Checks an uploaded image's size and content, and returns its MIME type. * Checks an uploaded image's size and content, and returns its MIME type,
* which the route stores to serve the image with.
* *
* @throws {APIException} 400 when it exceeds {@link MAX_IMAGE_SIZE_BYTES} or * @throws {APIException} 400 when it exceeds {@link MAX_IMAGE_SIZE_BYTES} or
* isn't a JPEG, PNG, GIF or WebP image. * isn't a JPEG, PNG, GIF or WebP image.
@@ -60,15 +47,13 @@ export function detectImageMime(data: Uint8Array): string | null {
export function validateImageUpload(data: Uint8Array): string { export function validateImageUpload(data: Uint8Array): string {
if (data.length > MAX_IMAGE_SIZE_BYTES) { if (data.length > MAX_IMAGE_SIZE_BYTES) {
throw new APIException( throw new APIException(
APIErrorCode.BAD_REQUEST,
400, 400,
"File too large (max 5 MB)", `File too large (max ${MAX_IMAGE_SIZE_BYTES / (1024 * 1024)} MB)`,
); );
} }
const mime = detectImageMime(data); const mime = detectImageMime(data);
if (!mime) { if (!mime) {
throw new APIException( throw new APIException(
APIErrorCode.BAD_REQUEST,
400, 400,
"File content is not a recognised image (JPEG, PNG, GIF, WebP)", "File content is not a recognised image (JPEG, PNG, GIF, WebP)",
); );
@@ -77,9 +62,10 @@ export function validateImageUpload(data: Uint8Array): string {
} }
/** /**
* Serves an uploaded file inline as `mime`, cached for `cacheMaxAge` seconds * Serves an uploaded image inline as `mime`, cached for `cacheMaxAge` seconds,
* (privately when `isPrivate`). Answers 404 and resolves false when the file * and only by the browser when `isPrivate`. It answers 404 and resolves false
* is missing. * when the file is missing. The caller checks first that the viewer may see
* it.
*/ */
export async function serveUploadedFile( export async function serveUploadedFile(
ctx: Context, ctx: Context,
@@ -89,6 +75,8 @@ export async function serveUploadedFile(
isPrivate = false, isPrivate = false,
): Promise<boolean> { ): Promise<boolean> {
ctx.response.headers.set("Content-Disposition", "inline"); ctx.response.headers.set("Content-Disposition", "inline");
// The browser must take the checked type, never guess another from the
// bytes, and a frontend on another origin may still show the image.
ctx.response.headers.set("X-Content-Type-Options", "nosniff"); ctx.response.headers.set("X-Content-Type-Options", "nosniff");
ctx.response.headers.set("Cross-Origin-Resource-Policy", "cross-origin"); ctx.response.headers.set("Cross-Origin-Resource-Policy", "cross-origin");
const served = await serveFile(ctx, filePath, { const served = await serveFile(ctx, filePath, {
@@ -98,3 +86,24 @@ export async function serveUploadedFile(
if (!served) ctx.response.status = 404; if (!served) ctx.response.status = 404;
return served; return served;
} }
/**
* Writes `data` to `filePath`, replacing whatever is there only once the whole
* file is written, so a failed write leaves the old file whole. The upload
* routes call it before recording the new file, so that clients told of it
* never fetch it early.
*/
export async function replaceUploadedFile(
filePath: string,
data: Uint8Array,
): Promise<void> {
const tmpPath = `${filePath}.${crypto.randomUUID()}.tmp`;
await Deno.mkdir(dirname(filePath), { recursive: true });
try {
await Deno.writeFile(tmpPath, data);
await Deno.rename(tmpPath, filePath);
} catch (err) {
await Deno.remove(tmpPath).catch(() => {});
throw err;
}
}

View File

@@ -1,7 +1,8 @@
/** /**
* The server's entry point: builds the Oak application — error and OG * Builds the Oak application and, when run directly, listens on
* middleware, CORS, every router, then the static frontend — and listens on * {@link LISTEN_HOST}:{@link PORT}. A request goes through the error
* {@link LISTEN_HOST}:{@link PORT} when run directly. * middleware, the OG tags and CORS, then the routers, and last the built
* frontend, which answers every path no router did.
* *
* @module * @module
*/ */
@@ -33,16 +34,19 @@ import { errorMiddleware } from "./middleware/error.ts";
import { ogMiddleware } from "./middleware/og.ts"; import { ogMiddleware } from "./middleware/og.ts";
import { routeStaticFilesFrom } from "./lib/static.ts"; import { routeStaticFilesFrom } from "./lib/static.ts";
/** The application, exported so the tests can drive it without listening. */ /**
* The whole server, exported so the tests' helpers can send it requests
* without listening.
*/
const app: Application = new Application(); const app: Application = new Application();
const cors = oakCors({ origin: ALLOWED_ORIGINS }); const cors = oakCors({ origin: ALLOWED_ORIGINS });
app.use(errorMiddleware); app.use(errorMiddleware);
app.use(ogMiddleware); app.use(ogMiddleware);
// Only invoke oakCors when the origin is present and allowed. // oakCors calls next() without awaiting it when there is no origin, which
// The library calls next() without await when origin is falsy, which // corrupts the response in Oak's async chain. So it only sees requests from an
// corrupts the response in Oak's async middleware chain. // allowed origin, and the others get no CORS headers.
app.use(async (ctx, next) => { app.use(async (ctx, next) => {
const origin = ctx.request.headers.get("origin"); const origin = ctx.request.headers.get("origin");
if (origin && ALLOWED_ORIGINS.includes(origin)) { if (origin && ALLOWED_ORIGINS.includes(origin)) {

View File

@@ -1,6 +1,7 @@
/** /**
* Middleware gating routes on authentication, the admin role, or a * Gates routes on a session token, and on a permission after it. The routers
* permission. * put these middleware before a handler; a route open to anonymous callers
* calls `parseOptionalAuth` (`lib/auth.ts`) instead.
* *
* @module * @module
*/ */
@@ -12,22 +13,22 @@ import {
type State, type State,
} from "@oak/oak"; } from "@oak/oak";
import { authenticate } from "../lib/auth.ts"; import { authenticate } from "../lib/auth.ts";
import { APIErrorCode } from "../../shared/api.ts";
import { APIException } from "../lib/api-exception.ts"; import { APIException } from "../lib/api-exception.ts";
import type { AuthPayload } from "../model/auth.ts"; import type { AuthPayload } from "../model/auth.ts";
import { can } from "../lib/permissions.ts"; import { can } from "../lib/permissions.ts";
import type { Permission } from "../../shared/permissions.ts"; import type { Permission } from "../../shared/permissions.ts";
/** The request state once {@link authMiddleware} has run. */ /**
* Set on `ctx.state` by {@link authMiddleware} for the handlers after it.
* `user` carries the caller's live role, not the one in their token.
*/
export interface AuthState extends State { export interface AuthState extends State {
/** The authenticated caller, with their live role. */
user: AuthPayload; user: AuthPayload;
} }
/** /**
* A router context whose state is guaranteed to carry the authenticated user * The context type of a handler behind {@link authMiddleware}. Pass the route
* (populated by {@link authMiddleware}). Generic over the route path so * path, as in `AuthContext<"/:userId/role">`, to keep `ctx.params` typed.
* `ctx.params` stays typed per-route — e.g. `AuthContext<"/:userId/role">`.
*/ */
export type AuthContext< export type AuthContext<
R extends string = string, R extends string = string,
@@ -35,8 +36,8 @@ export type AuthContext<
> = RouterContext<R, P, AuthState>; > = RouterContext<R, P, AuthState>;
/** /**
* Requires a valid session token in the `Authorization` header and puts its * Requires a valid session token in the `Authorization` header, and puts its
* caller in `ctx.state.user`. * payload in `ctx.state.user` for the rest of the route.
* *
* @throws {APIException} 401 when the token is missing or invalid. * @throws {APIException} 401 when the token is missing or invalid.
*/ */
@@ -47,20 +48,14 @@ export async function authMiddleware<R extends string>(
const authHeader = ctx.request.headers.get("Authorization"); const authHeader = ctx.request.headers.get("Authorization");
if (!authHeader || !authHeader.startsWith("Bearer ")) { if (!authHeader || !authHeader.startsWith("Bearer ")) {
throw new APIException( throw new APIException(401, "Missing or invalid token");
APIErrorCode.UNAUTHORIZED,
401,
"Missing or invalid token",
);
} }
const token = authHeader.substring(7); const token = authHeader.substring(7);
// Also checks the user still exists and hasn't changed their password since
// the token was issued, and takes the live role from the DB.
const payload = await authenticate(token); const payload = await authenticate(token);
if (!payload) { if (!payload) {
throw new APIException(APIErrorCode.UNAUTHORIZED, 401, "Invalid token"); throw new APIException(401, "Invalid token");
} }
ctx.state.user = payload; ctx.state.user = payload;
@@ -69,30 +64,11 @@ export async function authMiddleware<R extends string>(
} }
/** /**
* Requires the caller to be an admin. Must run after {@link authMiddleware}. * Returns a middleware that requires the caller to hold `permission`. It must
* * run after {@link authMiddleware}, as in
* @throws {APIException} 403 when the caller isn't an admin. * `router.patch(path, authMiddleware, requirePermission("user:manage"), …)`.
*/ * The returned middleware throws an {@link APIException} (403) when the caller
export async function adminOnlyMiddleware<R extends string>( * lacks the permission.
ctx: AuthContext<R>,
next: Next,
): Promise<void> {
if (ctx.state.user.role !== "admin") {
throw new APIException(
APIErrorCode.UNAUTHORIZED,
403,
"Admin access required",
);
}
await next();
}
/**
* Returns a middleware requiring the caller to hold `permission`, which throws
* an {@link APIException} (403) when they don't. Must run after
* {@link authMiddleware}. Generic over the route path so it composes with
* path-typed route handlers.
*/ */
export function requirePermission( export function requirePermission(
permission: Permission, permission: Permission,
@@ -102,11 +78,7 @@ export function requirePermission(
next: Next, next: Next,
): Promise<void> => { ): Promise<void> => {
if (!ctx.state.user || !can(ctx.state.user, permission)) { if (!ctx.state.user || !can(ctx.state.user, permission)) {
throw new APIException( throw new APIException(403, "Insufficient permissions");
APIErrorCode.UNAUTHORIZED,
403,
"Insufficient permissions",
);
} }
await next(); await next();
}; };

View File

@@ -1,5 +1,7 @@
/** /**
* Turns errors thrown while handling a request into `APIFailure` responses. * Turns the errors a request's handling throws into `APIFailure` responses.
* main.ts installs it first, so it catches what every later middleware and
* route throws.
* *
* @module * @module
*/ */
@@ -10,8 +12,9 @@ import { APIErrorCode, APIFailure } from "../../shared/api.ts";
import { APIException } from "../lib/api-exception.ts"; import { APIException } from "../lib/api-exception.ts";
/** /**
* Answers an {@link APIException} with its status and code, and anything else * Answers an {@link APIException} with its status, code and message. Anything
* with a 500 `SERVER_ERROR`, logging both. * else is logged with its stack and answered with a 500 `SERVER_ERROR` whose
* message gives nothing away.
*/ */
export async function errorMiddleware( export async function errorMiddleware(
ctx: Context, ctx: Context,

View File

@@ -1,7 +1,8 @@
/** /**
* Open Graph and Twitter card tags for shared links: a dump, user or public * Answers a link to a dump, a profile or a public playlist with `index.html`
* playlist page is answered with `index.html` carrying that page's title, * carrying that page's Open Graph and Twitter card tags, for the link previews
* description and image, so link unfurlers that don't run the app see them. * of chat apps and social sites, which don't run the app. main.ts installs it
* before the routers.
* *
* @module * @module
*/ */
@@ -10,7 +11,7 @@ import { Context, Next } from "@oak/oak";
import { getDump } from "../services/dump-service.ts"; import { getDump } from "../services/dump-service.ts";
import { getUserByUsername } from "../services/user-service.ts"; import { getUserByUsername } from "../services/user-service.ts";
import { getPlaylistById } from "../services/playlist-service.ts"; import { getPlaylistById } from "../services/playlist-service.ts";
import { OG_SITE_NAME } from "../config.ts"; import { FRONTEND_URL, OG_SITE_NAME, PUBLIC_URL } from "../config.ts";
import { loadIndexHtml } from "../lib/index-html.ts"; import { loadIndexHtml } from "../lib/index-html.ts";
interface OGMeta { interface OGMeta {
@@ -18,8 +19,8 @@ interface OGMeta {
description?: string; description?: string;
imageUrl?: string; imageUrl?: string;
/** /**
* True only for real artwork — a square site icon must not claim a wide * Set for a dump's artwork only. It asks for the large card, which would
* card. * stretch a square icon or avatar.
*/ */
imageIsWide?: boolean; imageIsWide?: boolean;
url: string; url: string;
@@ -39,7 +40,7 @@ function buildTags(meta: OGMeta): string {
: "summary"; : "summary";
const tags = [ const tags = [
`<title>${escapeAttr(meta.title)}</title>`, `<title>${escapeAttr(meta.title)}</title>`,
`<meta property="og:site_name" content="${OG_SITE_NAME}" />`, `<meta property="og:site_name" content="${escapeAttr(OG_SITE_NAME)}" />`,
`<meta property="og:type" content="website" />`, `<meta property="og:type" content="website" />`,
`<meta property="og:url" content="${escapeAttr(meta.url)}" />`, `<meta property="og:url" content="${escapeAttr(meta.url)}" />`,
`<meta property="og:title" content="${escapeAttr(meta.title)}" />`, `<meta property="og:title" content="${escapeAttr(meta.title)}" />`,
@@ -68,8 +69,8 @@ function inject(html: string, meta: OGMeta): string {
} }
/** /**
* Returns the filled `index.html` of the build, or of the working directory * Returns the filled `index.html` of dist/, or else the source one, which is
* when there is no build; null when there is neither. * all there is in development. Returns null when neither can be read.
*/ */
async function findIndexHtml(): Promise<string | null> { async function findIndexHtml(): Promise<string | null> {
for (const root of [`${Deno.cwd()}/dist`, Deno.cwd()]) { for (const root of [`${Deno.cwd()}/dist`, Deno.cwd()]) {
@@ -83,13 +84,16 @@ async function findIndexHtml(): Promise<string | null> {
} }
const DUMP_RE = /^\/dumps\/([^/]+)$/; const DUMP_RE = /^\/dumps\/([^/]+)$/;
/** A profile, with an optional `/~/<tab>` suffix so its tabs keep OG meta. */ /** Matches a profile, and its tabs under `/~/<tab>`. */
const USER_RE = /^\/users\/([^/]+)(?:\/~\/[^/]+)?$/; const USER_RE = /^\/users\/([^/]+)(?:\/~\/[^/]+)?$/;
const PLAYLIST_RE = /^\/playlists\/([^/]+)$/; const PLAYLIST_RE = /^\/playlists\/([^/]+)$/;
/** /**
* Serves `index.html` with OG tags for a dump, user or public playlist page * Serves `index.html` with the tags of the dump, user or public playlist a
* that exists, and passes every other request on. * path names, and passes the request on when it names none that exists or
* the visitor may see. The page's URL is on `FRONTEND_URL` and the images' on
* `PUBLIC_URL`, not on the request's origin, which a reverse proxy may have
* rewritten to an internal host or plain http.
*/ */
export async function ogMiddleware( export async function ogMiddleware(
ctx: Context, ctx: Context,
@@ -101,8 +105,8 @@ export async function ogMiddleware(
return await next(); return await next();
} }
const origin = ctx.request.url.origin; const origin = PUBLIC_URL;
const pageUrl = `${origin}${pathname}`; const pageUrl = `${FRONTEND_URL}${pathname}`;
let meta: OGMeta | null = null; let meta: OGMeta | null = null;
@@ -119,9 +123,8 @@ export async function ogMiddleware(
} else if (dump.richContent?.thumbnailUrl) { } else if (dump.richContent?.thumbnailUrl) {
imageUrl = dump.richContent.thumbnailUrl; imageUrl = dump.richContent.thumbnailUrl;
} }
// No artwork: fall back to our own site icon rather than the target's // Without artwork, the site's icon: the linked site's favicon is too
// favicon, which is typically too small to survive a social card (and may // small for a card, and may refuse hotlinking.
// be hotlink-protected).
meta = { meta = {
title: dump.title, title: dump.title,
description: dump.comment, description: dump.comment,
@@ -129,7 +132,7 @@ export async function ogMiddleware(
imageIsWide: !!imageUrl, imageIsWide: !!imageUrl,
url: pageUrl, url: pageUrl,
}; };
} catch { /* not found or private — serve default */ } } catch { /* Unknown, or private. */ }
} else if (userMatch) { } else if (userMatch) {
try { try {
const user = getUserByUsername(userMatch[1]); const user = getUserByUsername(userMatch[1]);
@@ -142,10 +145,11 @@ export async function ogMiddleware(
imageUrl: imageUrl ?? `${origin}/apple-touch-icon.png`, imageUrl: imageUrl ?? `${origin}/apple-touch-icon.png`,
url: pageUrl, url: pageUrl,
}; };
} catch { /* not found */ } } catch { /* Unknown. */ }
} else if (playlistMatch) { } else if (playlistMatch) {
try { try {
const playlist = getPlaylistById(playlistMatch[1]); const playlist = getPlaylistById(playlistMatch[1]);
// getPlaylistById doesn't check visibility, unlike getDump.
if (playlist.isPublic) { if (playlist.isPublic) {
const imageUrl = playlist.imageMime const imageUrl = playlist.imageMime
? `${origin}/api/playlists/${playlist.id}/image` ? `${origin}/api/playlists/${playlist.id}/image`
@@ -157,7 +161,7 @@ export async function ogMiddleware(
url: pageUrl, url: pageUrl,
}; };
} }
} catch { /* not found or private */ } } catch { /* Unknown. */ }
} }
if (!meta) return await next(); if (!meta) return await next();

View File

@@ -1,35 +1,35 @@
/** /**
* The payloads of the tokens the server signs, and their guards. * Types the payloads of session and password reset tokens, and guards them.
* `lib/jwt.ts` signs the tokens and runs the guards when it verifies one.
* *
* @module * @module
*/ */
import { isRole, type Role } from "../../shared/user.ts"; import { isRole, type Role } from "../../shared/user.ts";
/** What a session token carries. */ /**
export interface AuthPayload { * Signed into a session token by `createJWT`, and returned by `authenticate`
/** The user's id. */ * to the auth middleware as `ctx.state.user`. By then `role` is the user's live
userId: string; * role, while `username` is still the one at issue time.
/** The user's name when the token was issued. */
username: string;
/**
* The user's role; the live one replaces it once the token is authenticated.
*/ */
export interface AuthPayload {
userId: string;
username: string;
role: Role; role: Role;
/** When the token expires, in seconds since the epoch. */ /** Seconds since the epoch. */
exp: number; exp: number;
/** When the token was issued, in seconds since the epoch. */ /** Seconds since the epoch, compared with `users.tokens_valid_after`. */
iat?: number; iat?: number;
} }
/** /**
* Checks whether `obj` is an {@link AuthPayload}, and not a purpose-scoped * Checks whether `obj` is an {@link AuthPayload}. A media or reset token's
* token's payload. * payload, which has a `purpose`, never is.
*/ */
export function isAuthPayload(obj: unknown): obj is AuthPayload { export function isAuthPayload(obj: unknown): obj is AuthPayload {
return !!obj && return !!obj &&
typeof obj === "object" && typeof obj === "object" &&
// Purpose-scoped tokens (password reset, media) are never session tokens. // A media or reset token must not pass for a session token.
!("purpose" in obj) && !("purpose" in obj) &&
"userId" in obj && typeof obj.userId === "string" && "userId" in obj && typeof obj.userId === "string" &&
"username" in obj && typeof obj.username === "string" && "username" in obj && typeof obj.username === "string" &&
@@ -37,17 +37,21 @@ export function isAuthPayload(obj: unknown): obj is AuthPayload {
"exp" in obj && typeof obj.exp === "number"; "exp" in obj && typeof obj.exp === "number";
} }
/** What a password reset token carries. */ /**
* Signed into a reset link's token by `createPasswordResetToken`, and returned
* by `verifyPasswordResetToken` to `resetPassword`.
*/
export interface PasswordResetPayload { export interface PasswordResetPayload {
/** The token's purpose, which makes it a reset token. */
purpose: "password-reset"; purpose: "password-reset";
/** The id of the user whose password it resets. */
userId: string; userId: string;
/** When the token expires, in seconds since the epoch. */ /** Seconds since the epoch. */
exp: number; exp: number;
} }
/** Checks whether `obj` is a {@link PasswordResetPayload}. */ /**
* Checks whether `obj` is a {@link PasswordResetPayload}. It doesn't check
* `exp`, which `jwtVerify` has already enforced.
*/
export function isPasswordResetPayload( export function isPasswordResetPayload(
obj: unknown, obj: unknown,
): obj is PasswordResetPayload { ): obj is PasswordResetPayload {

View File

@@ -1,6 +1,6 @@
/** /**
* The `categories` table: its row type, its guard, and its mapper to the API's * Types the rows the category service reads from `categories`, and turns them
* {@link Category}. * into the {@link Category} the category routes send.
* *
* @module * @module
*/ */
@@ -8,25 +8,19 @@
import type { SQLOutputValue } from "node:sqlite"; import type { SQLOutputValue } from "node:sqlite";
import type { Category } from "../../shared/category.ts"; import type { Category } from "../../shared/category.ts";
/** A row of the `categories` table. */ /** Returned by the category service's `SELECT`s on `categories`. */
export interface CategoryRow { export interface CategoryRow {
/** The category's id. */
id: string; id: string;
/** The category's URL slug, at the site root. */
slug: string; slug: string;
/** The category's display name. */
name: string; name: string;
/** The category's place in the list, ascending. */ /** Sorts the categories, lowest first. */
position: number; position: number;
/** When the category was created. */
created_at: string; created_at: string;
/** When the category was last changed, if ever. */
updated_at: string | null; updated_at: string | null;
/** Any other column the query selects. */
[key: string]: SQLOutputValue; [key: string]: SQLOutputValue;
} }
/** Checks whether `obj` is a {@link CategoryRow}. */ /** Checks whether a row `node:sqlite` returned is a {@link CategoryRow}. */
export function isCategoryRow(obj: unknown): obj is CategoryRow { export function isCategoryRow(obj: unknown): obj is CategoryRow {
return !!obj && typeof obj === "object" && return !!obj && typeof obj === "object" &&
"id" in obj && typeof obj.id === "string" && "id" in obj && typeof obj.id === "string" &&
@@ -38,7 +32,7 @@ export function isCategoryRow(obj: unknown): obj is CategoryRow {
(typeof obj.updated_at === "string" || obj.updated_at === null); (typeof obj.updated_at === "string" || obj.updated_at === null);
} }
/** Converts a row to the API's {@link Category}. */ /** Converts `row` to the {@link Category} the category routes send. */
export function categoryRowToApi(row: CategoryRow): Category { export function categoryRowToApi(row: CategoryRow): Category {
return { return {
id: row.id, id: row.id,

View File

@@ -1,6 +1,6 @@
/** /**
* The `chat_messages` table: its row type, its guard, and its mapper to the * Maps the rows of `chat_messages` to the {@link ChatMessage} the API sends.
* API's {@link ChatMessage}. * Only `services/chat-service.ts` uses it, on the rows of its one query.
* *
* @module * @module
*/ */
@@ -9,44 +9,30 @@ import type { SQLOutputValue } from "node:sqlite";
import type { ChatMessage } from "../../shared/chat.ts"; import type { ChatMessage } from "../../shared/chat.ts";
/** /**
* A row of the `chat_messages` table, joined with its author and, when it is * Returned by chat-service's query, which joins each message to its author and
* a reply, the replied-to message and its author. * to the message it replies to. `reply_to_id` is kept when the message it
* points to is deleted, while `reply_to_author` and `reply_to_body` then read
* null.
*/ */
export interface ChatMessageRow { export interface ChatMessageRow {
/** The message's id. */
id: string; id: string;
/** The author's id. */
user_id: string; user_id: string;
/** The message, in Markdown. */
body: string; body: string;
/** When the message was sent. */
created_at: string; created_at: string;
/** When the message was last edited, if ever. */
updated_at: string | null; updated_at: string | null;
/** The author's name (joined). */
author_username: string; author_username: string;
/** The author's avatar type, null without one (joined). */
author_avatar_mime: string | null; author_avatar_mime: string | null;
/**
* The id of the message this replies to, kept even once that message is
* deleted.
*/
reply_to_id: string | null; reply_to_id: string | null;
/**
* The replied-to message's author (joined); null when there's no reply or it
* was since deleted.
*/
reply_to_author: string | null; reply_to_author: string | null;
/** /** Cut to chat-service's `REPLY_SNIPPET_LEN` characters. */
* The start of the replied-to message (joined); null when there's no reply or
* it was since deleted.
*/
reply_to_body: string | null; reply_to_body: string | null;
/** Any other column the query selects. */
[key: string]: SQLOutputValue; [key: string]: SQLOutputValue;
} }
/** Checks whether `obj` is a {@link ChatMessageRow}. */ /**
* Checks whether `obj`, a row read by chat-service, is a
* {@link ChatMessageRow}.
*/
export function isChatMessageRow(obj: unknown): obj is ChatMessageRow { export function isChatMessageRow(obj: unknown): obj is ChatMessageRow {
return !!obj && typeof obj === "object" && return !!obj && typeof obj === "object" &&
"id" in obj && typeof obj.id === "string" && "id" in obj && typeof obj.id === "string" &&
@@ -67,7 +53,10 @@ export function isChatMessageRow(obj: unknown): obj is ChatMessageRow {
(typeof obj.reply_to_body === "string" || obj.reply_to_body === null); (typeof obj.reply_to_body === "string" || obj.reply_to_body === null);
} }
/** Converts a row to the API's {@link ChatMessage}. */ /**
* Converts a row to the {@link ChatMessage} that chat-service returns, turning
* its timestamps into Dates and its nulls into absent fields.
*/
export function chatMessageRowToApi(row: ChatMessageRow): ChatMessage { export function chatMessageRowToApi(row: ChatMessageRow): ChatMessage {
return { return {
id: row.id, id: row.id,

View File

@@ -1,6 +1,6 @@
/** /**
* The `comments` table: its row type, its guard, and its mapper to the API's * Types the rows the comment service reads from `comments`, and turns them into
* {@link Comment}. * the {@link Comment} the comment routes send.
* *
* @module * @module
*/ */
@@ -8,38 +8,31 @@
import type { SQLOutputValue } from "node:sqlite"; import type { SQLOutputValue } from "node:sqlite";
import type { Comment } from "../../shared/comment.ts"; import type { Comment } from "../../shared/comment.ts";
/** A row of the `comments` table, joined with its author. */ /**
* Returned by the comment service's `SELECT`s on `comments`, joined with
* `users` for the `author_` columns.
*/
export interface CommentRow { export interface CommentRow {
/** The comment's id. */
id: string; id: string;
/** The id of the dump it is posted under. */
dump_id: string; dump_id: string;
/** The author's id. */
user_id: string; user_id: string;
/** The id of the comment it replies to; null at the top level. */ /** It is null on a top-level comment. */
parent_id: string | null; parent_id: string | null;
/** The comment, in Markdown. */
body: string; body: string;
/** When the comment was posted. */
created_at: string; created_at: string;
/** When the comment was last edited, if ever. */
updated_at: string | null; updated_at: string | null;
/** /**
* 1 once deleted: the body is emptied but the row stays, so its replies * 1 once deleted. The body is emptied but the row stays, so its replies keep
* keep their place. * their place in the thread.
*/ */
deleted: number; deleted: number;
/** How many users like the comment. */
like_count: number; like_count: number;
/** The author's name (joined). */
author_username: string; author_username: string;
/** The author's avatar type, null without one (joined). */
author_avatar_mime: string | null; author_avatar_mime: string | null;
/** Any other column the query selects. */
[key: string]: SQLOutputValue; [key: string]: SQLOutputValue;
} }
/** Checks whether `obj` is a {@link CommentRow}. */ /** Checks whether a row `node:sqlite` returned is a {@link CommentRow}. */
export function isCommentRow( export function isCommentRow(
obj: unknown, obj: unknown,
): obj is CommentRow { ): obj is CommentRow {
@@ -61,7 +54,10 @@ export function isCommentRow(
obj.author_avatar_mime === null); obj.author_avatar_mime === null);
} }
/** Converts a row to the API's {@link Comment}. */ /**
* Converts `row` to the {@link Comment} the comment routes send and the
* WebSocket broadcasts.
*/
export function commentRowToApi(row: CommentRow): Comment { export function commentRowToApi(row: CommentRow): Comment {
return { return {
id: row.id, id: row.id,

View File

@@ -1,6 +1,7 @@
/** /**
* The `dumps` table: its row type, its guard, the column lists services select * Maps the `dumps` table to the API's {@link Dump}. Every service that reads
* it with, and its mappers to and from the API's {@link Dump}. * dumps selects them with the column lists here, checks each row with
* {@link isDumpRow} and converts it with {@link dumpRowToApi}.
* *
* @module * @module
*/ */
@@ -8,67 +9,65 @@
import type { SQLOutputValue } from "node:sqlite"; import type { SQLOutputValue } from "node:sqlite";
import type { Dump, RichContent } from "../../shared/dump.ts"; import type { Dump, RichContent } from "../../shared/dump.ts";
/** A row of the `dumps` table, with its live comment count. */ /**
* Read from the `dumps` table with {@link DUMP_SELECT_COLUMNS}, or written back
* from a {@link Dump} by {@link dumpApiToRow}. It lacks `url_canonical`, which
* only the duplicate lookup reads.
*/
export interface DumpRow { export interface DumpRow {
/** The dump's id. */
id: string; id: string;
/** `url` for a link, `file` for an upload. */
kind: string; kind: string;
/** The dump's title. */
title: string; title: string;
/** The poster's description, in Markdown. */
comment: string | null; comment: string | null;
/** The poster's id. */
user_id: string; user_id: string;
/** When the dump was posted. */
created_at: string; created_at: string;
/** When the dump was last edited, if ever. */
updated_at: string | null; updated_at: string | null;
/** The dump's URL slug. */
slug: string | null; slug: string | null;
/** The link exactly as submitted, for a URL dump. */ /** The link as the poster submitted it. */
url: string | null; url: string | null;
/** The link's preview, as {@link RichContent} JSON. */ /** {@link RichContent} as JSON. */
rich_content: string | null; rich_content: string | null;
/** The uploaded file's name, for a file dump. */
file_name: string | null; file_name: string | null;
/** The uploaded file's type, for a file dump. */
file_mime: string | null; file_mime: string | null;
/** The uploaded file's size in bytes, for a file dump. */
file_size: number | null; file_size: number | null;
/** How many users upvoted the dump. */
vote_count: number; vote_count: number;
/** How many comments it has, not counting deleted ones (computed). */ /** Computed by a subquery, which counts the comments that aren't deleted. */
comment_count: number; comment_count: number;
/** 1 when only the poster sees the dump. */
is_private: number; is_private: number;
/** The uploaded thumbnail's type, null without one. */
custom_thumbnail_mime: string | null; custom_thumbnail_mime: string | null;
/** Any other column the query selects. */ /** Lets a query select more columns, such as a join's. */
[key: string]: SQLOutputValue; [key: string]: SQLOutputValue;
} }
/** /**
* The stored columns of {@link DumpRow} — keep in sync with it and * Lists the stored columns of {@link DumpRow}. A column added to the table must
* {@link isDumpRow}. Services select with these instead of repeating the * be added here, to {@link DumpRow} and to {@link isDumpRow}.
* column list themselves.
*/ */
export const DUMP_COLUMNS = export const DUMP_COLUMNS =
"id, kind, title, slug, comment, user_id, created_at, updated_at, url, rich_content, file_name, file_mime, file_size, vote_count, is_private, custom_thumbnail_mime"; "id, kind, title, slug, comment, user_id, created_at, updated_at, url, rich_content, file_name, file_mime, file_size, vote_count, is_private, custom_thumbnail_mime";
/** {@link DUMP_COLUMNS}, for a query aliasing `dumps` as `d`. */ // The same columns, for a query that aliases `dumps` as `d`.
export const DUMP_COLUMNS_ALIASED = const DUMP_COLUMNS_ALIASED = DUMP_COLUMNS.split(", ").map((c) => `d.${c}`)
"d.id, d.kind, d.title, d.slug, d.comment, d.user_id, d.created_at, d.updated_at, d.url, d.rich_content, d.file_name, d.file_mime, d.file_size, d.vote_count, d.is_private, d.custom_thumbnail_mime"; .join(", ");
/** {@link DUMP_COLUMNS} and the computed `comment_count`: a full {@link DumpRow}. */ /**
* Selects a whole {@link DumpRow}, the stored columns and the computed
* `comment_count`.
*/
export const DUMP_SELECT_COLUMNS: string = `${DUMP_COLUMNS}, export const DUMP_SELECT_COLUMNS: string = `${DUMP_COLUMNS},
(SELECT COUNT(*) FROM comments WHERE dump_id = dumps.id AND deleted = 0) as comment_count`; (SELECT COUNT(*) FROM comments WHERE dump_id = dumps.id AND deleted = 0) as comment_count`;
/** {@link DUMP_SELECT_COLUMNS}, for a query aliasing `dumps` as `d`. */ /**
* Selects a whole {@link DumpRow} in a query that aliases `dumps` as `d` to
* join it with other tables.
*/
export const DUMP_SELECT_COLUMNS_ALIASED: string = `${DUMP_COLUMNS_ALIASED}, export const DUMP_SELECT_COLUMNS_ALIASED: string = `${DUMP_COLUMNS_ALIASED},
(SELECT COUNT(*) FROM comments WHERE dump_id = d.id AND deleted = 0) as comment_count`; (SELECT COUNT(*) FROM comments WHERE dump_id = d.id AND deleted = 0) as comment_count`;
/** Checks whether `obj` is a {@link DumpRow}. */ /**
* Checks whether `obj`, a row from a query, is a {@link DumpRow}. It checks
* each column's type, not that `kind` is `url` or `file`.
*/
export function isDumpRow(obj: unknown): obj is DumpRow { export function isDumpRow(obj: unknown): obj is DumpRow {
return !!obj && return !!obj &&
typeof obj === "object" && typeof obj === "object" &&
@@ -99,7 +98,10 @@ export function isDumpRow(obj: unknown): obj is DumpRow {
obj.custom_thumbnail_mime === null); obj.custom_thumbnail_mime === null);
} }
/** Converts a row to the API's {@link Dump}. */ /**
* Converts a row to the API's {@link Dump}, parsing its preview's JSON. It
* leaves `categoryIds` out: the services add them with `attachCategoryIds`.
*/
export function dumpRowToApi(row: DumpRow): Dump { export function dumpRowToApi(row: DumpRow): Dump {
return { return {
id: row.id, id: row.id,
@@ -124,7 +126,10 @@ export function dumpRowToApi(row: DumpRow): Dump {
}; };
} }
/** Converts a {@link Dump} back to a row. */ /**
* Converts a {@link Dump} back to a row, for a service that writes an edited
* dump's columns.
*/
export function dumpApiToRow(dump: Dump): DumpRow { export function dumpApiToRow(dump: Dump): DumpRow {
return { return {
id: dump.id, id: dump.id,

View File

@@ -1,5 +1,6 @@
/** /**
* The `follows` table: its row type and its guard. * Types and guards the rows of `follows`, which `follow-service.ts` reads to
* answer what a user follows.
* *
* @module * @module
*/ */
@@ -7,25 +8,22 @@
import type { SQLOutputValue } from "node:sqlite"; import type { SQLOutputValue } from "node:sqlite";
/** /**
* A row of the `follows` table: one user following either another user or a * Read by `getFollowStatus`. Exactly one of `followed_user_id` and
* playlist. * `followed_playlist_id` is set, as the table's CHECK constraint requires.
*/ */
export interface FollowRow { export interface FollowRow {
/** The follow's id. */
id: string; id: string;
/** The id of the user following. */
follower_id: string; follower_id: string;
/** The id of the user followed, when it is a user. */
followed_user_id: string | null; followed_user_id: string | null;
/** The id of the playlist followed, when it is a playlist. */
followed_playlist_id: string | null; followed_playlist_id: string | null;
/** When the follow began. */
created_at: string; created_at: string;
/** Any other column the query selects. */
[key: string]: SQLOutputValue; [key: string]: SQLOutputValue;
} }
/** Checks whether `obj` is a {@link FollowRow}. */ /**
* Checks whether `obj`, a row `getFollowStatus` selected, is a
* {@link FollowRow}.
*/
export function isFollowRow( export function isFollowRow(
obj: unknown, obj: unknown,
): obj is FollowRow { ): obj is FollowRow {

View File

@@ -1,26 +1,32 @@
/** /**
* The `invites` table: its row type and its guard. * Types and guards the rows of `invites`, which `invite-service.ts` reads to
* check an invite before sign-up.
* *
* @module * @module
*/ */
import type { SQLOutputValue } from "node:sqlite"; import type { SQLOutputValue } from "node:sqlite";
/** A row of the `invites` table. */ /**
* Stored by `createInvite`, and read back by `validateInvite` when the invite
* link is opened or used to sign up.
*/
export interface InviteRow { export interface InviteRow {
/** The invite's token, which registering redeems. */ /** The random token the invite link carries, which is the primary key. */
token: string; token: string;
/** The id of the user who made the invite. */ /** Becomes the new user's `invited_by`. */
inviter_id: string; inviter_id: string;
/** When the invite was redeemed; null while unused. */ /** Set when an account is created with the invite. */
used_at: string | null; used_at: string | null;
/** When the invite was made. */ /** Starts the 7-day window in which the invite works. */
created_at: string; created_at: string;
/** Any other column the query selects. */
[key: string]: SQLOutputValue; [key: string]: SQLOutputValue;
} }
/** Checks whether `obj` is a {@link InviteRow}. */ /**
* Checks whether `obj`, a row `validateInvite` selected, is an
* {@link InviteRow}.
*/
export function isInviteRow( export function isInviteRow(
obj: unknown, obj: unknown,
): obj is InviteRow { ): obj is InviteRow {

View File

@@ -1,6 +1,6 @@
/** /**
* The `notifications` table: its row type, its guard, and its mapper to the * Types and guards the rows of `notifications`, and converts them to the
* API's {@link Notification}. * {@link Notification} that `GET /api/notifications` answers.
* *
* @module * @module
*/ */
@@ -12,30 +12,33 @@ import type {
NotificationType, NotificationType,
} from "../../shared/notification.ts"; } from "../../shared/notification.ts";
/** A row of the `notifications` table. */ /**
* Stored by the triggers of `notification-service.ts`, and read back by
* `getNotificationsForUser`.
*/
export interface NotificationRow { export interface NotificationRow {
/** The notification's id. */
id: string; id: string;
/** The id of the user notified. */
user_id: string; user_id: string;
/** The notification's {@link NotificationType}. */ /** A {@link NotificationType}, not checked by the guard. */
type: string; type: string;
/** The notification's {@link NotificationData}, as JSON. */ /** The {@link NotificationData} as JSON, not checked by the guard. */
data: string; data: string;
/** 1 once the user has read it. */ /** 0 or 1. */
read: number; read: number;
/** When the notification was made. */
created_at: string; created_at: string;
/** /**
* What caused the notification: a second one with the same * Names the event that caused the notification. A unique index keeps a
* `(user_id, source_key)` isn't stored. Null when duplicates are allowed. * second row with the same `(user_id, source_key)` out; null allows
* duplicates.
*/ */
source_key: string | null; source_key: string | null;
/** Any other column the query selects. */
[key: string]: SQLOutputValue; [key: string]: SQLOutputValue;
} }
/** Checks whether `obj` is a {@link NotificationRow}. */ /**
* Checks whether `obj`, a row `getNotificationsForUser` selected, is a
* {@link NotificationRow}.
*/
export function isNotificationRow( export function isNotificationRow(
obj: unknown, obj: unknown,
): obj is NotificationRow { ): obj is NotificationRow {
@@ -50,7 +53,11 @@ export function isNotificationRow(
(typeof obj.source_key === "string" || obj.source_key === null); (typeof obj.source_key === "string" || obj.source_key === null);
} }
/** Converts a row to the API's {@link Notification}. */ /**
* Converts a row to the {@link Notification} the client gets.
*
* @throws {SyntaxError} When `data` isn't valid JSON.
*/
export function notificationRowToApi(row: NotificationRow): Notification { export function notificationRowToApi(row: NotificationRow): Notification {
return { return {
id: row.id, id: row.id,

View File

@@ -1,26 +1,34 @@
/** /**
* The `password_reset_tokens` table: its row type and its guard. * Types and guards the rows of `password_reset_tokens`, which
* `password-reset-service.ts` reads to make each reset link work once.
* *
* @module * @module
*/ */
import type { SQLOutputValue } from "node:sqlite"; import type { SQLOutputValue } from "node:sqlite";
/** A row of the `password_reset_tokens` table. */ /**
* Stored by `requestPasswordReset` when it emails a reset link, and read back
* by `resetPassword` when the link is used.
*/
export interface PasswordResetTokenRow { export interface PasswordResetTokenRow {
/** The reset token. */ /** The signed token itself, which the reset link carries. */
token: string; token: string;
/** The id of the user whose password it resets. */
user_id: string; user_id: string;
/** When the token stops working. */ /**
* Lets `db/startup.ts` purge old rows. The token's own `exp` is what ends
* the link.
*/
expires_at: string; expires_at: string;
/** When the token was used; null while unused. */ /** Set when the link is used, and reset to null if the reset then fails. */
used_at: string | null; used_at: string | null;
/** Any other column the query selects. */
[key: string]: SQLOutputValue; [key: string]: SQLOutputValue;
} }
/** Checks whether `obj` is a {@link PasswordResetTokenRow}. */ /**
* Checks whether `obj`, a row `resetPassword` selected, is a
* {@link PasswordResetTokenRow}.
*/
export function isPasswordResetTokenRow( export function isPasswordResetTokenRow(
obj: unknown, obj: unknown,
): obj is PasswordResetTokenRow { ): obj is PasswordResetTokenRow {

View File

@@ -1,6 +1,6 @@
/** /**
* The `playlists` table: its row type, its guard, and its mapper to the API's * Types the rows the playlist and follow services read from `playlists`, and
* {@link Playlist}. * turns them into the {@link Playlist} the API sends.
* *
* @module * @module
*/ */
@@ -9,33 +9,24 @@ import type { SQLOutputValue } from "node:sqlite";
import type { Playlist } from "../../shared/playlist.ts"; import type { Playlist } from "../../shared/playlist.ts";
/** /**
* A row of the `playlists` table. Queries may add `dump_count` and * Returned by the playlist and follow services' `SELECT`s on `playlists`. A
* `owner_username`, which {@link playlistRowToApi} carries over. * query may add `dump_count` and `owner_username` columns, which
* {@link playlistRowToApi} carries over when present.
*/ */
export interface PlaylistRow { export interface PlaylistRow {
/** The playlist's id. */
id: string; id: string;
/** The owner's id. */
user_id: string; user_id: string;
/** The playlist's title. */
title: string; title: string;
/** The playlist's URL slug. */
slug: string | null; slug: string | null;
/** The owner's description, in Markdown. */
description: string | null; description: string | null;
/** 1 when anyone may see the playlist, 0 when only its owner does. */
is_public: number; is_public: number;
/** When the playlist was created. */
created_at: string; created_at: string;
/** When the playlist was last changed, if ever. */
updated_at: string | null; updated_at: string | null;
/** The cover image's type, null without one. */
image_mime: string | null; image_mime: string | null;
/** Any other column the query selects. */
[key: string]: SQLOutputValue; [key: string]: SQLOutputValue;
} }
/** Checks whether `obj` is a {@link PlaylistRow}. */ /** Checks whether a row `node:sqlite` returned is a {@link PlaylistRow}. */
export function isPlaylistRow( export function isPlaylistRow(
obj: unknown, obj: unknown,
): obj is PlaylistRow { ): obj is PlaylistRow {
@@ -54,7 +45,7 @@ export function isPlaylistRow(
(typeof obj.image_mime === "string" || obj.image_mime === null); (typeof obj.image_mime === "string" || obj.image_mime === null);
} }
/** Converts a row to the API's {@link Playlist}. */ /** Converts `row` to the {@link Playlist} the playlist routes send. */
export function playlistRowToApi(row: PlaylistRow): Playlist { export function playlistRowToApi(row: PlaylistRow): Playlist {
return { return {
id: row.id, id: row.id,

25
api/model/rows.ts Normal file
View File

@@ -0,0 +1,25 @@
/**
* Checks the rows of a list query, for the services. Every service answers a
* row that doesn't have its table's shape with a 500 rather than leaving it
* out, so that damage shows instead of making items silently vanish.
*
* @module
*/
import { APIException } from "../lib/api-exception.ts";
/**
* Returns `rows` turned by `toApi`, once `isRow` has accepted each.
*
* @throws {APIException} 500 when a row doesn't pass `isRow`.
*/
export function parseRows<R, T>(
rows: unknown[],
isRow: (row: unknown) => row is R,
toApi: (row: R) => T,
): T[] {
if (!rows.every(isRow)) {
throw new APIException(500, "Malformed data in the database");
}
return rows.map(toApi);
}

View File

@@ -1,6 +1,7 @@
/** /**
* The `users` table: its row type, its guard, its mappers to and from the * Types and guards the rows of `users`, and converts them to the {@link User}
* stored {@link User}, and the views of a user the API sends. * that `user-service.ts` returns. A route turns a `User` into what it may send
* with {@link toPublicUser} or {@link toSelfUser}.
* *
* @module * @module
*/ */
@@ -13,56 +14,65 @@ import {
type SelfUser, type SelfUser,
} from "../../shared/user.ts"; } from "../../shared/user.ts";
/** A user as stored: what they see of themselves, plus the password hash. */ /**
* Returned by the lookups of `user-service.ts`. It must never be sent as is:
* it carries the password hash.
*/
export interface User extends SelfUser { export interface User extends SelfUser {
/** The scrypt hash of the password, as `hash.salt`. */ /** From `hashPassword`, as `hash.salt`. */
passwordHash: string; passwordHash: string;
} }
/** Returns what anyone may see of `user`: no hash, no email. */ /**
* Returns what anyone may see of `user`, without the hash and the email. Every
* route that sends another user's profile goes through it.
*/
export function toPublicUser( export function toPublicUser(
{ passwordHash: _, email: _email, ...user }: User, { passwordHash: _, email: _email, ...user }: User,
): PublicUser { ): PublicUser {
return user; return user;
} }
/** Returns what `user` sees of themselves: everything but the hash. */ /**
* Returns what `user` sees of themselves, without the hash, for the routes
* that answer the caller's own account.
*/
export function toSelfUser({ passwordHash: _, ...user }: User): SelfUser { export function toSelfUser({ passwordHash: _, ...user }: User): SelfUser {
return user; return user;
} }
/** A row of the `users` table. */ /**
* Selected by `user-service.ts` and `follow-service.ts`, and converted by
* {@link userRowToApi}. It leaves out `chat_read_at` and `tokens_valid_after`,
* which only the chat's read state and `getSessionState` query.
*/
export interface UserRow { export interface UserRow {
/** The user's id. */
id: string; id: string;
/** The user's name, unique. */
username: string; username: string;
/** The scrypt hash of the password, as `hash.salt`. */
password_hash: string; password_hash: string;
/** The user's role. */
role: Role; role: Role;
/** When the user registered. */
created_at: string; created_at: string;
/** When the profile was last changed, if ever. */ /** Bumped by any profile or avatar change. */
updated_at: string | null; updated_at: string | null;
/** The avatar's type, null without one. */ /** Null without an avatar. */
avatar_mime: string | null; avatar_mime: string | null;
/** The user's profile text, in Markdown. */ /** Markdown. */
description: string | null; description: string | null;
/** The id of the user whose invite they registered with. */ /** Null for the seeded admin, who had no inviter. */
invited_by: string | null; invited_by: string | null;
/** /**
* The inviter's name; present only when the query joins it (`LEFT JOIN users * Joined from the inviter's row rather than stored, and undefined when the
* i ON i.id = u.invited_by`). * query doesn't join it. The guard doesn't check it.
*/ */
invited_by_username: string | null; invited_by_username: string | null;
/** The user's email address. */
email: string; email: string;
/** Any other column the query selects. */
[key: string]: SQLOutputValue; [key: string]: SQLOutputValue;
} }
/** Checks whether `obj` is a {@link UserRow}. */ /**
* Checks whether `obj`, a row selected with the columns of {@link UserRow}, is
* one.
*/
export function isUserRow(obj: unknown): obj is UserRow { export function isUserRow(obj: unknown): obj is UserRow {
return !!obj && return !!obj &&
typeof obj === "object" && typeof obj === "object" &&
@@ -82,7 +92,7 @@ export function isUserRow(obj: unknown): obj is UserRow {
"email" in obj && typeof obj.email === "string"; "email" in obj && typeof obj.email === "string";
} }
/** Converts a row to the stored {@link User}. */ /** Converts a row to a {@link User}, turning nulls into undefined. */
export function userRowToApi(row: UserRow): User { export function userRowToApi(row: UserRow): User {
return { return {
id: row.id, id: row.id,
@@ -100,7 +110,10 @@ export function userRowToApi(row: UserRow): User {
}; };
} }
/** Converts a {@link User} back to a row, without the join-only columns. */ /**
* Converts a {@link User} back to a row, for `updateUser`. A `User` has no
* inviter id, so `invited_by` comes out null: don't write it back.
*/
export function userApiToRow(user: User): UserRow { export function userApiToRow(user: User): UserRow {
return { return {
id: user.id, id: user.id,

View File

@@ -1,34 +1,31 @@
/** /**
* Resolves a Bandcamp page to its streamable tracks, at play time. * Resolves a Bandcamp page to its streamable tracks when the native player
* * starts it, for `GET /api/bandcamp/tracks`. The stream URLs are signed and
* Bandcamp's signed stream URLs expire 24h after the page fetch, so they can * expire 24 hours after the page is fetched, so unlike the embed URL they
* never be baked into `rich_content` at dump-creation time the way `embedUrl` * can't be stored in the dump's preview. The browser plays them from bcbits
* is. This service is the play-time counterpart: the frontend asks for a page, * directly.
* gets a whole tracklist back, and plays the mp3s directly from bcbits.
* *
* @module * @module
*/ */
import { APIErrorCode } from "../../shared/api.ts";
import { APIException } from "../lib/api-exception.ts"; import { APIException } from "../lib/api-exception.ts";
import { fetchWithTimeout, isValidHttpUrl, readTextCapped } from "./fetch.ts"; import { fetchWithTimeout, isValidHttpUrl, readTextCapped } from "./fetch.ts";
import { BANDCAMP_HOST_RE, parseTralbum } from "./bandcamp-tralbum.ts"; import { BANDCAMP_HOST_RE, parseTralbum } from "./bandcamp-tralbum.ts";
import type { Tralbum } from "../../shared/player.ts"; import type { Tralbum } from "../../shared/player.ts";
/** /**
* How long a resolved page is cached: comfortably inside the 24h signature * Half the signatures' life, so a cached URL still has at least 12 hours left
* life, so a cache hit is never a URL that is about to expire mid-listen. * when it is handed out.
*/ */
const TTL_MS = 12 * 60 * 60 * 1000; const TTL_MS = 12 * 60 * 60 * 1000;
/** The most pages cached; the oldest write goes first. */ /** The oldest write goes first. */
const MAX_ENTRIES = 200; const MAX_ENTRIES = 200;
/** Resolved pages, by {@link cacheKey}. */
const cache = new Map<string, Tralbum>(); const cache = new Map<string, Tralbum>();
/** /**
* Returns the cache key of `url`: a page is the same page regardless of query, * Returns `url` without its query, fragment and trailing slashes, and with its
* fragment or case. * host in lowercase.
*/ */
function cacheKey(url: string): string { function cacheKey(url: string): string {
try { try {
@@ -61,10 +58,10 @@ function writeCache(key: string, value: Tralbum): void {
} }
/** /**
* Fetches a Bandcamp page and returns its tracklist, cached. * Returns the tracklist of the Bandcamp page `pageUrl`, from the cache when it
* * was resolved in the last 12 hours. `force` skips the cache: the player sets
* `force` bypasses the cache — used by the client's re-resolve path, so a * it when it re-resolves a queue whose URLs expired or are about to, which a
* cached entry whose URLs have started 403ing can't keep being handed back. * cached entry could hand back.
* *
* @throws {APIException} 400 when `pageUrl` isn't a public bandcamp.com URL, * @throws {APIException} 400 when `pageUrl` isn't a public bandcamp.com URL,
* 502 when Bandcamp can't be reached or doesn't answer with a page, 404 when * 502 when Bandcamp can't be reached or doesn't answer with a page, 404 when
@@ -75,25 +72,17 @@ export async function resolveBandcamp(
{ force = false }: { force?: boolean } = {}, { force = false }: { force?: boolean } = {},
): Promise<Tralbum> { ): Promise<Tralbum> {
if (!isValidHttpUrl(pageUrl)) { if (!isValidHttpUrl(pageUrl)) {
throw new APIException( throw new APIException(400, "Invalid URL");
APIErrorCode.VALIDATION_ERROR,
400,
"Invalid URL",
);
} }
let hostname: string; let hostname: string;
try { try {
hostname = new URL(pageUrl).hostname; hostname = new URL(pageUrl).hostname;
} catch { } catch {
throw new APIException(APIErrorCode.VALIDATION_ERROR, 400, "Invalid URL"); throw new APIException(400, "Invalid URL");
} }
if (!BANDCAMP_HOST_RE.test(hostname)) { if (!BANDCAMP_HOST_RE.test(hostname)) {
throw new APIException( throw new APIException(400, "Not a Bandcamp URL");
APIErrorCode.VALIDATION_ERROR,
400,
"Not a Bandcamp URL",
);
} }
const key = cacheKey(pageUrl); const key = cacheKey(pageUrl);
@@ -106,42 +95,24 @@ export async function resolveBandcamp(
try { try {
res = await fetchWithTimeout(pageUrl, 8000); res = await fetchWithTimeout(pageUrl, 8000);
} catch { } catch {
throw new APIException( throw new APIException(502, "Could not reach Bandcamp");
APIErrorCode.SERVER_ERROR,
502,
"Could not reach Bandcamp",
);
} }
// Every path out of here that doesn't read the body has to release it, or the // A body left unread keeps its connection open until GC, and this endpoint
// connection stays open until GC. This endpoint is unauthenticated: a caller // is public: every early return cancels it.
// hammering random /track/<slug> URLs would otherwise leak one body a request.
if (!res.ok) { if (!res.ok) {
await res.body?.cancel(); await res.body?.cancel();
throw new APIException( throw new APIException(502, `Bandcamp returned ${res.status}`);
APIErrorCode.SERVER_ERROR,
502,
`Bandcamp returned ${res.status}`,
);
} }
if (!(res.headers.get("content-type") ?? "").startsWith("text/html")) { if (!(res.headers.get("content-type") ?? "").startsWith("text/html")) {
await res.body?.cancel(); await res.body?.cancel();
throw new APIException( throw new APIException(502, "Bandcamp did not return a page");
APIErrorCode.SERVER_ERROR,
502,
"Bandcamp did not return a page",
);
} }
const parsed = parseTralbum(await readTextCapped(res), pageUrl); const parsed = parseTralbum(await readTextCapped(res), pageUrl);
if (!parsed) { if (!parsed) {
// Covers "no data-tralbum", "hasAudio: false" and preorder-only releases // No data, no audio or nothing streamable yet (a preorder). The client
// with nothing streamable — all cases where the client falls back to the // plays the embed instead.
// iframe embed rather than showing an error. throw new APIException(404, "No streamable tracks on this page");
throw new APIException(
APIErrorCode.NOT_FOUND,
404,
"No streamable tracks on this page",
);
} }
writeCache(key, parsed); writeCache(key, parsed);

View File

@@ -1,18 +1,8 @@
/** /**
* Parser for Bandcamp's `data-tralbum` attribute. * Reads a release out of a Bandcamp page, for `bandcamp-streams.ts`. Every
* * track and album page carries the whole release, stream URLs included, as
* Every Bandcamp track and album page carries the whole release — titles, * HTML-escaped JSON in a `data-tralbum` attribute, so one fetch covers an
* durations and signed mp3 stream URLs — as one HTML-escaped JSON blob on a * album. The format is undocumented, so every field is treated as optional.
* `data-tralbum` attribute. One fetch of an album page therefore yields every
* track's stream URL; there is no per-track request.
*
* The stream URLs are signed and expire 24h after the page was fetched, which
* is why nothing here is ever persisted to `rich_content` — see
* `api/routes/bandcamp.ts` for the resolve-at-play-time endpoint.
*
* The shape is undocumented, so every field is treated as optional and the
* whole parse is best-effort: a `null` return means "not a Bandcamp page",
* which callers turn into a fallback to the iframe embed.
* *
* @module * @module
*/ */
@@ -20,19 +10,16 @@
import type { Tralbum, TralbumTrack } from "../../shared/player.ts"; import type { Tralbum, TralbumTrack } from "../../shared/player.ts";
/** /**
* The hostnames Bandcamp serves its own sites from, anchored at the end so that `bandcamp.com.example.org` cannot pass. Artists on custom * Matches the hostnames `bandcamp-streams.ts` accepts: bandcamp.com and its
* domains do not match this and keep using the embed. * subdomains, and not `bandcamp.com.example.org`. An artist on a custom domain
* doesn't match, and the player falls back to the embed.
*/ */
export const BANDCAMP_HOST_RE = /(?:^|\.)bandcamp\.com$/; export const BANDCAMP_HOST_RE = /(?:^|\.)bandcamp\.com$/;
/** /**
* Decodes HTML entities in a single pass. * Decodes HTML entities in a single pass. `html.ts` decodes `&amp;` before
* * `&quot;`, which turns a title's literal `&quot;` (escaped as `&amp;quot;`)
* The single pass is the point: `decodeHtmlEntities` in remote/html.ts * into a bare `"` that breaks the JSON.
* replaces `&amp;` before `&quot;`, so an escaped-ampersand-then-quot sequence
* (`&amp;quot;`, i.e. the literal text `&quot;` inside a title) decodes twice
* and yields a bare `"` — which lands inside the JSON string and breaks the
* parse. Matching every entity form in one regex means each is replaced once.
*/ */
function decodeEntitiesOnce(input: string): string { function decodeEntitiesOnce(input: string): string {
const named: Record<string, string> = { const named: Record<string, string> = {
@@ -53,7 +40,7 @@ function decodeEntitiesOnce(input: string): string {
); );
} }
/** Returns the URL of the large square artwork numbered `artId`. */ /** Returns the URL of the artwork `artId`; undefined when it isn't numeric. */
function artworkUrl(artId: unknown): string | undefined { function artworkUrl(artId: unknown): string | undefined {
if (typeof artId !== "number" && typeof artId !== "string") return undefined; if (typeof artId !== "number" && typeof artId !== "string") return undefined;
const id = String(artId).trim(); const id = String(artId).trim();
@@ -63,8 +50,9 @@ function artworkUrl(artId: unknown): string | undefined {
} }
/** /**
* Pulls the `data-tralbum` blob out of a Bandcamp page and normalises it. * Returns the release on the Bandcamp page `html`, fetched from `sourceUrl`;
* Returns null when the page has no blob, no audio, or nothing parseable. * null when the page has no release data, no audio or no streamable track.
* Stream URLs outside bcbits.com are dropped.
*/ */
export function parseTralbum(html: string, sourceUrl: string): Tralbum | null { export function parseTralbum(html: string, sourceUrl: string): Tralbum | null {
const match = html.match(/\bdata-tralbum=(["'])([\s\S]*?)\1/); const match = html.match(/\bdata-tralbum=(["'])([\s\S]*?)\1/);
@@ -84,8 +72,8 @@ export function parseTralbum(html: string, sourceUrl: string): Tralbum | null {
const t = (entry ?? {}) as Record<string, unknown>; const t = (entry ?? {}) as Record<string, unknown>;
const file = (t.file ?? {}) as Record<string, unknown>; const file = (t.file ?? {}) as Record<string, unknown>;
const mp3 = typeof file["mp3-128"] === "string" ? file["mp3-128"] : null; const mp3 = typeof file["mp3-128"] === "string" ? file["mp3-128"] : null;
// Only ever hand out bcbits URLs. Without this the endpoint would relay // A hostile page could put any host here, and the endpoint would hand it
// whatever host a hostile page put in the blob straight to the browser. // to the browser.
const streamUrl = mp3 && isBcbitsUrl(mp3) ? mp3 : null; const streamUrl = mp3 && isBcbitsUrl(mp3) ? mp3 : null;
return { return {
index, index,
@@ -117,7 +105,6 @@ export function parseTralbum(html: string, sourceUrl: string): Tralbum | null {
}; };
} }
/** Checks whether `url` is an https URL on Bandcamp's bcbits.com CDN. */
function isBcbitsUrl(url: string): boolean { function isBcbitsUrl(url: string): boolean {
try { try {
const { protocol, hostname } = new URL(url); const { protocol, hostname } = new URL(url);

View File

@@ -1,7 +1,8 @@
/** /**
* The guarded fetch every request to another site goes through: each hop must * Fetches pages and images from other sites for link previews, the image proxy
* resolve to a public address, the whole exchange has a deadline, and pages are * and the Bandcamp resolver, without letting a user-supplied URL reach the
* read up to a cap. * server's own network. Each hop must resolve to a public address, the whole
* exchange has a deadline, and pages are read up to a cap.
* *
* @module * @module
*/ */
@@ -20,15 +21,11 @@ const FETCH_HEADERS = {
const MAX_REDIRECTS = 5; const MAX_REDIRECTS = 5;
/** /**
* Fetches `url` through a `curl --insecure` subprocess and returns a minimal * Fetches `url` with `curl --insecure`, for a site whose TLS certificate Deno
* Response. * rejects. Returns a Response carrying only the `Location` and `Content-Type`
* Used as a last resort when Deno's rustls rejects the server's TLS certificate * headers, or null when curl is missing or fails. The connection is pinned to
* due to an unsupported algorithm (e.g. UnsupportedSignatureAlgorithm). * `address`, the one {@link resolvePublicAddress} checked, and redirects come
* Returns null if curl is unavailable or exits non-zero. * back unfollowed so that {@link fetchWithTimeout} checks each hop.
*
* `address` pins the connection to the IP already checked by
* resolvePublicAddress, and curl doesn't follow redirects itself: a 3xx comes
* back with its Location so fetchWithTimeout can check the next hop too.
*/ */
async function fetchViaCurl( async function fetchViaCurl(
url: URL, url: URL,
@@ -49,6 +46,8 @@ async function fetchViaCurl(
`${url.hostname.replace(/^\[(.*)\]$/, "$1")}:${port}:${pinned}`, `${url.hostname.replace(/^\[(.*)\]$/, "$1")}:${port}:${pinned}`,
"--max-time", "--max-time",
String(Math.ceil(timeoutMs / 1000)), String(Math.ceil(timeoutMs / 1000)),
// curl fails on a larger file rather than stopping at a cap, so this is
// looser than MAX_PAGE_BYTES, where readTextCapped still cuts.
"--max-filesize", "--max-filesize",
String(2 * MAX_PAGE_BYTES), String(2 * MAX_PAGE_BYTES),
"--user-agent", "--user-agent",
@@ -78,20 +77,7 @@ async function fetchViaCurl(
if (contentType) headers.set("content-type", contentType); if (contentType) headers.set("content-type", contentType);
const bytes = await Deno.readFile(tmpPath); const bytes = await Deno.readFile(tmpPath);
if (!contentType.startsWith("text/")) {
return new Response(bytes, { status: statusCode, headers }); return new Response(bytes, { status: statusCode, headers });
}
// Decode using the charset declared in the Content-Type header so that
// pages served in ISO-8859-1, windows-1252, etc. are read correctly.
const charset = /charset=([\w-]+)/i.exec(contentType)?.[1] ?? "utf-8";
let bodyText: string;
try {
bodyText = new TextDecoder(charset, { fatal: false }).decode(bytes);
} catch {
bodyText = new TextDecoder("utf-8", { fatal: false }).decode(bytes);
}
headers.set("content-type", contentType.replace(/;.*$/, "; charset=utf-8"));
return new Response(bodyText, { status: statusCode, headers });
} catch { } catch {
return null; return null;
} finally { } finally {
@@ -100,15 +86,15 @@ async function fetchViaCurl(
} }
/** /**
* The most bytes of a fetched page that are read. Metadata lives in the <head>, * The default cap of {@link readTextCapped}. The metadata previews read is in
* and a "page" past this is a download, not something to scrape. * the `<head>`, and a page past this is a download.
*/ */
export const MAX_PAGE_BYTES = 4 * 1024 * 1024; export const MAX_PAGE_BYTES = 4 * 1024 * 1024;
/** /**
* Hands `body` on, and clears the request's deadline once it has been read to * Wraps `body` so that the request's deadline `timer` is cleared only once the
* the end or cancelled. Until then the deadline still applies: a server that * body is read to the end or cancelled. A server that answers at once and then
* answers at once and then drips its body can't hold the request open. * drips its body is still cut off.
*/ */
function withDeadline( function withDeadline(
body: ReadableStream<Uint8Array>, body: ReadableStream<Uint8Array>,
@@ -138,14 +124,14 @@ function withDeadline(
} }
/** /**
* Reads a response's body as text, stopping at `maxBytes` (the rest is * Reads the body of `res` up to `maxBytes`; the rest is cancelled, never
* cancelled, never downloaded). * downloaded. The image proxy reads through it, and {@link readTextCapped}.
*/ */
export async function readTextCapped( export async function readBytesCapped(
res: Response, res: Response,
maxBytes = MAX_PAGE_BYTES, maxBytes: number,
): Promise<string> { ): Promise<Uint8Array> {
if (!res.body) return ""; if (!res.body) return new Uint8Array();
const reader = res.body.getReader(); const reader = res.body.getReader();
const chunks: Uint8Array[] = []; const chunks: Uint8Array[] = [];
let size = 0; let size = 0;
@@ -167,10 +153,45 @@ export async function readTextCapped(
offset += part.byteLength; offset += part.byteLength;
if (offset >= bytes.byteLength) break; if (offset >= bytes.byteLength) break;
} }
return new TextDecoder().decode(bytes); return bytes;
} }
/** Makes one request, following no redirects. The timeout covers the body too. */ /**
* Returns the charset a page declares, in its `Content-Type` header or else in
* a `<meta>` near its top, as HTML allows; UTF-8 when it declares none.
*/
function declaredCharset(contentType: string, bytes: Uint8Array): string {
const fromHeader = /charset=["']?([\w-]+)/i.exec(contentType)?.[1];
if (fromHeader) return fromHeader;
// Latin-1 maps every byte to a character, so the ASCII of the tag reads
// right whatever the page's encoding.
const head = new TextDecoder("latin1").decode(bytes.subarray(0, 1024));
return /<meta[^>]+charset=["']?([\w-]+)/i.exec(head)?.[1] ?? "utf-8";
}
/**
* Reads the body of `res` as text, up to `maxBytes`, in the charset the page
* declares; the rest is cancelled, never downloaded. The providers and the
* Bandcamp and YouTube resolvers read pages through it.
*/
export async function readTextCapped(
res: Response,
maxBytes = MAX_PAGE_BYTES,
): Promise<string> {
const bytes = await readBytesCapped(res, maxBytes);
const charset = declaredCharset(res.headers.get("content-type") ?? "", bytes);
try {
return new TextDecoder(charset).decode(bytes);
} catch {
// An unknown label: TextDecoder throws rather than guess.
return new TextDecoder().decode(bytes);
}
}
/**
* Makes one request to `url`, following no redirects, with a deadline that
* covers the body too.
*/
async function fetchOnce( async function fetchOnce(
url: URL, url: URL,
address: string, address: string,
@@ -191,10 +212,9 @@ async function fetchOnce(
throw err; throw err;
} }
// Deno/rustls rejected the server's TLS certificate (expired, self-signed, // Deno can't be told to skip certificate checks for one request, so a
// or an unsupported signature algorithm) and offers no way to override // site with an expired, self-signed or unusually signed certificate is
// verification programmatically. Fall back to a `curl --insecure` // fetched with curl instead.
// subprocess, which uses its own TLS stack.
const curlRes = await fetchViaCurl(url, address, timeoutMs); const curlRes = await fetchViaCurl(url, address, timeoutMs);
if (curlRes) return curlRes; if (curlRes) return curlRes;
@@ -212,10 +232,10 @@ async function fetchOnce(
} }
/** /**
* Fetches a user-supplied URL, following up to five redirects. Every hop — the * Fetches a user-supplied URL, following up to five redirects, each of which
* URL itself and each redirect — must resolve to a public address (see * must resolve to a public address too. `timeoutMs` bounds each hop, body
* {@link resolvePublicAddress}), so a link or a redirect can't point the * included. The caller must read or cancel the body, or its connection stays
* server at its own network. * open.
* *
* @throws {TypeError} When a hop isn't http(s), there are too many redirects, * @throws {TypeError} When a hop isn't http(s), there are too many redirects,
* or the request fails. * or the request fails.
@@ -242,16 +262,17 @@ export async function fetchWithTimeout(
} }
/** /**
* Checks whether `raw` is an http(s) URL a dump may link to: not on a private * Checks whether `raw` is an http(s) URL on a host that isn't private, or a
* host, unless it is one of this site's own pages. * link to one of this site's own pages. Routes and services check a URL with it
* before storing, previewing or proxying it; it does no DNS lookup, which
* {@link fetchWithTimeout} does.
*/ */
export function isValidHttpUrl(raw: string): boolean { export function isValidHttpUrl(raw: string): boolean {
try { try {
const u = new URL(raw); const u = new URL(raw);
if (u.protocol !== "http:" && u.protocol !== "https:") return false; if (u.protocol !== "http:" && u.protocol !== "https:") return false;
// This site's own pages are fine even on a private host (e.g. localhost in // This site may run on a private host (localhost in development), and the
// development): selfProvider reads them from the DB, no HTTP involved. // self provider reads its pages from the database without fetching them.
// Every actual fetch is checked again, after DNS, in fetchWithTimeout.
return !isPrivateHostname(u) || isSelfUrl(u); return !isPrivateHostname(u) || isSelfUrl(u);
} catch { } catch {
return false; return false;

View File

@@ -1,11 +1,17 @@
/** /**
* Scraping helpers: the metadata a link preview reads from a page's HTML — * Reads a link preview's metadata out of a page's HTML with regular
* meta and OG tags, the title, JSON-LD, images, icons and brand colors. * expressions, without parsing the document. The generic provider uses most of
* it; the Bandcamp, SoundCloud and YouTube providers read only OG tags. Each
* function returns undefined when the page doesn't say.
* *
* @module * @module
*/ */
/** Decodes the named and numeric HTML entities in `str`. */ /**
* Decodes the five XML entities and numeric ones in `str`. `&amp;` goes first,
* so `&amp;quot;` decodes twice; `bandcamp-tralbum.ts`, which decodes JSON,
* has its own single-pass decoder.
*/
function decodeHtmlEntities(str: string): string { function decodeHtmlEntities(str: string): string {
return str return str
.replace(/&amp;/gi, "&") .replace(/&amp;/gi, "&")
@@ -21,9 +27,9 @@ function decodeHtmlEntities(str: string): string {
} }
/** /**
* Returns the `content` of the first `<meta>` tag whose `attr` equals `value` * Returns the `content` of the first `<meta>` tag whose `attr` equals `value`,
* (either attribute ordering). Each tag is matched in isolation so * in whichever order the attributes come. Each tag is matched on its own, so
* the `content` capture can never span across tag boundaries. * the capture can't run into the next tag.
*/ */
function extractMetaContent( function extractMetaContent(
html: string, html: string,
@@ -44,7 +50,7 @@ function extractMetaContent(
return undefined; return undefined;
} }
/** Returns the content of `<meta property="og:<tag>">`. */ /** Returns the content of `<meta property="og:<tag>">`, entities decoded. */
export function extractOgTag( export function extractOgTag(
html: string, html: string,
tag: string, tag: string,
@@ -53,8 +59,8 @@ export function extractOgTag(
} }
/** /**
* Returns the content of `<meta name="…" content="…">` (both attribute * Returns the content of `<meta name="<name>">`, entities decoded. Twitter
* orderings). * card tags use `name`, not `property`.
*/ */
export function extractMetaName( export function extractMetaName(
html: string, html: string,
@@ -63,20 +69,16 @@ export function extractMetaName(
return extractMetaContent(html, "name", name); return extractMetaContent(html, "name", name);
} }
/** Returns the text content of the `<title>` element. */ /** Returns the text of the `<title>` element, trimmed and entities decoded. */
export function extractPageTitle(html: string): string | undefined { export function extractPageTitle(html: string): string | undefined {
const match = html.match(/<title[^>]*>([^<]+)<\/title>/i); const match = html.match(/<title[^>]*>([^<]+)<\/title>/i);
return match ? decodeHtmlEntities(match[1].trim()) : undefined; return match ? decodeHtmlEntities(match[1].trim()) : undefined;
} }
// ── Brand color helpers ───────────────────────────────────────────────────────
/** /**
* Returns the page's `theme-color`. * Returns the page's `theme-color`, as written: pass it to
* * {@link normalizeCssColor}. Sites often ship one per color scheme with the
* Sites routinely ship the tag twice, scoped by `media`, and often list the * dark one first, so the unscoped tag wins, then the light one, then any.
* dark-scheme one first — so unlike `extractMetaName` this can't just take the
* first match. Prefer the unscoped tag, then the light-scheme one, then any.
*/ */
export function extractThemeColor(html: string): string | undefined { export function extractThemeColor(html: string): string | undefined {
const tagRe = /<meta\b[^>]*>/gi; const tagRe = /<meta\b[^>]*>/gi;
@@ -100,7 +102,10 @@ export function extractThemeColor(html: string): string | undefined {
return unscoped ?? light ?? any; return unscoped ?? light ?? any;
} }
/** Returns the `color` of `<link rel="mask-icon">` (Safari pinned tabs). */ /**
* Returns the `color` of `<link rel="mask-icon">`, Safari's pinned-tab icon,
* as written. The generic provider's last resort for a brand color.
*/
export function extractMaskIconColor(html: string): string | undefined { export function extractMaskIconColor(html: string): string | undefined {
const linkRe = /<link[^>]+>/gi; const linkRe = /<link[^>]+>/gi;
let m: RegExpExecArray | null; let m: RegExpExecArray | null;
@@ -114,8 +119,8 @@ export function extractMaskIconColor(html: string): string | undefined {
} }
/** /**
* The handful of named colors that actually show up in `theme-color`. The full * The named colors sites actually put in `theme-color`; the rest of CSS's names
* 148-name table isn't worth carrying for the long tail. * aren't worth carrying.
*/ */
const NAMED_COLORS: Record<string, string> = { const NAMED_COLORS: Record<string, string> = {
black: "#000000", black: "#000000",
@@ -145,9 +150,10 @@ function clampByte(n: number): number {
} }
/** /**
* Normalizes a CSS color to canonical lowercase `#rrggbb`, or `undefined` when * Returns `raw` as a lowercase `#rrggbb`, from a hex, `rgb()`, `rgba()` or
* it isn't one of the forms sites actually use. Alpha is dropped — the color is * common named color; undefined for any other form. The generic provider
* only ever used as a tint over the app's own surface. * stores it as the preview's `accentColor`, a tint over the app's own surface,
* so alpha is dropped.
*/ */
export function normalizeCssColor(raw: string | undefined): string | undefined { export function normalizeCssColor(raw: string | undefined): string | undefined {
if (!raw) return undefined; if (!raw) return undefined;
@@ -182,17 +188,17 @@ export function normalizeCssColor(raw: string | undefined): string | undefined {
return undefined; return undefined;
} }
// ── JSON-LD helpers ─────────────────────────────────────────────────────────── /**
* Returned to the generic provider by {@link extractJsonLd}, which ranks each
/** What a page's JSON-LD says about it. */ * field below the page's OG and Twitter tags.
*/
export interface JsonLdResult { export interface JsonLdResult {
/** The node's `name` or `headline`. */ /** From `name`, or else `headline`. */
title?: string; title?: string;
/** The node's `description`. */
description?: string; description?: string;
/** /**
* The node's `image`, `thumbnailUrl` or `thumbnail`, when it is an absolute * From `image`, or else `thumbnailUrl` or `thumbnail`, and only when it is
* URL. * an absolute URL.
*/ */
thumbnailUrl?: string; thumbnailUrl?: string;
} }
@@ -237,9 +243,10 @@ function ldExtractNode(data: unknown): JsonLdResult {
} }
/** /**
* Parses every `<script type="application/ld+json">` block and returns the first * Returns the first JSON-LD node in `html` that has a title or an image,
* node that yields a title or image. Handles `@graph`, arrays, and the common * looking through `@graph` and arrays; empty when there is none. An image may
* `image` shapes (string, string[], ImageObject). * be a URL, a list of them or an `ImageObject`. Blocks that aren't valid JSON
* are skipped.
*/ */
export function extractJsonLd(html: string): JsonLdResult { export function extractJsonLd(html: string): JsonLdResult {
const pattern = const pattern =
@@ -249,14 +256,15 @@ export function extractJsonLd(html: string): JsonLdResult {
try { try {
const result = ldExtractNode(JSON.parse(match[1])); const result = ldExtractNode(JSON.parse(match[1]));
if (result.title || result.thumbnailUrl) return result; if (result.title || result.thumbnailUrl) return result;
} catch { /* invalid JSON — skip */ } } catch { /* invalid JSON: try the next block */ }
} }
return {}; return {};
} }
/** /**
* Returns the `src` of the first `<img>` whose declared width or height is at * Returns the absolute URL of the first `<img>` whose declared width and height
* least `minSize` pixels (default 200). Skips data URIs. Resolves relative URLs. * are both at least `minSize` pixels; an image without them doesn't count.
* Data URIs are skipped.
*/ */
export function extractLargeImage( export function extractLargeImage(
html: string, html: string,
@@ -283,13 +291,12 @@ export function extractLargeImage(
} }
/** /**
* Collects all `<link rel="icon">` / `<link rel="apple-touch-icon">` tags, ranks * Returns the absolute URL of the page's largest icon, among its `icon` and
* them by declared size (largest wins), and returns the best resolved URL. * `apple-touch-icon` links, by declared `sizes`; the first one when none
* Falls back to the first match when no `sizes` attribute is present. * declares a size. The generic provider stores it as the preview's
* `faviconUrl`, which the thumbnail placeholder draws.
* *
* SVG icons are ranked above everything: they carry no `sizes` attribute, so * An SVG icon wins over any size: it declares none, yet scales to any.
* they'd otherwise score 0 and lose to a 16×16 PNG, yet they scale cleanly to
* whatever size the placeholder renders them at.
*/ */
export function extractBestIcon( export function extractBestIcon(
html: string, html: string,
@@ -330,7 +337,7 @@ export function extractBestIcon(
return candidates[0].href; return candidates[0].href;
} }
/** Filename tokens that mark an image as UI chrome rather than content. */ /** Filename words that mark an image as page furniture rather than content. */
const UI_IMAGE_KEYWORDS = new Set([ const UI_IMAGE_KEYWORDS = new Set([
"logo", "logo",
"icon", "icon",
@@ -350,24 +357,22 @@ const UI_IMAGE_KEYWORDS = new Set([
]); ]);
/** /**
* Checks whether `src` looks like UI chrome: an SVG, or a filename with one of * Checks whether `src` looks like page furniture: an SVG, or a filename with
* the {@link UI_IMAGE_KEYWORDS}. * one of the {@link UI_IMAGE_KEYWORDS} between its `-`, `_` or `.`.
*/ */
function isUiImage(src: string): boolean { function isUiImage(src: string): boolean {
if (/\.svg(\?|$)/i.test(src)) return true; if (/\.svg(\?|$)/i.test(src)) return true;
const filename = src.split("?")[0].split("/").pop() ?? ""; const filename = src.split("?")[0].split("/").pop() ?? "";
const baseName = filename.replace(/\.[^.]+$/, ""); // strip extension const baseName = filename.replace(/\.[^.]+$/, "");
// Split on common filename separators (-, _, .) and check each token
return baseName.toLowerCase().split(/[-_.]/).some((t) => return baseName.toLowerCase().split(/[-_.]/).some((t) =>
UI_IMAGE_KEYWORDS.has(t) UI_IMAGE_KEYWORDS.has(t)
); );
} }
/** /**
* Returns the `src` of the first `<img>` that looks like content rather than UI * Returns the absolute URL of the first `<img>` that looks like content rather
* chrome. Skips SVGs, data URIs, and images whose filename tokens match common * than a logo, icon or other page furniture. Data URIs are skipped. The generic
* icon/logo/nav patterns (logo, icon, sprite, etc.). * provider's last resort for a thumbnail.
* Resolves relative and protocol-relative URLs.
*/ */
export function extractFirstContentImage( export function extractFirstContentImage(
html: string, html: string,
@@ -388,36 +393,3 @@ export function extractFirstContentImage(
} }
return undefined; return undefined;
} }
/**
* Returns the `href` of the first `<link rel="…">` whose rel contains
* `relFragment`, resolved to an absolute URL using `baseUrl`.
*/
export function extractLinkHref(
html: string,
relFragment: string,
baseUrl: string,
): string | undefined {
const escaped = relFragment.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
const patterns = [
new RegExp(
`<link[^>]+rel=["'][^"']*${escaped}[^"']*["'][^>]+href=["']([^"']+)["']`,
"i",
),
new RegExp(
`<link[^>]+href=["']([^"']+)["'][^>]+rel=["'][^"']*${escaped}[^"']*["']`,
"i",
),
];
for (const pattern of patterns) {
const match = html.match(pattern);
if (match) {
try {
return new URL(match[1], baseUrl).toString();
} catch {
return undefined;
}
}
}
return undefined;
}

View File

@@ -1,17 +1,21 @@
/** /**
* The image proxy's fetch and cache: remote thumbnails are served through * Fetches and caches the remote images `GET /api/proxy-image` serves. The
* this site, so viewers' browsers never contact the hosts. * frontend routes a thumbnail through it when the image is on plain http,
* which an https page would block, or when loading it directly failed, often
* to hotlink protection.
* *
* @module * @module
*/ */
import { fetchWithTimeout } from "./fetch.ts"; import { fetchWithTimeout, readBytesCapped } from "./fetch.ts";
/** An image fetched for the proxy. */ /**
* Returned to `GET /api/proxy-image` by {@link fetchProxiedImage}, which sends
* it back under `contentType`.
*/
export interface ProxiedImage { export interface ProxiedImage {
/** The image's bytes. */
bytes: Uint8Array; bytes: Uint8Array;
/** The image's MIME type, as its host sent it. */ /** One of the raster types, without parameters. */
contentType: string; contentType: string;
} }
@@ -26,14 +30,12 @@ const PROXYABLE_IMAGE_TYPES = new Set([
"image/vnd.microsoft.icon", "image/vnd.microsoft.icon",
]); ]);
/** How long a proxied image is served from the cache. */
const IMAGE_CACHE_TTL_MS = 24 * 60 * 60 * 1000; const IMAGE_CACHE_TTL_MS = 24 * 60 * 60 * 1000;
/** The cache's size bound, in image bytes. */
const IMAGE_CACHE_MAX_BYTES = 32 * 1024 * 1024; const IMAGE_CACHE_MAX_BYTES = 32 * 1024 * 1024;
/** /**
* Proxied images, by URL: every viewer of a feed asks for the same thumbnails, * Proxied images, by URL, since every viewer of a feed asks for the same
* which were fetched from their hosts again for each one. Bounded by total * thumbnails. The least recently used go first, and the Map's order is that
* size; the least recently used go first. Map order is recency order. * of use.
*/ */
const imageCache = new Map< const imageCache = new Map<
string, string,
@@ -51,7 +53,7 @@ function readImageCache(
imageCacheBytes -= hit.bytes.byteLength; imageCacheBytes -= hit.bytes.byteLength;
return undefined; return undefined;
} }
imageCache.set(url, hit); // most recently used imageCache.set(url, hit); // Moves it to the end: most recently used.
return hit; return hit;
} }
@@ -77,7 +79,7 @@ function writeImageCache(
imageCacheBytes += bytes.byteLength; imageCacheBytes += bytes.byteLength;
} }
/** Forgets every proxied image (for the tests). */ /** Empties the cache, for the tests' reset between cases. */
export function clearImageCache(): void { export function clearImageCache(): void {
imageCache.clear(); imageCache.clear();
imageCacheBytes = 0; imageCacheBytes = 0;
@@ -85,9 +87,12 @@ export function clearImageCache(): void {
/** /**
* Returns the image at `url`, from the cache or else from its host; null when * Returns the image at `url`, from the cache or else from its host; null when
* the host answers with something that isn't a raster image of 5 MB at most. * the host answers with an error, or with anything but a raster image of 5 MB
* at most. The route
* checks `url` with `isValidHttpUrl` first.
* *
* @throws {Error} When the host can't be reached (see {@link fetchWithTimeout}). * @throws {Error} When the host can't be reached, as
* {@link fetchWithTimeout} reports it.
*/ */
export async function fetchProxiedImage( export async function fetchProxiedImage(
url: string, url: string,
@@ -96,9 +101,12 @@ export async function fetchProxiedImage(
if (cached) return cached; if (cached) return cached;
const res = await fetchWithTimeout(url, 8000); const res = await fetchWithTimeout(url, 8000);
// Raster formats only. The bytes go out under this site's origin, so an if (!res.ok) {
// SVG (which can carry script) or anything else a browser might execute await res.body?.cancel();
// would run with the app's origin if opened directly. return null;
}
// Raster formats only: the bytes go out under this site's origin, where an
// SVG, which can carry script, would run if opened directly.
const contentType = (res.headers.get("content-type") ?? "") const contentType = (res.headers.get("content-type") ?? "")
.split(";")[0].trim().toLowerCase(); .split(";")[0].trim().toLowerCase();
if (!PROXYABLE_IMAGE_TYPES.has(contentType)) { if (!PROXYABLE_IMAGE_TYPES.has(contentType)) {
@@ -106,14 +114,15 @@ export async function fetchProxiedImage(
return null; return null;
} }
const MAX_SIZE = 5 * 1024 * 1024; // 5 MB const MAX_SIZE = 5 * 1024 * 1024;
const contentLength = Number(res.headers.get("content-length") ?? "0"); const contentLength = Number(res.headers.get("content-length") ?? "0");
if (contentLength > MAX_SIZE) { if (contentLength > MAX_SIZE) {
await res.body?.cancel(); await res.body?.cancel();
return null; return null;
} }
const bytes = new Uint8Array(await res.arrayBuffer()); // Without a Content-Length, the cap is found by reading one byte past it.
const bytes = await readBytesCapped(res, MAX_SIZE + 1);
if (bytes.length > MAX_SIZE) return null; if (bytes.length > MAX_SIZE) return null;
writeImageCache(url, bytes, contentType); writeImageCache(url, bytes, contentType);

View File

@@ -1,5 +1,6 @@
/** /**
* The Bandcamp link provider: previews from a release page's OG tags. * Previews Bandcamp links from their page's OG tags, for
* `rich-content-service.ts`.
* *
* @module * @module
*/ */
@@ -8,16 +9,19 @@ import type { RichContent } from "../../../shared/dump.ts";
import type { RichContentProvider } from "./provider.ts"; import type { RichContentProvider } from "./provider.ts";
import { extractOgTag } from "../html.ts"; import { extractOgTag } from "../html.ts";
import { fetchWithTimeout, readTextCapped } from "../fetch.ts"; import { fetchWithTimeout, readTextCapped } from "../fetch.ts";
import { BANDCAMP_HOST_RE } from "../bandcamp-tralbum.ts";
const BANDCAMP_REGEX = /(?:^|\.)bandcamp\.com/; /**
* Claims bandcamp.com and its subdomains, where artists have their pages. Takes
/** The provider for bandcamp.com pages, with the embed player from `og:video`. */ * the embed player's URL from the page's `og:video`. The native player fetches the page's streams
* itself at play time, through `GET /api/bandcamp/tracks`.
*/
export const bandcampProvider: RichContentProvider = { export const bandcampProvider: RichContentProvider = {
name: "bandcamp", name: "bandcamp",
matches(url: string): boolean { matches(url: string): boolean {
try { try {
return BANDCAMP_REGEX.test(new URL(url).hostname); return BANDCAMP_HOST_RE.test(new URL(url).hostname);
} catch { } catch {
return false; return false;
} }

View File

@@ -1,5 +1,6 @@
/** /**
* The fallback link provider, for any page no other provider claims. * Previews any page no other provider claims, for `rich-content-service.ts`,
* from whatever metadata its HTML carries.
* *
* @module * @module
*/ */
@@ -21,15 +22,16 @@ import {
import { fetchWithTimeout, readTextCapped } from "../fetch.ts"; import { fetchWithTimeout, readTextCapped } from "../fetch.ts";
/** /**
* The provider for any HTML page, previewed from its OG and Twitter tags, JSON-LD, and markup: * Matches every URL, so it comes last in the service's list. Each field falls
* title, description, artwork, icon and brand color. Matches every URL, so it * back from OG tags to Twitter tags, JSON-LD and the markup itself. The icon
* must come last. * and brand color feed the placeholder the frontend draws when there is no
* thumbnail.
*/ */
export const genericProvider: RichContentProvider = { export const genericProvider: RichContentProvider = {
name: "generic", name: "generic",
matches(_url: string): boolean { matches(_url: string): boolean {
return true; // fallback — always matches return true;
}, },
async fetch(url: string): Promise<RichContent> { async fetch(url: string): Promise<RichContent> {
@@ -44,9 +46,9 @@ export const genericProvider: RichContentProvider = {
const html = await readTextCapped(res); const html = await readTextCapped(res);
const ld = extractJsonLd(html); const ld = extractJsonLd(html);
// If og:url is present but points to a different page (e.g. the homepage), // An og:url naming another page (often the home page) means the OG tags
// the og: block is a site-level fallback, not page-specific metadata. // are the site's defaults, not this page's: its title and image are
// In that case skip og:title and og:image so page-level signals win. // skipped.
const ogUrl = extractOgTag(html, "url"); const ogUrl = extractOgTag(html, "url");
const useOg = !ogUrl || (() => { const useOg = !ogUrl || (() => {
try { try {
@@ -58,36 +60,29 @@ export const genericProvider: RichContentProvider = {
} }
})(); })();
// Title: og:title (page-matched) → twitter:title → JSON-LD → <title>
const title = (useOg ? extractOgTag(html, "title") : undefined) ?? const title = (useOg ? extractOgTag(html, "title") : undefined) ??
extractMetaName(html, "twitter:title") ?? extractMetaName(html, "twitter:title") ??
ld.title ?? ld.title ??
extractPageTitle(html); extractPageTitle(html);
// Site name: og:site_name → hostname
const siteName = extractOgTag(html, "site_name") ?? const siteName = extractOgTag(html, "site_name") ??
new URL(url).hostname.replace(/^www\./, ""); new URL(url).hostname.replace(/^www\./, "");
// Description: og:description → twitter:description → JSON-LD → <meta name="description">
const description = extractOgTag(html, "description") ?? const description = extractOgTag(html, "description") ??
extractMetaName(html, "twitter:description") ?? extractMetaName(html, "twitter:description") ??
ld.description ?? ld.description ??
extractMetaName(html, "description"); extractMetaName(html, "description");
// Image: og:image (page-matched) → twitter:image → JSON-LD → large <img> → // Never the favicon: a 16×16 icon cropped into a thumbnail looks broken,
// first content <img>. The chain deliberately stops there: a favicon is not // and the placeholder drawn instead shows the icon properly.
// artwork, and pretending otherwise means every art-less page gets a 16×16
// icon cover-cropped into a 128×72 box. No match here means "no artwork",
// and the frontend draws a generated placeholder instead.
const thumbnailUrl = (useOg ? extractOgTag(html, "image") : undefined) ?? const thumbnailUrl = (useOg ? extractOgTag(html, "image") : undefined) ??
extractMetaName(html, "twitter:image") ?? extractMetaName(html, "twitter:image") ??
ld.thumbnailUrl ?? ld.thumbnailUrl ??
extractLargeImage(html, url) ?? extractLargeImage(html, url) ??
extractFirstContentImage(html, url); extractFirstContentImage(html, url);
// Icon and brand color are independent facts about the page, so they're // `/favicon.ico` is a guess. When it doesn't load, the placeholder shows
// collected whether or not there's artwork. `/favicon.ico` is a guess; when // the site's initial.
// it 404s the placeholder falls back to the site's initial.
const faviconUrl = extractBestIcon(html, url) ?? const faviconUrl = extractBestIcon(html, url) ??
`${new URL(url).origin}/favicon.ico`; `${new URL(url).origin}/favicon.ico`;

View File

@@ -1,20 +1,25 @@
/** /**
* The interface every link provider implements. * Declares what a link provider implements. `rich-content-service.ts` asks its
* providers in turn and fetches a link's preview from the first that matches.
* *
* @module * @module
*/ */
import type { RichContent } from "../../../shared/dump.ts"; import type { RichContent } from "../../../shared/dump.ts";
/** Metadata for the links one kind of site serves. */ /**
* Implemented by each file of `remote/providers/`, and listed in priority order
* by `rich-content-service.ts`.
*/
export interface RichContentProvider { export interface RichContentProvider {
/** The provider's name, for logs. */
name: string; name: string;
/** Checks whether this provider handles `url`. */ /** Called on a URL that `isValidHttpUrl` accepted; never throws. */
matches(url: string): boolean; matches(url: string): boolean;
/** /**
* Fetches `url`'s preview. Rejects when the page can't be fetched; a page * Returns the preview of `url`, which ends up in the dump's `rich_content` or
* that isn't HTML yields a preview with no metadata. * the create form's preview. It rejects when the page can't be fetched, and
* the service then falls back to a stub. A page that isn't HTML yields a
* preview without metadata.
*/ */
fetch(url: string): Promise<RichContent>; fetch(url: string): Promise<RichContent>;
} }

View File

@@ -1,6 +1,7 @@
/** /**
* The link provider for this site's own pages, read from the database * Previews links to this site's own dump, user and playlist pages from the
* instead of fetched. * database, for `rich-content-service.ts`. No request goes out, so this works
* even when the site runs on a private host that `fetchWithTimeout` refuses.
* *
* @module * @module
*/ */
@@ -10,6 +11,7 @@ import type { RichContentProvider } from "./provider.ts";
import { getDump } from "../../services/dump-service.ts"; import { getDump } from "../../services/dump-service.ts";
import { getUserByUsername } from "../../services/user-service.ts"; import { getUserByUsername } from "../../services/user-service.ts";
import { getPlaylistById } from "../../services/playlist-service.ts"; import { getPlaylistById } from "../../services/playlist-service.ts";
import { APIException } from "../../lib/api-exception.ts";
import { PUBLIC_URL, THEME_COLOR } from "../../config.ts"; import { PUBLIC_URL, THEME_COLOR } from "../../config.ts";
import { isSelfUrl } from "../../lib/self-url.ts"; import { isSelfUrl } from "../../lib/self-url.ts";
@@ -18,8 +20,8 @@ const USER_RE = /^\/users\/([^/]+)$/;
const PLAYLIST_RE = /^\/playlists\/([^/]+)$/; const PLAYLIST_RE = /^\/playlists\/([^/]+)$/;
/** /**
* Internal links carry gerbeur's own branding, so an art-less one still gets a * Gives a link to this site without a thumbnail a placeholder in the site's
* recognizable placeholder instead of a generic tint. * icon and color.
*/ */
const SELF_BRANDING = { const SELF_BRANDING = {
siteName: "gerbeur", siteName: "gerbeur",
@@ -28,9 +30,10 @@ const SELF_BRANDING = {
} as const; } as const;
/** /**
* The provider for links to this site's dump, user and playlist pages, read * Comes before the generic provider, and claims what `isSelfUrl` accepts. Its
* from the database. Its `fetch` rejects with the services' `APIException` (404) for a * `fetch` rejects with a 404 `APIException` for a missing page, a private dump
* page that doesn't exist or is private. * or a private playlist: the preview is stored in the linking dump, which
* others may see.
*/ */
export const selfProvider: RichContentProvider = { export const selfProvider: RichContentProvider = {
name: "self", name: "self",
@@ -84,6 +87,9 @@ export const selfProvider: RichContentProvider = {
const playlistMatch = pathname.match(PLAYLIST_RE); const playlistMatch = pathname.match(PLAYLIST_RE);
if (playlistMatch) { if (playlistMatch) {
const playlist = getPlaylistById(playlistMatch[1]); const playlist = getPlaylistById(playlistMatch[1]);
if (!playlist.isPublic) {
throw new APIException(404, "Playlist not found");
}
const thumbnailUrl = playlist.imageMime const thumbnailUrl = playlist.imageMime
? `${PUBLIC_URL}/api/playlists/${playlist.id}/image` ? `${PUBLIC_URL}/api/playlists/${playlist.id}/image`
: undefined; : undefined;
@@ -97,7 +103,7 @@ export const selfProvider: RichContentProvider = {
}); });
} }
// Should not reach here if matches() is correct // Unreachable: isSelfUrl accepts only the three paths above.
return Promise.resolve({ type: "generic", url }); return Promise.resolve({ type: "generic", url });
}, },
}; };

View File

@@ -1,5 +1,6 @@
/** /**
* The SoundCloud link provider: previews from a track page's OG tags. * Previews SoundCloud links from their page's OG tags, for
* `rich-content-service.ts`.
* *
* @module * @module
*/ */
@@ -9,7 +10,10 @@ import type { RichContentProvider } from "./provider.ts";
import { extractOgTag } from "../html.ts"; import { extractOgTag } from "../html.ts";
import { fetchWithTimeout, readTextCapped } from "../fetch.ts"; import { fetchWithTimeout, readTextCapped } from "../fetch.ts";
/** The provider for soundcloud.com pages, with SoundCloud's embed player. */ /**
* Claims soundcloud.com only, not www. or m.soundcloud.com, and builds the
* embed player's URL from the link itself.
*/
export const soundcloudProvider: RichContentProvider = { export const soundcloudProvider: RichContentProvider = {
name: "soundcloud", name: "soundcloud",

View File

@@ -1,5 +1,7 @@
/** /**
* The YouTube link provider: videos, playlists and channels. * Previews YouTube videos, playlists and channels, for
* `rich-content-service.ts`. Videos and playlists are read from YouTube's
* oEmbed endpoint, channels from their page's OG tags.
* *
* @module * @module
*/ */
@@ -8,43 +10,32 @@ import type { RichContent } from "../../../shared/dump.ts";
import type { RichContentProvider } from "./provider.ts"; import type { RichContentProvider } from "./provider.ts";
import { extractOgTag } from "../html.ts"; import { extractOgTag } from "../html.ts";
import { fetchWithTimeout, readTextCapped } from "../fetch.ts"; import { fetchWithTimeout, readTextCapped } from "../fetch.ts";
import { youtubeHost, youtubeVideoId } from "../../lib/canonical-url.ts";
/** Returns the id of the video a youtu.be or youtube.com URL points at. */ /** Returns the id of the video `url` points at, in any shape YouTube uses. */
function extractVideoId(url: string): string | null { function extractVideoId(url: string): string | null {
try { try {
const u = new URL(url); const u = new URL(url);
if (u.hostname === "youtu.be") { const host = u.hostname.toLowerCase().replace(/^www\./, "");
return u.pathname.slice(1).split("/")[0] || null; return youtubeVideoId(host, u.pathname, u.searchParams);
}
if (u.hostname === "youtube.com" || u.hostname === "www.youtube.com") {
if (u.pathname === "/watch" || u.pathname.startsWith("/watch?")) {
return u.searchParams.get("v");
}
if (
u.pathname.startsWith("/embed/") || u.pathname.startsWith("/shorts/")
) {
return u.pathname.split("/")[2] || null;
}
}
} catch { } catch {
// invalid URL
}
return null; return null;
}
} }
/** Returns the `list` parameter of a URL: the playlist it plays. */ /** Returns the `list` parameter of a YouTube URL, and null for any other. */
function extractPlaylistId(url: string): string | null { function extractPlaylistId(url: string): string | null {
try { try {
return new URL(url).searchParams.get("list"); const u = new URL(url);
return youtubeHost(u) ? u.searchParams.get("list") : null;
} catch { } catch {
return null; return null;
} }
} }
/** /**
* Returns the playback offset in seconds of a YouTube URL. * Returns the start time of a YouTube URL in seconds, from `t` (`90`, `90s`,
* Accepts `t` (plain seconds, `90s`, or `1h2m3s` / `20m5s` forms) and `start` * `1h2m3s`) or `start`; null when there is none that parses.
* (plain seconds). Returns null when absent or unparseable.
*/ */
function extractStartSeconds(url: string): number | null { function extractStartSeconds(url: string): number | null {
let raw: string | null; let raw: string | null;
@@ -66,22 +57,15 @@ function extractStartSeconds(url: string): number | null {
} }
/** /**
* Returns the path of a channel URL: /channel/UC…, /@handle, /c/name or * Returns the path of a YouTube channel URL (`/channel/UC…`, `/@handle`,
* /user/name. * `/c/name` or `/user/name`); null for any other URL.
*/ */
function extractChannelPath(url: string): string | null { function extractChannelPath(url: string): string | null {
try { try {
const u = new URL(url); const u = new URL(url);
if (u.hostname === "youtube.com" || u.hostname === "www.youtube.com") { if (youtubeHost(u) && /^\/(channel\/|@|c\/|user\/)/.test(u.pathname)) {
if (
u.pathname.startsWith("/channel/") ||
u.pathname.startsWith("/@") ||
u.pathname.startsWith("/c/") ||
u.pathname.startsWith("/user/")
) {
return u.pathname; return u.pathname;
} }
}
} catch { } catch {
// invalid URL // invalid URL
} }
@@ -90,7 +74,7 @@ function extractChannelPath(url: string): string | null {
/** /**
* Fetches the title and thumbnail of `url` from YouTube's oEmbed endpoint; * Fetches the title and thumbnail of `url` from YouTube's oEmbed endpoint;
* empty when that fails. * empty when that fails, since the preview can do without them.
*/ */
async function fetchOEmbed( async function fetchOEmbed(
url: string, url: string,
@@ -109,14 +93,16 @@ async function fetchOEmbed(
} }
await res.body?.cancel(); await res.body?.cancel();
} catch { } catch {
// oembed failed — carry on // YouTube unreachable, or its answer not JSON.
} }
return {}; return {};
} }
/** /**
* The provider for YouTube videos (with their playlist and start time in the embed), * Comes first in the service's list. A video's embed keeps the link's playlist
* playlists (as a `videoseries` embed) and channels (not embeddable). * and start time, a playlist embeds as a `videoseries`, and a channel gets no
* embed. Its `fetch` never rejects: whatever YouTube doesn't answer is left
* out.
*/ */
export const youtubeProvider: RichContentProvider = { export const youtubeProvider: RichContentProvider = {
name: "youtube", name: "youtube",
@@ -134,9 +120,8 @@ export const youtubeProvider: RichContentProvider = {
const listId = extractPlaylistId(url); const listId = extractPlaylistId(url);
const channelPath = extractChannelPath(url); const channelPath = extractChannelPath(url);
// ── Channel ───────────────────────────────────────────────────────────────
if (channelPath && !videoId) { if (channelPath && !videoId) {
// oEmbed doesn't support channel URLs — scrape og:image from the page instead // oEmbed doesn't take channel URLs.
let title: string | undefined; let title: string | undefined;
let thumbnailUrl: string | undefined; let thumbnailUrl: string | undefined;
try { try {
@@ -149,7 +134,7 @@ export const youtubeProvider: RichContentProvider = {
await res.body?.cancel(); await res.body?.cancel();
} }
} catch { } catch {
// scrape failed — carry on with undefined values // YouTube unreachable: the preview goes without.
} }
return { return {
type: "youtube", type: "youtube",
@@ -157,11 +142,9 @@ export const youtubeProvider: RichContentProvider = {
url, url,
title, title,
thumbnailUrl, thumbnailUrl,
// channels are not embeddable as a player
}; };
} }
// ── Playlist (no specific video) ──────────────────────────────────────────
if (listId && !videoId) { if (listId && !videoId) {
const { title, thumbnailUrl } = await fetchOEmbed(url); const { title, thumbnailUrl } = await fetchOEmbed(url);
return { return {
@@ -175,7 +158,7 @@ export const youtubeProvider: RichContentProvider = {
}; };
} }
// ── Video (with optional playlist context) ──────────────────────────────── // Built, not taken from oEmbed, so it is there even when oEmbed fails.
const thumbnailUrl = `https://img.youtube.com/vi/${videoId}/hqdefault.jpg`; const thumbnailUrl = `https://img.youtube.com/vi/${videoId}/hqdefault.jpg`;
const { title } = await fetchOEmbed(url); const { title } = await fetchOEmbed(url);

View File

@@ -1,8 +1,8 @@
/** /**
* Guards for server-side fetches of user-supplied URLs (link previews, the * Tells private addresses from public ones, so that a user-supplied URL can't
* image proxy, Bandcamp resolution): the server must not become a way to reach * make the server fetch what only it can reach: loopback, the LAN, container
* addresses only it can see — loopback, the LAN, container networks, or the * networks or the cloud metadata endpoint at 169.254.169.254. `fetch.ts` checks
* cloud metadata endpoint at 169.254.169.254. * every hop with it.
* *
* @module * @module
*/ */
@@ -83,8 +83,9 @@ function isPrivateIPv6(b: number[]): boolean {
} }
/** /**
* Checks whether `ip` is an IP literal outside the public internet. Non-IPs * Checks whether `ip` is an IPv4 or IPv6 address outside the public internet.
* are false. * An IPv4 address carried in IPv6 is judged as IPv4. Anything that isn't an IP
* address is false, so a hostname must be resolved first.
*/ */
export function isPrivateAddress(ip: string): boolean { export function isPrivateAddress(ip: string): boolean {
const v4 = parseIPv4(ip); const v4 = parseIPv4(ip);
@@ -106,9 +107,9 @@ function bareHostname(url: URL): string {
} }
/** /**
* Checks whether `url`'s host is private without DNS: a private IP literal, * Checks whether the host of `url` is private without a DNS lookup: a private
* or a name that is local by definition. Names still need resolving (see * IP address, or `localhost` and its subdomains. `isValidHttpUrl` uses it to
* resolvePublicAddress) — this only rejects what is obvious without DNS. * reject a link early; only {@link resolvePublicAddress} is a real guard.
*/ */
export function isPrivateHostname(url: URL): boolean { export function isPrivateHostname(url: URL): boolean {
const host = bareHostname(url).toLowerCase(); const host = bareHostname(url).toLowerCase();
@@ -117,13 +118,13 @@ export function isPrivateHostname(url: URL): boolean {
} }
/** /**
* Resolves `url`'s host and returns one of its addresses, or throws if the * Resolves the host of `url` and returns one of its addresses, all of which
* host is private or any address it resolves to is (a name with both a public * must be public: the fetch may pick any of them. `fetchWithTimeout` calls it
* and a private record is refused outright, since the fetch may pick either). * before each hop.
* *
* The fetch itself resolves again, so a DNS server answering differently the * Only the curl fallback connects to the address returned. Deno's fetch
* second time (rebinding) isn't fully excluded; the curl fallback pins the * resolves the name again, so a DNS server that answers differently the second
* address this returns. * time (rebinding) can still get through.
* *
* @throws {Error} When the host is private, can't be resolved, or resolves to * @throws {Error} When the host is private, can't be resolved, or resolves to
* a private address. * a private address.

View File

@@ -1,18 +1,23 @@
/** /**
* Twemoji glyphs, fetched from the jsDelivr CDN. * Fetches Twemoji glyphs from the jsDelivr CDN, for `lib/site-icons.ts` to
* render the favicons from.
* *
* @module * @module
*/ */
/** The pinned Twemoji release. */ /** Pinned, so a new Twemoji release can't change the icons unasked. */
const TWEMOJI_VERSION = "16.0.1"; const TWEMOJI_VERSION = "16.0.1";
/** Fetched glyphs by codepoint, so repeated runs don't re-hit the network. */ /**
* Glyphs by codepoint. It only lives as long as the process, and the `icons`
* task renders one emoji per process.
*/
const glyphCache = new Map<string, string>(); const glyphCache = new Map<string, string>();
/** /**
* Fetches Twemoji's plain vector-path SVG for `emoji`, whose file is named * Returns Twemoji's SVG for `emoji`, whose file name `cp` comes from
* `cp` (see `emojiToCodepoint` in lib/site-icons.ts). * `emojiToCodepoint`. `emoji` only serves the error message. It fetches with
* plain `fetch`, since the host is fixed, and gives up after 15 s.
* *
* @throws {Error} When the glyph can't be fetched. * @throws {Error} When the glyph can't be fetched.
*/ */
@@ -26,7 +31,9 @@ export async function fetchTwemojiSvg(
`https://cdn.jsdelivr.net/gh/jdecked/twemoji@${TWEMOJI_VERSION}/assets/svg/${cp}.svg`; `https://cdn.jsdelivr.net/gh/jdecked/twemoji@${TWEMOJI_VERSION}/assets/svg/${cp}.svg`;
let res: Response; let res: Response;
try { try {
res = await fetch(url); // The icons task runs before the server starts, so a stalled CDN
// mustn't hold the start up forever.
res = await fetch(url, { signal: AbortSignal.timeout(15_000) });
} catch (cause) { } catch (cause) {
throw new Error( throw new Error(
`Failed to fetch Twemoji glyph for "${emoji}" (${cp}) from ${url}`, `Failed to fetch Twemoji glyph for "${emoji}" (${cp}) from ${url}`,

View File

@@ -1,22 +1,17 @@
/** /**
* Resolves a YouTube playlist to its tracklist, at play time. * Resolves a YouTube playlist to its entries' titles and lengths, for
* `GET /api/youtube/playlist`. The embed plays the list but doesn't expose
* them, and the global player needs them to draw a tracklist.
* *
* Playback itself stays in the iframe embed — YouTube plays the list and * They are read from the `ytInitialData` JSON of the public playlist page,
* advances through it on its own. This only supplies what the embed doesn't * whose undocumented shape changes, so both the current `lockupViewModel` rows
* expose: every entry's title and length, so the global player can draw a * and the older `playlistVideoRenderer` ones are accepted. Only the first 100
* tracklist the way it does for a Bandcamp album. The client syncs the * entries are there; the client adds the rest, untitled, as the player reports
* highlighted row and row clicks through the IFrame Player API. * them.
*
* The data comes from the `ytInitialData` blob on the public playlist page. Its
* shape is undocumented and changes, so the parse is best-effort and accepts
* both the current `lockupViewModel` rows and the older `playlistVideoRenderer`
* ones. Only the first page (100 entries) is read; the client appends any
* further IDs the player reports, untitled.
* *
* @module * @module
*/ */
import { APIErrorCode } from "../../shared/api.ts";
import { APIException } from "../lib/api-exception.ts"; import { APIException } from "../lib/api-exception.ts";
import { readTextCapped } from "./fetch.ts"; import { readTextCapped } from "./fetch.ts";
import type { import type {
@@ -24,28 +19,25 @@ import type {
YouTubePlaylistTrack, YouTubePlaylistTrack,
} from "../../shared/player.ts"; } from "../../shared/player.ts";
/** /** Titles and lengths rarely change, and a stale one only mislabels a row. */
* How long a resolved playlist is cached. Titles and lengths rarely change; a
* stale entry only mislabels a row.
*/
const TTL_MS = 6 * 60 * 60 * 1000; const TTL_MS = 6 * 60 * 60 * 1000;
/** The most playlists cached; the oldest write goes first. */ /** The oldest write goes first. */
const MAX_ENTRIES = 200; const MAX_ENTRIES = 200;
/** Resolved playlists, by list id. */
const cache = new Map<string, { value: YouTubePlaylist; at: number }>(); const cache = new Map<string, { value: YouTubePlaylist; at: number }>();
/** A playlist id: URL-safe base64-ish, PL…, OLAK5uy_…, RD… (mixes). */ /**
* Admits every kind of list id (`PL…`, `OLAK5uy_…`, `RD…`) and nothing else.
*/
const LIST_ID_RE = /^[\w-]{2,64}$/; const LIST_ID_RE = /^[\w-]{2,64}$/;
/** The headers the playlist page is requested with. */
const HEADERS = { const HEADERS = {
"User-Agent": "User-Agent":
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/140.0.0.0 Safari/537.36", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/140.0.0.0 Safari/537.36",
"Accept": "text/html", "Accept": "text/html",
"Accept-Language": "en-US,en;q=0.8", "Accept-Language": "en-US,en;q=0.8",
// Pre-answered consent: without it EU requests are redirected to // Declines the consent prompt, which EU requests are otherwise redirected
// consent.youtube.com and never see the playlist. // to instead of the playlist.
"Cookie": "SOCS=CAI", "Cookie": "SOCS=CAI",
}; };
@@ -55,7 +47,7 @@ function parseClock(text: string): number | undefined {
return text.split(":").reduce((acc, part) => acc * 60 + Number(part), 0); return text.split(":").reduce((acc, part) => acc * 60 + Number(part), 0);
} }
/** Collects, depth-first, every value stored under `key` into `out`. */ /** Collects, depth first, every value stored under `key` into `out`. */
function collect(node: unknown, key: string, out: unknown[]): void { function collect(node: unknown, key: string, out: unknown[]): void {
if (Array.isArray(node)) { if (Array.isArray(node)) {
for (const child of node) collect(child, key, out); for (const child of node) collect(child, key, out);
@@ -74,10 +66,7 @@ function firstString(node: unknown, key: string): string | undefined {
return found.find((v): v is string => typeof v === "string"); return found.find((v): v is string => typeof v === "string");
} }
/** /** Returns the value at `path` in `node`; undefined when a step is missing. */
* Returns the value at `path` in parsed JSON; undefined wherever a step is
* missing.
*/
function at(node: unknown, ...path: (string | number)[]): unknown { function at(node: unknown, ...path: (string | number)[]): unknown {
let cur = node; let cur = node;
for (const key of path) { for (const key of path) {
@@ -124,8 +113,8 @@ function fromLegacyRenderer(r: unknown): YouTubePlaylistTrack | null {
} }
/** /**
* Reads the tracklist from a playlist page's `ytInitialData`; null when the * Returns the playlist `listId` as read from its page `html`; null when the
* page has none or no track in it. * page has no `ytInitialData` or no track in it.
*/ */
export function parsePlaylistPage( export function parsePlaylistPage(
html: string, html: string,
@@ -148,8 +137,7 @@ export function parsePlaylistPage(
const legacy: unknown[] = []; const legacy: unknown[] = [];
collect(at(data, "contents"), "playlistVideoRenderer", legacy); collect(at(data, "contents"), "playlistVideoRenderer", legacy);
// A video can appear in a playlist twice; both rows are kept, so the list // A video listed twice keeps both rows, so that row n is the embed's entry n.
// stays positional and matches what the embed plays.
const tracks = [ const tracks = [
...lockups.map(fromLockup), ...lockups.map(fromLockup),
...legacy.map(fromLegacyRenderer), ...legacy.map(fromLegacyRenderer),
@@ -164,7 +152,8 @@ export function parsePlaylistPage(
} }
/** /**
* Fetches the playlist `listId` and returns its tracklist, cached. * Returns the tracklist of the playlist `listId`, from the cache when it was
* resolved in the last 6 hours.
* *
* @throws {APIException} 400 when `listId` isn't a playlist id, 502 when * @throws {APIException} 400 when `listId` isn't a playlist id, 502 when
* YouTube can't be reached or answers with an error, 404 when the playlist has * YouTube can't be reached or answers with an error, 404 when the playlist has
@@ -174,11 +163,7 @@ export async function resolveYouTubePlaylist(
listId: string, listId: string,
): Promise<YouTubePlaylist> { ): Promise<YouTubePlaylist> {
if (!LIST_ID_RE.test(listId)) { if (!LIST_ID_RE.test(listId)) {
throw new APIException( throw new APIException(400, "Invalid playlist ID");
APIErrorCode.VALIDATION_ERROR,
400,
"Invalid playlist ID",
);
} }
const hit = cache.get(listId); const hit = cache.get(listId);
@@ -189,39 +174,28 @@ export async function resolveYouTubePlaylist(
const timer = setTimeout(() => controller.abort(), 8000); const timer = setTimeout(() => controller.abort(), 8000);
let html: string; let html: string;
try { try {
// youtube.com needs no address check, and fetchWithTimeout can't send the
// consent cookie.
const res = await fetch( const res = await fetch(
`https://www.youtube.com/playlist?list=${encodeURIComponent(listId)}`, `https://www.youtube.com/playlist?list=${encodeURIComponent(listId)}`,
{ headers: HEADERS, signal: controller.signal }, { headers: HEADERS, signal: controller.signal },
); );
if (!res.ok) { if (!res.ok) {
await res.body?.cancel(); await res.body?.cancel();
throw new APIException( throw new APIException(502, `YouTube returned ${res.status}`);
APIErrorCode.SERVER_ERROR,
502,
`YouTube returned ${res.status}`,
);
} }
html = await readTextCapped(res); html = await readTextCapped(res);
} catch (err) { } catch (err) {
if (err instanceof APIException) throw err; if (err instanceof APIException) throw err;
throw new APIException( throw new APIException(502, "Could not reach YouTube");
APIErrorCode.SERVER_ERROR,
502,
"Could not reach YouTube",
);
} finally { } finally {
clearTimeout(timer); clearTimeout(timer);
} }
const parsed = parsePlaylistPage(html, listId); const parsed = parsePlaylistPage(html, listId);
if (!parsed) { if (!parsed) {
// Private, deleted, or auto-generated mixes (RD…) that have no playlist // The client then shows the embed without a tracklist.
// page — the client just shows the embed without a tracklist. throw new APIException(404, "No tracks on this playlist");
throw new APIException(
APIErrorCode.NOT_FOUND,
404,
"No tracks on this playlist",
);
} }
if (cache.size >= MAX_ENTRIES) { if (cache.size >= MAX_ENTRIES) {

View File

@@ -1,10 +1,12 @@
/** /**
* The attachment routes, for images pasted into Markdown bodies. * Stores and serves the images pasted or dropped into the text editor, which
* puts each one in the body as a link to its `GET` URL.
* *
* - `POST /api/attachments` — signed in. Stores an image (multipart `file`, at * - `POST /api/attachments` — signed in. Stores an image (multipart `file`, at
* most 5 MB) and answers its id; it is bound to a resource once a saved body * most 5 MB) and answers its id.
* links it. * - `GET /api/attachments/:attachmentId?token=` — public, but an image in a
* - `GET /api/attachments/:attachmentId` — public. Serves the image. * private dump or playlist, or in a comment on a private dump, needs its
* owner's media `token`. Serves the image.
* *
* @module * @module
*/ */
@@ -13,35 +15,32 @@ import { Router } from "@oak/oak";
import { authMiddleware } from "../middleware/auth.ts"; import { authMiddleware } from "../middleware/auth.ts";
import { import {
createAttachment, createAttachment,
getAttachment, getAttachmentAccess,
} from "../services/attachment-service.ts"; } from "../services/attachment-service.ts";
import { APIErrorCode, type APIResponse } from "../../shared/api.ts"; import { authenticateMedia } from "../lib/auth.ts";
import { type APIResponse } from "../../shared/api.ts";
import { APIException } from "../lib/api-exception.ts"; import { APIException } from "../lib/api-exception.ts";
import type { AttachmentResponse } from "../../shared/dump.ts"; import type { AttachmentResponse } from "../../shared/dump.ts";
import { import { serveUploadedFile, validateImageUpload } from "../lib/upload.ts";
ATTACHMENTS_DIR, import { ATTACHMENTS_DIR } from "../config.ts";
serveUploadedFile,
validateImageUpload,
} from "../lib/upload.ts";
/** The router serving the endpoints the module doc lists. */ /**
* Mounted by `api/main.ts` behind the error, Open Graph and CORS middleware.
* An `APIException` thrown by a handler becomes the JSON error response there.
*/
const router: Router = new Router(); const router: Router = new Router();
router.post("/api/attachments", authMiddleware, async (ctx) => { router.post("/api/attachments", authMiddleware, async (ctx) => {
const contentType = ctx.request.headers.get("content-type") ?? ""; const contentType = ctx.request.headers.get("content-type") ?? "";
if (!contentType.includes("multipart/form-data")) { if (!contentType.includes("multipart/form-data")) {
throw new APIException( throw new APIException(400, "Expected multipart/form-data");
APIErrorCode.BAD_REQUEST,
400,
"Expected multipart/form-data",
);
} }
const body = await ctx.request.body.formData(); const body = await ctx.request.body.formData();
const file = body.get("file"); const file = body.get("file");
if (!(file instanceof File)) { if (!(file instanceof File)) {
throw new APIException(APIErrorCode.BAD_REQUEST, 400, "Missing file"); throw new APIException(400, "Missing file");
} }
const data = new Uint8Array(await file.arrayBuffer()); const data = new Uint8Array(await file.arrayBuffer());
@@ -68,18 +67,30 @@ router.post("/api/attachments", authMiddleware, async (ctx) => {
router.get("/api/attachments/:attachmentId", async (ctx) => { router.get("/api/attachments/:attachmentId", async (ctx) => {
const { attachmentId } = ctx.params; const { attachmentId } = ctx.params;
let attachment; let access;
try { try {
attachment = getAttachment(attachmentId); access = getAttachmentAccess(attachmentId);
} catch { } catch {
ctx.response.status = 404; ctx.response.status = 404;
return; return;
} }
if (access.ownerId) {
// An <img> can't send an Authorization header, so the viewer's media
// token comes in the query string.
const token = ctx.request.url.searchParams.get("token");
const viewerId = token ? await authenticateMedia(token) : null;
if (viewerId !== access.ownerId) {
ctx.response.status = 404;
return;
}
}
await serveUploadedFile( await serveUploadedFile(
ctx, ctx,
`${ATTACHMENTS_DIR}/${attachmentId}`, `${ATTACHMENTS_DIR}/${attachmentId}`,
attachment.mime, access.mime,
3600,
!!access.ownerId,
); );
}); });

View File

@@ -1,8 +1,11 @@
/** /**
* The avatar routes. * Serves the avatar endpoints. The profile page uploads the avatar, and the
* client's `Avatar` component loads it, with the user's `updatedAt` as a
* cache-busting version.
* *
* - `POST /api/avatars/me` — signed in. Replaces the caller's avatar (multipart * - `POST /api/avatars/me` — signed in. Replaces the caller's avatar (multipart
* `file`, at most 5 MB), announces it, and answers the caller's user. * `file`, at most 5 MB), sends `user_updated` to everyone, and answers the
* caller's user.
* - `GET /api/avatars/:userId` — public. Serves a user's avatar; 404 without * - `GET /api/avatars/:userId` — public. Serves a user's avatar; 404 without
* one. * one.
* *
@@ -16,56 +19,49 @@ import {
broadcastUserUpdated, broadcastUserUpdated,
updateClientAvatar, updateClientAvatar,
} from "../services/ws-service.ts"; } from "../services/ws-service.ts";
import { APIErrorCode, type APIResponse } from "../../shared/api.ts"; import { type APIResponse } from "../../shared/api.ts";
import { APIException } from "../lib/api-exception.ts"; import { APIException } from "../lib/api-exception.ts";
import type { SelfUser } from "../../shared/user.ts"; import type { SelfUser } from "../../shared/user.ts";
import { toPublicUser, toSelfUser } from "../model/user.ts"; import { toPublicUser, toSelfUser } from "../model/user.ts";
import { import {
AVATARS_DIR, replaceUploadedFile,
serveUploadedFile, serveUploadedFile,
validateImageUpload, validateImageUpload,
} from "../lib/upload.ts"; } from "../lib/upload.ts";
import { AVATARS_DIR } from "../config.ts";
/** The router serving the endpoints the module doc lists. */ /**
* Mounted by `api/main.ts` behind the error, Open Graph and CORS middleware.
* An `APIException` thrown by a handler becomes the JSON error response there.
*/
const router: Router = new Router(); const router: Router = new Router();
router.post("/api/avatars/me", authMiddleware, async (ctx) => { router.post("/api/avatars/me", authMiddleware, async (ctx) => {
const authPayload = ctx.state.user; const authPayload = ctx.state.user;
const contentType = ctx.request.headers.get("content-type") ?? ""; const contentType = ctx.request.headers.get("content-type") ?? "";
if (!contentType.includes("multipart/form-data")) { if (!contentType.includes("multipart/form-data")) {
throw new APIException( throw new APIException(400, "Expected multipart/form-data");
APIErrorCode.BAD_REQUEST,
400,
"Expected multipart/form-data",
);
} }
const body = await ctx.request.body.formData(); const body = await ctx.request.body.formData();
const file = body.get("file"); const file = body.get("file");
if (!(file instanceof File)) { if (!(file instanceof File)) {
throw new APIException(APIErrorCode.BAD_REQUEST, 400, "Missing file field"); throw new APIException(400, "Missing file field");
} }
const data = new Uint8Array(await file.arrayBuffer()); const data = new Uint8Array(await file.arrayBuffer());
const mime = validateImageUpload(data); const mime = validateImageUpload(data);
const filePath = `${AVATARS_DIR}/${authPayload.userId}`; // Written before the record announces it, or clients would fetch it too
await Deno.mkdir(AVATARS_DIR, { recursive: true }); // early.
await Deno.writeFile(filePath, data); await replaceUploadedFile(`${AVATARS_DIR}/${authPayload.userId}`, data);
try {
updateUserAvatar(authPayload.userId, mime); updateUserAvatar(authPayload.userId, mime);
} catch (err) {
// DB write failed — clean up the orphaned file
await Deno.remove(filePath).catch(() => {});
throw err;
}
updateClientAvatar(authPayload.userId, mime); updateClientAvatar(authPayload.userId, mime);
const user = getUserById(authPayload.userId); const user = getUserById(authPayload.userId);
// Let every client refresh this user's avatar wherever it's shown (feeds, // Presence alone would update only the online list; `user_updated` reaches
// comments, OP, profiles) — not just the presence row. updated_at, bumped on // every avatar on screen, and its bumped `updatedAt` busts their cache.
// the avatar write, doubles as the cache-busting version.
broadcastUserUpdated(toPublicUser(user)); broadcastUserUpdated(toPublicUser(user));
const responseBody: APIResponse<SelfUser> = { const responseBody: APIResponse<SelfUser> = {
success: true, success: true,

View File

@@ -1,8 +1,9 @@
/** /**
* The Bandcamp route, for the native player. * Serves the Bandcamp release the native player plays, resolved from its page
* when playback starts. On any failure the player falls back to the embed.
* *
* - `GET /api/bandcamp/tracks?url=&force=` — public. Resolves a Bandcamp page * - `GET /api/bandcamp/tracks?url=&force=` — public. Answers the page's
* to its streamable tracks, at play time; `force=1` bypasses the cache. * tracks and their stream URLs; `force=1` skips the server's cache.
* *
* @module * @module
*/ */
@@ -12,13 +13,14 @@ import type { APIResponse } from "../../shared/api.ts";
import { resolveBandcamp } from "../remote/bandcamp-streams.ts"; import { resolveBandcamp } from "../remote/bandcamp-streams.ts";
import type { Tralbum } from "../../shared/player.ts"; import type { Tralbum } from "../../shared/player.ts";
/** The router serving the endpoints the module doc lists. */ /**
* Mounted by `api/main.ts` behind the error, Open Graph and CORS middleware.
* An `APIException` thrown by a handler becomes the JSON error response there.
*/
const bandcampRouter: Router = new Router(); const bandcampRouter: Router = new Router();
// Public: it only reads a public page and returns URLs the same page already // Public: it reads a public page and answers the URLs that page gives any
// hands any visitor. Errors are thrown as APIException and rendered by // visitor.
// errorMiddleware; the client treats any failure as "fall back to the iframe
// embed".
bandcampRouter.get("/api/bandcamp/tracks", async (ctx) => { bandcampRouter.get("/api/bandcamp/tracks", async (ctx) => {
const url = ctx.request.url.searchParams.get("url") ?? ""; const url = ctx.request.url.searchParams.get("url") ?? "";
const force = ctx.request.url.searchParams.get("force") === "1"; const force = ctx.request.url.searchParams.get("force") === "1";
@@ -28,8 +30,8 @@ bandcampRouter.get("/api/bandcamp/tracks", async (ctx) => {
data: await resolveBandcamp(url, { force }), data: await resolveBandcamp(url, { force }),
}; };
// Private: the signed URLs in the body are per-fetch, so they must not be // Not in a shared cache: the signed URLs belong to one page fetch. An hour
// held in a shared cache. Well inside the 24h signature life either way. // is well inside their 24 hours.
ctx.response.headers.set("Cache-Control", "private, max-age=3600"); ctx.response.headers.set("Cache-Control", "private, max-age=3600");
ctx.response.body = responseBody; ctx.response.body = responseBody;
}); });

View File

@@ -16,11 +16,7 @@
import { Router } from "@oak/oak"; import { Router } from "@oak/oak";
import { import { type APIResponse, type PaginatedData } from "../../shared/api.ts";
APIErrorCode,
type APIResponse,
type PaginatedData,
} from "../../shared/api.ts";
import { APIException } from "../lib/api-exception.ts"; import { APIException } from "../lib/api-exception.ts";
import type { Category } from "../../shared/category.ts"; import type { Category } from "../../shared/category.ts";
import type { Dump } from "../../shared/dump.ts"; import type { Dump } from "../../shared/dump.ts";
@@ -41,7 +37,10 @@ import {
updateCategory, updateCategory,
} from "../services/category-service.ts"; } from "../services/category-service.ts";
/** The router serving the endpoints the module doc lists. */ /**
* Mounted by `api/main.ts` behind the error, Open Graph and CORS middleware.
* An `APIException` thrown by a handler becomes the JSON error response there.
*/
const router: Router = new Router({ prefix: "/api/categories" }); const router: Router = new Router({ prefix: "/api/categories" });
router.get("/", (ctx) => { router.get("/", (ctx) => {
@@ -84,11 +83,7 @@ router.post(
async (ctx) => { async (ctx) => {
const body: unknown = await ctx.request.body.json(); const body: unknown = await ctx.request.body.json();
if (!isCreateCategoryRequest(body)) { if (!isCreateCategoryRequest(body)) {
throw new APIException( throw new APIException(400, "Invalid category data");
APIErrorCode.VALIDATION_ERROR,
400,
"Invalid category data",
);
} }
const category = createCategory(body); const category = createCategory(body);
const responseBody: APIResponse<Category> = { const responseBody: APIResponse<Category> = {
@@ -107,11 +102,7 @@ router.put(
async (ctx) => { async (ctx) => {
const body: unknown = await ctx.request.body.json(); const body: unknown = await ctx.request.body.json();
if (!isUpdateCategoryRequest(body)) { if (!isUpdateCategoryRequest(body)) {
throw new APIException( throw new APIException(400, "Invalid category data");
APIErrorCode.VALIDATION_ERROR,
400,
"Invalid category data",
);
} }
const category = updateCategory(ctx.params.id, body); const category = updateCategory(ctx.params.id, body);
const responseBody: APIResponse<Category> = { const responseBody: APIResponse<Category> = {

View File

@@ -1,13 +1,14 @@
/** /**
* The chat history route; messages are posted and edited over the socket (see * Serves the chat's history to the chatbox (`ChatModal`). Messages are sent,
* routes/ws.ts). * edited and deleted over the socket instead (`routes/ws.ts`).
* *
* - `GET /api/chat?limit=&before=&beforeId=&after=&afterId=` — signed in. * - `GET /api/chat?limit=&before=&beforeId=&after=&afterId=`, signed in.
* Answers messages in ascending order: the most recent by default; with * Returns messages oldest first, at most `limit` (50 by default, 100 at
* `before`, the page of older messages preceding that cursor ("load older"); * most). With no cursor, it returns the latest. With `before`, it returns
* with `after`, the messages following it (resync after a reconnect). A * the page just older than that cursor, for "load older". With `after`, it
* cursor is an ISO `created_at` plus the message id to break ties; without * returns what follows it, for the resync after a reconnect. A cursor is a
* `afterId` the `after` timestamp itself is included. * message's ISO `created_at`, and its id breaks ties. Without `afterId`,
* messages at the `after` timestamp itself are included.
* *
* @module * @module
*/ */
@@ -21,7 +22,10 @@ import {
getRecentChatMessages, getRecentChatMessages,
} from "../services/chat-service.ts"; } from "../services/chat-service.ts";
/** The router serving the endpoints the module doc lists. */ /**
* Mounted by `api/main.ts` behind the error, Open Graph and CORS middleware.
* An `APIException` thrown by a handler becomes the JSON error response there.
*/
const router: Router = new Router({ prefix: "/api/chat" }); const router: Router = new Router({ prefix: "/api/chat" });
router.get("/", authMiddleware, (ctx) => { router.get("/", authMiddleware, (ctx) => {

View File

@@ -1,6 +1,7 @@
/** /**
* The comment routes. A private dump's comments exist only for its poster, and * Serves the comments on dumps. The comments of a private dump 404 for anyone
* changes to them aren't broadcast. * but its poster, and changes to them aren't broadcast. Comments are otherwise
* broadcast over the WebSocket as they are posted, edited and deleted.
* *
* - `GET /api/dumps/:dumpId/comments` — optional sign-in. Answers every comment * - `GET /api/dumps/:dumpId/comments` — optional sign-in. Answers every comment
* on a dump, oldest first. * on a dump, oldest first.
@@ -17,11 +18,7 @@
*/ */
import { Router } from "@oak/oak"; import { Router } from "@oak/oak";
import { import { type APIResponse, type PaginatedData } from "../../shared/api.ts";
APIErrorCode,
type APIResponse,
type PaginatedData,
} from "../../shared/api.ts";
import { APIException } from "../lib/api-exception.ts"; import { APIException } from "../lib/api-exception.ts";
import type { Comment } from "../../shared/comment.ts"; import type { Comment } from "../../shared/comment.ts";
import { import {
@@ -49,7 +46,10 @@ import {
broadcastCommentUpdated, broadcastCommentUpdated,
} from "../services/ws-service.ts"; } from "../services/ws-service.ts";
/** The router serving the endpoints the module doc lists. */ /**
* Mounted by `api/main.ts` behind the error, Open Graph and CORS middleware.
* An `APIException` thrown by a handler becomes the JSON error response there.
*/
const router: Router = new Router({ prefix: "/api" }); const router: Router = new Router({ prefix: "/api" });
router.get("/dumps/:dumpId/comments", async (ctx) => { router.get("/dumps/:dumpId/comments", async (ctx) => {
@@ -68,11 +68,7 @@ router.post("/dumps/:dumpId/comments", authMiddleware, async (ctx) => {
const dump = getDump(ctx.params.dumpId, userId); const dump = getDump(ctx.params.dumpId, userId);
const body: unknown = await ctx.request.body.json(); const body: unknown = await ctx.request.body.json();
if (!isCreateCommentRequest(body)) { if (!isCreateCommentRequest(body)) {
throw new APIException( throw new APIException(400, "Invalid comment data");
APIErrorCode.VALIDATION_ERROR,
400,
"Invalid comment data",
);
} }
const comment = createComment( const comment = createComment(
dump.id, dump.id,
@@ -91,11 +87,7 @@ router.patch("/comments/:commentId", authMiddleware, async (ctx) => {
const canModerate = can(ctx.state.user, "comment:moderate"); const canModerate = can(ctx.state.user, "comment:moderate");
const body: unknown = await ctx.request.body.json(); const body: unknown = await ctx.request.body.json();
if (!isUpdateCommentRequest(body)) { if (!isUpdateCommentRequest(body)) {
throw new APIException( throw new APIException(400, "Invalid comment data");
APIErrorCode.VALIDATION_ERROR,
400,
"Invalid comment data",
);
} }
const { comment, isPrivate } = updateComment( const { comment, isPrivate } = updateComment(
ctx.params.commentId, ctx.params.commentId,
@@ -111,7 +103,7 @@ router.patch("/comments/:commentId", authMiddleware, async (ctx) => {
router.get("/comments/:commentId/likers", async (ctx) => { router.get("/comments/:commentId/likers", async (ctx) => {
const requestingUserId = await parseOptionalAuth(ctx) ?? undefined; const requestingUserId = await parseOptionalAuth(ctx) ?? undefined;
const dumpId = getCommentDumpId(ctx.params.commentId); const dumpId = getCommentDumpId(ctx.params.commentId);
getDump(dumpId, requestingUserId); // enforces privacy, 404s if not visible getDump(dumpId, requestingUserId); // 404s unless the caller may see the dump.
const { page, limit } = parsePagination(ctx.request.url.searchParams); const { page, limit } = parsePagination(ctx.request.url.searchParams);
const { items, hasMore } = getCommentLikers( const { items, hasMore } = getCommentLikers(
ctx.params.commentId, ctx.params.commentId,

View File

@@ -1,6 +1,7 @@
/** /**
* The dump routes. `:dumpId` takes an id or a slug; a private dump exists only * Posts, reads, edits and deletes dumps, for the index feed, the dump page, its
* for its poster. * edit page and the create form. `:dumpId` takes an id or a slug. A private
* dump answers 404 to anyone but its poster, moderators included.
* *
* - `POST /api/dumps` — signed in. Posts a link (JSON body) or a file * - `POST /api/dumps` — signed in. Posts a link (JSON body) or a file
* (multipart `file`, with `title`, `comment`, `isPrivate` and `categoryIds`). * (multipart `file`, with `title`, `comment`, `isPrivate` and `categoryIds`).
@@ -14,9 +15,10 @@
* - `GET /api/dumps/:dumpId/playlists` — optional sign-in. Answers the * - `GET /api/dumps/:dumpId/playlists` — optional sign-in. Answers the
* playlists it appears in. * playlists it appears in.
* - `GET /api/dumps?page=&limit=` — optional sign-in. Answers a page of the * - `GET /api/dumps?page=&limit=` — optional sign-in. Answers a page of the
* dumps, newest first. * dumps the caller may see, newest first.
* - `PUT /api/dumps/:dumpId/file` — signed in, as its poster or with * - `PUT /api/dumps/:dumpId/file` — signed in, as its poster or with
* `dump:moderate`. Replaces a file dump's upload (multipart). * `dump:moderate`. Replaces a file dump's upload and comment, and any of its
* title, categories and visibility (multipart).
* - `PUT /api/dumps/:dumpId` — signed in, as its poster or with * - `PUT /api/dumps/:dumpId` — signed in, as its poster or with
* `dump:moderate`. Updates its title, comment, visibility, categories, and a * `dump:moderate`. Updates its title, comment, visibility, categories, and a
* link's URL. * link's URL.
@@ -35,11 +37,7 @@
import { Router } from "@oak/oak"; import { Router } from "@oak/oak";
import { import { type APIResponse, type PaginatedData } from "../../shared/api.ts";
APIErrorCode,
type APIResponse,
type PaginatedData,
} from "../../shared/api.ts";
import { APIException } from "../lib/api-exception.ts"; import { APIException } from "../lib/api-exception.ts";
import type { Dump, DumpUrlMatch } from "../../shared/dump.ts"; import type { Dump, DumpUrlMatch } from "../../shared/dump.ts";
import { import {
@@ -54,7 +52,7 @@ import { authMiddleware } from "../middleware/auth.ts";
import { can } from "../lib/permissions.ts"; import { can } from "../lib/permissions.ts";
import { parseOptionalAuth } from "../lib/auth.ts"; import { parseOptionalAuth } from "../lib/auth.ts";
import { parsePagination } from "../lib/pagination.ts"; import { parsePagination } from "../lib/pagination.ts";
import { validateImageUpload } from "../lib/upload.ts"; import { replaceUploadedFile, validateImageUpload } from "../lib/upload.ts";
import { THUMBNAILS_DIR } from "../config.ts"; import { THUMBNAILS_DIR } from "../config.ts";
import { import {
createFileDump, createFileDump,
@@ -73,16 +71,18 @@ import { getDumpVoters } from "../services/vote-service.ts";
import { getRelatedDumps } from "../services/backlink-service.ts"; import { getRelatedDumps } from "../services/backlink-service.ts";
import { getPlaylistsForDump } from "../services/playlist-service.ts"; import { getPlaylistsForDump } from "../services/playlist-service.ts";
/** The router serving the endpoints the module doc lists. */ /**
* Mounted by `api/main.ts` behind the error, Open Graph and CORS middleware.
* An `APIException` thrown by a handler becomes the JSON error response there.
*/
const router: Router = new Router({ prefix: "/api/dumps" }); const router: Router = new Router({ prefix: "/api/dumps" });
// Every route here takes a slug or an id; the services below them key on the // Each handler resolves `:dumpId` with getDump, which checks privacy and takes
// id, so mutations always pass `dump.id` from the resolved dump, never the raw // a slug, and passes `dump.id` on: the services' writes match on the id alone.
// parameter.
/** /**
* Reads the category ids that arrive on the multipart create path as a * Parses a multipart form's `categoryIds`, a JSON array sent as a string. A
* JSON-encoded string field; empty when it is missing or malformed. * missing or malformed value counts as no category.
*/ */
function parseCategoryIds(value: FormDataEntryValue | null): string[] { function parseCategoryIds(value: FormDataEntryValue | null): string[] {
if (typeof value !== "string" || !value) return []; if (typeof value !== "string" || !value) return [];
@@ -113,11 +113,7 @@ router.post(
const isPrivate = formData.get("isPrivate") === "true"; const isPrivate = formData.get("isPrivate") === "true";
if (!(file instanceof File)) { if (!(file instanceof File)) {
throw new APIException( throw new APIException(400, "A file is required");
APIErrorCode.VALIDATION_ERROR,
400,
"A file is required",
);
} }
dump = await createFileDump( dump = await createFileDump(
@@ -132,11 +128,7 @@ router.post(
const body: unknown = await ctx.request.body.json(); const body: unknown = await ctx.request.body.json();
if (!isCreateUrlDumpRequest(body)) { if (!isCreateUrlDumpRequest(body)) {
throw new APIException( throw new APIException(400, "Invalid dump data");
APIErrorCode.VALIDATION_ERROR,
400,
"Invalid dump data",
);
} }
dump = await createUrlDump(body, userId); dump = await createUrlDump(body, userId);
@@ -191,7 +183,6 @@ router.get("/:dumpId/related", async (ctx) => {
router.get("/:dumpId/playlists", async (ctx) => { router.get("/:dumpId/playlists", async (ctx) => {
const requestingUserId = await parseOptionalAuth(ctx) ?? undefined; const requestingUserId = await parseOptionalAuth(ctx) ?? undefined;
// Resolve through getDump so private dumps 404 for anyone but their owner.
const dump = getDump(ctx.params.dumpId, requestingUserId); const dump = getDump(ctx.params.dumpId, requestingUserId);
const playlists = getPlaylistsForDump(dump.id, requestingUserId); const playlists = getPlaylistsForDump(dump.id, requestingUserId);
const responseBody: APIResponse<Playlist[]> = { const responseBody: APIResponse<Playlist[]> = {
@@ -218,11 +209,7 @@ router.put("/:dumpId/file", authMiddleware, async (ctx) => {
const dump = getDump(dumpId, userId); const dump = getDump(dumpId, userId);
if (userId !== dump.userId && !can(ctx.state.user, "dump:moderate")) { if (userId !== dump.userId && !can(ctx.state.user, "dump:moderate")) {
throw new APIException( throw new APIException(403, "Not authorized to update dump");
APIErrorCode.UNAUTHORIZED,
403,
"Not authorized to update dump",
);
} }
const formData = await ctx.request.body.formData(); const formData = await ctx.request.body.formData();
@@ -231,11 +218,7 @@ router.put("/:dumpId/file", authMiddleware, async (ctx) => {
const title = formData.get("title"); const title = formData.get("title");
if (!(file instanceof File)) { if (!(file instanceof File)) {
throw new APIException( throw new APIException(400, "A file is required");
APIErrorCode.VALIDATION_ERROR,
400,
"A file is required",
);
} }
const updatedDump = await replaceFileDump( const updatedDump = await replaceFileDump(
@@ -260,21 +243,13 @@ router.put("/:dumpId", authMiddleware, async (ctx) => {
const body: unknown = await ctx.request.body.json(); const body: unknown = await ctx.request.body.json();
if (!isUpdateDumpRequest(body)) { if (!isUpdateDumpRequest(body)) {
throw new APIException( throw new APIException(400, "Erroneous user input");
APIErrorCode.VALIDATION_ERROR,
422,
"Erroneous user input",
);
} }
const dump = getDump(dumpId, userId); const dump = getDump(dumpId, userId);
if (userId !== dump.userId && !can(ctx.state.user, "dump:moderate")) { if (userId !== dump.userId && !can(ctx.state.user, "dump:moderate")) {
throw new APIException( throw new APIException(403, "Not authorized to update dump");
APIErrorCode.UNAUTHORIZED,
403,
"Not authorized to update dump",
);
} }
const updatedDump = await updateDump(dump.id, body); const updatedDump = await updateDump(dump.id, body);
@@ -288,37 +263,23 @@ router.put("/:dumpId/thumbnail", authMiddleware, async (ctx) => {
const dump = getDump(dumpId, userId); const dump = getDump(dumpId, userId);
if (userId !== dump.userId && !can(ctx.state.user, "dump:moderate")) { if (userId !== dump.userId && !can(ctx.state.user, "dump:moderate")) {
throw new APIException( throw new APIException(403, "Not authorized to update dump");
APIErrorCode.UNAUTHORIZED,
403,
"Not authorized to update dump",
);
} }
const formData = await ctx.request.body.formData(); const formData = await ctx.request.body.formData();
const file = formData.get("file"); const file = formData.get("file");
if (!(file instanceof File)) { if (!(file instanceof File)) {
throw new APIException( throw new APIException(400, "Missing file field");
APIErrorCode.VALIDATION_ERROR,
400,
"Missing file field",
);
} }
const data = new Uint8Array(await file.arrayBuffer()); const data = new Uint8Array(await file.arrayBuffer());
const mime = validateImageUpload(data); const mime = validateImageUpload(data);
// DB update first (resolves slug → id), then file write. // Written before setDumpThumbnail announces it, or clients would fetch it
// too early.
await replaceUploadedFile(`${THUMBNAILS_DIR}/${dump.id}-custom`, data);
const updatedDump = setDumpThumbnail(dump.id, mime); const updatedDump = setDumpThumbnail(dump.id, mime);
const filePath = `${THUMBNAILS_DIR}/${dump.id}-custom`;
await Deno.mkdir(THUMBNAILS_DIR, { recursive: true });
try {
await Deno.writeFile(filePath, data);
} catch (err) {
await Deno.remove(filePath).catch(() => {});
throw err;
}
const responseBody: APIResponse<Dump> = { success: true, data: updatedDump }; const responseBody: APIResponse<Dump> = { success: true, data: updatedDump };
ctx.response.body = responseBody; ctx.response.body = responseBody;
@@ -330,11 +291,7 @@ router.delete("/:dumpId/thumbnail", authMiddleware, async (ctx) => {
const dump = getDump(dumpId, userId); const dump = getDump(dumpId, userId);
if (userId !== dump.userId && !can(ctx.state.user, "dump:moderate")) { if (userId !== dump.userId && !can(ctx.state.user, "dump:moderate")) {
throw new APIException( throw new APIException(403, "Not authorized to update dump");
APIErrorCode.UNAUTHORIZED,
403,
"Not authorized to update dump",
);
} }
const updatedDump = await removeDumpThumbnail(dump.id); const updatedDump = await removeDumpThumbnail(dump.id);
@@ -348,11 +305,7 @@ router.post("/:dumpId/refresh-metadata", authMiddleware, async (ctx) => {
const dump = getDump(dumpId, userId); const dump = getDump(dumpId, userId);
if (userId !== dump.userId && !can(ctx.state.user, "dump:moderate")) { if (userId !== dump.userId && !can(ctx.state.user, "dump:moderate")) {
throw new APIException( throw new APIException(403, "Not authorized to update dump");
APIErrorCode.UNAUTHORIZED,
403,
"Not authorized to update dump",
);
} }
const updatedDump = await refreshDumpMetadata(dump.id); const updatedDump = await refreshDumpMetadata(dump.id);
@@ -366,11 +319,7 @@ router.delete("/:dumpId", authMiddleware, async (ctx) => {
const dump = getDump(dumpId, userId); const dump = getDump(dumpId, userId);
if (userId !== dump.userId && !can(ctx.state.user, "dump:moderate")) { if (userId !== dump.userId && !can(ctx.state.user, "dump:moderate")) {
throw new APIException( throw new APIException(403, "Not authorized to delete dump");
APIErrorCode.UNAUTHORIZED,
403,
"Not authorized to delete dump",
);
} }
await deleteDump(dump.id); await deleteDump(dump.id);

View File

@@ -1,31 +1,31 @@
/** /**
* The route serving file dumps' uploads. * Serves file dumps' uploads to the players, viewers and download links of the
* frontend, which builds the URL with `dumpFileUrl`.
* *
* - `GET /api/files/:dumpId?token=` — public; a private dump's file needs its * - `GET /api/files/:dumpId?v=&token=` — public; a private dump's file needs
* poster's media `token`. Serves the file, inline when that is safe and as a * its poster's media `token`. Serves the file, inline when that is safe and
* download otherwise, with byte ranges. * as a download otherwise, with byte ranges.
* *
* @module * @module
*/ */
import { Router } from "@oak/oak"; import { Router } from "@oak/oak";
import { APIErrorCode } from "../../shared/api.ts";
import { APIException } from "../lib/api-exception.ts"; import { APIException } from "../lib/api-exception.ts";
import { DUMPS_DIR } from "../lib/upload.ts"; import { DUMPS_DIR } from "../config.ts";
import { authenticateMedia } from "../lib/auth.ts"; import { authenticateMedia } from "../lib/auth.ts";
import { serveFile } from "../lib/serve-file.ts"; import { serveFile } from "../lib/serve-file.ts";
import { dumpFileVersion, getDump } from "../services/dump-service.ts"; import { dumpFileVersion, getDump } from "../services/dump-service.ts";
/** The router serving the endpoints the module doc lists. */ /**
* Mounted by `api/main.ts` behind the error, Open Graph and CORS middleware.
* An `APIException` thrown by a handler becomes the JSON error response there.
*/
const router: Router = new Router({ prefix: "/api/files" }); const router: Router = new Router({ prefix: "/api/files" });
/** /**
* Builds an RFC 6266 `Content-Disposition` value that is always a valid * Builds a `Content-Disposition` value for a file called `name`. `Headers.set`
* ByteString. `headers.set` throws a TypeError if the value contains any * throws on any character above U+00FF, so `filename` gets an ASCII copy of the
* character > U+00FF, so a filename with an emoji / accent / CJK character * name and `filename*` (RFC 5987) the full one, percent-encoded.
* (e.g. an uploaded file) would otherwise crash the response. We provide an
* ASCII-only `filename="…"` fallback plus an RFC 5987 `filename*` with the
* full UTF-8 name for modern browsers.
*/ */
function contentDisposition( function contentDisposition(
type: "inline" | "attachment", type: "inline" | "attachment",
@@ -40,17 +40,16 @@ function contentDisposition(
} }
/** /**
* The types served inline. The MIME type of an upload is whatever the * Matches the types served inline. An upload's type is whatever the uploader's
* uploader's browser claimed, and these responses come from the app's own * browser claimed, and the file comes from the app's own origin, where the
* origin — where the session token lives. Only types a browser renders without * session token lives. So only types a browser shows without running code in
* running the file's own code are served inline; anything else (text/html, * them are inline, and anything else (HTML, SVG, XML…) downloads. `<img>`,
* image/svg+xml, XML, …) downloads. <img>, <audio> and fetch() ignore * `<audio>` and `fetch()` ignore `Content-Disposition`, so the players and
* Content-Disposition, so previews that embed or read the file still work. * previews still work.
*/ */
const INLINE_MIME_RE = const INLINE_MIME_RE =
/^(image\/(png|jpeg|gif|webp|avif|bmp)|audio\/[\w.+-]+|video\/[\w.+-]+|application\/pdf|text\/plain)$/; /^(image\/(png|jpeg|gif|webp|avif|bmp)|audio\/[\w.+-]+|video\/[\w.+-]+|application\/pdf|text\/plain)$/;
/** Checks whether a file of type `mime` may be served inline. */
function isInlineSafe(mime: string): boolean { function isInlineSafe(mime: string): boolean {
return INLINE_MIME_RE.test(mime.split(";")[0].trim().toLowerCase()); return INLINE_MIME_RE.test(mime.split(";")[0].trim().toLowerCase());
} }
@@ -58,24 +57,19 @@ function isInlineSafe(mime: string): boolean {
router.get("/:dumpId", async (ctx) => { router.get("/:dumpId", async (ctx) => {
const { dumpId } = ctx.params; const { dumpId } = ctx.params;
// Guard against path traversal (UUIDs are safe, but be explicit) // The id names a file on disk: nothing but an id's characters gets through.
if (!/^[0-9a-f-]{36}$/.test(dumpId)) { if (!/^[0-9a-f-]{36}$/.test(dumpId)) {
throw new APIException(APIErrorCode.BAD_REQUEST, 400, "Invalid dump ID"); throw new APIException(400, "Invalid dump ID");
} }
// Media elements can't send an Authorization header, so the viewer's token // Media elements can't send an Authorization header, so the viewer's media
// arrives as a query param (same as the WebSocket route). Needed to authorize // token comes in the query string.
// access to private dumps the requester owns.
const token = ctx.request.url.searchParams.get("token"); const token = ctx.request.url.searchParams.get("token");
const viewerId = token ? await authenticateMedia(token) : null; const viewerId = token ? await authenticateMedia(token) : null;
const dump = getDump(dumpId, viewerId ?? undefined); const dump = getDump(dumpId, viewerId ?? undefined);
if (dump.kind !== "file" || !dump.fileMime || !dump.fileName) { if (dump.kind !== "file" || !dump.fileMime || !dump.fileName) {
throw new APIException( throw new APIException(404, "No file for this dump");
APIErrorCode.NOT_FOUND,
404,
"No file for this dump",
);
} }
const path = `${DUMPS_DIR}/${dumpId}`; const path = `${DUMPS_DIR}/${dumpId}`;
@@ -98,10 +92,9 @@ router.get("/:dumpId", async (ctx) => {
// links. // links.
ctx.response.headers.set("Referrer-Policy", "no-referrer"); ctx.response.headers.set("Referrer-Policy", "no-referrer");
// The app's URLs carry the file's version (see dumpFileVersion), so one // A URL naming the file's current version (dumpFileVersion) always serves the
// naming the current version never changes and is cached for good; any // same bytes, so it is cached for good; any other is revalidated against the
// other URL is revalidated against the ETag. A private dump's bytes must not // ETag. A private dump's bytes must not sit in a shared cache.
// sit in a shared cache.
const scope = dump.isPrivate ? "private" : "public"; const scope = dump.isPrivate ? "private" : "public";
const current = ctx.request.url.searchParams.get("v") === const current = ctx.request.url.searchParams.get("v") ===
dumpFileVersion(dump); dumpFileVersion(dump);
@@ -113,7 +106,7 @@ router.get("/:dumpId", async (ctx) => {
ranges: true, ranges: true,
}); });
if (!served) { if (!served) {
throw new APIException(APIErrorCode.NOT_FOUND, 404, "File not found"); throw new APIException(404, "File not found");
} }
}); });

View File

@@ -1,5 +1,6 @@
/** /**
* The follow routes, all for what the caller follows. * Serves the endpoints for what the caller follows, and the index's "followed"
* tab built from it. What another user follows is served by `routes/users.ts`.
* *
* - `GET /api/follows/status` — signed in. Answers the ids of the users and * - `GET /api/follows/status` — signed in. Answers the ids of the users and
* playlists they follow. * playlists they follow.
@@ -37,10 +38,9 @@ import {
import { getCategoryIdBySlug } from "../services/category-service.ts"; import { getCategoryIdBySlug } from "../services/category-service.ts";
/** /**
* Resolves the optional ?category=<slug> filter. Returns the matching category * Returns the id of the category the `?category=` slug names, undefined without
* id, or "missing" when a slug was given but matches nothing (so the feed comes * one, or "missing" when it names none, so that the feed comes back empty
* back empty rather than unfiltered), or undefined when no filter was * rather than unfiltered.
* requested.
*/ */
function resolveCategoryFilter( function resolveCategoryFilter(
params: URLSearchParams, params: URLSearchParams,
@@ -50,14 +50,14 @@ function resolveCategoryFilter(
return getCategoryIdBySlug(slug) ?? "missing"; return getCategoryIdBySlug(slug) ?? "missing";
} }
/** The feed a filter naming no category yields. */
const EMPTY_FEED: Page<Dump> = { items: [], hasMore: false }; const EMPTY_FEED: Page<Dump> = { items: [], hasMore: false };
/** The router serving the endpoints the module doc lists. */ /**
* Mounted by `api/main.ts` behind the error, Open Graph and CORS middleware.
* An `APIException` thrown by a handler becomes the JSON error response there.
*/
const router: Router = new Router({ prefix: "/api/follows" }); const router: Router = new Router({ prefix: "/api/follows" });
// Static routes first to prevent Oak matching "status"/"feed" as a :param
router.get("/status", authMiddleware, (ctx) => { router.get("/status", authMiddleware, (ctx) => {
const status = getFollowStatus(ctx.state.user.userId); const status = getFollowStatus(ctx.state.user.userId);
const body: APIResponse<FollowStatus> = { success: true, data: status }; const body: APIResponse<FollowStatus> = { success: true, data: status };

View File

@@ -1,27 +1,28 @@
/** /**
* The invite routes. * Serves the invite endpoints. Any signed-in user can invite someone, from
* their profile page, and the invite link opens the `/register` page.
* *
* - `POST /api/invites` — signed in. Makes an invite and answers its token. * - `POST /api/invites` — signed in. Makes an invite and answers its token.
* - `GET /api/invites/:token` — public. Checks an invite is valid, before the * - `GET /api/invites/:token` — public. Checks that an invite still works,
* register page shows its form; 404 when it isn't. * before the register page shows its form; 404 when it doesn't.
* *
* @module * @module
*/ */
import { Router } from "@oak/oak"; import { Router } from "@oak/oak";
import { APIErrorCode, type APIResponse } from "../../shared/api.ts"; import { type APIResponse } from "../../shared/api.ts";
import { APIException } from "../lib/api-exception.ts"; import { APIException } from "../lib/api-exception.ts";
import type { InviteResponse } from "../../shared/user.ts"; import type { InviteResponse } from "../../shared/user.ts";
import { type AuthContext, authMiddleware } from "../middleware/auth.ts"; import { type AuthContext, authMiddleware } from "../middleware/auth.ts";
import { createInvite, validateInvite } from "../services/invite-service.ts"; import { createInvite, validateInvite } from "../services/invite-service.ts";
/** The router serving the endpoints the module doc lists. */ /**
* Mounted by `api/main.ts` behind the error, Open Graph and CORS middleware.
* An `APIException` thrown by a handler becomes the JSON error response there.
*/
const router: Router = new Router({ prefix: "/api/invites" }); const router: Router = new Router({ prefix: "/api/invites" });
router.post("/", authMiddleware, (ctx: AuthContext) => { router.post("/", authMiddleware, (ctx: AuthContext) => {
if (!ctx.state.user) {
throw new APIException(APIErrorCode.UNAUTHORIZED, 401, "Not authenticated");
}
const responseBody: APIResponse<InviteResponse> = { const responseBody: APIResponse<InviteResponse> = {
success: true, success: true,
data: { token: createInvite(ctx.state.user.userId) }, data: { token: createInvite(ctx.state.user.userId) },
@@ -31,16 +32,13 @@ router.post("/", authMiddleware, (ctx: AuthContext) => {
}); });
router.get("/:token", (ctx) => { router.get("/:token", (ctx) => {
// A used invite answers 404 too: the page only needs to know it won't work.
try { try {
validateInvite(ctx.params.token); validateInvite(ctx.params.token);
const responseBody: APIResponse<null> = { success: true, data: null }; const responseBody: APIResponse<null> = { success: true, data: null };
ctx.response.body = responseBody; ctx.response.body = responseBody;
} catch { } catch {
throw new APIException( throw new APIException(404, "Invalid or expired invite");
APIErrorCode.NOT_FOUND,
404,
"Invalid or expired invite",
);
} }
}); });

View File

@@ -1,37 +1,32 @@
/** /**
* The notification routes, all for the caller's own notifications. * Serves the caller's own notifications to the client's `/notifications`
* page, which marks them all read when it opens. New ones arrive over the
* WebSocket instead, as `notification_created`.
* *
* - `GET /api/notifications?page=&limit=` — signed in. Answers a page of them, * - `GET /api/notifications?page=&limit=` — signed in. Answers a page of them,
* newest first. * newest first.
* - `POST /api/notifications/read-all` — signed in. Marks them all read. * - `POST /api/notifications/read-all` — signed in. Marks them all read.
* - `PATCH /api/notifications/:id/read` — signed in. Marks one read.
* *
* @module * @module
*/ */
import { Router } from "@oak/oak"; import { Router } from "@oak/oak";
import { import { type APIResponse, type PaginatedData } from "../../shared/api.ts";
APIErrorCode,
type APIResponse,
type PaginatedData,
} from "../../shared/api.ts";
import { APIException } from "../lib/api-exception.ts";
import type { Notification } from "../../shared/notification.ts"; import type { Notification } from "../../shared/notification.ts";
import { parsePagination } from "../lib/pagination.ts"; import { parsePagination } from "../lib/pagination.ts";
import { type AuthContext, authMiddleware } from "../middleware/auth.ts"; import { type AuthContext, authMiddleware } from "../middleware/auth.ts";
import { import {
getNotificationsForUser, getNotificationsForUser,
markAllRead, markAllRead,
markOneRead,
} from "../services/notification-service.ts"; } from "../services/notification-service.ts";
/** The router serving the endpoints the module doc lists. */ /**
* Mounted by `api/main.ts` behind the error, Open Graph and CORS middleware.
* An `APIException` thrown by a handler becomes the JSON error response there.
*/
const router: Router = new Router({ prefix: "/api/notifications" }); const router: Router = new Router({ prefix: "/api/notifications" });
router.get("/", authMiddleware, (ctx: AuthContext) => { router.get("/", authMiddleware, (ctx: AuthContext) => {
if (!ctx.state.user) {
throw new APIException(APIErrorCode.UNAUTHORIZED, 401, "Not authenticated");
}
const { page, limit } = parsePagination(ctx.request.url.searchParams); const { page, limit } = parsePagination(ctx.request.url.searchParams);
const { items, hasMore } = getNotificationsForUser( const { items, hasMore } = getNotificationsForUser(
ctx.state.user.userId, ctx.state.user.userId,
@@ -46,20 +41,8 @@ router.get("/", authMiddleware, (ctx: AuthContext) => {
}); });
router.post("/read-all", authMiddleware, (ctx: AuthContext) => { router.post("/read-all", authMiddleware, (ctx: AuthContext) => {
if (!ctx.state.user) {
throw new APIException(APIErrorCode.UNAUTHORIZED, 401, "Not authenticated");
}
markAllRead(ctx.state.user.userId); markAllRead(ctx.state.user.userId);
ctx.response.status = 204; ctx.response.status = 204;
}); });
router.patch("/:id/read", authMiddleware, (ctx) => {
const user = ctx.state.user;
if (!user) {
throw new APIException(APIErrorCode.UNAUTHORIZED, 401, "Not authenticated");
}
markOneRead(ctx.params.id, user.userId);
ctx.response.status = 204;
});
export default router; export default router;

View File

@@ -1,11 +1,13 @@
/** /**
* The playlist routes. `:playlistId` takes an id or a slug. * Serves playlists, their dumps and their covers. `:playlistId` takes an id or
* a slug.
* *
* - `GET /api/playlists/by-dump/:dumpId/memberships` — signed in. Answers each * - `GET /api/playlists/by-dump/:dumpId/memberships` — signed in. Answers each
* of the caller's playlists and whether it holds the dump. * of the caller's playlists and whether it holds the dump.
* - `POST /api/playlists` — signed in. Creates a playlist. * - `POST /api/playlists` — signed in. Creates a playlist.
* - `GET /api/playlists/:playlistId` — optional sign-in. Answers a playlist * - `GET /api/playlists/:playlistId` — optional sign-in. Answers a playlist
* with the dumps the caller may see in it; a private one only to its owner. * with the dumps the caller may see in it. A private one 404s but for its
* owner.
* - `PUT /api/playlists/:playlistId` — signed in, as its owner or with * - `PUT /api/playlists/:playlistId` — signed in, as its owner or with
* `playlist:moderate`. Updates its title, description and visibility. * `playlist:moderate`. Updates its title, description and visibility.
* - `DELETE /api/playlists/:playlistId` — signed in, as its owner or with * - `DELETE /api/playlists/:playlistId` — signed in, as its owner or with
@@ -16,8 +18,8 @@
* or with `playlist:moderate`. Removes a dump. * or with `playlist:moderate`. Removes a dump.
* - `POST /api/playlists/:playlistId/image` — signed in, as its owner or with * - `POST /api/playlists/:playlistId/image` — signed in, as its owner or with
* `playlist:moderate`. Replaces its cover (multipart `file`, at most 5 MB). * `playlist:moderate`. Replaces its cover (multipart `file`, at most 5 MB).
* - `GET /api/playlists/:playlistId/image?token=` — public; a private * - `GET /api/playlists/:playlistId/image?token=` — public, but a private
* playlist's cover needs its owner's media `token`. Serves its cover; 404 * playlist's cover needs its owner's media `token`. Serves its cover, or 404s
* without one. * without one.
* - `PUT /api/playlists/:playlistId/order` — signed in, as its owner or with * - `PUT /api/playlists/:playlistId/order` — signed in, as its owner or with
* `playlist:moderate`. Reorders the dumps the caller can see in it. * `playlist:moderate`. Reorders the dumps the caller can see in it.
@@ -27,7 +29,7 @@
import { Router } from "@oak/oak"; import { Router } from "@oak/oak";
import { authenticateMedia, parseOptionalAuth } from "../lib/auth.ts"; import { authenticateMedia, parseOptionalAuth } from "../lib/auth.ts";
import { APIErrorCode, type APIResponse } from "../../shared/api.ts"; import { type APIResponse } from "../../shared/api.ts";
import { APIException } from "../lib/api-exception.ts"; import { APIException } from "../lib/api-exception.ts";
import { import {
isCreatePlaylistRequest, isCreatePlaylistRequest,
@@ -55,12 +57,16 @@ import {
updatePlaylist, updatePlaylist,
} from "../services/playlist-service.ts"; } from "../services/playlist-service.ts";
import { import {
PLAYLIST_IMAGES_DIR, replaceUploadedFile,
serveUploadedFile, serveUploadedFile,
validateImageUpload, validateImageUpload,
} from "../lib/upload.ts"; } from "../lib/upload.ts";
import { PLAYLIST_IMAGES_DIR } from "../config.ts";
/** The router serving the endpoints the module doc lists. */ /**
* Mounted by `api/main.ts` behind the error, Open Graph and CORS middleware.
* An `APIException` thrown by a handler becomes the JSON error response there.
*/
const router: Router<AuthState> = new Router<AuthState>({ const router: Router<AuthState> = new Router<AuthState>({
prefix: "/api/playlists", prefix: "/api/playlists",
}); });
@@ -79,11 +85,7 @@ router.get("/by-dump/:dumpId/memberships", authMiddleware, (ctx) => {
router.post("/", authMiddleware, async (ctx) => { router.post("/", authMiddleware, async (ctx) => {
const body: unknown = await ctx.request.body.json(); const body: unknown = await ctx.request.body.json();
if (!isCreatePlaylistRequest(body)) { if (!isCreatePlaylistRequest(body)) {
throw new APIException( throw new APIException(400, "Invalid request");
APIErrorCode.VALIDATION_ERROR,
400,
"Invalid request",
);
} }
const responseBody: APIResponse<Playlist> = { const responseBody: APIResponse<Playlist> = {
success: true, success: true,
@@ -105,11 +107,7 @@ router.get("/:playlistId", async (ctx) => {
router.put("/:playlistId", authMiddleware, async (ctx) => { router.put("/:playlistId", authMiddleware, async (ctx) => {
const body: unknown = await ctx.request.body.json(); const body: unknown = await ctx.request.body.json();
if (!isUpdatePlaylistRequest(body)) { if (!isUpdatePlaylistRequest(body)) {
throw new APIException( throw new APIException(400, "Invalid request");
APIErrorCode.VALIDATION_ERROR,
400,
"Invalid request",
);
} }
const playlist = updatePlaylist( const playlist = updatePlaylist(
ctx.params.playlistId, ctx.params.playlistId,
@@ -153,43 +151,27 @@ router.delete("/:playlistId/dumps/:dumpId", authMiddleware, (ctx) => {
router.post("/:playlistId/image", authMiddleware, async (ctx) => { router.post("/:playlistId/image", authMiddleware, async (ctx) => {
const contentType = ctx.request.headers.get("content-type") ?? ""; const contentType = ctx.request.headers.get("content-type") ?? "";
if (!contentType.includes("multipart/form-data")) { if (!contentType.includes("multipart/form-data")) {
throw new APIException( throw new APIException(400, "Expected multipart/form-data");
APIErrorCode.BAD_REQUEST,
400,
"Expected multipart/form-data",
);
} }
const body = await ctx.request.body.formData(); const body = await ctx.request.body.formData();
const file = body.get("file"); const file = body.get("file");
if (!(file instanceof File)) { if (!(file instanceof File)) {
throw new APIException(APIErrorCode.BAD_REQUEST, 400, "Missing file field"); throw new APIException(400, "Missing file field");
} }
const data = new Uint8Array(await file.arrayBuffer()); const data = new Uint8Array(await file.arrayBuffer());
const mime = validateImageUpload(data); const mime = validateImageUpload(data);
// The file goes in before the row changes. Setting image_mime and bumping // Written before setPlaylistImage announces it, or clients would fetch the
// updated_at announces the cover (the `?v=` on its URL, the socket event), // new `?v=` URL too early and cache the old cover under it.
// so the bytes must already be there: an earlier fetch would cache the old
// cover under the new URL, and a failed write would leave a cover that 404s.
// Written beside and renamed over, so the old cover is never half-replaced.
const { id } = getEditablePlaylist( const { id } = getEditablePlaylist(
ctx.params.playlistId, ctx.params.playlistId,
ctx.state.user.userId, ctx.state.user.userId,
can(ctx.state.user, "playlist:moderate"), can(ctx.state.user, "playlist:moderate"),
); );
const filePath = `${PLAYLIST_IMAGES_DIR}/${id}`; await replaceUploadedFile(`${PLAYLIST_IMAGES_DIR}/${id}`, data);
const tmpPath = `${filePath}.${crypto.randomUUID()}.tmp`;
await Deno.mkdir(PLAYLIST_IMAGES_DIR, { recursive: true });
try {
await Deno.writeFile(tmpPath, data);
await Deno.rename(tmpPath, filePath);
} catch (err) {
await Deno.remove(tmpPath).catch(() => {});
throw err;
}
const playlist = setPlaylistImage( const playlist = setPlaylistImage(
id, id,
mime, mime,
@@ -210,7 +192,7 @@ router.get("/:playlistId/image", async (ctx) => {
ctx.response.status = 404; ctx.response.status = 404;
return; return;
} }
// The URL may carry a media token; keep it out of anyone else's logs. // The URL may carry a media token, which a Referer would leak.
ctx.response.headers.set("Referrer-Policy", "no-referrer"); ctx.response.headers.set("Referrer-Policy", "no-referrer");
await serveUploadedFile( await serveUploadedFile(
ctx, ctx,
@@ -224,11 +206,7 @@ router.get("/:playlistId/image", async (ctx) => {
router.put("/:playlistId/order", authMiddleware, async (ctx) => { router.put("/:playlistId/order", authMiddleware, async (ctx) => {
const body: unknown = await ctx.request.body.json(); const body: unknown = await ctx.request.body.json();
if (!isReorderPlaylistRequest(body)) { if (!isReorderPlaylistRequest(body)) {
throw new APIException( throw new APIException(400, "Invalid request");
APIErrorCode.VALIDATION_ERROR,
400,
"Invalid request",
);
} }
reorderPlaylist( reorderPlaylist(
ctx.params.playlistId, ctx.params.playlistId,

View File

@@ -1,5 +1,6 @@
/** /**
* The link preview routes. * Fetches link previews for the create form, and proxies the remote images
* that the frontend's thumbnails can't load directly.
* *
* - `GET /api/preview?url=` — signed in. Answers the preview of a link, and * - `GET /api/preview?url=` — signed in. Answers the preview of a link, and
* whether its page could be reached. * whether its page could be reached.
@@ -21,26 +22,24 @@ import {
import type { LinkPreview } from "../../shared/dump.ts"; import type { LinkPreview } from "../../shared/dump.ts";
import { authMiddleware } from "../middleware/auth.ts"; import { authMiddleware } from "../middleware/auth.ts";
/** The router serving the endpoints the module doc lists. */ /**
* Mounted by `api/main.ts` behind the error, Open Graph and CORS middleware.
* An `APIException` thrown by a handler becomes the JSON error response there.
*/
const previewRouter: Router = new Router(); const previewRouter: Router = new Router();
// Signed-in only: it's the create form's preview, and an open one lets anyone // Signed-in only: an open preview would let anyone make the server fetch pages.
// use the server to fetch pages for them.
previewRouter.get("/api/preview", authMiddleware, async (ctx) => { previewRouter.get("/api/preview", authMiddleware, async (ctx) => {
const url = ctx.request.url.searchParams.get("url") ?? ""; const url = ctx.request.url.searchParams.get("url") ?? "";
if (!isValidHttpUrl(url)) { if (!isValidHttpUrl(url)) {
const responseBody: APIFailure = { const responseBody: APIFailure = {
success: false, success: false,
error: { code: APIErrorCode.VALIDATION_ERROR, message: "Invalid URL" }, error: { code: APIErrorCode.BAD_REQUEST, message: "Invalid URL" },
}; };
ctx.response.status = 400; ctx.response.status = 400;
ctx.response.body = responseBody; ctx.response.body = responseBody;
return; return;
} }
// `reached` is reported separately because a failed fetch still yields a
// usable stub (hostname only). Without the flag the create form cannot tell
// "this page has no preview" from "this link is dead", and shows the same
// bare card for both.
const { ok, content } = await tryFetchRichContent(url); const { ok, content } = await tryFetchRichContent(url);
const responseBody: APIResponse<LinkPreview> = { const responseBody: APIResponse<LinkPreview> = {
success: true, success: true,
@@ -49,7 +48,10 @@ previewRouter.get("/api/preview", authMiddleware, async (ctx) => {
ctx.response.body = responseBody; ctx.response.body = responseBody;
}); });
/** Answers with a proxied image, sandboxed so it can't run as a document. */ /**
* Answers with a proxied image. It is sent from this site's origin, so it is
* sandboxed in case a browser opens it as a document.
*/
function sendImage( function sendImage(
ctx: { response: { headers: Headers; body?: unknown } }, ctx: { response: { headers: Headers; body?: unknown } },
bytes: Uint8Array, bytes: Uint8Array,
@@ -65,8 +67,8 @@ function sendImage(
ctx.response.body = bytes; ctx.response.body = bytes;
} }
// Proxies an external image through the server so HTTP thumbnail URLs don't // The frontend sends a thumbnail here when it is plain HTTP, which an HTTPS
// trigger mixed-content blocks when the frontend is served over HTTPS. // page won't load, or when its host's hotlink protection refused the browser.
previewRouter.get("/api/proxy-image", async (ctx) => { previewRouter.get("/api/proxy-image", async (ctx) => {
const url = ctx.request.url.searchParams.get("url") ?? ""; const url = ctx.request.url.searchParams.get("url") ?? "";
if (!isValidHttpUrl(url)) { if (!isValidHttpUrl(url)) {

View File

@@ -1,9 +1,11 @@
/** /**
* The search route. * Searches dumps, users and playlists at once, for the client's `Search` page.
* Only the dumps are paged. Private dumps and playlists are found only by
* their owner.
* *
* - `GET /api/search?q=&page=&limit=` — optional sign-in. Answers a page of the * - `GET /api/search?q=&page=&limit=` — optional sign-in. Answers a page of the
* matching dumps with their total, and up to 20 matching users and playlists; * matching dumps with their total, and up to 20 matching users and
* private dumps and playlists only the caller's own. * playlists. A blank `q` finds nothing.
* *
* @module * @module
*/ */
@@ -17,7 +19,10 @@ import { searchPlaylists } from "../services/playlist-service.ts";
import type { APIResponse } from "../../shared/api.ts"; import type { APIResponse } from "../../shared/api.ts";
import type { SearchResults } from "../../shared/search.ts"; import type { SearchResults } from "../../shared/search.ts";
/** The router serving the endpoints the module doc lists. */ /**
* Mounted by `api/main.ts` behind the error, Open Graph and CORS middleware.
* An `APIException` thrown by a handler becomes the JSON error response there.
*/
const router: Router = new Router({ prefix: "/api/search" }); const router: Router = new Router({ prefix: "/api/search" });
router.get("/", async (ctx) => { router.get("/", async (ctx) => {

View File

@@ -1,7 +1,9 @@
/** /**
* The route serving dumps' thumbnails, generated with ffmpeg where needed. * Serves dumps' thumbnails to the cards and pages that show a file dump, which
* build the URL with `dumpThumbnailUrl`. Thumbnails made from an upload are
* generated with ffmpeg on first request and kept in `THUMBNAILS_DIR`.
* *
* - `GET /api/thumbnails/:dumpId?token=&v=` — public; a private dump's * - `GET /api/thumbnails/:dumpId?v=&token=` — public; a private dump's
* thumbnail needs its poster's media `token`. Serves the dump's custom * thumbnail needs its poster's media `token`. Serves the dump's custom
* thumbnail, else one made from its upload: a still of a video, or a * thumbnail, else one made from its upload: a still of a video, or a
* scaled-down copy of an image (the image itself when none can be made); 404 * scaled-down copy of an image (the image itself when none can be made); 404
@@ -11,7 +13,6 @@
*/ */
import { Router } from "@oak/oak"; import { Router } from "@oak/oak";
import { APIErrorCode } from "../../shared/api.ts";
import { APIException } from "../lib/api-exception.ts"; import { APIException } from "../lib/api-exception.ts";
import { import {
dumpFileVersion, dumpFileVersion,
@@ -23,16 +24,18 @@ import { serveUploadedFile } from "../lib/upload.ts";
import { DUMPS_DIR, THUMBNAILS_DIR } from "../config.ts"; import { DUMPS_DIR, THUMBNAILS_DIR } from "../config.ts";
import { authenticateMedia } from "../lib/auth.ts"; import { authenticateMedia } from "../lib/auth.ts";
/** The router serving the endpoints the module doc lists. */ /**
* Mounted by `api/main.ts` behind the error, Open Graph and CORS middleware.
* An `APIException` thrown by a handler becomes the JSON error response there.
*/
const router: Router = new Router({ prefix: "/api/thumbnails" }); const router: Router = new Router({ prefix: "/api/thumbnails" });
/** /**
* Whether ffmpeg runs, probed once per process: spawning `ffmpeg -version` on * Holds whether ffmpeg runs, probed once per process: probing on every uncached
* every uncached thumbnail request cost a process per card. * thumbnail request cost a process per card.
*/ */
let ffmpegProbe: Promise<boolean> | undefined; let ffmpegProbe: Promise<boolean> | undefined;
/** Checks whether ffmpeg is available. */
function ffmpegAvailable(): Promise<boolean> { function ffmpegAvailable(): Promise<boolean> {
ffmpegProbe ??= (async () => { ffmpegProbe ??= (async () => {
try { try {
@@ -50,21 +53,23 @@ function ffmpegAvailable(): Promise<boolean> {
} }
/** /**
* The container formats ffmpeg may read a video as. The upload's type is only * Lists the container formats ffmpeg may read a video as. The upload's type is
* what the uploader claimed, and ffmpeg probes the real format from the bytes — * only what the uploader claimed, and ffmpeg guesses the format from the bytes,
* a "video" could be an HLS or concat playlist pointing ffmpeg at local files * so a "video" could be an HLS or concat playlist pointing it at local files or
* or internal URLs. Read nothing but the file itself, and only as the kind of * internal URLs. With this list, and `-protocol_whitelist file`, ffmpeg reads
* container it's meant to be. * nothing but the file, and only as a video.
*/ */
const VIDEO_FORMATS = const VIDEO_FORMATS =
"mov,mp4,m4a,3gp,3g2,mj2,matroska,webm,avi,flv,mpegts,mpeg,ogg,asf,h264,hevc"; "mov,mp4,m4a,3gp,3g2,mj2,matroska,webm,avi,flv,mpegts,mpeg,ogg,asf,h264,hevc";
/** The formats ffmpeg may read an image as (see {@link VIDEO_FORMATS}). */ /**
* Lists the formats ffmpeg may read an image as, for the reason given at
* {@link VIDEO_FORMATS}.
*/
const IMAGE_FORMATS = "jpeg_pipe,png_pipe,webp_pipe,bmp_pipe"; const IMAGE_FORMATS = "jpeg_pipe,png_pipe,webp_pipe,bmp_pipe";
/** /**
* The image types the feed shows through a scaled-down preview, and the * Maps the image types that get a scaled-down preview to the preview's format.
* preview's format. GIFs keep their animation (served as they are); anything * A GIF is served as it is, to keep its animation.
* else has no preview to make.
*/ */
const PREVIEWABLE_IMAGES: Record<string, "jpg" | "png"> = { const PREVIEWABLE_IMAGES: Record<string, "jpg" | "png"> = {
"image/jpeg": "jpg", "image/jpeg": "jpg",
@@ -75,13 +80,13 @@ const PREVIEWABLE_IMAGES: Record<string, "jpg" | "png"> = {
}; };
/** /**
* The width of an image preview: comfortably covers the widest feed card on a * Caps an image preview's width. It covers the widest feed card on a
* high-density screen, a fraction of a phone photo's bytes. * high-density screen, at a fraction of a phone photo's bytes.
*/ */
const PREVIEW_WIDTH = 960; const PREVIEW_WIDTH = 960;
/** /**
* Runs ffmpeg with `args` into `outPath`. * Runs ffmpeg with `args`, writing to `outPath`.
* *
* @throws {Error} When ffmpeg fails. * @throws {Error} When ffmpeg fails.
*/ */
@@ -121,7 +126,7 @@ function videoStill(srcPath: string, outPath: string): Promise<void> {
], outPath); ], outPath);
} }
/** Writes a copy of an image scaled down to {@link PREVIEW_WIDTH}. */ /** Writes a copy of an image at most {@link PREVIEW_WIDTH} wide. */
function imagePreview(srcPath: string, outPath: string): Promise<void> { function imagePreview(srcPath: string, outPath: string): Promise<void> {
return runFfmpeg([ return runFfmpeg([
"-format_whitelist", "-format_whitelist",
@@ -141,15 +146,14 @@ function imagePreview(srcPath: string, outPath: string): Promise<void> {
} }
/** /**
* The generations under way, by output path: one ffmpeg run per output, so a * Holds the generations under way, by output path. A feed full of cards asking
* feed full of cards asking for the same new still at once waits on a single * for the same new still at once then waits on a single ffmpeg run.
* generation.
*/ */
const inFlight = new Map<string, Promise<void>>(); const inFlight = new Map<string, Promise<void>>();
/** /**
* Makes `outPath` with `generate` unless it exists; false when it can't be * Makes `outPath` with `generate` unless it exists. Resolves false when it
* made (no ffmpeg, or ffmpeg failed). * can't be made, because ffmpeg is missing or failed.
*/ */
async function ensureGenerated( async function ensureGenerated(
outPath: string, outPath: string,
@@ -168,20 +172,22 @@ async function ensureGenerated(
router.get("/:dumpId", async (ctx) => { router.get("/:dumpId", async (ctx) => {
const { dumpId } = ctx.params; const { dumpId } = ctx.params;
// The id names files on disk: nothing but an id's characters gets through.
if (!/^[0-9a-f-]{36}$/.test(dumpId)) { if (!/^[0-9a-f-]{36}$/.test(dumpId)) {
throw new APIException(APIErrorCode.BAD_REQUEST, 400, "Invalid dump ID"); throw new APIException(400, "Invalid dump ID");
} }
// Token via query param (media elements can't send headers) — authorizes the // Media elements can't send an Authorization header, so the viewer's media
// owner's private dumps and gates the custom-thumbnail branch below too. // token comes in the query string. This privacy check covers the custom
// thumbnail too, since getDumpThumbnailInfo makes none.
const token = ctx.request.url.searchParams.get("token"); const token = ctx.request.url.searchParams.get("token");
const viewerId = token ? await authenticateMedia(token) : null; const viewerId = token ? await authenticateMedia(token) : null;
const dump = getDump(dumpId, viewerId ?? undefined); const dump = getDump(dumpId, viewerId ?? undefined);
// The URL may carry a media token; keep it out of anyone else's logs. // The URL may carry a media token, which must not leak through a Referer.
ctx.response.headers.set("Referrer-Policy", "no-referrer"); ctx.response.headers.set("Referrer-Policy", "no-referrer");
// A private dump's thumbnail must not sit in a shared cache. The app's URLs // What is generated from the file is cached for good under a URL naming its
// carry the file's version (see dumpFileVersion): what's generated from the // current version (dumpFileVersion), and revalidated under any other. A
// file is cached for good under the current one, and revalidated otherwise. // private dump's thumbnail must not sit in a shared cache.
const scope = dump.isPrivate ? "private" : "public"; const scope = dump.isPrivate ? "private" : "public";
const generatedCache = const generatedCache =
ctx.request.url.searchParams.get("v") === dumpFileVersion(dump) ctx.request.url.searchParams.get("v") === dumpFileVersion(dump)
@@ -190,8 +196,8 @@ router.get("/:dumpId", async (ctx) => {
const customThumb = getDumpThumbnailInfo(dumpId); const customThumb = getDumpThumbnailInfo(dumpId);
if (customThumb) { if (customThumb) {
// Replaceable without the URL changing: always revalidated (a 304 when // Replacing it doesn't change the file's version, so it is always
// unchanged). // revalidated.
const served = await serveUploadedFile( const served = await serveUploadedFile(
ctx, ctx,
`${THUMBNAILS_DIR}/${customThumb.id}-custom`, `${THUMBNAILS_DIR}/${customThumb.id}-custom`,
@@ -210,11 +216,7 @@ router.get("/:dumpId", async (ctx) => {
if ( if (
!await ensureGenerated(thumbPath, () => videoStill(srcPath, thumbPath)) !await ensureGenerated(thumbPath, () => videoStill(srcPath, thumbPath))
) { ) {
throw new APIException( throw new APIException(404, "Could not generate thumbnail");
APIErrorCode.NOT_FOUND,
404,
"Could not generate thumbnail",
);
} }
if ( if (
await serveFile(ctx, thumbPath, { await serveFile(ctx, thumbPath, {
@@ -248,11 +250,7 @@ router.get("/:dumpId", async (ctx) => {
) return; ) return;
} }
throw new APIException( throw new APIException(404, "No thumbnail for this dump");
APIErrorCode.NOT_FOUND,
404,
"No thumbnail for this dump",
);
}); });
export default router; export default router;

View File

@@ -1,6 +1,7 @@
/** /**
* The user and account routes. Public profiles never carry the password hash or * Serves accounts (sign-up, sign-in, tokens, passwords) and user profiles,
* the email. * with what each user posted, upvoted, follows and invited. A profile sent to
* anyone but its user goes through `toPublicUser`, which drops the email.
* *
* - `POST /api/users/register` — public, with a valid invite. Creates an * - `POST /api/users/register` — public, with a valid invite. Creates an
* account, sends the welcome email, and answers the user and their tokens. * account, sends the welcome email, and answers the user and their tokens.
@@ -42,11 +43,7 @@
import { Router } from "@oak/oak"; import { Router } from "@oak/oak";
import { import { type APIResponse, type PaginatedData } from "../../shared/api.ts";
APIErrorCode,
type APIResponse,
type PaginatedData,
} from "../../shared/api.ts";
import { APIException } from "../lib/api-exception.ts"; import { APIException } from "../lib/api-exception.ts";
import { import {
type AuthResponse, type AuthResponse,
@@ -69,7 +66,8 @@ import {
import type { Playlist } from "../../shared/playlist.ts"; import type { Playlist } from "../../shared/playlist.ts";
import { toPublicUser, toSelfUser, type User } from "../model/user.ts"; import { toPublicUser, toSelfUser, type User } from "../model/user.ts";
import { createJWT, createMediaToken, verifyPassword } from "../lib/jwt.ts"; import { createJWT, createMediaToken } from "../lib/jwt.ts";
import { hashPassword, verifyPassword } from "../lib/password.ts";
import { type RateLimit, takeToken } from "../lib/rate-limit.ts"; import { type RateLimit, takeToken } from "../lib/rate-limit.ts";
import { import {
type AuthContext, type AuthContext,
@@ -106,15 +104,21 @@ import {
getFollowedUsersByUser, getFollowedUsersByUser,
} from "../services/follow-service.ts"; } from "../services/follow-service.ts";
/** The router serving the endpoints the module doc lists. */ // Hashed once, at startup, for the sign-ins of usernames that don't exist.
const UNKNOWN_USER_HASH: Promise<string> = hashPassword(crypto.randomUUID());
/**
* Mounted by `api/main.ts` behind the error, Open Graph and CORS middleware.
* An `APIException` thrown by a handler becomes the JSON error response there.
*/
const router: Router = new Router({ prefix: "/api/users" }); const router: Router = new Router({ prefix: "/api/users" });
/** Login attempts per account: a burst of 10, then one every 30s. */ /** Allows a burst of 10 login attempts per account, then one every 30s. */
const LOGIN_LIMIT: RateLimit = { capacity: 10, refill: 1 / 30 }; const LOGIN_LIMIT: RateLimit = { capacity: 10, refill: 1 / 30 };
/** /**
* Issues `user` a session token, plus the media-only token that goes in file * Returns a new session token and media token for `user`, as sign-up, sign-in
* and thumbnail URLs (see {@link createMediaToken}). * and a password change answer them.
*/ */
async function issueTokens(user: User): Promise<AuthTokens> { async function issueTokens(user: User): Promise<AuthTokens> {
return { return {
@@ -132,14 +136,13 @@ router.post("/register", async (ctx) => {
if (!isRegisterUserRequest(body)) { if (!isRegisterUserRequest(body)) {
throw new APIException( throw new APIException(
APIErrorCode.VALIDATION_ERROR,
400, 400,
validateRegisterUserRequest(body) ?? "Invalid request", validateRegisterUserRequest(body) ?? "Invalid request",
); );
} }
// Claim the invite up front (throws 404/409 if bad) and hand it back if the // Claimed before the async account creation, so that one invite can't open
// account can't be created, so one invite can't open two accounts. // two accounts, and handed back if the creation fails.
const inviterId = claimInvite(body.inviteToken); const inviterId = claimInvite(body.inviteToken);
let user; let user;
@@ -150,7 +153,7 @@ router.post("/register", async (ctx) => {
throw err; throw err;
} }
// Send welcome email (fire-and-forget) // Not awaited: a slow or failing SMTP server mustn't hold up the sign-up.
if (isEmailEnabled()) { if (isEmailEnabled()) {
const emailMarkdown = WELCOME_EMAIL_BODY const emailMarkdown = WELCOME_EMAIL_BODY
.replaceAll("{{username}}", user.username) .replaceAll("{{username}}", user.username)
@@ -176,18 +179,13 @@ router.post("/login", async (ctx) => {
const body: unknown = await ctx.request.body.json(); const body: unknown = await ctx.request.body.json();
if (!isLoginUserRequest(body)) { if (!isLoginUserRequest(body)) {
throw new APIException( throw new APIException(400, "Invalid request");
APIErrorCode.VALIDATION_ERROR,
400,
"Invalid request",
);
} }
// Per account, so guessing one user's password is slow however many // Per account, so guessing one user's password is slow however many
// connections it comes from. // connections it comes from.
if (!takeToken(`login:${body.username.toLowerCase()}`, LOGIN_LIMIT)) { if (!takeToken(`login:${body.username.toLowerCase()}`, LOGIN_LIMIT)) {
throw new APIException( throw new APIException(
APIErrorCode.VALIDATION_ERROR,
429, 429,
"Too many login attempts — try again in a minute", "Too many login attempts — try again in a minute",
); );
@@ -197,21 +195,16 @@ router.post("/login", async (ctx) => {
try { try {
user = getUserByUsername(body.username); user = getUserByUsername(body.username);
} catch { } catch {
// Same answer as a wrong password: don't confirm which usernames exist. user = undefined;
throw new APIException(
APIErrorCode.VALIDATION_ERROR,
401,
"Invalid username or password",
);
} }
// An unknown username gets the same answer as a wrong password, after the
const valid = await verifyPassword(body.password, user.passwordHash); // same work, so neither the message nor the timing tells which exist.
if (!valid) { const valid = await verifyPassword(
throw new APIException( body.password,
APIErrorCode.VALIDATION_ERROR, user?.passwordHash ?? await UNKNOWN_USER_HASH,
401,
"Invalid username or password",
); );
if (!user || !valid) {
throw new APIException(401, "Invalid username or password");
} }
const responseBody: APIResponse<AuthResponse> = { const responseBody: APIResponse<AuthResponse> = {
@@ -222,29 +215,11 @@ router.post("/login", async (ctx) => {
}); });
router.get("/me", authMiddleware, (ctx: AuthContext) => { router.get("/me", authMiddleware, (ctx: AuthContext) => {
try {
if (!ctx.state.user) {
throw new APIException(
APIErrorCode.UNAUTHORIZED,
401,
"Not authenticated",
);
}
const responseBody: APIResponse<SelfUser> = { const responseBody: APIResponse<SelfUser> = {
success: true, success: true,
data: toSelfUser(getUserById(ctx.state.user.userId)), data: toSelfUser(getUserById(ctx.state.user.userId)),
}; };
ctx.response.body = responseBody; ctx.response.body = responseBody;
} catch (err) {
console.error(err);
throw new APIException(
APIErrorCode.SERVER_ERROR,
500,
"Failed to fetch user profile",
);
}
}); });
router.get("/me/media-token", authMiddleware, async (ctx: AuthContext) => { router.get("/me/media-token", authMiddleware, async (ctx: AuthContext) => {
@@ -255,6 +230,8 @@ router.get("/me/media-token", authMiddleware, async (ctx: AuthContext) => {
ctx.response.body = responseBody; ctx.response.body = responseBody;
}); });
// Answers the same whatever happens, so it can't tell which addresses have an
// account.
router.post("/request-password-reset", async (ctx) => { router.post("/request-password-reset", async (ctx) => {
const body: unknown = await ctx.request.body.json(); const body: unknown = await ctx.request.body.json();
const email = fieldOf(body, "email"); const email = fieldOf(body, "email");
@@ -273,11 +250,7 @@ router.post("/reset-password", async (ctx) => {
const token = fieldOf(body, "token"); const token = fieldOf(body, "token");
const newPassword = fieldOf(body, "newPassword"); const newPassword = fieldOf(body, "newPassword");
if (typeof token !== "string" || typeof newPassword !== "string") { if (typeof token !== "string" || typeof newPassword !== "string") {
throw new APIException( throw new APIException(400, "Invalid request");
APIErrorCode.VALIDATION_ERROR,
400,
"Invalid request",
);
} }
await resetPassword(token, newPassword); await resetPassword(token, newPassword);
const responseBody: APIResponse<null> = { success: true, data: null }; const responseBody: APIResponse<null> = { success: true, data: null };
@@ -290,11 +263,7 @@ router.post("/me/change-password", authMiddleware, async (ctx: AuthContext) => {
const newPassword = fieldOf(body, "newPassword"); const newPassword = fieldOf(body, "newPassword");
if (typeof currentPassword !== "string" || typeof newPassword !== "string") { if (typeof currentPassword !== "string" || typeof newPassword !== "string") {
throw new APIException( throw new APIException(400, "Invalid request");
APIErrorCode.VALIDATION_ERROR,
400,
"Invalid request",
);
} }
if ( if (
@@ -302,7 +271,6 @@ router.post("/me/change-password", authMiddleware, async (ctx: AuthContext) => {
newPassword.length > VALIDATION.PASSWORD_MAX newPassword.length > VALIDATION.PASSWORD_MAX
) { ) {
throw new APIException( throw new APIException(
APIErrorCode.VALIDATION_ERROR,
400, 400,
`Password must be ${VALIDATION.PASSWORD_MIN}–${VALIDATION.PASSWORD_MAX} characters`, `Password must be ${VALIDATION.PASSWORD_MIN}–${VALIDATION.PASSWORD_MAX} characters`,
); );
@@ -311,11 +279,7 @@ router.post("/me/change-password", authMiddleware, async (ctx: AuthContext) => {
const user = getUserById(ctx.state.user.userId); const user = getUserById(ctx.state.user.userId);
const valid = await verifyPassword(currentPassword, user.passwordHash); const valid = await verifyPassword(currentPassword, user.passwordHash);
if (!valid) { if (!valid) {
throw new APIException( throw new APIException(403, "Current password is incorrect");
APIErrorCode.VALIDATION_ERROR,
401,
"Current password is incorrect",
);
} }
// Changing the password revokes every token issued before it, this // Changing the password revokes every token issued before it, this
@@ -333,16 +297,11 @@ router.post("/me/change-password", authMiddleware, async (ctx: AuthContext) => {
router.patch("/me", authMiddleware, async (ctx: AuthContext) => { router.patch("/me", authMiddleware, async (ctx: AuthContext) => {
const body: unknown = await ctx.request.body.json(); const body: unknown = await ctx.request.body.json();
if (!isUpdateUserRequest(body)) { if (!isUpdateUserRequest(body)) {
throw new APIException( throw new APIException(400, "Invalid request");
APIErrorCode.VALIDATION_ERROR,
400,
"Invalid request",
);
} }
// Only the profile fields. The role is assigned by an admin, the password // Only the profile fields: a stolen token alone must not change the role,
// changes through /me/change-password (which asks for the current one), and // the password (which needs the current one) or the username (which is
// the username is fixed — a token alone must not be able to change any of // fixed).
// them.
const updated = await updateUser(ctx.state.user.userId, { const updated = await updateUser(ctx.state.user.userId, {
description: body.description, description: body.description,
email: body.email, email: body.email,
@@ -364,20 +323,11 @@ router.patch(
const body: unknown = await ctx.request.body.json(); const body: unknown = await ctx.request.body.json();
const role = fieldOf(body, "role"); const role = fieldOf(body, "role");
if (!isRole(role)) { if (!isRole(role)) {
throw new APIException( throw new APIException(400, "Invalid role");
APIErrorCode.VALIDATION_ERROR,
400,
"Invalid role",
);
} }
// Disallow changing your own role: prevents an admin from accidentally // So that an admin can't demote themselves by mistake and leave no admin.
// locking themselves out, and self-promotion is meaningless anyway.
if (ctx.params.userId === ctx.state.user.userId) { if (ctx.params.userId === ctx.state.user.userId) {
throw new APIException( throw new APIException(400, "Cannot change your own role");
APIErrorCode.VALIDATION_ERROR,
400,
"Cannot change your own role",
);
} }
const updated = await updateUser(ctx.params.userId, { role }); const updated = await updateUser(ctx.params.userId, { role });
const publicUser = toPublicUser(updated); const publicUser = toPublicUser(updated);

View File

@@ -1,11 +1,12 @@
/** /**
* The WebSocket route, and the handling of what clients send over it. * Opens the WebSocket that `WSProvider` keeps for each tab, and handles what
* the client sends over it. Voting, comment likes and every chat write go
* through here, not through HTTP. The messages are typed in `shared/ws.ts`,
* and `services/ws-service.ts` sends the broadcasts.
* *
* - `GET /ws?token=` — public; a session `token` signs the socket in, and an * - `GET /ws?token=`, public. A valid session `token` signs the socket in, and
* invalid one leaves it anonymous. Opens the socket that carries presence, * an invalid one leaves it anonymous. Answers 403 to an origin not in
* live updates, votes, comment likes and the chat (see * {@link ALLOWED_ORIGINS}, and 426 to a request that isn't an upgrade.
* `ClientToServerMessage` and `ServerToClientMessage` in shared/ws.ts); 403
* from an origin not in {@link ALLOWED_ORIGINS}.
* *
* @module * @module
*/ */
@@ -62,10 +63,17 @@ import type {
} from "../../shared/ws.ts"; } from "../../shared/ws.ts";
import type { User } from "../model/user.ts"; import type { User } from "../model/user.ts";
/** The router serving the endpoints the module doc lists. */ /**
* Mounted by `api/main.ts` behind the error, Open Graph and CORS middleware.
* An `APIException` thrown by a handler becomes the JSON error response there.
*/
const router: Router = new Router(); const router: Router = new Router();
/** Checks whether a socket may be opened from `origin`. */ /**
* Checks whether a socket may be opened from `origin`. Browsers don't apply
* CORS to WebSockets, so this is what stops another site from opening one with
* its visitors' tokens.
*/
function isAllowedOrigin(origin: string): boolean { function isAllowedOrigin(origin: string): boolean {
return ALLOWED_ORIGINS.includes(origin); return ALLOWED_ORIGINS.includes(origin);
} }
@@ -82,9 +90,8 @@ router.get("/ws", async (ctx) => {
return; return;
} }
// authenticate() refuses a token whose user is gone or which predates a // A revoked token (deleted user, changed password) opens an anonymous
// password change, and reads the live role from the DB. Such a socket is // socket. The client signs out on its next API call, which gets a 401.
// simply anonymous; the client's next API call gets the 401.
const token = ctx.request.url.searchParams.get("token"); const token = ctx.request.url.searchParams.get("token");
const authPayload = token ? await authenticate(token) : null; const authPayload = token ? await authenticate(token) : null;
let user: User | undefined; let user: User | undefined;
@@ -104,9 +111,11 @@ router.get("/ws", async (ctx) => {
avatarMime: user?.avatarMime, avatarMime: user?.avatarMime,
}; };
// Use addEventListener — more reliable than onopen= with Deno.serve // addEventListener, since `onopen =` proved unreliable under Deno.serve.
socket.addEventListener("open", () => { socket.addEventListener("open", () => {
register(client); register(client);
// The new socket is registered, so it gets this presence_update too,
// before its welcome.
broadcastPresence(); broadcastPresence();
try { try {
@@ -141,12 +150,16 @@ router.get("/ws", async (ctx) => {
socket.addEventListener("close", () => { socket.addEventListener("close", () => {
unregister(client); unregister(client);
broadcastPresence(); broadcastPresence();
// If they were mid-compose, drop them from the typing set right away. // The sweep only sees registered clients, so it would never clear this one.
if (client.typingAt) broadcastTyping(); if (client.typingAt) broadcastTyping();
}); });
}); });
/** Parses one message from a client and dispatches it by `type`. */ /**
* Parses one message from a client and dispatches it by `type`. A message that
* isn't JSON, or whose `type` is unknown, is ignored. Its fields are still
* untrusted, so each handler checks their types.
*/
function handleMessage(client: WsClient, data: unknown): void { function handleMessage(client: WsClient, data: unknown): void {
if (typeof data !== "string") return; if (typeof data !== "string") return;
let parsed: unknown; let parsed: unknown;
@@ -155,8 +168,6 @@ function handleMessage(client: WsClient, data: unknown): void {
} catch { } catch {
return; return;
} }
// Anything but an object with a string `type` (`null`, a number, an array)
// is ignored rather than read from.
if ( if (
typeof parsed !== "object" || parsed === null || typeof parsed !== "object" || parsed === null ||
typeof (parsed as { type?: unknown }).type !== "string" typeof (parsed as { type?: unknown }).type !== "string"
@@ -187,7 +198,7 @@ function handleMessage(client: WsClient, data: unknown): void {
handleChatSend(client, msg.body, msg.replyToId, msg.nonce); handleChatSend(client, msg.body, msg.replyToId, msg.nonce);
break; break;
case "chat_typing": case "chat_typing":
// Typing is cosmetic: past the limit it's just dropped. // Typing is cosmetic: past the limit it's dropped, with no error.
if ( if (
client.userId && client.userId &&
takeToken(`typing:${client.userId}`, TYPING_LIMIT) takeToken(`typing:${client.userId}`, TYPING_LIMIT)
@@ -210,14 +221,15 @@ function handleMessage(client: WsClient, data: unknown): void {
} }
} }
/** Sends a message only while the socket is open: `send` on a closing socket throws. */ /** Sends `data` unless the socket is closing, where `send` would throw. */
function send(socket: WebSocket, data: ServerToClientMessage): void { function send(socket: WebSocket, data: ServerToClientMessage): void {
if (socket.readyState === WebSocket.OPEN) socket.send(JSON.stringify(data)); if (socket.readyState === WebSocket.OPEN) socket.send(JSON.stringify(data));
} }
/** /**
* Casts or removes the client's upvote, acknowledges it, and broadcasts the new * Handles `vote_cast` and `vote_remove`. It acks the voter with `vote_ack` and
* count. * broadcasts `votes_update`. A failure goes back as an `error` naming the dump
* and the action, so the client can undo its optimistic vote at once.
*/ */
function handleVote( function handleVote(
client: WsClient, client: WsClient,
@@ -244,8 +256,7 @@ function handleVote(
voteCount, voteCount,
}); });
// Only a private dump's owner can vote on it, so they're its whole // Only a private dump's owner can see it, and so vote on it.
// audience.
broadcastVoteUpdate( broadcastVoteUpdate(
dumpId, dumpId,
voteCount, voteCount,
@@ -260,8 +271,8 @@ function handleVote(
} }
/** /**
* Casts or removes the client's like on a comment, acknowledges it, and * Handles `comment_like_cast` and `comment_like_remove`, as
* broadcasts the new count. * {@link handleVote} does votes.
*/ */
function handleCommentLike( function handleCommentLike(
client: WsClient, client: WsClient,
@@ -302,20 +313,23 @@ function handleCommentLike(
} }
/** /**
* Chat writes per user. Sending, editing and deleting share one bucket: a * Limits each user's chat writes, sends, edits and deletes together, to a burst
* burst of 8, then one every 1.5s — room for a quick back-and-forth, not for a * of 8, then one every 1.5 s. The buckets are per user, so more tabs don't buy
* flood. * more.
*/ */
const CHAT_WRITE_LIMIT: RateLimit = { capacity: 8, refill: 1 / 1.5 }; const CHAT_WRITE_LIMIT: RateLimit = { capacity: 8, refill: 1 / 1.5 };
/** Typing signals per user. */
const TYPING_LIMIT: RateLimit = { capacity: 10, refill: 1 }; const TYPING_LIMIT: RateLimit = { capacity: 10, refill: 1 };
/** /**
* Read markers per user. They are cheap for the client to send often; past * Limits each user's `chat_read`. One over the limit is dropped, and the next
* this they're dropped, and the next one catches up. * one catches up, since a marker only moves forward.
*/ */
const CHAT_READ_LIMIT: RateLimit = { capacity: 10, refill: 1 }; const CHAT_READ_LIMIT: RateLimit = { capacity: 10, refill: 1 };
/** Reports a failed chat write to the client, with the nonce of its pending copy. */ /**
* Sends a chat-scoped `error`. With the `nonce` of a `chat_send`, the chatbox
* marks that pending message failed. Without one, it shows `message` as a
* notice.
*/
function sendChatError( function sendChatError(
client: WsClient, client: WsClient,
message: string, message: string,
@@ -325,8 +339,8 @@ function sendChatError(
} }
/** /**
* Validates the common preconditions of a chat write: signed in and under the * Returns the id of the user behind a chat write, or undefined after sending
* rate limit. Returns the user id, or undefined after reporting why not. * the error when they are signed out or over {@link CHAT_WRITE_LIMIT}.
*/ */
function chatWriter(client: WsClient, nonce?: string): string | undefined { function chatWriter(client: WsClient, nonce?: string): string | undefined {
if (!client.userId) { if (!client.userId) {
@@ -344,7 +358,9 @@ function chatWriter(client: WsClient, nonce?: string): string | undefined {
return client.userId; return client.userId;
} }
/** Returns `value` when it is a string of at most `maxLength`, else undefined. */ /**
* Returns `value` when it is a string of at most `maxLength`, else undefined.
*/
function optionalString(value: unknown, maxLength: number): string | undefined { function optionalString(value: unknown, maxLength: number): string | undefined {
return typeof value === "string" && value.length <= maxLength return typeof value === "string" && value.length <= maxLength
? value ? value
@@ -352,8 +368,8 @@ function optionalString(value: unknown, maxLength: number): string | undefined {
} }
/** /**
* Posts a chat message, broadcasts it, and notifies the @mentioned users not * Handles `chat_send`. It stores the message, broadcasts it, and notifies the
* already reading the chat. * @mentioned users who aren't reading the chat.
*/ */
function handleChatSend( function handleChatSend(
client: WsClient, client: WsClient,
@@ -375,14 +391,10 @@ function handleChatSend(
body, body,
optionalString(replyToId, 64), optionalString(replyToId, 64),
); );
// Sending a message ends composition — clear the typing flag before the // Before the broadcast, so the typing indicator doesn't linger beside the
// message broadcast so the indicator doesn't linger next to it. // message.
setClientTyping(client, false); setClientTyping(client, false);
// Broadcast to everyone including the sender, whose pending copy the nonce
// retires.
broadcastChatMessage(message, nonce); broadcastChatMessage(message, nonce);
// Skip mention notifications for users who already have the chat open —
// they've seen the message live.
notifyMentions( notifyMentions(
userId, userId,
message.body, message.body,
@@ -402,10 +414,9 @@ function handleChatSend(
} }
/** /**
* Checks whether the client's user may moderate the chat. Reads the role from * Checks whether the client's user may edit and delete others' chat messages.
* the DB on every check rather than trusting the one captured at connect: a * The role is read from the database, not from the socket's connect, so that a
* demoted moderator loses the power right away, not whenever their socket next * demoted moderator loses the right at once.
* reconnects.
*/ */
function canModerateChat(client: WsClient): boolean { function canModerateChat(client: WsClient): boolean {
if (!client.userId) return false; if (!client.userId) return false;
@@ -417,7 +428,10 @@ function canModerateChat(client: WsClient): boolean {
return !!client.role && hasPermission(client.role, "chat:moderate"); return !!client.role && hasPermission(client.role, "chat:moderate");
} }
/** Edits a chat message, broadcasts it, and notifies newly @mentioned users. */ /**
* Handles `chat_edit`. It stores the new body, broadcasts it, and notifies, in
* the author's name, the users the edit newly mentions.
*/
function handleChatEdit(client: WsClient, id: unknown, body: unknown): void { function handleChatEdit(client: WsClient, id: unknown, body: unknown): void {
const userId = chatWriter(client); const userId = chatWriter(client);
if (!userId) return; if (!userId) return;
@@ -434,10 +448,11 @@ function handleChatEdit(client: WsClient, id: unknown, body: unknown): void {
canModerateChat(client), canModerateChat(client),
); );
broadcastChatMessageUpdated(message); broadcastChatMessageUpdated(message);
// A newly added @mention in an edit should still notify (deduped per // Mentions already notified for this message are deduplicated, so only
// message+user), skipping anyone already reading the chat. // the new ones notify. A moderator's edit keeps the author's words, so the
// author mentions.
notifyMentions( notifyMentions(
userId, message.userId,
message.body, message.body,
"chat", "chat",
message.id, message.id,
@@ -453,7 +468,7 @@ function handleChatEdit(client: WsClient, id: unknown, body: unknown): void {
} }
} }
/** Deletes a chat message and broadcasts it. */ /** Handles `chat_delete`. It deletes the message and broadcasts its id. */
function handleChatDelete(client: WsClient, id: unknown): void { function handleChatDelete(client: WsClient, id: unknown): void {
const userId = chatWriter(client); const userId = chatWriter(client);
if (!userId) return; if (!userId) return;
@@ -474,8 +489,9 @@ function handleChatDelete(client: WsClient, id: unknown): void {
} }
/** /**
* Moves the user's read marker and tells all their sockets, so every tab and * Handles `chat_read`. It moves the user's read marker and sends
* device drops its badge together. * `chat_read_update` to all their sockets, so every tab and device updates its
* badge together.
*/ */
function handleChatRead(client: WsClient, at: unknown): void { function handleChatRead(client: WsClient, at: unknown): void {
if (!client.userId || typeof at !== "string") return; if (!client.userId || typeof at !== "string") return;
@@ -488,7 +504,8 @@ function handleChatRead(client: WsClient, at: unknown): void {
unread: state.unread, unread: state.unread,
}); });
} catch { } catch {
// A malformed marker changes nothing; there's nothing to report. // A malformed marker changes nothing, and the client doesn't wait for an
// answer.
} }
} }

View File

@@ -1,8 +1,10 @@
/** /**
* The YouTube route, for the player's tracklist. * Serves the titles and lengths of a YouTube playlist's entries, which the
* global player lists beside the embed. On any failure it shows the embed
* without a tracklist.
* *
* - `GET /api/youtube/playlist?list=` — public. Resolves a YouTube playlist id * - `GET /api/youtube/playlist?list=` — public. Answers the playlist's
* to its tracklist (titles and lengths). * tracklist.
* *
* @module * @module
*/ */
@@ -12,11 +14,13 @@ import type { APIResponse } from "../../shared/api.ts";
import { resolveYouTubePlaylist } from "../remote/youtube-playlist.ts"; import { resolveYouTubePlaylist } from "../remote/youtube-playlist.ts";
import type { YouTubePlaylist } from "../../shared/player.ts"; import type { YouTubePlaylist } from "../../shared/player.ts";
/** The router serving the endpoints the module doc lists. */ /**
* Mounted by `api/main.ts` behind the error, Open Graph and CORS middleware.
* An `APIException` thrown by a handler becomes the JSON error response there.
*/
const youtubeRouter: Router = new Router(); const youtubeRouter: Router = new Router();
// Public, like /api/bandcamp/tracks: it only reads a public page. The client // Public, like /api/bandcamp/tracks: it only reads a public page.
// treats any failure as "show the embed without a tracklist".
youtubeRouter.get("/api/youtube/playlist", async (ctx) => { youtubeRouter.get("/api/youtube/playlist", async (ctx) => {
const listId = ctx.request.url.searchParams.get("list") ?? ""; const listId = ctx.request.url.searchParams.get("list") ?? "";
const responseBody: APIResponse<YouTubePlaylist> = { const responseBody: APIResponse<YouTubePlaylist> = {

View File

@@ -1,12 +1,10 @@
/** /**
* Generates the favicon set and manifest from the runtime `GERBEUR_SITE_EMOJI`. * Draws the favicons and writes the manifest from {@link SITE_EMOJI} and
* Runs before the server starts, so changing the emoji only requires a restart * {@link OG_SITE_NAME}, so changing them needs a restart and no rebuild.
* — no rebuild. * `deno task icons <dir>` runs it before the server starts, into the directory
* * the frontend is served from, relative to the working directory. That is
* The output directory is passed as the first argument (relative to cwd), and * dist/ for `start` and `serve`, and public/, which Vite serves, for `dev`.
* defaults to `dist`: * It does nothing when the directory doesn't exist.
* - prod (`deno task start` / `serve`): `dist` — what the Oak server serves
* - dev (`deno task dev`): `public` — what the Vite server serves
* *
* @module * @module
*/ */

View File

@@ -1,59 +1,131 @@
/** /**
* Images pasted into Markdown bodies (chat, comments, descriptions): their * Records the images pasted into Markdown bodies, and binds each one to the
* records, and binding them to the resource whose body links them, so the * resource whose body links it. `routes/attachments.ts` stores and serves the
* startup purge keeps them. * files; the services that save a body call {@link linkAttachments}.
* *
* @module * @module
*/ */
import { db, transaction } from "../db/connection.ts"; import { db, transaction } from "../db/connection.ts";
import { APIErrorCode } from "../../shared/api.ts";
import { APIException } from "../lib/api-exception.ts"; import { APIException } from "../lib/api-exception.ts";
/** Records the upload of attachment `id`, not yet bound to any resource. */ /**
* Records attachment `id` once `POST /api/attachments` has written its file.
* Until a saved body links it, the startup purge deletes it after
* `ORPHANED_ATTACHMENTS_RETENTION_HOURS`.
*/
export function createAttachment(id: string, mime: string): void { export function createAttachment(id: string, mime: string): void {
db.prepare( db.prepare(
"INSERT INTO attachments (id, mime, created_at) VALUES (?, ?, ?)", "INSERT INTO attachments (id, mime, created_at) VALUES (?, ?, ?)",
).run(id, mime, new Date().toISOString()); ).run(id, mime, new Date().toISOString());
} }
/** The kinds of resource whose saved text can link an attachment. */
export type AttachmentResourceType =
| "dump"
| "comment"
| "playlist"
| "chat"
| "user";
/** /**
* Returns attachment `attachmentId`'s MIME type. * Returned to `GET /api/attachments/:attachmentId` by
* * {@link getAttachmentAccess}. `ownerId` is set when only that user may see
* @throws {APIException} 404 when there is no such attachment. * the image, because the resource linking it is private.
*/ */
export function getAttachment(attachmentId: string): { mime: string } { export interface AttachmentAccess {
const row = db.prepare( mime: string;
"SELECT mime FROM attachments WHERE id = ?", ownerId?: string;
).get(attachmentId) as { mime: string } | undefined; }
if (!row) { /**
throw new APIException( * Returns attachment `attachmentId`'s type and who may see it. An attachment
APIErrorCode.NOT_FOUND, * nothing links yet is public, so the editor can show it before the text is
404, * saved; its id is unguessable. One linked by a private dump or playlist, or
"Attachment not found", * by a comment on a private dump, is its owner's.
); *
} * @throws {APIException} 404 when there is no such attachment, or when the
* resource linking it has been deleted.
return { mime: row.mime }; */
export function getAttachmentAccess(attachmentId: string): AttachmentAccess {
const row = db.prepare(
"SELECT mime, resource_type, resource_id FROM attachments WHERE id = ?",
).get(attachmentId) as
| { mime: string; resource_type: string | null; resource_id: string | null }
| undefined;
if (!row) throw new APIException(404, "Attachment not found");
if (!row.resource_id) return { mime: row.mime };
const owner = resourceOwner(row.resource_type, row.resource_id);
if (owner === undefined) {
throw new APIException(404, "Attachment not found");
}
return owner ? { mime: row.mime, ownerId: owner } : { mime: row.mime };
}
/**
* Returns the owner of a private resource, null for a public one, and
* undefined for one that no longer exists.
*/
function resourceOwner(
type: string | null,
id: string,
): string | null | undefined {
switch (type) {
case "dump": {
const dump = db.prepare(
"SELECT user_id, is_private FROM dumps WHERE id = ?",
).get(id) as { user_id: string; is_private: number } | undefined;
return dump && (dump.is_private ? dump.user_id : null);
}
case "comment": {
const comment = db.prepare(
`SELECT d.user_id, d.is_private FROM comments c
JOIN dumps d ON d.id = c.dump_id WHERE c.id = ? AND c.deleted = 0`,
).get(id) as { user_id: string; is_private: number } | undefined;
return comment && (comment.is_private ? comment.user_id : null);
}
case "playlist": {
const playlist = db.prepare(
"SELECT user_id, is_public FROM playlists WHERE id = ?",
).get(id) as { user_id: string; is_public: number } | undefined;
return playlist && (playlist.is_public ? null : playlist.user_id);
}
case "chat":
return db.prepare("SELECT 1 FROM chat_messages WHERE id = ?").get(id)
? null
: undefined;
case "user":
return db.prepare("SELECT 1 FROM users WHERE id = ?").get(id)
? null
: undefined;
default:
return undefined;
}
} }
/** An attachment URL in a Markdown body, capturing its id. */
const ATTACHMENT_RE = const ATTACHMENT_RE =
/\/api\/attachments\/([0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12})/g; /\/api\/attachments\/([0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12})/g;
/** /**
* Binds the unowned attachments whose URLs appear in `text` to * Binds the attachments whose URLs appear in `text` to the resource that saved
* `resourceId`. Only updates rows where resource_id IS NULL so a user cannot * it, so the startup purge keeps them as long as that resource exists, and
* claim attachments that already belong to another resource. * they are seen by whoever may see it. It is called whenever a dump, comment,
* chat message, playlist or profile description is saved. An attachment
* already bound stays with its resource, so pasting another's URL claims
* nothing, and removing the link from a body never unbinds it.
*/ */
export function linkAttachments(text: string, resourceId: string): void { export function linkAttachments(
text: string,
resourceType: AttachmentResourceType,
resourceId: string,
): void {
const ids = [...text.matchAll(ATTACHMENT_RE)].map((m) => m[1]); const ids = [...text.matchAll(ATTACHMENT_RE)].map((m) => m[1]);
if (ids.length === 0) return; if (ids.length === 0) return;
const update = db.prepare( const update = db.prepare(
"UPDATE attachments SET resource_id = ? WHERE id = ? AND resource_id IS NULL", "UPDATE attachments SET resource_type = ?, resource_id = ? WHERE id = ? AND resource_id IS NULL",
); );
transaction(() => { transaction(() => {
for (const id of ids) update.run(resourceId, id); for (const id of ids) update.run(resourceType, resourceId, id);
}); });
} }

View File

@@ -1,6 +1,7 @@
/** /**
* Backlinks between dumps: which dumps a dump's description and comments link * Records which dumps a dump's note and comments link to, and finds the dumps
* to, and the dumps related to one through them. * related to one through those links. The dump and comment services keep the
* links up to date, and the dump page lists the related dumps.
* *
* @module * @module
*/ */
@@ -13,32 +14,23 @@ import {
isDumpRow, isDumpRow,
} from "../model/dump.ts"; } from "../model/dump.ts";
import { UUID_RE } from "../lib/slugify.ts"; import { UUID_RE } from "../lib/slugify.ts";
import { canonicalizeUrl } from "../lib/canonical-url.ts";
import { parseRows } from "../model/rows.ts";
/** /**
* An http(s) URL in free text — stops at whitespace or markdown-link * Matches an http(s) URL in Markdown. It stops at the delimiters of
* delimiters (so `[text](url)` and `<url>` don't swallow the closing char). * `[text](url)` and `<url>`, which would otherwise end up in the URL.
*/ */
const URL_RE = /https?:\/\/[^\s)\]<>"']+/g; const URL_RE = /https?:\/\/[^\s)\]<>"']+/g;
/** /**
* A link to another dump's page on this same site (e.g. pasted from * Matches the path of a dump's page, `/dumps/{slug-or-id}`, and ignores what
* the address bar) — host-agnostic, since the configured public URL can differ * follows it, so an `/edit` or `#comment-x` link counts too. The host isn't
* between dev/prod/reverse-proxy setups. Matches `/dumps/{slug-or-id}`, * checked, since the site's public URL differs between setups.
* ignoring any trailing path/query/hash (so `/edit` or `#comment-x` links
* still resolve to the same target dump).
*/ */
const DUMP_PATH_RE = /^\/dumps\/([^/?#]+)/; const DUMP_PATH_RE = /^\/dumps\/([^/?#]+)/;
/** Returns `url` as the URL parser normalises it, or null when it isn't one. */ /** Returns the id of the dump with id or slug `idOrSlug`, or null. */
function canonicalize(url: string): string | null {
try {
return new URL(url).toString();
} catch {
return null;
}
}
/** Resolves a same-site `/dumps/{slug-or-id}` path segment to a dump id. */
function resolveDumpPathId(idOrSlug: string): string | null { function resolveDumpPathId(idOrSlug: string): string | null {
const row = UUID_RE.test(idOrSlug) const row = UUID_RE.test(idOrSlug)
? db.prepare(`SELECT id FROM dumps WHERE id = ?;`).get(idOrSlug) ? db.prepare(`SELECT id FROM dumps WHERE id = ?;`).get(idOrSlug)
@@ -47,11 +39,11 @@ function resolveDumpPathId(idOrSlug: string): string | null {
} }
/** /**
* Re-derives the backlinks contributed by one text source (a dump's own * Replaces the backlinks from one text, a dump's note (`dump_comment`) or one
* `comment` field, or a single comment's `body`) and persist them. Always * of its comments (`comment`), with the links its current `text` holds. A link
* deletes the source's previous contribution first, so editing/clearing the * counts when it is a dump's page on this site, or when it is the URL a dump
* text correctly removes stale links — unlike `linkAttachments`/`notifyMentions`, * was posted with, compared by `canonicalizeUrl` as reposts are. The dump and comment services call it on every save, and with
* which are purely additive. * an empty `text` when a comment is deleted.
*/ */
export function relinkBacklinks( export function relinkBacklinks(
sourceType: "dump_comment" | "comment", sourceType: "dump_comment" | "comment",
@@ -63,11 +55,9 @@ export function relinkBacklinks(
const externalUrls = new Set<string>(); const externalUrls = new Set<string>();
for (const match of text.matchAll(URL_RE)) { for (const match of text.matchAll(URL_RE)) {
const canonical = canonicalize(match[0]);
if (!canonical) continue;
let parsed: URL; let parsed: URL;
try { try {
parsed = new URL(canonical); parsed = new URL(match[0]);
} catch { } catch {
continue; continue;
} }
@@ -76,19 +66,20 @@ export function relinkBacklinks(
const resolved = resolveDumpPathId(decodeURIComponent(pathMatch[1])); const resolved = resolveDumpPathId(decodeURIComponent(pathMatch[1]));
if (resolved) targetIds.add(resolved); if (resolved) targetIds.add(resolved);
} else { } else {
externalUrls.add(canonical); const canonical = canonicalizeUrl(match[0]);
if (canonical) externalUrls.add(canonical);
} }
} }
if (externalUrls.size > 0) { if (externalUrls.size > 0) {
const placeholders = [...externalUrls].map(() => "?").join(", "); const placeholders = [...externalUrls].map(() => "?").join(", ");
const rows = db.prepare( const rows = db.prepare(
`SELECT id FROM dumps WHERE url IN (${placeholders});`, `SELECT id FROM dumps WHERE url_canonical IN (${placeholders});`,
).all(...externalUrls) as { id: string }[]; ).all(...externalUrls) as { id: string }[];
for (const row of rows) targetIds.add(row.id); for (const row of rows) targetIds.add(row.id);
} }
targetIds.delete(fromDumpId); // no self-links targetIds.delete(fromDumpId);
transaction(() => { transaction(() => {
db.prepare( db.prepare(
@@ -107,7 +98,8 @@ export function relinkBacklinks(
/** /**
* Returns the dumps linked to or from `dumpId`, newest first and at most 20, * Returns the dumps linked to or from `dumpId`, newest first and at most 20,
* without the private ones the requesting user can't see. * without the private ones the requesting user can't see. It doesn't check
* `dumpId` itself: `GET /api/dumps/:dumpId/related` does that first.
*/ */
export function getRelatedDumps( export function getRelatedDumps(
dumpId: string, dumpId: string,
@@ -122,5 +114,5 @@ export function getRelatedDumps(
ORDER BY created_at DESC LIMIT 20;`, ORDER BY created_at DESC LIMIT 20;`,
).all(dumpId, dumpId, requestingUserId ?? ""); ).all(dumpId, dumpId, requestingUserId ?? "");
return rows.filter(isDumpRow).map(dumpRowToApi); return parseRows(rows, isDumpRow, dumpRowToApi);
} }

View File

@@ -1,10 +1,11 @@
/** /**
* Categories: managing them, and which dumps belong to which. * Stores the categories and which dumps belong to them. The category routes
* manage and list them, the dump service files each dump it writes under its
* categories, and every service that lists dumps fills in their `categoryIds`.
* *
* @module * @module
*/ */
import { APIErrorCode } from "../../shared/api.ts";
import { APIException } from "../lib/api-exception.ts"; import { APIException } from "../lib/api-exception.ts";
import type { import type {
Category, Category,
@@ -22,41 +23,37 @@ import {
import { type Page, pageOf } from "../lib/pagination.ts"; import { type Page, pageOf } from "../lib/pagination.ts";
import { isReservedSlug, isValidSlugFormat } from "../lib/reserved-slugs.ts"; import { isReservedSlug, isValidSlugFormat } from "../lib/reserved-slugs.ts";
import { VALIDATION } from "../../shared/validation.ts"; import { VALIDATION } from "../../shared/validation.ts";
import { parseRows } from "../model/rows.ts";
/** Throws a 400 unless `slug` is well-formed, short enough, and not reserved. */ /**
* Throws a 400 unless `slug` is well-formed, short enough and not reserved,
* with a message for each case.
*/
function validateSlug(slug: string): void { function validateSlug(slug: string): void {
if (!isValidSlugFormat(slug)) { if (!isValidSlugFormat(slug)) {
throw new APIException( throw new APIException(
APIErrorCode.BAD_REQUEST,
400, 400,
"Slug must be lowercase letters, numbers and hyphens", "Slug must be lowercase letters, numbers and hyphens",
); );
} }
if (slug.length > VALIDATION.CATEGORY_SLUG_MAX) { if (slug.length > VALIDATION.CATEGORY_SLUG_MAX) {
throw new APIException( throw new APIException(
APIErrorCode.BAD_REQUEST,
400, 400,
`Slug must be at most ${VALIDATION.CATEGORY_SLUG_MAX} characters`, `Slug must be at most ${VALIDATION.CATEGORY_SLUG_MAX} characters`,
); );
} }
if (isReservedSlug(slug)) { if (isReservedSlug(slug)) {
throw new APIException( throw new APIException(400, `Slug "${slug}" is reserved`);
APIErrorCode.BAD_REQUEST,
400,
`Slug "${slug}" is reserved`,
);
} }
} }
/** Returns `name` trimmed, or throws a 400 when it is blank or too long. */
function requireName(name: string): string { function requireName(name: string): string {
const trimmed = name.trim(); const trimmed = name.trim();
if (!trimmed) { if (!trimmed) {
throw new APIException(APIErrorCode.BAD_REQUEST, 400, "Name is required"); throw new APIException(400, "Name is required");
} }
if (trimmed.length > VALIDATION.CATEGORY_NAME_MAX) { if (trimmed.length > VALIDATION.CATEGORY_NAME_MAX) {
throw new APIException( throw new APIException(
APIErrorCode.BAD_REQUEST,
400, 400,
`Name must be at most ${VALIDATION.CATEGORY_NAME_MAX} characters`, `Name must be at most ${VALIDATION.CATEGORY_NAME_MAX} characters`,
); );
@@ -65,55 +62,51 @@ function requireName(name: string): string {
} }
/** /**
* Rethrows a SQLite UNIQUE violation as a friendly 409 naming the offending * Rethrows a UNIQUE violation from SQLite as a 409 naming the slug or name
* field, and anything else as is. * that is taken, and any other error as is. SQLite's message names the
* column, so this matches on it.
*/ */
function uniqueConflict(err: unknown, slug: string, name: string): never { function uniqueConflict(err: unknown, slug: string, name: string): never {
const msg = err instanceof Error ? err.message : String(err); const msg = err instanceof Error ? err.message : String(err);
if (msg.includes("categories.name")) { if (msg.includes("categories.name")) {
throw new APIException( throw new APIException(409, `Name "${name}" is already in use`);
APIErrorCode.VALIDATION_ERROR,
409,
`Name "${name}" is already in use`,
);
} }
if (msg.includes("UNIQUE") || msg.includes("unique")) { if (msg.includes("UNIQUE") || msg.includes("unique")) {
throw new APIException( throw new APIException(409, `Slug "${slug}" is already in use`);
APIErrorCode.VALIDATION_ERROR,
409,
`Slug "${slug}" is already in use`,
);
} }
throw err as Error; throw err as Error;
} }
/** /**
* Returns category `id`. * Returns category `id`, for {@link updateCategory}.
* *
* @throws {APIException} 404 when there is no such category. * @throws {APIException} 404 when there is no such category.
*/ */
export function getCategoryById(id: string): Category { export function getCategoryById(id: string): Category {
const row = db.prepare(`SELECT * FROM categories WHERE id = ?;`).get(id); const row = db.prepare(`SELECT * FROM categories WHERE id = ?;`).get(id);
if (!row || !isCategoryRow(row)) { if (!row || !isCategoryRow(row)) {
throw new APIException(APIErrorCode.NOT_FOUND, 404, "Category not found"); throw new APIException(404, "Category not found");
} }
return categoryRowToApi(row); return categoryRowToApi(row);
} }
/** /**
* Returns the category at `slug`. * Returns the category at `slug`, for `GET /api/categories/:slug`.
* *
* @throws {APIException} 404 when there is no such category. * @throws {APIException} 404 when there is no such category.
*/ */
export function getCategoryBySlug(slug: string): Category { export function getCategoryBySlug(slug: string): Category {
const row = db.prepare(`SELECT * FROM categories WHERE slug = ?;`).get(slug); const row = db.prepare(`SELECT * FROM categories WHERE slug = ?;`).get(slug);
if (!row || !isCategoryRow(row)) { if (!row || !isCategoryRow(row)) {
throw new APIException(APIErrorCode.NOT_FOUND, 404, "Category not found"); throw new APIException(404, "Category not found");
} }
return categoryRowToApi(row); return categoryRowToApi(row);
} }
/** Returns the id of the category at `slug`, or undefined when there is none. */ /**
* Returns the id of the category at `slug`, or undefined when there is none.
* The follows routes use it for their `?category=` feed filter.
*/
export function getCategoryIdBySlug(slug: string): string | undefined { export function getCategoryIdBySlug(slug: string): string | undefined {
const row = db.prepare(`SELECT id FROM categories WHERE slug = ?;`).get(slug); const row = db.prepare(`SELECT id FROM categories WHERE slug = ?;`).get(slug);
return row && typeof (row as { id?: unknown }).id === "string" return row && typeof (row as { id?: unknown }).id === "string"
@@ -121,16 +114,19 @@ export function getCategoryIdBySlug(slug: string): string | undefined {
: undefined; : undefined;
} }
/** Returns every category, by position then slug. */ /**
* Returns every category, by position then slug, for `GET /api/categories`.
*/
export function listCategories(): Category[] { export function listCategories(): Category[] {
const rows = db.prepare( const rows = db.prepare(
`SELECT * FROM categories ORDER BY position ASC, slug ASC;`, `SELECT * FROM categories ORDER BY position ASC, slug ASC;`,
).all(); ).all();
return rows.filter(isCategoryRow).map(categoryRowToApi); return parseRows(rows, isCategoryRow, categoryRowToApi);
} }
/** /**
* Creates a category. * Creates a category, for `POST /api/categories`. The name is stored trimmed,
* and the position defaults to 0.
* *
* @throws {APIException} 400 when the slug or name is invalid, 409 when * @throws {APIException} 400 when the slug or name is invalid, 409 when
* either is taken. * either is taken.
@@ -166,7 +162,8 @@ export function createCategory(req: CreateCategoryRequest): Category {
} }
/** /**
* Applies `req` to category `id`. * Applies `req` to category `id`, for `PUT /api/categories/:id`, and returns
* the category as it now is. Fields `req` leaves out keep their value.
* *
* @throws {APIException} 404 when there is no such category, 400 when the new * @throws {APIException} 404 when there is no such category, 400 when the new
* slug or name is invalid, 409 when either is taken. * slug or name is invalid, 409 when either is taken.
@@ -211,21 +208,22 @@ export function updateCategory(
} }
/** /**
* Deletes category `id`; its dumps stay, outside it. * Deletes category `id`, for `DELETE /api/categories/:id`. Its dumps stay, and
* lose it from their `categoryIds`.
* *
* @throws {APIException} 404 when there is no such category. * @throws {APIException} 404 when there is no such category.
*/ */
export function deleteCategory(id: string): void { export function deleteCategory(id: string): void {
const result = db.prepare(`DELETE FROM categories WHERE id = ?;`).run(id); const result = db.prepare(`DELETE FROM categories WHERE id = ?;`).run(id);
if (result.changes === 0) { if (result.changes === 0) {
throw new APIException(APIErrorCode.NOT_FOUND, 404, "Category not found"); throw new APIException(404, "Category not found");
} }
} }
/** /**
* Checks that every id names an existing category. Callers that write * Checks that every id in `categoryIds` names a category. The dump service
* anything else first (a dump row, an uploaded file) run this before those * runs it before it writes a dump row or stores an uploaded file, so a bad id
* writes, so a bad id can't leave them half-done. * fails the request before anything is written.
* *
* @throws {APIException} 400 when an id names no category. * @throws {APIException} 400 when an id names no category.
*/ */
@@ -238,18 +236,15 @@ export function assertCategoriesExist(categoryIds: string[]): void {
`SELECT COUNT(*) as count FROM categories WHERE id IN (${placeholders});`, `SELECT COUNT(*) as count FROM categories WHERE id IN (${placeholders});`,
).get(...unique) as { count: number } | undefined; ).get(...unique) as { count: number } | undefined;
if ((found?.count ?? 0) !== unique.length) { if ((found?.count ?? 0) !== unique.length) {
throw new APIException( throw new APIException(400, "One or more categories do not exist");
APIErrorCode.BAD_REQUEST,
400,
"One or more categories do not exist",
);
} }
} }
} }
/** /**
* Replaces the set of categories a dump belongs to. An empty list clears all * Replaces the categories of dump `dumpId` with `categoryIds`, duplicates
* category assignments. * dropped. An empty list takes the dump out of every category. Called inside
* the dump service's transaction, it joins that one.
* *
* @throws {APIException} 400 when an id names no category. * @throws {APIException} 400 when an id names no category.
*/ */
@@ -271,7 +266,10 @@ export function setDumpCategories(
}); });
} }
/** Fills `categoryIds` on each dump with a single batched lookup. */ /**
* Sets `categoryIds` on each of `dumps` in place, in one query. The dump row
* doesn't carry them, so every service that returns dumps calls this first.
*/
export function attachCategoryIds(dumps: Dump[]): void { export function attachCategoryIds(dumps: Dump[]): void {
if (dumps.length === 0) return; if (dumps.length === 0) return;
const ids = dumps.map((d) => d.id); const ids = dumps.map((d) => d.id);
@@ -292,8 +290,9 @@ export function attachCategoryIds(dumps: Dump[]): void {
} }
/** /**
* Returns a page of the dumps in the category at `slug`, newest first, with * Returns a page of the dumps in the category at `slug`, newest first, for
* the requesting user's private ones. * `GET /api/categories/:slug/dumps`. Private dumps are left out, except
* `requestingUserId`'s own.
* *
* @throws {APIException} 404 when there is no such category. * @throws {APIException} 404 when there is no such category.
*/ */
@@ -319,7 +318,7 @@ export function listDumpsByCategory(
ORDER BY d.created_at DESC LIMIT ? OFFSET ?;`, ORDER BY d.created_at DESC LIMIT ? OFFSET ?;`,
).all(category.id, ...privacyParams, limit + 1, offset); ).all(category.id, ...privacyParams, limit + 1, offset);
const items = rows.filter(isDumpRow).map(dumpRowToApi); const items = parseRows(rows, isDumpRow, dumpRowToApi);
attachCategoryIds(items); attachCategoryIds(items);
return pageOf(items, limit); return pageOf(items, limit);
} }

View File

@@ -1,10 +1,13 @@
/** /**
* The site-wide chat: messages, their history, and each user's read marker. * Stores the site-wide chat's messages and each user's read marker.
*
* The WebSocket route (`routes/ws.ts`) writes messages and moves read markers,
* then broadcasts the result. `routes/chat.ts` reads the history, a page at a
* time.
* *
* @module * @module
*/ */
import { APIErrorCode } from "../../shared/api.ts";
import { APIException } from "../lib/api-exception.ts"; import { APIException } from "../lib/api-exception.ts";
import type { ChatMessage } from "../../shared/chat.ts"; import type { ChatMessage } from "../../shared/chat.ts";
import { chatMessageRowToApi, isChatMessageRow } from "../model/chat.ts"; import { chatMessageRowToApi, isChatMessageRow } from "../model/chat.ts";
@@ -12,49 +15,46 @@ import { db } from "../db/connection.ts";
import { linkAttachments } from "./attachment-service.ts"; import { linkAttachments } from "./attachment-service.ts";
import { VALIDATION } from "../../shared/validation.ts"; import { VALIDATION } from "../../shared/validation.ts";
/** How many messages a page holds when the caller names no limit. */
const DEFAULT_LIMIT = 50; const DEFAULT_LIMIT = 50;
/** The most messages a page may hold. */
const MAX_LIMIT = 100; const MAX_LIMIT = 100;
/** /**
* The length cap of the replied-to preview snippet, so a reply to a long * How much of the replied-to message a reply carries. The client shows one line
* message doesn't ship the whole 2000-char body over the wire for every reply. * of it, once stripped of its Markdown, so the cut leaves room for a long link
* This is raw markdown: the client strips it to text before showing a line of * or image URL before the text.
* it, so it's cut generously enough that an image URL or link doesn't eat it
* all.
*/ */
const REPLY_SNIPPET_LEN = 300; const REPLY_SNIPPET_LEN = 300;
/** The columns of a full `ChatMessageRow`, over {@link FROM_JOINS}. */ /** The columns of a `ChatMessageRow`, to select over {@link FROM_JOINS}. */
const SELECT_COLS = const SELECT_COLS =
`m.id, m.user_id, m.body, m.created_at, m.updated_at, m.reply_to_id, `m.id, m.user_id, m.body, m.created_at, m.updated_at, m.reply_to_id,
u.username as author_username, u.avatar_mime as author_avatar_mime, u.username as author_username, u.avatar_mime as author_avatar_mime,
ru.username as reply_to_author, substr(rm.body, 1, ${REPLY_SNIPPET_LEN}) as reply_to_body`; ru.username as reply_to_author, substr(rm.body, 1, ${REPLY_SNIPPET_LEN}) as reply_to_body`;
/** /**
* The common joins: author (required) plus the optional replied-to message and * Joins a message to its author, and to the message it replies to and that
* its author (LEFT — null when there's no reply or the target was deleted). * message's author. Those two are left joins: a reply outlives its target.
*/ */
const FROM_JOINS = `FROM chat_messages m const FROM_JOINS = `FROM chat_messages m
JOIN users u ON m.user_id = u.id JOIN users u ON m.user_id = u.id
LEFT JOIN chat_messages rm ON m.reply_to_id = rm.id LEFT JOIN chat_messages rm ON m.reply_to_id = rm.id
LEFT JOIN users ru ON rm.user_id = ru.id`; LEFT JOIN users ru ON rm.user_id = ru.id`;
/** Returns message `id`, or throws a 404. */ /** Returns message `id` as the broadcast sends it, or throws a 404. */
function fetchMessage(id: string): ChatMessage { function fetchMessage(id: string): ChatMessage {
const row = db.prepare( const row = db.prepare(
`SELECT ${SELECT_COLS} ${FROM_JOINS} WHERE m.id = ?;`, `SELECT ${SELECT_COLS} ${FROM_JOINS} WHERE m.id = ?;`,
).get(id); ).get(id);
if (!row || !isChatMessageRow(row)) { if (!row || !isChatMessageRow(row)) {
throw new APIException(APIErrorCode.NOT_FOUND, 404, "Message not found"); throw new APIException(404, "Message not found");
} }
return chatMessageRowToApi(row); return chatMessageRowToApi(row);
} }
/** /**
* Posts a message from `userId`, replying to `replyToId` when that message * Stores a message from `userId` on a `chat_send`, and claims the pasted images
* exists, and binds its attachments. * it links to. A `replyToId` naming no message is dropped, and the message is
* posted as no reply. The caller broadcasts the result.
* *
* @throws {APIException} 400 when the body is blank or longer than * @throws {APIException} 400 when the body is blank or longer than
* `VALIDATION.CHAT_MESSAGE_MAX`. * `VALIDATION.CHAT_MESSAGE_MAX`.
@@ -66,21 +66,14 @@ export function createChatMessage(
): ChatMessage { ): ChatMessage {
const trimmed = body.trim(); const trimmed = body.trim();
if (!trimmed) { if (!trimmed) {
throw new APIException( throw new APIException(400, "Message cannot be empty");
APIErrorCode.VALIDATION_ERROR,
400,
"Message cannot be empty",
);
} }
if (trimmed.length > VALIDATION.CHAT_MESSAGE_MAX) { if (trimmed.length > VALIDATION.CHAT_MESSAGE_MAX) {
throw new APIException( throw new APIException(
APIErrorCode.VALIDATION_ERROR,
400, 400,
`Message exceeds ${VALIDATION.CHAT_MESSAGE_MAX} characters`, `Message exceeds ${VALIDATION.CHAT_MESSAGE_MAX} characters`,
); );
} }
// Only store the reply target if it actually exists — drop dangling ids so a
// message never references a phantom parent.
let replyTo: string | null = null; let replyTo: string | null = null;
if (replyToId) { if (replyToId) {
const exists = db.prepare(`SELECT 1 FROM chat_messages WHERE id = ?;`).get( const exists = db.prepare(`SELECT 1 FROM chat_messages WHERE id = ?;`).get(
@@ -93,28 +86,28 @@ export function createChatMessage(
db.prepare( db.prepare(
`INSERT INTO chat_messages (id, user_id, body, created_at, reply_to_id) VALUES (?, ?, ?, ?, ?);`, `INSERT INTO chat_messages (id, user_id, body, created_at, reply_to_id) VALUES (?, ?, ?, ?, ?);`,
).run(id, userId, trimmed, createdAt, replyTo); ).run(id, userId, trimmed, createdAt, replyTo);
// Claim any pasted/dropped image attachments so the startup cleanup doesn't // Unclaimed attachments are deleted as orphans at startup.
// purge them as orphans (mirrors dumps/comments/playlists). linkAttachments(trimmed, "chat", id);
linkAttachments(trimmed, id);
return fetchMessage(id); return fetchMessage(id);
} }
/** Returns the author of message `id`, or throws a 404. */ /** Returns the id of message `id`'s author, or throws a 404. */
function getMessageOwner(id: string): string { function getMessageOwner(id: string): string {
const row = db.prepare(`SELECT user_id FROM chat_messages WHERE id = ?;`).get( const row = db.prepare(`SELECT user_id FROM chat_messages WHERE id = ?;`).get(
id, id,
) as { user_id: string } | undefined; ) as { user_id: string } | undefined;
if (!row) { if (!row) {
throw new APIException(APIErrorCode.NOT_FOUND, 404, "Message not found"); throw new APIException(404, "Message not found");
} }
return row.user_id; return row.user_id;
} }
/** /**
* Replaces a message's body, as its author or a moderator (`canModerate`), and * Replaces a message's body on a `chat_edit`, and claims any image the new body
* binds any new attachment. * links to. `canModerate` lets a moderator edit anyone's message. The caller
* broadcasts the result.
* *
* @throws {APIException} 404 when there is no such message, 401 when the user * @throws {APIException} 404 when there is no such message, 403 when the user
* is neither its author nor a moderator, 400 when the body is blank or too * is neither its author nor a moderator, 400 when the body is blank or too
* long. * long.
*/ */
@@ -126,38 +119,29 @@ export function updateChatMessage(
): ChatMessage { ): ChatMessage {
const ownerId = getMessageOwner(id); const ownerId = getMessageOwner(id);
if (ownerId !== requestingUserId && !canModerate) { if (ownerId !== requestingUserId && !canModerate) {
throw new APIException( throw new APIException(403, "Not authorized to edit this message");
APIErrorCode.UNAUTHORIZED,
401,
"Not authorized to edit this message",
);
} }
const trimmed = body.trim(); const trimmed = body.trim();
if (!trimmed) { if (!trimmed) {
throw new APIException( throw new APIException(400, "Message cannot be empty");
APIErrorCode.VALIDATION_ERROR,
400,
"Message cannot be empty",
);
} }
if (trimmed.length > VALIDATION.CHAT_MESSAGE_MAX) { if (trimmed.length > VALIDATION.CHAT_MESSAGE_MAX) {
throw new APIException( throw new APIException(
APIErrorCode.VALIDATION_ERROR,
400, 400,
`Message exceeds ${VALIDATION.CHAT_MESSAGE_MAX} characters`, `Message exceeds ${VALIDATION.CHAT_MESSAGE_MAX} characters`,
); );
} }
db.prepare(`UPDATE chat_messages SET body = ?, updated_at = ? WHERE id = ?;`) db.prepare(`UPDATE chat_messages SET body = ?, updated_at = ? WHERE id = ?;`)
.run(trimmed, new Date().toISOString(), id); .run(trimmed, new Date().toISOString(), id);
// An edit may introduce a freshly pasted image; claim its attachment too. linkAttachments(trimmed, "chat", id);
linkAttachments(trimmed, id);
return fetchMessage(id); return fetchMessage(id);
} }
/** /**
* Deletes a message, as its author or a moderator (`canModerate`). * Deletes a message on a `chat_delete`. `canModerate` lets a moderator delete
* anyone's message. Replies to it keep their `replyToId`, without its preview.
* *
* @throws {APIException} 404 when there is no such message, 401 when the user * @throws {APIException} 404 when there is no such message, 403 when the user
* is neither its author nor a moderator. * is neither its author nor a moderator.
*/ */
export function deleteChatMessage( export function deleteChatMessage(
@@ -165,53 +149,47 @@ export function deleteChatMessage(
requestingUserId: string, requestingUserId: string,
canModerate: boolean, canModerate: boolean,
): void { ): void {
const ownerId = getMessageOwner(id); // 404s if the message doesn't exist const ownerId = getMessageOwner(id);
if (ownerId !== requestingUserId && !canModerate) { if (ownerId !== requestingUserId && !canModerate) {
throw new APIException( throw new APIException(403, "Not authorized to delete this message");
APIErrorCode.UNAUTHORIZED,
401,
"Not authorized to delete this message",
);
} }
db.prepare(`DELETE FROM chat_messages WHERE id = ?;`).run(id); db.prepare(`DELETE FROM chat_messages WHERE id = ?;`).run(id);
} }
/** /**
* A position in the chat stream. Messages are ordered by (createdAt, id), so * Passed by `routes/chat.ts` to page through the history, from the message the
* two sharing a millisecond still have a strict order and a page boundary * client already has at one end. Messages are ordered by `createdAt`, then by
* between them can't skip either. * id, so a page boundary between two messages sent in the same millisecond
* skips neither.
*/ */
export interface ChatCursor { export interface ChatCursor {
/** The `created_at` of the message the cursor sits at. */ /** An ISO timestamp, compared as a string with `created_at`. */
createdAt: string; createdAt: string;
/** /**
* The id of that message. Omitted: the cursor sits before (for `after`) or * Without it, the cursor takes in every message at `createdAt`: they are
* after (for `before`) every message sharing that timestamp. * all older for {@link getRecentChatMessages} and all newer for
* {@link getChatMessagesAfter}.
*/ */
id?: string; id?: string;
} }
/** Converts query rows to messages. */ /** Converts query rows to messages, or throws a 500 if one doesn't parse. */
function toMessages(rows: unknown[]): ChatMessage[] { function toMessages(rows: unknown[]): ChatMessage[] {
if (!rows.every(isChatMessageRow)) { if (!rows.every(isChatMessageRow)) {
throw new APIException( throw new APIException(500, "Malformed chat message data");
APIErrorCode.SERVER_ERROR,
500,
"Malformed chat message data",
);
} }
return rows.map(chatMessageRowToApi); return rows.map(chatMessageRowToApi);
} }
/** Clamps a page size to [1, {@link MAX_LIMIT}]. */
function capLimit(limit: number): number { function capLimit(limit: number): number {
return Math.min(Math.max(Math.floor(limit), 1), MAX_LIMIT); return Math.min(Math.max(Math.floor(limit), 1), MAX_LIMIT);
} }
/** /**
* Returns up to `limit` most recent messages in ascending (display) order. * Returns the latest `limit` messages, or with `before` the `limit` messages
* When `before` is given, returns the page of older messages immediately * just older than it, oldest first either way. `limit` is clamped to 1–100.
* preceding it — used for "load older" history. *
* @throws {APIException} 500 when a row doesn't parse as a message.
*/ */
export function getRecentChatMessages( export function getRecentChatMessages(
limit = DEFAULT_LIMIT, limit = DEFAULT_LIMIT,
@@ -226,7 +204,7 @@ export function getRecentChatMessages(
).all( ).all(
before.createdAt, before.createdAt,
before.createdAt, before.createdAt,
// No id: nothing at that timestamp sorts below the empty string. // Nothing sorts below "", so no message at that timestamp is taken.
before.id ?? "", before.id ?? "",
capped, capped,
) )
@@ -234,14 +212,15 @@ export function getRecentChatMessages(
`SELECT ${SELECT_COLS} ${FROM_JOINS} `SELECT ${SELECT_COLS} ${FROM_JOINS}
ORDER BY m.created_at DESC, m.id DESC LIMIT ?;`, ORDER BY m.created_at DESC, m.id DESC LIMIT ?;`,
).all(capped); ).all(capped);
// Newest-first from SQL; reverse to ascending for display.
return toMessages(rows).reverse(); return toMessages(rows).reverse();
} }
/** /**
* Returns up to `limit` messages following `after`, in ascending order — used * Returns the first `limit` messages after `after`, oldest first. The chatbox
* to resync the chat after a reconnect. Without an id the cursor is inclusive * uses it to fetch what it missed while the socket was down. `limit` is
* of its timestamp. * clamped to 1–100.
*
* @throws {APIException} 500 when a row doesn't parse as a message.
*/ */
export function getChatMessagesAfter( export function getChatMessagesAfter(
after: ChatCursor, after: ChatCursor,
@@ -255,17 +234,21 @@ export function getChatMessagesAfter(
return toMessages(rows); return toMessages(rows);
} }
/** How far a user has read the chat. */ /**
* Returned to the WebSocket route by {@link getChatReadState} and
* {@link markChatRead}. The route sends it in `welcome` and in
* `chat_read_update`, and the client shows `unread` on the chat's badge.
*/
export interface ChatReadState { export interface ChatReadState {
/** The `created_at` of the newest message the user has seen (ISO). */ /** The `created_at` of the newest message the user has read. */
readAt: string; readAt: string;
/** Messages from other people since then. */ /** Counts only the messages of other users after `readAt`. */
unread: number; unread: number;
} }
/** /**
* Returns how far `userId` has read the chat; a user who never opened it has * Returns how far `userId` has read the chat. A user who never opened it has
* read up to their registration. * read up to their registration, so the history from before it isn't unread.
* *
* @throws {APIException} 404 when there is no such user. * @throws {APIException} 404 when there is no such user.
*/ */
@@ -278,15 +261,16 @@ export function getChatReadState(userId: string): ChatReadState {
FROM users u WHERE u.id = ?;`, FROM users u WHERE u.id = ?;`,
).get(userId) as { read_at: string; unread: number } | undefined; ).get(userId) as { read_at: string; unread: number } | undefined;
if (!row) { if (!row) {
throw new APIException(APIErrorCode.NOT_FOUND, 404, "User not found"); throw new APIException(404, "User not found");
} }
return { readAt: row.read_at, unread: row.unread }; return { readAt: row.read_at, unread: row.unread };
} }
/** /**
* Moves the user's read marker forward to `at` (a message's created_at) — * Moves `userId`'s read marker to `at`, a message's `created_at`, on a
* never backward, so a lagging tab can't undo a newer tab's read, and never * `chat_read`. It never moves back, so a lagging tab can't undo a newer tab's
* past the newest message, so a bogus timestamp can't pre-read the future. * read. It never passes the newest message, so a timestamp from the future
* can't mark messages read before they are sent.
* *
* @throws {APIException} 400 when `at` isn't a date, 404 when there is no * @throws {APIException} 400 when `at` isn't a date, 404 when there is no
* such user. * such user.
@@ -294,17 +278,12 @@ export function getChatReadState(userId: string): ChatReadState {
export function markChatRead(userId: string, at: string): ChatReadState { export function markChatRead(userId: string, at: string): ChatReadState {
const parsed = new Date(at); const parsed = new Date(at);
if (Number.isNaN(parsed.getTime())) { if (Number.isNaN(parsed.getTime())) {
throw new APIException( throw new APIException(400, "Invalid read marker");
APIErrorCode.VALIDATION_ERROR,
400,
"Invalid read marker",
);
} }
const newest = db.prepare( const newest = db.prepare(
`SELECT MAX(created_at) AS newest FROM chat_messages;`, `SELECT MAX(created_at) AS newest FROM chat_messages;`,
).get() as { newest: string | null }; ).get() as { newest: string | null };
// No messages yet: there's nothing to have read, and no newest message to // With no message to cap it at, a marker from the future would stay stuck.
// cap a bogus far-future marker at — which could never be moved back.
if (!newest.newest) return getChatReadState(userId); if (!newest.newest) return getChatReadState(userId);
const iso = parsed.toISOString(); const iso = parsed.toISOString();
const target = iso > newest.newest ? newest.newest : iso; const target = iso > newest.newest ? newest.newest : iso;

View File

@@ -1,26 +1,32 @@
/** /**
* Likes on comments. * Stores likes on comments and keeps each comment's `like_count` in step with
* them. Likes have no HTTP endpoint to cast or remove. The WebSocket route
* (`routes/ws.ts`) calls this module on `comment_like_cast` and
* `comment_like_remove` messages, then broadcasts the new count. The comment
* routes use it only to list a comment's likers.
* *
* @module * @module
*/ */
import { APIErrorCode } from "../../shared/api.ts";
import { APIException } from "../lib/api-exception.ts"; import { APIException } from "../lib/api-exception.ts";
import { isUserRow, type User, userRowToApi } from "../model/user.ts"; import { isUserRow, type User, userRowToApi } from "../model/user.ts";
import { db } from "../db/connection.ts"; import { db, transaction } from "../db/connection.ts";
import { type Page, pageOf } from "../lib/pagination.ts"; import { type Page, pageOf } from "../lib/pagination.ts";
import { notifyCommentOwnerLike } from "./notification-service.ts"; import { notifyCommentOwnerLike } from "./notification-service.ts";
/** A comment's likes after a like was cast or removed. */ /**
* Returned to the WebSocket route by {@link castCommentLike} and
* {@link removeCommentLike}. The route acks the liker with `likeCount`, then
* broadcasts it to every client, or to the liker's alone when the comment's
* dump `isPrivate`.
*/
export interface LikeResult { export interface LikeResult {
/** The comment's like count now. */
likeCount: number; likeCount: number;
/** Whether the comment's dump is private: its likes go to the dump's owner only. */
isPrivate: boolean; isPrivate: boolean;
} }
/** /**
* Returns whether the comment's dump is private, or throws a 404 when the user * Returns whether the comment's dump is private, or throws a 404 when `userId`
* can't see it. * can't see it.
*/ */
function commentVisibility(commentId: string, userId: string): boolean { function commentVisibility(commentId: string, userId: string): boolean {
@@ -29,13 +35,14 @@ function commentVisibility(commentId: string, userId: string): boolean {
WHERE c.id = ?;`, WHERE c.id = ?;`,
).get(commentId) as { is_private: number; user_id: string } | undefined; ).get(commentId) as { is_private: number; user_id: string } | undefined;
if (!row || (row.is_private && row.user_id !== userId)) { if (!row || (row.is_private && row.user_id !== userId)) {
throw new APIException(APIErrorCode.NOT_FOUND, 404, "Comment not found"); throw new APIException(404, "Comment not found");
} }
return Boolean(row.is_private); return Boolean(row.is_private);
} }
/** /**
* Records `userId`'s like on a comment and notifies its author. * Records `userId`'s like on a comment and notifies the comment's author. The
* notification is best-effort: if it fails, the like still counts.
* *
* @throws {APIException} 404 when the comment doesn't exist or sits on someone * @throws {APIException} 404 when the comment doesn't exist or sits on someone
* else's private dump, 409 when the user already likes it. * else's private dump, 409 when the user already likes it.
@@ -47,7 +54,7 @@ export function castCommentLike(
const isPrivate = commentVisibility(commentId, userId); const isPrivate = commentVisibility(commentId, userId);
let likeCount: number; let likeCount: number;
try { try {
db.exec("BEGIN;"); likeCount = transaction(() => {
db.prepare( db.prepare(
`INSERT INTO comment_likes (comment_id, user_id, created_at) VALUES (?, ?, ?);`, `INSERT INTO comment_likes (comment_id, user_id, created_at) VALUES (?, ?, ?);`,
).run(commentId, userId, new Date().toISOString()); ).run(commentId, userId, new Date().toISOString());
@@ -56,23 +63,16 @@ export function castCommentLike(
).run(commentId); ).run(commentId);
const row = db.prepare( const row = db.prepare(
`SELECT like_count FROM comments WHERE id = ?;`, `SELECT like_count FROM comments WHERE id = ?;`,
).get(commentId) as { like_count: number } | undefined; ).get(commentId) as { like_count: number };
db.exec("COMMIT;"); return row.like_count;
likeCount = row?.like_count ?? 0; });
} catch (err) { } catch (err) {
try {
db.exec("ROLLBACK;");
} catch { /* ignore if no active transaction */ }
if (err instanceof Error && err.message.includes("UNIQUE constraint")) { if (err instanceof Error && err.message.includes("UNIQUE constraint")) {
throw new APIException( throw new APIException(409, "Already liked");
APIErrorCode.VALIDATION_ERROR,
409,
"Already liked",
);
} }
throw err; throw err;
} }
// Notification is best-effort — must not prevent like_ack from being sent // The like is committed: a failed notification must not report it as failed.
try { try {
notifyCommentOwnerLike(userId, commentId); notifyCommentOwnerLike(userId, commentId);
} catch { /* ignore */ } } catch { /* ignore */ }
@@ -80,8 +80,9 @@ export function castCommentLike(
} }
/** /**
* Takes back `userId`'s like on a comment. No visibility check: taking back a * Takes back `userId`'s like on a comment. It doesn't check that the user can
* like is harmless, and it's the user's own row. * still see the comment, since a like on a dump that has gone private is still
* theirs to take back.
* *
* @throws {APIException} 404 when the user doesn't like the comment. * @throws {APIException} 404 when the user doesn't like the comment.
*/ */
@@ -94,31 +95,28 @@ export function removeCommentLike(
WHERE c.id = ?;`, WHERE c.id = ?;`,
).get(commentId) as { is_private: number } | undefined; ).get(commentId) as { is_private: number } | undefined;
const isPrivate = Boolean(dumpRow?.is_private); const isPrivate = Boolean(dumpRow?.is_private);
try { return transaction(() => {
db.exec("BEGIN;");
const result = db.prepare( const result = db.prepare(
`DELETE FROM comment_likes WHERE comment_id = ? AND user_id = ?;`, `DELETE FROM comment_likes WHERE comment_id = ? AND user_id = ?;`,
).run(commentId, userId); ).run(commentId, userId);
if (result.changes === 0) { if (result.changes === 0) {
db.exec("ROLLBACK;"); throw new APIException(404, "Like not found");
throw new APIException(APIErrorCode.NOT_FOUND, 404, "Like not found");
} }
db.prepare( db.prepare(
`UPDATE comments SET like_count = like_count - 1 WHERE id = ?;`, `UPDATE comments SET like_count = like_count - 1 WHERE id = ?;`,
).run(commentId); ).run(commentId);
const row = db.prepare( const row = db.prepare(
`SELECT like_count FROM comments WHERE id = ?;`, `SELECT like_count FROM comments WHERE id = ?;`,
).get(commentId) as { like_count: number } | undefined; ).get(commentId) as { like_count: number };
db.exec("COMMIT;"); return { likeCount: row.like_count, isPrivate };
return { likeCount: row?.like_count ?? 0, isPrivate }; });
} catch (err) {
if (err instanceof APIException) throw err;
db.exec("ROLLBACK;");
throw err;
}
} }
/** Returns the ids of every comment `userId` likes. */ /**
* Returns the ids of the comments `userId` likes. The WebSocket route sends
* them in its `welcome` message, and the client shows those like buttons
* pressed.
*/
export function getUserCommentLikes(userId: string): string[] { export function getUserCommentLikes(userId: string): string[] {
const rows = db.prepare( const rows = db.prepare(
`SELECT comment_id FROM comment_likes WHERE user_id = ?;`, `SELECT comment_id FROM comment_likes WHERE user_id = ?;`,
@@ -126,7 +124,13 @@ export function getUserCommentLikes(userId: string): string[] {
return rows.map((r) => r.comment_id); return rows.map((r) => r.comment_id);
} }
/** Returns a page of the users who like a comment, latest first. */ /**
* Returns a page of the users who like a comment, latest like first, for
* `GET /api/comments/:commentId/likers`. It doesn't check that the caller may
* see the comment: the route does that first, with `getDump`.
*
* @throws {APIException} 500 when a row doesn't parse as a user.
*/
export function getCommentLikers( export function getCommentLikers(
commentId: string, commentId: string,
page: number, page: number,
@@ -146,11 +150,7 @@ export function getCommentLikers(
).all(commentId, limit + 1, offset); ).all(commentId, limit + 1, offset);
if (!rawRows.every(isUserRow)) { if (!rawRows.every(isUserRow)) {
throw new APIException( throw new APIException(500, "Malformed user data");
APIErrorCode.SERVER_ERROR,
500,
"Malformed user data",
);
} }
return pageOf(rawRows.map(userRowToApi), limit); return pageOf(rawRows.map(userRowToApi), limit);

View File

@@ -1,13 +1,13 @@
/** /**
* Comments on dumps, threaded by `parent_id`. * Stores the comments on dumps, threaded by `parent_id`. The comment routes
* call it and broadcast what it returns. It leaves checking that the caller
* may see the dump to them.
* *
* @module * @module
*/ */
import { APIErrorCode } from "../../shared/api.ts";
import { APIException } from "../lib/api-exception.ts"; import { APIException } from "../lib/api-exception.ts";
import type { Comment } from "../../shared/comment.ts"; import type { Comment } from "../../shared/comment.ts";
import { type SQLOutputValue } from "node:sqlite";
import { commentRowToApi, isCommentRow } from "../model/comment.ts"; import { commentRowToApi, isCommentRow } from "../model/comment.ts";
import { db } from "../db/connection.ts"; import { db } from "../db/connection.ts";
import { import {
@@ -17,31 +17,25 @@ import {
import { linkAttachments } from "./attachment-service.ts"; import { linkAttachments } from "./attachment-service.ts";
import { relinkBacklinks } from "./backlink-service.ts"; import { relinkBacklinks } from "./backlink-service.ts";
/** The columns of a full `CommentRow`, from `comments c` and `users u`. */ /** The select list of a `CommentRow`, from `comments c` joined to `users u`. */
const SELECT_COLS = const SELECT_COLS =
`c.id, c.dump_id, c.user_id, c.parent_id, c.body, c.created_at, c.updated_at, c.deleted, c.like_count, `c.id, c.dump_id, c.user_id, c.parent_id, c.body, c.created_at, c.updated_at, c.deleted, c.like_count,
u.username as author_username, u.avatar_mime as author_avatar_mime`; u.username as author_username, u.avatar_mime as author_avatar_mime`;
/** Returns comment `commentId`, or throws a 404. */
function fetchComment(commentId: string): Comment { function fetchComment(commentId: string): Comment {
const row = db.prepare( const row = db.prepare(
`SELECT ${SELECT_COLS} FROM comments c JOIN users u ON c.user_id = u.id WHERE c.id = ?;`, `SELECT ${SELECT_COLS} FROM comments c JOIN users u ON c.user_id = u.id WHERE c.id = ?;`,
).get(commentId); ).get(commentId);
if (!row || !isCommentRow(row as Record<string, SQLOutputValue>)) { if (!row) throw new APIException(404, "Comment not found");
throw new APIException(APIErrorCode.NOT_FOUND, 404, "Comment not found");
}
if (!isCommentRow(row)) { if (!isCommentRow(row)) {
throw new APIException( throw new APIException(500, "Malformed comment data");
APIErrorCode.SERVER_ERROR,
500,
"Malformed comment data",
);
} }
return commentRowToApi(row); return commentRowToApi(row);
} }
/** /**
* Returns the id of the dump comment `commentId` is posted under. * Returns the id of the dump comment `commentId` is posted under. The likers
* route uses it to check that the caller may see that dump.
* *
* @throws {APIException} 404 when there is no such comment. * @throws {APIException} 404 when there is no such comment.
*/ */
@@ -50,14 +44,17 @@ export function getCommentDumpId(commentId: string): string {
commentId, commentId,
) as { dump_id: string } | undefined; ) as { dump_id: string } | undefined;
if (!row) { if (!row) {
throw new APIException(APIErrorCode.NOT_FOUND, 404, "Comment not found"); throw new APIException(404, "Comment not found");
} }
return row.dump_id; return row.dump_id;
} }
/** /**
* Returns every comment on dump `dumpId`, deleted ones included, oldest * Returns every comment on dump `dumpId`, oldest first, for
* first. * `GET /api/dumps/:dumpId/comments`. Deleted comments are included, emptied,
* so the client can keep their replies in place.
*
* @throws {APIException} 500 when a row doesn't parse as a comment.
*/ */
export function getComments(dumpId: string): Comment[] { export function getComments(dumpId: string): Comment[] {
const rows = db.prepare( const rows = db.prepare(
@@ -65,19 +62,19 @@ export function getComments(dumpId: string): Comment[] {
WHERE c.dump_id = ? ORDER BY c.created_at ASC;`, WHERE c.dump_id = ? ORDER BY c.created_at ASC;`,
).all(dumpId); ).all(dumpId);
if (!rows.every(isCommentRow)) { if (!rows.every(isCommentRow)) {
throw new APIException( throw new APIException(500, "Malformed comment data");
APIErrorCode.SERVER_ERROR,
500,
"Malformed comment data",
);
} }
return rows.map(commentRowToApi); return rows.map(commentRowToApi);
} }
/** /**
* Posts a comment on dump `dumpId`, replying to `parentId` if given; notifies * Posts a comment on dump `dumpId`, replying to `parentId` if given, for
* the dump's poster and anyone @mentioned, and binds its attachments and * `POST /api/dumps/:dumpId/comments`. It notifies the dump's poster and the
* backlinks. * users it mentions, unless the dump is private, and links the comment's
* attachments and backlinks. The route has checked that the caller may see
* the dump.
*
* @throws {APIException} 400 when `parentId` isn't a comment on the same dump.
*/ */
export function createComment( export function createComment(
dumpId: string, dumpId: string,
@@ -85,6 +82,13 @@ export function createComment(
body: string, body: string,
parentId?: string, parentId?: string,
): Comment { ): Comment {
if (parentId !== undefined) {
const parent = db.prepare(`SELECT dump_id FROM comments WHERE id = ?;`)
.get(parentId) as { dump_id: string } | undefined;
if (parent?.dump_id !== dumpId) {
throw new APIException(400, "The comment replied to isn't on this dump");
}
}
const id = crypto.randomUUID(); const id = crypto.randomUUID();
const createdAt = new Date(); const createdAt = new Date();
db.prepare( db.prepare(
@@ -98,22 +102,27 @@ export function createComment(
createdAt.toISOString(), createdAt.toISOString(),
); );
const comment = fetchComment(id); const comment = fetchComment(id);
const dumpRow = db.prepare(`SELECT title FROM dumps WHERE id = ?;`).get( const dumpRow = db.prepare(
dumpId, `SELECT title, is_private FROM dumps WHERE id = ?;`,
) as { title: string } | undefined; ).get(dumpId) as { title: string; is_private: number } | undefined;
notifyDumpOwnerNewComment(userId, id, dumpId); notifyDumpOwnerNewComment(userId, id, dumpId);
notifyMentions(userId, body, "comment", id, dumpRow?.title ?? "", dumpId); if (dumpRow && !dumpRow.is_private) {
linkAttachments(body, id); notifyMentions(userId, body, "comment", id, dumpRow.title, dumpId);
}
linkAttachments(body, "comment", id);
relinkBacklinks("comment", id, dumpId, body); relinkBacklinks("comment", id, dumpId, body);
return comment; return comment;
} }
/** /**
* Replaces a comment's body, as its author or a moderator (`canModerate`); * Replaces a comment's body, for `PATCH /api/comments/:commentId`.
* notifies anyone newly @mentioned, and rebinds its attachments and backlinks. * `canModerate` lets someone other than the author do it. It notifies, in the
* author's name, the users it mentions for the first time, unless the dump is
* private, and relinks the attachments and backlinks. The route broadcasts the
* comment unless `isPrivate`.
* *
* @throws {APIException} 404 when there is no such comment, 400 when it is * @throws {APIException} 404 when there is no such comment, 400 when it is
* deleted, 401 when the user is neither its author nor a moderator. * deleted, 403 when the user is neither its author nor a moderator.
*/ */
export function updateComment( export function updateComment(
commentId: string, commentId: string,
@@ -125,22 +134,14 @@ export function updateComment(
`SELECT c.dump_id, d.is_private FROM comments c JOIN dumps d ON c.dump_id = d.id WHERE c.id = ?;`, `SELECT c.dump_id, d.is_private FROM comments c JOIN dumps d ON c.dump_id = d.id WHERE c.id = ?;`,
).get(commentId) as { dump_id: string; is_private: number } | undefined; ).get(commentId) as { dump_id: string; is_private: number } | undefined;
if (!row) { if (!row) {
throw new APIException(APIErrorCode.NOT_FOUND, 404, "Comment not found"); throw new APIException(404, "Comment not found");
} }
const existing = fetchComment(commentId); const existing = fetchComment(commentId);
if (existing.deleted) { if (existing.deleted) {
throw new APIException( throw new APIException(400, "Cannot edit a deleted comment");
APIErrorCode.VALIDATION_ERROR,
400,
"Cannot edit a deleted comment",
);
} }
if (existing.userId !== requestingUserId && !canModerate) { if (existing.userId !== requestingUserId && !canModerate) {
throw new APIException( throw new APIException(403, "Not authorized to edit this comment");
APIErrorCode.UNAUTHORIZED,
401,
"Not authorized to edit this comment",
);
} }
const now = new Date().toISOString(); const now = new Date().toISOString();
db.prepare(`UPDATE comments SET body = ?, updated_at = ? WHERE id = ?;`).run( db.prepare(`UPDATE comments SET body = ?, updated_at = ? WHERE id = ?;`).run(
@@ -148,18 +149,21 @@ export function updateComment(
now, now,
commentId, commentId,
); );
if (!row.is_private) {
const dumpRow = db.prepare(`SELECT title FROM dumps WHERE id = ?;`).get( const dumpRow = db.prepare(`SELECT title FROM dumps WHERE id = ?;`).get(
row.dump_id, row.dump_id,
) as { title: string } | undefined; ) as { title: string } | undefined;
// A moderator's edit keeps the author's words, so the author mentions.
notifyMentions( notifyMentions(
requestingUserId, existing.userId,
body, body,
"comment", "comment",
commentId, commentId,
dumpRow?.title ?? "", dumpRow?.title ?? "",
row.dump_id, row.dump_id,
); );
linkAttachments(body, commentId); }
linkAttachments(body, "comment", commentId);
relinkBacklinks("comment", commentId, row.dump_id, body); relinkBacklinks("comment", commentId, row.dump_id, body);
return { return {
comment: fetchComment(commentId), comment: fetchComment(commentId),
@@ -169,11 +173,12 @@ export function updateComment(
} }
/** /**
* Deletes a comment, as its author or a moderator (`canModerate`): its body is * Deletes a comment, for `DELETE /api/comments/:commentId`. `canModerate` lets
* emptied and its backlinks dropped, but the row stays so its replies keep * someone other than the author do it. The body is emptied and the backlinks
* their place. * dropped, but the row stays so its replies keep their place. The route
* broadcasts the deletion unless `isPrivate`.
* *
* @throws {APIException} 404 when there is no such comment, 401 when the user * @throws {APIException} 404 when there is no such comment, 403 when the user
* is neither its author nor a moderator. * is neither its author nor a moderator.
*/ */
export function deleteComment( export function deleteComment(
@@ -185,15 +190,11 @@ export function deleteComment(
`SELECT c.dump_id, d.is_private FROM comments c JOIN dumps d ON c.dump_id = d.id WHERE c.id = ?;`, `SELECT c.dump_id, d.is_private FROM comments c JOIN dumps d ON c.dump_id = d.id WHERE c.id = ?;`,
).get(commentId) as { dump_id: string; is_private: number } | undefined; ).get(commentId) as { dump_id: string; is_private: number } | undefined;
if (!row) { if (!row) {
throw new APIException(APIErrorCode.NOT_FOUND, 404, "Comment not found"); throw new APIException(404, "Comment not found");
} }
const comment = fetchComment(commentId); const comment = fetchComment(commentId);
if (comment.userId !== requestingUserId && !canModerate) { if (comment.userId !== requestingUserId && !canModerate) {
throw new APIException( throw new APIException(403, "Not authorized to delete this comment");
APIErrorCode.UNAUTHORIZED,
401,
"Not authorized to delete this comment",
);
} }
db.prepare(`UPDATE comments SET deleted = 1, body = '' WHERE id = ?;`).run( db.prepare(`UPDATE comments SET deleted = 1, body = '' WHERE id = ?;`).run(
commentId, commentId,

View File

@@ -1,14 +1,14 @@
/** /**
* Dumps: posting links and files, editing and deleting them, and reading them * Posts, edits and deletes dumps, and reads them as each viewer may see them.
* as each viewer may see them. A private dump exists only for its poster. * `routes/dumps.ts` calls it, and so do the search, user, file and thumbnail
* routes to read dumps. It broadcasts the changes to public dumps over the
* WebSocket.
* *
* The functions that change a dump don't check who asks: the routes do, before * The functions that change a dump don't check who asks: the routes do first.
* calling them.
* *
* @module * @module
*/ */
import { APIErrorCode } from "../../shared/api.ts";
import { APIException } from "../lib/api-exception.ts"; import { APIException } from "../lib/api-exception.ts";
import type { import type {
CreateUrlDumpRequest, CreateUrlDumpRequest,
@@ -30,11 +30,7 @@ import {
tryFetchRichContent, tryFetchRichContent,
} from "./rich-content-service.ts"; } from "./rich-content-service.ts";
import { isValidHttpUrl } from "../remote/fetch.ts"; import { isValidHttpUrl } from "../remote/fetch.ts";
import { import { broadcastDumpChange } from "./ws-service.ts";
broadcastDumpDeleted,
broadcastDumpUpdated,
broadcastNewDump,
} from "./ws-service.ts";
import { import {
notifyMentions, notifyMentions,
notifyUserFollowersNewDump, notifyUserFollowersNewDump,
@@ -58,36 +54,34 @@ import {
attachCategoryIds, attachCategoryIds,
setDumpCategories, setDumpCategories,
} from "./category-service.ts"; } from "./category-service.ts";
import { LIKE, likePattern } from "../lib/like.ts";
import { parseRows } from "../model/rows.ts";
/** Checks whether a file dump may have type `mime`. */
function isAllowedMime(mime: string): boolean { function isAllowedMime(mime: string): boolean {
return DUMP_ALLOWED_MIME_PREFIXES.some((p) => mime.startsWith(p)) || return DUMP_ALLOWED_MIME_PREFIXES.some((p) => mime.startsWith(p)) ||
DUMP_ALLOWED_MIME_TYPES.has(mime); DUMP_ALLOWED_MIME_TYPES.has(mime);
} }
/** /**
* Throws a 400 when a file dump's title or comment is too long. The JSON routes * Throws a 400 when a file dump's title or comment is too long. The multipart
* validate these in isCreateUrlDumpRequest/isUpdateDumpRequest; the multipart * routes that post and replace a file don't check them, unlike the JSON ones.
* ones (file create and replace) arrive here unchecked.
*/ */
function assertFileDumpText(title?: string, comment?: string): void { function assertFileDumpText(title?: string, comment?: string): void {
if (title && title.trim().length > VALIDATION.DUMP_TITLE_MAX) { if (title && title.trim().length > VALIDATION.DUMP_TITLE_MAX) {
throw new APIException( throw new APIException(
APIErrorCode.VALIDATION_ERROR,
400, 400,
`Title must be at most ${VALIDATION.DUMP_TITLE_MAX} characters`, `Title must be at most ${VALIDATION.DUMP_TITLE_MAX} characters`,
); );
} }
if (comment && comment.length > VALIDATION.DUMP_COMMENT_MAX) { if (comment && comment.length > VALIDATION.DUMP_COMMENT_MAX) {
throw new APIException( throw new APIException(
APIErrorCode.VALIDATION_ERROR,
400, 400,
`Comment must be at most ${VALIDATION.DUMP_COMMENT_MAX} characters`, `Comment must be at most ${VALIDATION.DUMP_COMMENT_MAX} characters`,
); );
} }
} }
/** Returns the title a link gets without a better one: its host. */ /** Returns a link's host name, the title it gets when its page has none. */
function titleFromUrl(url: string): string { function titleFromUrl(url: string): string {
try { try {
return new URL(url).hostname.replace(/^www\./, ""); return new URL(url).hostname.replace(/^www\./, "");
@@ -97,8 +91,10 @@ function titleFromUrl(url: string): string {
} }
/** /**
* Posts a link as `userId`, with the preview fetched from it; a public one is * Posts a link as `userId`, for `POST /api/dumps`, with its preview fetched
* announced and notified to the poster's followers. * from the page. A public dump is broadcast as `dump_created` and notified to
* the poster's followers. Anyone @mentioned in the note is notified even when
* the dump is private.
* *
* @throws {APIException} 400 when the URL isn't a public http(s) URL or a * @throws {APIException} 400 when the URL isn't a public http(s) URL or a
* category doesn't exist. * category doesn't exist.
@@ -108,7 +104,7 @@ export async function createUrlDump(
userId: string, userId: string,
): Promise<Dump> { ): Promise<Dump> {
if (!isValidHttpUrl(request.url)) { if (!isValidHttpUrl(request.url)) {
throw new APIException(APIErrorCode.BAD_REQUEST, 400, "Invalid URL"); throw new APIException(400, "Invalid URL");
} }
assertCategoriesExist(request.categoryIds ?? []); assertCategoriesExist(request.categoryIds ?? []);
@@ -158,21 +154,23 @@ export async function createUrlDump(
isPrivate, isPrivate,
categoryIds, categoryIds,
}; };
if (!isPrivate) { broadcastDumpChange(null, dump);
broadcastNewDump(dump); if (!isPrivate) notifyUserFollowersNewDump(userId, dumpId, title);
notifyUserFollowersNewDump(userId, dumpId, title);
}
if (request.comment) { if (request.comment) {
// A private dump's mentions are notified if it goes public.
if (!isPrivate) {
notifyMentions(userId, request.comment, "dump", dumpId, title); notifyMentions(userId, request.comment, "dump", dumpId, title);
linkAttachments(request.comment, dumpId); }
linkAttachments(request.comment, "dump", dumpId);
} }
relinkBacklinks("dump_comment", dumpId, dumpId, request.comment ?? ""); relinkBacklinks("dump_comment", dumpId, dumpId, request.comment ?? "");
return dump; return dump;
} }
/** /**
* Posts an uploaded file as `userId`, stored under {@link DUMPS_DIR}; a public * Posts an uploaded file as `userId`, for `POST /api/dumps`, and stores it in
* one is announced and notified to the poster's followers. * `DUMPS_DIR` under the dump's id. It is announced and notified as
* {@link createUrlDump} does.
* *
* @throws {APIException} 400 when the title or comment is too long, the file's * @throws {APIException} 400 when the title or comment is too long, the file's
* type isn't allowed, it exceeds 50 MB, or a category doesn't exist. * type isn't allowed, it exceeds 50 MB, or a category doesn't exist.
@@ -187,18 +185,10 @@ export async function createFileDump(
): Promise<Dump> { ): Promise<Dump> {
assertFileDumpText(title, comment); assertFileDumpText(title, comment);
if (!isAllowedMime(file.type)) { if (!isAllowedMime(file.type)) {
throw new APIException( throw new APIException(400, `File type '${file.type}' is not allowed`);
APIErrorCode.BAD_REQUEST,
400,
`File type '${file.type}' is not allowed`,
);
} }
if (file.size > DUMP_MAX_FILE_SIZE_BYTES) { if (file.size > DUMP_MAX_FILE_SIZE_BYTES) {
throw new APIException( throw new APIException(400, "File too large (max 50 MB)");
APIErrorCode.BAD_REQUEST,
400,
"File too large (max 50 MB)",
);
} }
assertCategoriesExist(categoryIds); assertCategoriesExist(categoryIds);
@@ -234,7 +224,7 @@ export async function createFileDump(
setDumpCategories(dumpId, dedupedCategoryIds); setDumpCategories(dumpId, dedupedCategoryIds);
}); });
} catch (err) { } catch (err) {
// Roll back the file if DB insert fails // Leave no file behind when the write or the insert fails.
await Deno.remove(`${DUMPS_DIR}/${dumpId}`).catch(() => {}); await Deno.remove(`${DUMPS_DIR}/${dumpId}`).catch(() => {});
throw err; throw err;
} }
@@ -255,22 +245,24 @@ export async function createFileDump(
isPrivate, isPrivate,
categoryIds: dedupedCategoryIds, categoryIds: dedupedCategoryIds,
}; };
if (!isPrivate) { broadcastDumpChange(null, dump);
broadcastNewDump(dump); if (!isPrivate) notifyUserFollowersNewDump(userId, dumpId, finalTitle);
notifyUserFollowersNewDump(userId, dumpId, finalTitle);
}
if (comment) { if (comment) {
// A private dump's mentions are notified if it goes public.
if (!isPrivate) {
notifyMentions(userId, comment, "dump", dumpId, finalTitle); notifyMentions(userId, comment, "dump", dumpId, finalTitle);
linkAttachments(comment, dumpId); }
linkAttachments(comment, "dump", dumpId);
} }
relinkBacklinks("dump_comment", dumpId, dumpId, comment ?? ""); relinkBacklinks("dump_comment", dumpId, dumpId, comment ?? "");
return dump; return dump;
} }
/** /**
* Returns a file dump's version as the app puts it in file and thumbnail URLs * Returns a file dump's version, which the app's file and thumbnail URLs carry
* (`?v=`): it changes whenever the file can have. Mirrored by * as `?v=`. It changes whenever the file may have, so the file and thumbnail
* `dumpFileVersion` in src/utils/shared/urls.ts. * routes cache a URL naming the current one for good. The frontend's
* `dumpFileVersion` must compute the same.
*/ */
export function dumpFileVersion( export function dumpFileVersion(
dump: Pick<Dump, "fileSize" | "createdAt" | "updatedAt">, dump: Pick<Dump, "fileSize" | "createdAt" | "updatedAt">,
@@ -281,9 +273,9 @@ export function dumpFileVersion(
} }
/** /**
* Removes the files generated from a dump's upload (see * Deletes the thumbnails that `routes/thumbnails.ts` generated from a dump's
* api/routes/thumbnails.ts) when it's replaced or gone, so they're made again * upload, once the upload is replaced or deleted. The route makes them again
* from the new one. * from a new one.
*/ */
async function removeGeneratedThumbnails(dumpId: string): Promise<void> { async function removeGeneratedThumbnails(dumpId: string): Promise<void> {
await Promise.all( await Promise.all(
@@ -294,8 +286,8 @@ async function removeGeneratedThumbnails(dumpId: string): Promise<void> {
} }
/** /**
* Returns the dump with id or slug `idOrSlug`, or throws a 404. No privacy * Returns the dump with id or slug `idOrSlug`, without its categories, or
* check: use only when ownership is already enforced. * throws a 404. It doesn't check privacy: the routes do, with {@link getDump}.
*/ */
function fetchDump(idOrSlug: string): Dump { function fetchDump(idOrSlug: string): Dump {
const row = UUID_RE.test(idOrSlug) const row = UUID_RE.test(idOrSlug)
@@ -304,29 +296,31 @@ function fetchDump(idOrSlug: string): Dump {
idOrSlug, idOrSlug,
); );
if (!row || !isDumpRow(row)) { if (!row || !isDumpRow(row)) {
throw new APIException(APIErrorCode.NOT_FOUND, 404, "Dump not found"); throw new APIException(404, "Dump not found");
} }
return dumpRowToApi(row); return dumpRowToApi(row);
} }
/** /**
* Returns a dump, by id or slug, as `requestingUserId` may see it. * Returns a dump, by id or slug, as `requestingUserId` may see it. The routes
* resolve every `:dumpId` with it before acting on the dump.
* *
* @throws {APIException} 404 when there is no such dump, or it is private and not theirs. * @throws {APIException} 404 when there is no such dump, or it is private and
* not theirs.
*/ */
export function getDump(dumpId: string, requestingUserId?: string): Dump { export function getDump(dumpId: string, requestingUserId?: string): Dump {
const dump = fetchDump(dumpId); const dump = fetchDump(dumpId);
if (dump.isPrivate && dump.userId !== requestingUserId) { if (dump.isPrivate && dump.userId !== requestingUserId) {
throw new APIException(APIErrorCode.NOT_FOUND, 404, "Dump not found"); throw new APIException(404, "Dump not found");
} }
attachCategoryIds([dump]); attachCategoryIds([dump]);
return dump; return dump;
} }
/** /**
* Returns a page of the dumps whose title, comment, or previewed title or * Returns a page of the dumps `requestingUserId` may see whose title, note, or
* description contains `query`, newest first, and how many match in all: the * previewed title or description contains `query`, newest first, with how many
* public ones, and the requesting user's own. * match in all. `GET /api/search` calls it.
*/ */
export function searchDumps( export function searchDumps(
query: string, query: string,
@@ -335,11 +329,11 @@ export function searchDumps(
requestingUserId?: string, requestingUserId?: string,
): { items: Dump[]; total: number } { ): { items: Dump[]; total: number } {
if (!query.trim()) return { items: [], total: 0 }; if (!query.trim()) return { items: [], total: 0 };
const pattern = `%${query}%`; const pattern = likePattern(query);
const offset = (page - 1) * limit; const offset = (page - 1) * limit;
const searchClause = const searchClause =
`(title LIKE ? OR comment LIKE ? OR json_extract(rich_content,'$.title') LIKE ? OR json_extract(rich_content,'$.description') LIKE ?)`; `(title ${LIKE} OR comment ${LIKE} OR json_extract(rich_content,'$.title') ${LIKE} OR json_extract(rich_content,'$.description') ${LIKE})`;
const searchParams = [pattern, pattern, pattern, pattern] as const; const searchParams = [pattern, pattern, pattern, pattern] as const;
if (requestingUserId) { if (requestingUserId) {
@@ -349,7 +343,7 @@ export function searchDumps(
const totalRow = db.prepare( const totalRow = db.prepare(
`SELECT COUNT(*) as count FROM dumps WHERE (is_private = 0 OR user_id = ?) AND ${searchClause};`, `SELECT COUNT(*) as count FROM dumps WHERE (is_private = 0 OR user_id = ?) AND ${searchClause};`,
).get(requestingUserId, ...searchParams) as { count: number } | undefined; ).get(requestingUserId, ...searchParams) as { count: number } | undefined;
const items = rows.filter(isDumpRow).map(dumpRowToApi); const items = parseRows(rows, isDumpRow, dumpRowToApi);
attachCategoryIds(items); attachCategoryIds(items);
return { items, total: totalRow?.count ?? 0 }; return { items, total: totalRow?.count ?? 0 };
} else { } else {
@@ -359,15 +353,17 @@ export function searchDumps(
const totalRow = db.prepare( const totalRow = db.prepare(
`SELECT COUNT(*) as count FROM dumps WHERE is_private = 0 AND ${searchClause};`, `SELECT COUNT(*) as count FROM dumps WHERE is_private = 0 AND ${searchClause};`,
).get(...searchParams) as { count: number } | undefined; ).get(...searchParams) as { count: number } | undefined;
const items = rows.filter(isDumpRow).map(dumpRowToApi); const items = parseRows(rows, isDumpRow, dumpRowToApi);
attachCategoryIds(items); attachCategoryIds(items);
return { items, total: totalRow?.count ?? 0 }; return { items, total: totalRow?.count ?? 0 };
} }
} }
/** /**
* Returns a page of the dumps, newest first: the public ones, and the * Returns a page of the dumps `requestingUserId` may see, newest first, for the
* requesting user's own. * index feed (`GET /api/dumps`).
*
* @throws {APIException} 500 when a row doesn't parse as a dump.
*/ */
export function listDumps( export function listDumps(
page: number, page: number,
@@ -375,7 +371,6 @@ export function listDumps(
requestingUserId?: string, requestingUserId?: string,
): Page<Dump> { ): Page<Dump> {
const offset = (page - 1) * limit; const offset = (page - 1) * limit;
// Show public dumps + the requesting user's own private dumps
const rows = requestingUserId const rows = requestingUserId
? db.prepare( ? db.prepare(
`SELECT ${SELECT_COLS} FROM dumps WHERE (is_private = 0 OR user_id = ?) ORDER BY created_at DESC LIMIT ? OFFSET ?;`, `SELECT ${SELECT_COLS} FROM dumps WHERE (is_private = 0 OR user_id = ?) ORDER BY created_at DESC LIMIT ? OFFSET ?;`,
@@ -385,11 +380,7 @@ export function listDumps(
).all(limit + 1, offset); ).all(limit + 1, offset);
if (!rows || !rows.every(isDumpRow)) { if (!rows || !rows.every(isDumpRow)) {
throw new APIException( throw new APIException(500, "Malformed dump data");
APIErrorCode.SERVER_ERROR,
500,
"Malformed dump data",
);
} }
const items = rows.map(dumpRowToApi); const items = rows.map(dumpRowToApi);
@@ -398,16 +389,11 @@ export function listDumps(
} }
/** /**
* Returns the dumps that already point at the same thing as `url`, newest * Returns the dumps `requestingUserId` may see that were posted with the same
* first. * link as `url`, newest first, for the create form's duplicate notice
* * (`GET /api/dumps/by-url`). Links are compared in their canonical form, so
* Matched on the canonical key rather than the raw URL, so a link reposted with * tracking parameters, the scheme, `www.` or a YouTube share format don't hide
* different tracking parameters, scheme, `www.`, or (for YouTube) in a * a repost.
* different share shape still counts as the same dump. Private dumps are
* visible only to their owner, exactly as everywhere else.
*
* Returns a small display-only projection: this feeds a hint on the create
* form, not a feed.
*/ */
export function findDumpsByUrl( export function findDumpsByUrl(
url: string, url: string,
@@ -447,12 +433,14 @@ export function findDumpsByUrl(
} }
/** /**
* Applies `request` to a dump: its title, comment, visibility and categories, * Applies `request` to a dump, for `PUT /api/dumps/:dumpId`. A new link has its
* and a URL dump's link (fetching the new preview). Announces the change as the * preview fetched and gives the dump its page's title, unless `request` sets
* visibility requires, and notifies anyone newly @mentioned. * one. A file dump ignores `url`. Clients get the dump as `dump_updated`, or a
* `dump_deleted` when it turns private and a `dump_created` when it turns
* public. Anyone @mentioned in the note is notified.
* *
* @throws {APIException} 404 when there is no such dump, 400 when a new URL isn't a public http(s) URL * @throws {APIException} 404 when there is no such dump, 400 when a new URL
* or a category doesn't exist. * isn't a public http(s) URL or a category doesn't exist.
*/ */
export async function updateDump( export async function updateDump(
dumpId: string, dumpId: string,
@@ -465,7 +453,6 @@ export async function updateDump(
const now = new Date(); const now = new Date();
// File dumps: title, comment and isPrivate are editable
if (dump.kind === "file") { if (dump.kind === "file") {
const title = request.title?.trim() || dump.title; const title = request.title?.trim() || dump.title;
const slug = title !== dump.title ? makeSlug(title, dumpId) : dump.slug; const slug = title !== dump.title ? makeSlug(title, dumpId) : dump.slug;
@@ -498,11 +485,9 @@ export async function updateDump(
} else { } else {
attachCategoryIds([updatedDump]); attachCategoryIds([updatedDump]);
} }
if (updatedDump.isPrivate && !dump.isPrivate) broadcastDumpDeleted(dumpId); broadcastDumpChange(dump, updatedDump);
else if (!updatedDump.isPrivate) broadcastDumpUpdated(updatedDump);
// Going public is an arrival for everyone who couldn't see it before.
if (dump.isPrivate && !updatedDump.isPrivate) broadcastNewDump(updatedDump);
if (updatedDump.comment) { if (updatedDump.comment) {
if (!updatedDump.isPrivate) {
notifyMentions( notifyMentions(
dump.userId, dump.userId,
updatedDump.comment, updatedDump.comment,
@@ -510,17 +495,17 @@ export async function updateDump(
dumpId, dumpId,
updatedDump.title, updatedDump.title,
); );
linkAttachments(updatedDump.comment, dumpId); }
linkAttachments(updatedDump.comment, "dump", dumpId);
} }
relinkBacklinks("dump_comment", dumpId, dumpId, updatedDump.comment ?? ""); relinkBacklinks("dump_comment", dumpId, dumpId, updatedDump.comment ?? "");
return updatedDump; return updatedDump;
} }
// URL dumps
const newUrl = request.url ?? dump.url!; const newUrl = request.url ?? dump.url!;
if (!isValidHttpUrl(newUrl)) { if (!isValidHttpUrl(newUrl)) {
throw new APIException(APIErrorCode.BAD_REQUEST, 400, "Invalid URL"); throw new APIException(400, "Invalid URL");
} }
let { richContent, title } = dump; let { richContent, title } = dump;
@@ -566,7 +551,7 @@ export async function updateDump(
); );
if (result.changes === 0) { if (result.changes === 0) {
throw new APIException(APIErrorCode.NOT_FOUND, 404, "Dump not found"); throw new APIException(404, "Dump not found");
} }
if (request.categoryIds !== undefined) { if (request.categoryIds !== undefined) {
@@ -577,11 +562,9 @@ export async function updateDump(
attachCategoryIds([updatedDump]); attachCategoryIds([updatedDump]);
} }
if (updatedDump.isPrivate && !dump.isPrivate) broadcastDumpDeleted(dumpId); broadcastDumpChange(dump, updatedDump);
else if (!updatedDump.isPrivate) broadcastDumpUpdated(updatedDump);
// Going public is an arrival for everyone who couldn't see it before.
if (dump.isPrivate && !updatedDump.isPrivate) broadcastNewDump(updatedDump);
if (updatedDump.comment) { if (updatedDump.comment) {
if (!updatedDump.isPrivate) {
notifyMentions( notifyMentions(
dump.userId, dump.userId,
updatedDump.comment, updatedDump.comment,
@@ -589,22 +572,23 @@ export async function updateDump(
dumpId, dumpId,
updatedDump.title, updatedDump.title,
); );
linkAttachments(updatedDump.comment, dumpId); }
linkAttachments(updatedDump.comment, "dump", dumpId);
} }
relinkBacklinks("dump_comment", dumpId, dumpId, updatedDump.comment ?? ""); relinkBacklinks("dump_comment", dumpId, dumpId, updatedDump.comment ?? "");
return updatedDump; return updatedDump;
} }
/** /**
* Replaces a file dump's upload, and with it its comment and optionally its * Replaces a file dump's upload and its comment, for
* title, categories and visibility. `isPrivate` left out, visibility stays as * `PUT /api/dumps/:dumpId/file`. A `title`, `categoryIds` or `isPrivate` left
* it is; it is taken here rather than by a second request so a new file and a * out stays as it is. Visibility is taken here so that a new file and a change
* change of visibility land together. The old file is restored if the update * of visibility land together. The old file is put back if the database update
* fails. * fails, and clients are told as {@link updateDump} tells them.
* *
* @throws {APIException} 404 when there is no such dump; 400 when it isn't a file dump, the title or * @throws {APIException} 404 when there is no such dump, 400 when it isn't a
* comment is too long, the file's type isn't allowed, it exceeds 50 MB, or a * file dump, the title or comment is too long, the file's type isn't allowed,
* category doesn't exist. * it exceeds 50 MB, or a category doesn't exist.
*/ */
export async function replaceFileDump( export async function replaceFileDump(
dumpId: string, dumpId: string,
@@ -616,23 +600,15 @@ export async function replaceFileDump(
): Promise<Dump> { ): Promise<Dump> {
assertFileDumpText(title, comment); assertFileDumpText(title, comment);
if (!isAllowedMime(file.type)) { if (!isAllowedMime(file.type)) {
throw new APIException( throw new APIException(400, `File type '${file.type}' is not allowed`);
APIErrorCode.BAD_REQUEST,
400,
`File type '${file.type}' is not allowed`,
);
} }
if (file.size > DUMP_MAX_FILE_SIZE_BYTES) { if (file.size > DUMP_MAX_FILE_SIZE_BYTES) {
throw new APIException( throw new APIException(400, "File too large (max 50 MB)");
APIErrorCode.BAD_REQUEST,
400,
"File too large (max 50 MB)",
);
} }
const dump = fetchDump(dumpId); const dump = fetchDump(dumpId);
if (dump.kind !== "file") { if (dump.kind !== "file") {
throw new APIException(APIErrorCode.BAD_REQUEST, 400, "Not a file dump"); throw new APIException(400, "Not a file dump");
} }
if (categoryIds !== undefined) assertCategoriesExist(categoryIds); if (categoryIds !== undefined) assertCategoriesExist(categoryIds);
@@ -681,7 +657,6 @@ export async function replaceFileDump(
} }
}); });
} catch (err) { } catch (err) {
// Put the previous file back on DB failure
if (hadOld) await Deno.rename(backupPath, filePath).catch(() => {}); if (hadOld) await Deno.rename(backupPath, filePath).catch(() => {});
else await Deno.remove(filePath).catch(() => {}); else await Deno.remove(filePath).catch(() => {});
throw err; throw err;
@@ -696,8 +671,10 @@ export async function replaceFileDump(
} }
if (comment) { if (comment) {
if (!finalPrivate) {
notifyMentions(dump.userId, comment, "dump", dumpId, finalTitle); notifyMentions(dump.userId, comment, "dump", dumpId, finalTitle);
linkAttachments(comment, dumpId); }
linkAttachments(comment, "dump", dumpId);
} }
relinkBacklinks("dump_comment", dumpId, dumpId, comment ?? ""); relinkBacklinks("dump_comment", dumpId, dumpId, comment ?? "");
const updatedDump: Dump = { const updatedDump: Dump = {
@@ -711,15 +688,16 @@ export async function replaceFileDump(
isPrivate: finalPrivate, isPrivate: finalPrivate,
updatedAt: now, updatedAt: now,
}; };
if (updatedDump.isPrivate && !dump.isPrivate) broadcastDumpDeleted(dumpId); broadcastDumpChange(dump, updatedDump);
else if (!updatedDump.isPrivate) broadcastDumpUpdated(updatedDump);
if (dump.isPrivate && !updatedDump.isPrivate) broadcastNewDump(updatedDump);
return updatedDump; return updatedDump;
} }
/** /**
* Returns a page of `userId`'s dumps, newest first, the private ones only with * Returns a page of `userId`'s dumps, newest first, for
* `includePrivate`. * `GET /api/users/:username/dumps`. The route sets `includePrivate` when users
* ask for their own.
*
* @throws {APIException} 500 when a row doesn't parse as a dump.
*/ */
export function getDumpsByUser( export function getDumpsByUser(
userId: string, userId: string,
@@ -733,11 +711,7 @@ export function getDumpsByUser(
`SELECT ${SELECT_COLS} FROM dumps WHERE user_id = ?${privacyFilter} ORDER BY created_at DESC LIMIT ? OFFSET ?;`, `SELECT ${SELECT_COLS} FROM dumps WHERE user_id = ?${privacyFilter} ORDER BY created_at DESC LIMIT ? OFFSET ?;`,
).all(userId, limit + 1, offset); ).all(userId, limit + 1, offset);
if (!rows.every(isDumpRow)) { if (!rows.every(isDumpRow)) {
throw new APIException( throw new APIException(500, "Malformed dump data");
APIErrorCode.SERVER_ERROR,
500,
"Malformed dump data",
);
} }
const items = rows.map(dumpRowToApi); const items = rows.map(dumpRowToApi);
attachCategoryIds(items); attachCategoryIds(items);
@@ -745,8 +719,11 @@ export function getDumpsByUser(
} }
/** /**
* Returns a page of the dumps `userId` upvoted, latest vote first — their own * Returns a page of the dumps `userId` upvoted, latest vote first, for
* private ones only when they are the one asking. * `GET /api/users/:username/votes`. Their own private dumps are included only
* when they are the one asking.
*
* @throws {APIException} 500 when a row doesn't parse as a dump.
*/ */
export function getVotedDumpsByUser( export function getVotedDumpsByUser(
userId: string, userId: string,
@@ -759,7 +736,6 @@ export function getVotedDumpsByUser(
let rows: unknown[]; let rows: unknown[];
if (requestingUserId === userId) { if (requestingUserId === userId) {
// Own profile: include private dumps the user themselves voted on and owns.
rows = db.prepare( rows = db.prepare(
`SELECT ${dumpCols} `SELECT ${dumpCols}
FROM dumps d FROM dumps d
@@ -778,11 +754,7 @@ export function getVotedDumpsByUser(
} }
if (!rows.every(isDumpRow)) { if (!rows.every(isDumpRow)) {
throw new APIException( throw new APIException(500, "Malformed dump data");
APIErrorCode.SERVER_ERROR,
500,
"Malformed dump data",
);
} }
const items = rows.map(dumpRowToApi); const items = rows.map(dumpRowToApi);
attachCategoryIds(items); attachCategoryIds(items);
@@ -790,32 +762,26 @@ export function getVotedDumpsByUser(
} }
/** /**
* Fetches a URL dump's preview again and replaces its title and preview. * Fetches a URL dump's preview again, bypassing the cache, and replaces its
* title and preview, for `POST /api/dumps/:dumpId/refresh-metadata`.
* *
* @throws {APIException} 404 when there is no such dump, 400 when it isn't a URL dump, 502 when the * @throws {APIException} 404 when there is no such dump, 400 when it isn't a
* page can't be reached. * URL dump, 502 when the page can't be reached.
*/ */
export async function refreshDumpMetadata(dumpId: string): Promise<Dump> { export async function refreshDumpMetadata(dumpId: string): Promise<Dump> {
const dump = fetchDump(dumpId); const dump = fetchDump(dumpId);
if (dump.kind !== "url" || !dump.url) { if (dump.kind !== "url" || !dump.url) {
throw new APIException( throw new APIException(400, "Only URL dumps support metadata refresh");
APIErrorCode.BAD_REQUEST,
400,
"Only URL dumps support metadata refresh",
);
} }
// A failed fetch yields a stub with no title and no thumbnail. Writing that // A failed fetch yields a stub with no title or thumbnail. Written over the
// over a dump that already has good metadata would silently destroy it and // dump's metadata, it would rename the dump to its host name.
// rename the dump to its bare hostname, so bail out instead — the caller
// surfaces the error and nothing is lost.
const { ok, content: richContent } = await tryFetchRichContent(dump.url, { const { ok, content: richContent } = await tryFetchRichContent(dump.url, {
fresh: true, fresh: true,
}); });
if (!ok) { if (!ok) {
throw new APIException( throw new APIException(
APIErrorCode.SERVER_ERROR,
502, 502,
"Could not reach the page to refresh its metadata", "Could not reach the page to refresh its metadata",
); );
@@ -829,13 +795,15 @@ export async function refreshDumpMetadata(dumpId: string): Promise<Dump> {
`UPDATE dumps SET title = ?, rich_content = ? WHERE id = ?;`, `UPDATE dumps SET title = ?, rich_content = ? WHERE id = ?;`,
).run(row.title, row.rich_content, row.id); ).run(row.title, row.rich_content, row.id);
if (!updatedDump.isPrivate) broadcastDumpUpdated(updatedDump); broadcastDumpChange(dump, updatedDump);
return updatedDump; return updatedDump;
} }
/** /**
* Returns a dump's id and custom thumbnail type, or undefined without one. * Returns a dump's id and custom thumbnail type, or undefined without one, for
* the thumbnail route. It doesn't check privacy: the route calls
* {@link getDump} first.
* *
* @throws {APIException} 404 when there is no such dump. * @throws {APIException} 404 when there is no such dump.
*/ */
@@ -849,7 +817,8 @@ export function getDumpThumbnailInfo(
} }
/** /**
* Records a dump's new custom thumbnail, once its file is in place. * Records that a dump has a custom thumbnail of type `mime`, for
* `PUT /api/dumps/:dumpId/thumbnail`, once the route has written the file.
* *
* @throws {APIException} 404 when there is no such dump. * @throws {APIException} 404 when there is no such dump.
*/ */
@@ -860,12 +829,13 @@ export function setDumpThumbnail(dumpId: string, mime: string): Dump {
dump.id, dump.id,
); );
const updatedDump: Dump = { ...dump, thumbnailMime: mime }; const updatedDump: Dump = { ...dump, thumbnailMime: mime };
if (!updatedDump.isPrivate) broadcastDumpUpdated(updatedDump); broadcastDumpChange(dump, updatedDump);
return updatedDump; return updatedDump;
} }
/** /**
* Removes a dump's custom thumbnail, and its file. * Removes a dump's custom thumbnail and its file, for
* `DELETE /api/dumps/:dumpId/thumbnail`.
* *
* @throws {APIException} 404 when there is no such dump. * @throws {APIException} 404 when there is no such dump.
*/ */
@@ -875,12 +845,13 @@ export async function removeDumpThumbnail(dumpId: string): Promise<Dump> {
.run(dump.id); .run(dump.id);
await Deno.remove(`${THUMBNAILS_DIR}/${dump.id}-custom`).catch(() => {}); await Deno.remove(`${THUMBNAILS_DIR}/${dump.id}-custom`).catch(() => {});
const updatedDump: Dump = { ...dump, thumbnailMime: undefined }; const updatedDump: Dump = { ...dump, thumbnailMime: undefined };
if (!updatedDump.isPrivate) broadcastDumpUpdated(updatedDump); broadcastDumpChange(dump, updatedDump);
return updatedDump; return updatedDump;
} }
/** /**
* Deletes a dump, its files and thumbnails, and announces it. * Deletes a dump with its files and thumbnails, for
* `DELETE /api/dumps/:dumpId`, and broadcasts `dump_deleted`.
* *
* @throws {APIException} 404 when there is no such dump. * @throws {APIException} 404 when there is no such dump.
*/ */
@@ -890,7 +861,7 @@ export async function deleteDump(dumpId: string): Promise<void> {
const result = db.prepare(`DELETE FROM dumps WHERE id = ?;`).run(dumpId); const result = db.prepare(`DELETE FROM dumps WHERE id = ?;`).run(dumpId);
if (result.changes === 0) { if (result.changes === 0) {
throw new APIException(APIErrorCode.NOT_FOUND, 404, "Dump not found"); throw new APIException(404, "Dump not found");
} }
if (dump.kind === "file") { if (dump.kind === "file") {
@@ -901,5 +872,5 @@ export async function deleteDump(dumpId: string): Promise<void> {
await Deno.remove(`${THUMBNAILS_DIR}/${dump.id}-custom`).catch(() => {}); await Deno.remove(`${THUMBNAILS_DIR}/${dump.id}-custom`).catch(() => {});
} }
broadcastDumpDeleted(dumpId); broadcastDumpChange(dump, null);
} }

View File

@@ -1,5 +1,7 @@
/** /**
* Sending mail through the SMTP server in {@link SMTPS_URL}. * Sends mail through the SMTP server of {@link SMTPS_URL}, which may be unset.
* Its two callers, the welcome email at sign-up and the password reset link,
* check {@link isEmailEnabled} first and don't wait for delivery.
* *
* @module * @module
*/ */
@@ -8,42 +10,43 @@ import nodemailer, { type SendMailOptions, type Transporter } from "nodemailer";
import { SMTPS_URL } from "../config.ts"; import { SMTPS_URL } from "../config.ts";
/** One address, a comma-separated list of them, or an array. */ /**
* One address, a comma-separated list of them, or an array, as nodemailer
* takes them.
*/
export type EmailRecipient = string | string[]; export type EmailRecipient = string | string[];
/** A message to send. It needs a recipient, a subject, and a text or HTML body. */ /**
* Passed to {@link sendEmail}. It needs a recipient, a subject, and a text or
* HTML body; the callers send the same Markdown as both, rendered for `html`.
*/
export interface EmailMessage { export interface EmailMessage {
/** The sender's address. */
from: string; from: string;
/** The recipients. */
to: EmailRecipient; to: EmailRecipient;
/** The subject line. */
subject: string; subject: string;
/** The plain-text body. */
text?: string; text?: string;
/** The HTML body. */
html?: string; html?: string;
/** The carbon-copy recipients. */
cc?: EmailRecipient; cc?: EmailRecipient;
/** The blind-carbon-copy recipients. */
bcc?: EmailRecipient; bcc?: EmailRecipient;
/** The address replies go to. */
replyTo?: string; replyTo?: string;
} }
/** What the SMTP server said about a sent message. */ /**
* Returned by {@link sendEmail} once the SMTP server has taken the message.
* No caller reads it yet.
*/
export interface EmailSendResult { export interface EmailSendResult {
/** The message's `Message-ID`. */
messageId: string; messageId: string;
/** The recipients the server accepted. */
accepted: string[]; accepted: string[];
/** The recipients the server rejected. */
rejected: string[]; rejected: string[];
} }
let transporter: Transporter | null = null; let transporter: Transporter | null = null;
/** Checks whether an SMTP server is configured. */ /**
* Checks whether an SMTP server is configured. Without one, sign-up sends no
* welcome email and a password reset request does nothing.
*/
export function isEmailEnabled(): boolean { export function isEmailEnabled(): boolean {
return SMTPS_URL.length > 0; return SMTPS_URL.length > 0;
} }
@@ -76,19 +79,8 @@ function getTransporter(): Transporter {
} }
/** /**
* Checks the connection to the SMTP server. Resolves false when email is * Sends `message` and resolves once the SMTP server has accepted it. The
* disabled. * transport is created on the first send.
*
* @throws {Error} When the server can't be reached or refuses the login.
*/
export async function verifyEmailTransport(): Promise<boolean> {
if (!isEmailEnabled()) return false;
await getTransporter().verify();
return true;
}
/**
* Sends `message`.
* *
* @throws {Error} When email is disabled, the message lacks a recipient, * @throws {Error} When email is disabled, the message lacks a recipient,
* subject or body, or the server refuses it. * subject or body, or the server refuses it.

Some files were not shown because too many files have changed in this diff Show More