vincent/gerbeur
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

v4: every image the API serves comes scaled to the size it is shown at — a first visit to the feed loaded 4.8 MB of images out of 5.1 MB in all: an avatar shown at 32 pixels was the upload itself, a 1.4 MB GIF among them, and so were a cover, a custom thumbnail, an attachment and a proxied image, only an image dump having a preview, 960 pixels wide. Every route that answers an image now takes ?w= and sends a WebP copy at most that wide, of the widths shared/image.ts lists, 128, 320, 640 and 960, a width between two taking the one above, and never wider than the image; without it the upload goes out as before, a dump's thumbnail alone being scaled to the widest by default, the custom ones included. serveImage (lib/upload.ts) is where they all end, and the image proxy scales what it fetched the same way, keeping each width in its cache. lib/ffmpeg.ts makes the copies as it made the previews, which they replace: on first request, under uploads/scaled/ at the upload's own path, each given its upload's date and made again when the two differ, so a new avatar or cover needs nobody to delete the old ones. WebP keeps a PNG's transparency and a GIF's animation. A JPEG is turned by its EXIF orientation here, ffmpeg's own rotation of images depending on its version. An animated WebP, which ffmpeg can't read, and an image it fails on go out as they are, a failure being remembered until the upload changes. ?f=compat asks for the copy as a JPEG, or a PNG for what may be transparent: the digest's emails take their thumbnails that way, at 640 pixels, not every mail client reading WebP. On the client, imageProps (utils/shared/urls.ts) gives an image its srcset of those copies and the width it is laid out at, which config/imageSizes.ts states for each place from App.css: Avatar, Thumbnail, which scales an API image and leaves another site's as that site sends it, and the playlists' covers use it. The images of notes and comments still load as uploaded. The copies go with the dump, the custom thumbnail or the attachment they were made from, and generatedMediaSweepJob, a minute after the start then daily, takes those whose upload is gone, the previews under their former name and the temporary files a generation left unfinished.

Browse Source 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
khannurien
2026-10-10 19:01:36 +00:00
parent 6ab398e687
commit 90fc281e90
43 changed files with 1147 additions and 211 deletions
Download Patch File Download Diff File Expand all files Collapse all files

10
API.md
View File

@@ -105,8 +105,14 @@ status. A list answers `{ "items", "hasMore" }` and takes `?page=` (from 1) and
thumbnail, waveform, cover or attachment takes the owner's media token as
`?token=`.
JSON answers of a kilobyte or more are compressed, in brotli or gzip, for a
client that accepts it.
An image the API serves, an avatar, a thumbnail, a cover, an attachment or a
remote image, takes `?w=` and answers a WebP copy scaled down to that width,
snapped up to one of `shared/image.ts`'s (128, 320, 640, 960), and never wider
than the image; without it, the image as it was uploaded, a dump's thumbnail
alone being scaled to the widest by default. `?f=compat` asks for the copy as a
JPEG or PNG, for a mail client that reads no WebP. An image no copy can be made
of goes out as it is. JSON answers of a kilobyte or more are compressed, in
brotli or gzip, for a client that accepts it.
### me

2
README.md
View File

@@ -199,7 +199,7 @@ api/
test/ # Backend tests (Deno): test/<layer>/<module>.test.ts
sql/
gerbeur.db # SQLite database (not committed); <name>.db per dev instance
uploads/ # User-uploaded files — avatars, dumps, thumbnails, … (not committed); <name>/ per dev instance
uploads/ # User-uploaded files — avatars, dumps, thumbnails, … and scaled/, their copies made on request (not committed); <name>/ per dev instance
src/ # React frontend (Vite)
config/ # API base URL, default feed tab, player mode, instance settings
pages/ # One component per route, nothing else

6
api/config.ts
View File

@@ -120,6 +120,12 @@ export const ATTACHMENTS_DIR = `${UPLOADS_DIR}/attachments`;
* thumbnails route generates from file dumps.
*/
export const THUMBNAILS_DIR = `${UPLOADS_DIR}/thumbnails`;
/**
* Holds the scaled-down copies of the images above, which lib/ffmpeg.ts makes
* as they are first asked for, each at its upload's path with its width. It
* can be emptied at any time: the copies are made again.
*/
export const SCALED_DIR = `${UPLOADS_DIR}/scaled`;
/**
* Caps the size of an image upload, in bytes, for avatars, covers, thumbnails

2
api/db/startup.ts
View File

@@ -11,6 +11,7 @@ import {
ORPHANED_ATTACHMENTS_RETENTION_HOURS,
UNUSED_INVITES_RETENTION_DAYS,
} from "../config.ts";
import { removeScaledImages } from "../lib/ffmpeg.ts";
import { hashPassword } from "../lib/password.ts";
/**
@@ -60,6 +61,7 @@ export async function runStartupTasks(db: DatabaseSync): Promise<void> {
if (orphanedAttachments.length > 0) {
for (const { id } of orphanedAttachments) {
await Deno.remove(`${ATTACHMENTS_DIR}/${id}`).catch(() => {});
await removeScaledImages(`${ATTACHMENTS_DIR}/${id}`);
}
db.prepare(`DELETE FROM attachments WHERE ${ORPHANED};`)
.run(attachmentCutoff);

307
api/lib/ffmpeg.ts
View File

@@ -1,15 +1,26 @@
/**
* Makes what the server derives from a file dump's upload with ffmpeg: a
* video's still and an image's scaled-down copy, which the thumbnails route
* serves, and an audio file's waveform, which the waveform route serves. Each
* is made on its first request and kept in `THUMBNAILS_DIR` under a name this
* module gives, until dump-service replaces or deletes the upload and
* {@link removeGeneratedMedia} drops them.
* Makes what the server derives from an upload with ffmpeg: a video's still,
* which the thumbnails route serves, an audio file's waveform, which the
* waveform route serves, and the scaled-down copies of any image, which
* `serveImage` (lib/upload.ts) serves for every route that answers one. Each
* is made on its first request and kept under a name this module gives, a
* dump's in `THUMBNAILS_DIR` and the scaled copies in `SCALED_DIR`, until
* dump-service replaces or deletes the upload and
* {@link removeGeneratedMedia} drops them. {@link generatedMediaSweepJob}
* takes what that leaves behind.
*
* @module
*/
import { THUMBNAILS_DIR } from "../config.ts";
import { dirname, relative } from "node:path";
import {
DUMPS_DIR,
SCALED_DIR,
THUMBNAILS_DIR,
UPLOADS_DIR,
} from "../config.ts";
import { IMAGE_WIDTHS, type ImageWidth } from "../../shared/image.ts";
import type { Job } from "./scheduler.ts";
import { type DumpWaveform, peaksOf } from "../../shared/waveform.ts";
import type { APIResponse } from "../../shared/api.ts";
@@ -48,7 +59,7 @@ const VIDEO_FORMATS =
* Lists the formats ffmpeg may read an image as, for the reason given at
* {@link VIDEO_FORMATS}.
*/
const IMAGE_FORMATS = "jpeg_pipe,png_pipe,webp_pipe,bmp_pipe";
const IMAGE_FORMATS = "jpeg_pipe,png_pipe,webp_pipe,bmp_pipe,gif";
/**
* Lists the formats ffmpeg may read an audio file as, for the reason given at
* {@link VIDEO_FORMATS}; an `.m4a` or a `.webm` is read by its container's.
@@ -57,35 +68,50 @@ const AUDIO_FORMATS =
"mp3,ogg,flac,wav,aiff,aac,asf,caf,w64,wv,ape,mov,mp4,m4a,matroska,webm";
/**
* Maps the image types that get a scaled-down preview to the preview's format.
* A GIF is served as it is, to keep its animation.
* Returns where the still and the waveform generated from dump `dumpId`'s
* upload are kept.
*/
export const PREVIEWABLE_IMAGES: Record<string, "jpg" | "png"> = {
"image/jpeg": "jpg",
"image/bmp": "jpg",
// May carry transparency, which JPEG would fill with black.
"image/png": "png",
"image/webp": "png",
};
/**
* Caps an image preview's width. It covers the widest feed card on a
* high-density screen, at a fraction of a phone photo's bytes.
*/
const PREVIEW_WIDTH = 960;
/** Returns where the media generated from dump `dumpId`'s upload are kept. */
export function generatedPaths(dumpId: string): {
still: string;
preview: Record<"jpg" | "png", string>;
waveform: string;
} {
const base = `${THUMBNAILS_DIR}/${dumpId}`;
return {
still: `${base}.jpg`,
preview: { jpg: `${base}.preview.jpg`, png: `${base}.preview.png` },
waveform: `${base}.waveform.json`,
};
return { still: `${base}.jpg`, waveform: `${base}.waveform.json` };
}
/**
* The formats a scaled copy comes in: WebP for a browser, and a JPEG or a PNG
* for what doesn't read it, as some mail clients don't.
*/
export type ScaledFormat = "webp" | "jpg" | "png";
const SCALED_FORMATS: readonly ScaledFormat[] = ["webp", "jpg", "png"];
/**
* Returns where the copy of the upload at `srcPath` scaled to `width` is
* kept, in `format`: under `SCALED_DIR`, at the upload's own path below
* `UPLOADS_DIR`.
*/
export function scaledImagePath(
srcPath: string,
width: ImageWidth,
format: ScaledFormat = "webp",
): string {
return `${SCALED_DIR}/${relative(UPLOADS_DIR, srcPath)}.w${width}.${format}`;
}
/**
* Deletes the scaled copies of the upload at `srcPath`, once it is deleted.
* One that is replaced needs no call: its copies are made again, being older
* than it ({@link ensureGenerated}).
*/
export async function removeScaledImages(srcPath: string): Promise<void> {
await Promise.all(
IMAGE_WIDTHS.flatMap((width) =>
SCALED_FORMATS.map((format) =>
Deno.remove(scaledImagePath(srcPath, width, format)).catch(() => {})
)
),
);
}
/**
@@ -93,12 +119,11 @@ export function generatedPaths(dumpId: string): {
* has replaced or deleted it. The routes make them again from a new one.
*/
export async function removeGeneratedMedia(dumpId: string): Promise<void> {
const { still, preview, waveform } = generatedPaths(dumpId);
await Promise.all(
[still, preview.jpg, preview.png, waveform].map((path) =>
Deno.remove(path).catch(() => {})
),
);
const { still, waveform } = generatedPaths(dumpId);
await Promise.all([
...[still, waveform].map((path) => Deno.remove(path).catch(() => {})),
removeScaledImages(`${DUMPS_DIR}/${dumpId}`),
]);
}
/**
@@ -109,7 +134,7 @@ async function writeAtomically(
outPath: string,
write: (tmpPath: string) => Promise<void>,
): Promise<void> {
await Deno.mkdir(THUMBNAILS_DIR, { recursive: true });
await Deno.mkdir(dirname(outPath), { recursive: true });
const tmpPath = `${outPath}.${crypto.randomUUID()}.tmp`;
try {
await write(tmpPath);
@@ -156,25 +181,87 @@ export function videoStill(srcPath: string, outPath: string): Promise<void> {
], outPath);
}
/** Writes a copy of an image at most {@link PREVIEW_WIDTH} wide. */
export function imagePreview(srcPath: string, outPath: string): Promise<void> {
/**
* Maps a JPEG's EXIF orientation to the filters that turn its pixels upright.
* A browser applies the orientation to the upload itself; the copy carries no
* EXIF, so it is turned here, and not by ffmpeg's own autorotation, which
* only its recent versions apply to images.
*/
const ORIENTATION_FILTERS: Record<number, string> = {
2: "hflip",
3: "hflip,vflip",
4: "vflip",
5: "transpose=0",
6: "transpose=1",
7: "transpose=3",
8: "transpose=2",
};
/** What ffmpeg is told to write each {@link ScaledFormat} with. */
const ENCODERS: Record<ScaledFormat, string[]> = {
webp: ["-c:v", "libwebp", "-q:v", "80", "-loop", "0", "-f", "webp"],
// A still: the first frame of an animation.
jpg: ["-frames:v", "1", "-c:v", "mjpeg", "-q:v", "4", "-f", "image2"],
png: ["-frames:v", "1", "-c:v", "png", "-f", "image2"],
};
/**
* Writes a copy of an image at most `width` wide, never wider than the image
* itself, as a WebP unless `format` names another. WebP keeps a PNG's
* transparency and a GIF's animation, at a fraction of either's bytes.
* `orientation` is the EXIF orientation of a JPEG, as `scalableImage`
* (lib/upload.ts) reads it.
*/
export function scaleImage(
srcPath: string,
outPath: string,
width: ImageWidth,
orientation = 1,
format: ScaledFormat = "webp",
): Promise<void> {
const upright = ORIENTATION_FILTERS[orientation];
// A JPEG's colour planes halve its height, which they need even.
const height = format === "jpg" ? -2 : -1;
return runFfmpeg([
"-noautorotate",
"-format_whitelist",
IMAGE_FORMATS,
"-i",
srcPath,
"-frames:v",
"1",
"-vf",
`scale='min(${PREVIEW_WIDTH},iw)':-2`,
"-f",
"image2",
...(outPath.endsWith(".png")
? ["-c:v", "png"]
: ["-c:v", "mjpeg", "-q:v", "4"]),
`${upright ? `${upright},` : ""}scale='min(${width},iw)':${height}`,
"-an",
...ENCODERS[format],
], outPath);
}
/**
* Returns `bytes`, an image held in memory, scaled as {@link scaleImage}
* does, or null when it can't be, because ffmpeg is missing or failed. The
* image proxy scales what it fetched with it, and keeps the result itself.
*/
export async function scaleImageBytes(
bytes: Uint8Array,
width: ImageWidth,
orientation = 1,
): Promise<Uint8Array | null> {
if (!await ffmpegAvailable()) return null;
const srcPath = `${SCALED_DIR}/tmp/${crypto.randomUUID()}`;
const outPath = `${srcPath}.webp`;
try {
await Deno.mkdir(dirname(srcPath), { recursive: true });
await Deno.writeFile(srcPath, bytes);
await scaleImage(srcPath, outPath, width, orientation);
return await Deno.readFile(outPath);
} catch {
return null;
} finally {
await Promise.all(
[srcPath, outPath].map((path) => Deno.remove(path).catch(() => {})),
);
}
}
/**
* The sample rate the audio is decoded at for its waveform. A bar stands for
* a second or more of a song, so 8 kHz keeps every peak worth drawing at a
@@ -269,20 +356,140 @@ export function audioWaveform(srcPath: string, outPath: string): Promise<void> {
*/
const inFlight = new Map<string, Promise<void>>();
/**
* Spares an ffmpeg run per request for an image it can't read: the
* generations that failed, by output path, with the date of the source they
* failed on. A new upload gets a new try.
*/
const failed = new Map<string, number>();
/** Returns the date `path` was last written, or null when it doesn't exist. */
async function writtenAt(path: string): Promise<number | null> {
try {
return (await Deno.stat(path)).mtime?.getTime() ?? 0;
} catch {
return null;
}
}
/**
* Makes `outPath` with `generate` unless it exists. Resolves false when it
* can't be made, because ffmpeg is missing or failed.
*
* With `srcPath`, the file it is made from, `outPath` takes that file's date,
* and is made again when the two differ: an upload replaced in place, as an
* avatar or a cover is, gets new copies without anyone deleting the old.
* Without it, the caller deletes `outPath` when its source changes.
*/
export async function ensureGenerated(
outPath: string,
generate: () => Promise<void>,
srcPath?: string,
): Promise<boolean> {
if (await Deno.stat(outPath).then(() => true, () => false)) return true;
const made = await writtenAt(outPath);
let version: number | null = null;
if (srcPath) {
version = await writtenAt(srcPath);
if (version === null) return false;
if (made === version) return true;
if (failed.get(outPath) === version) return false;
} else if (made !== null) {
return true;
}
if (!await ffmpegAvailable()) return false;
let pending = inFlight.get(outPath);
if (!pending) {
pending = generate().finally(() => inFlight.delete(outPath));
pending = generate()
.then(() =>
version === null
? undefined
: Deno.utime(outPath, new Date(), new Date(version))
)
.catch((err) => {
if (version !== null) failed.set(outPath, version);
throw err;
})
.finally(() => inFlight.delete(outPath));
inFlight.set(outPath, pending);
}
return await pending.then(() => true, () => false);
}
/**
* How old a temporary file must be for the sweep to take it: one being
* written, by a generation under way, is seconds old, and one a crash left
* is never finished.
*/
const ABANDONED_AFTER_MS = 60 * 60_000;
/** Yields the path of every file under `dir`, and nothing when it is missing. */
async function* filesUnder(dir: string): AsyncGenerator<string> {
let entries: Deno.DirEntry[];
try {
entries = await Array.fromAsync(Deno.readDir(dir));
} catch {
return;
}
for (const entry of entries) {
const path = `${dir}/${entry.name}`;
if (entry.isDirectory) yield* filesUnder(path);
else if (entry.isFile) yield path;
}
}
/**
* Deletes the generated files nothing serves any more, and returns how many:
* the scaled copies of an upload that is gone, which the services' own
* removals missed or never made, as for a user's avatar; the previews of
* image dumps under their former name, `<id>.preview.jpg` or `.png`, which
* the scaled copies replaced; and the temporary files a generation left
* unfinished. Nothing else of `THUMBNAILS_DIR` is touched, the custom
* thumbnails kept there being uploads.
*/
export async function sweepGeneratedMedia(): Promise<number> {
let removed = 0;
const remove = (path: string) =>
Deno.remove(path).then(() => {
removed++;
}, () => {});
const abandoned = async (path: string) => {
const at = await writtenAt(path);
return at !== null && Date.now() - at > ABANDONED_AFTER_MS;
};
for await (const path of filesUnder(THUMBNAILS_DIR)) {
if (
/\.preview\.(jpg|png)$/.test(path) ||
(path.endsWith(".tmp") && await abandoned(path))
) await remove(path);
}
for await (const path of filesUnder(SCALED_DIR)) {
// A copy is named after its upload's path: see scaledImagePath.
const copy = relative(SCALED_DIR, path).match(
/^(.+)\.w\d+\.(?:webp|jpg|png)$/,
);
const unused = copy
? await writtenAt(`${UPLOADS_DIR}/${copy[1]}`) === null
: await abandoned(path);
if (unused) await remove(path);
}
return removed;
}
/**
* Runs {@link sweepGeneratedMedia} a minute after the server starts, which
* clears what a new version no longer uses as soon as it is deployed, then
* once a day. `main.ts` starts it.
*/
export const generatedMediaSweepJob: Job = {
name: "media-sweep",
everyMs: 24 * 60 * 60_000,
firstAfterMs: 60_000,
async run() {
const removed = await sweepGeneratedMedia();
if (removed > 0) {
console.log(`[media-sweep] removed ${removed} generated files unused`);
}
},
};

202
api/lib/upload.ts
View File

@@ -1,6 +1,7 @@
/**
* Checks the images users upload and serves them back. The avatar, playlist
* cover, dump thumbnail and attachment routes call it.
* Checks the images users upload and serves them back, scaled down to the
* width a request names. The avatar, playlist cover, dump thumbnail and
* attachment routes call it.
*
* @module
*/
@@ -10,6 +11,13 @@ import { APIException } from "./api-exception.ts";
import { serveFile } from "./serve-file.ts";
import { dirname } from "node:path";
import { MAX_IMAGE_SIZE_BYTES } from "../config.ts";
import { type ImageWidth, parseImageWidth } from "../../shared/image.ts";
import {
ensureGenerated,
type ScaledFormat,
scaledImagePath,
scaleImage,
} from "./ffmpeg.ts";
/**
* Returns the MIME type of a JPEG, PNG, GIF or WebP image from its magic
@@ -62,10 +70,178 @@ export function validateImageUpload(data: Uint8Array): string {
}
/**
* Serves an uploaded image inline as `mime`, cached for `cacheMaxAge` seconds,
* and only by the browser when `isPrivate`. It answers 404 and resolves false
* when the file is missing. The caller checks first that the viewer may see
* it.
* Returns the EXIF orientation of a JPEG, 1 to 8, from the start of its
* bytes, and 1, upright, when it states none. A phone stores a photo as its
* sensor saw it and says here how to turn it.
*/
function jpegOrientation(data: Uint8Array): number {
const view = new DataView(data.buffer, data.byteOffset, data.byteLength);
// The segments follow the two-byte SOI, each a marker and its length.
let at = 2;
while (at + 4 <= data.length && data[at] === 0xFF) {
const marker = data[at + 1];
const length = view.getUint16(at + 2);
// APP1 holding "Exif\0\0", then a TIFF header and its first directory.
if (marker === 0xE1 && view.getUint32(at + 4) === 0x45786966) {
const tiff = at + 10;
if (tiff + 8 > data.length) return 1;
const little = view.getUint16(tiff) === 0x4949;
const ifd = tiff + view.getUint32(tiff + 4, little);
if (ifd + 2 > data.length) return 1;
const entries = view.getUint16(ifd, little);
for (let i = 0; i < entries; i++) {
const entry = ifd + 2 + i * 12;
if (entry + 12 > data.length) return 1;
if (view.getUint16(entry, little) === 0x0112) {
const value = view.getUint16(entry + 8, little);
return value >= 1 && value <= 8 ? value : 1;
}
}
return 1;
}
// The image data starts: no EXIF came before it.
if (marker === 0xDA) return 1;
at += 2 + length;
}
return 1;
}
/**
* How much of a file {@link scalableImage} needs: the EXIF segment, which
* comes first in a JPEG, is 64 KB at most.
*/
const IMAGE_HEAD_BYTES = 64 * 1024 + 32;
/** Returned by {@link scalableImage}, for `scaleImage`. */
export interface ScalableImage {
/** The EXIF orientation to turn the image by, 1 when it is upright. */
orientation: number;
/**
* The format of its copy for a reader without WebP: a JPEG, or a PNG for
* an image that may be transparent, which a JPEG would fill with black.
*/
compat: "jpg" | "png";
}
/**
* Returns how to scale the image whose first {@link IMAGE_HEAD_BYTES} are
* `head`, or null when it is to be served as it is: an animated WebP, which
* ffmpeg can't read, or anything but a JPEG, PNG, GIF, WebP or BMP.
*/
export function scalableImage(head: Uint8Array): ScalableImage | null {
const mime = detectImageMime(head);
// A file dump may be a BMP ("BM"), which no upload route here accepts.
if (!mime) {
return head[0] === 0x42 && head[1] === 0x4D
? { orientation: 1, compat: "jpg" }
: null;
}
// "RIFF....WEBPVP8X", whose flags then say whether frames follow.
const animated = mime === "image/webp" && head[15] === 0x58 &&
(head[20] & 0x02) !== 0;
if (animated) return null;
return mime === "image/jpeg"
? { orientation: jpegOrientation(head), compat: "jpg" }
: { orientation: 1, compat: "png" };
}
/** Reads the first {@link IMAGE_HEAD_BYTES} of the file at `path`. */
async function readHead(path: string): Promise<Uint8Array | null> {
let file: Deno.FsFile;
try {
file = await Deno.open(path, { read: true });
} catch {
return null;
}
try {
const head = new Uint8Array(IMAGE_HEAD_BYTES);
let filled = 0;
while (filled < head.length) {
const n = await file.read(head.subarray(filled));
if (n === null) break;
filled += n;
}
return head.subarray(0, filled);
} finally {
file.close();
}
}
/** The type each format of a scaled copy is served as. */
const SCALED_MIMES: Record<ScaledFormat, string> = {
webp: "image/webp",
jpg: "image/jpeg",
png: "image/png",
};
/**
* Returns the copy of the image at `path` scaled to `width`, made now unless
* it was already, as its path and type, or null when there is none to serve:
* the image can't be scaled ({@link scalableImage}), or ffmpeg is missing or
* failed on it. The copy is a WebP, or with `compat` the JPEG or PNG a reader
* without WebP takes.
*/
async function scaledCopy(
path: string,
width: ImageWidth,
compat: boolean,
): Promise<{ path: string; mime: string } | null> {
// The other format depends on the image, so it is read before the copy is
// looked for; a WebP's path is known without.
let how = compat ? await describe(path) : undefined;
if (how === null) return null;
const format: ScaledFormat = how ? how.compat : "webp";
const scaledPath = scaledImagePath(path, width, format);
const made = await ensureGenerated(scaledPath, async () => {
how ??= await describe(path);
if (!how) throw new Error("Not an image that can be scaled");
await scaleImage(path, scaledPath, width, how.orientation, format);
}, path);
return made ? { path: scaledPath, mime: SCALED_MIMES[format] } : null;
}
/** Returns how to scale the image at `path`: see {@link scalableImage}. */
async function describe(path: string): Promise<ScalableImage | null> {
const head = await readHead(path);
return head && scalableImage(head);
}
/**
* Sends the image at `path`, of type `mime`, with `cacheControl`: a WebP copy
* at most as wide as the request's `?w=` names (`shared/image.ts`), or as
* `defaultWidth` without one, and the image as it is when the request names
* no width or the copy can't be made. With `?f=compat`, the copy is a JPEG,
* or a PNG for an image that may be transparent, which the digest's emails
* ask for, since not every mail client reads WebP. Every route that answers
* one of the site's own images ends here, so each takes both. It resolves
* false, having sent nothing, when the file is missing.
*/
export async function serveImage(
ctx: Context,
path: string,
mime: string,
cacheControl: string,
defaultWidth?: ImageWidth,
): Promise<boolean> {
// The browser must take the checked type, never guess another from the
// bytes.
ctx.response.headers.set("X-Content-Type-Options", "nosniff");
const width = parseImageWidth(ctx.request.url.searchParams.get("w")) ??
defaultWidth;
const compat = ctx.request.url.searchParams.get("f") === "compat";
const scaled = width ? await scaledCopy(path, width, compat) : null;
if (
scaled &&
await serveFile(ctx, scaled.path, { mime: scaled.mime, cacheControl })
) return true;
return await serveFile(ctx, path, { mime, cacheControl });
}
/**
* Serves an uploaded image inline as `mime`, or scaled down as
* {@link serveImage} does, cached for `cacheMaxAge` seconds, and only by the
* browser when `isPrivate`. It answers 404 and resolves false when the file
* is missing. The caller checks first that the viewer may see it.
*/
export async function serveUploadedFile(
ctx: Context,
@@ -73,16 +249,18 @@ export async function serveUploadedFile(
mime: string,
cacheMaxAge = 3600,
isPrivate = false,
defaultWidth?: ImageWidth,
): Promise<boolean> {
ctx.response.headers.set("Content-Disposition", "inline");
// The browser must take the checked type, never guess another from the
// bytes, and a frontend on another origin may still show the image.
ctx.response.headers.set("X-Content-Type-Options", "nosniff");
// A frontend on another origin may still show the image.
ctx.response.headers.set("Cross-Origin-Resource-Policy", "cross-origin");
const served = await serveFile(ctx, filePath, {
const served = await serveImage(
ctx,
filePath,
mime,
cacheControl: `${isPrivate ? "private" : "public"}, max-age=${cacheMaxAge}`,
});
`${isPrivate ? "private" : "public"}, max-age=${cacheMaxAge}`,
defaultWidth,
);
if (!served) ctx.response.status = 404;
return served;
}

2
api/main.ts
View File

@@ -53,6 +53,7 @@ import { ogMiddleware } from "./middleware/og.ts";
import { routeStaticFilesFrom } from "./lib/static.ts";
import { startJobs } from "./lib/scheduler.ts";
import { rateLimitSweepJob } from "./lib/rate-limit.ts";
import { generatedMediaSweepJob } from "./lib/ffmpeg.ts";
import { digestJob } from "./services/digest-service.ts";
import { communityJob } from "./services/community-service.ts";
import { pingJob, typingExpiryJob } from "./services/ws-service.ts";
@@ -141,6 +142,7 @@ if (import.meta.main) {
pingJob,
typingExpiryJob,
rateLimitSweepJob,
generatedMediaSweepJob,
outboxJob,
]);
await app.listen({ hostname: LISTEN_HOST, port: PORT });

34
api/remote/image-proxy.ts
View File

@@ -3,12 +3,15 @@
* http, which an https page would block, and those behind hotlink protection.
* It fetches and caches the images `GET /api/remote/image` serves, which the
* frontend routes a thumbnail through when it is on http or when loading it
* directly failed.
* directly failed, and scales them down as the site's own are.
*
* @module
*/
import { fetchWithTimeout, readBytesCapped } from "./fetch.ts";
import type { ImageWidth } from "../../shared/image.ts";
import { scaleImageBytes } from "../lib/ffmpeg.ts";
import { scalableImage } from "../lib/upload.ts";
/**
* Returned to `GET /api/remote/image` by {@link fetchProxiedImage}, which sends
@@ -34,9 +37,9 @@ const PROXYABLE_IMAGE_TYPES = new Set([
const IMAGE_CACHE_TTL_MS = 24 * 60 * 60 * 1000;
const IMAGE_CACHE_MAX_BYTES = 32 * 1024 * 1024;
/**
* Proxied images, by URL, since every viewer of a feed asks for the same
* thumbnails. The least recently used go first, and the Map's order is that
* of use.
* Proxied images, by URL and by the width they were scaled to, since every
* viewer of a feed asks for the same thumbnails. The least recently used go
* first, and the Map's order is that of use.
*/
const imageCache = new Map<
string,
@@ -89,15 +92,34 @@ export function clearImageCache(): void {
/**
* Returns the image at `url`, from the cache or else from its host; null when
* the host answers with an error, or with anything but a raster image of 5 MB
* at most. The route
* checks `url` with `isValidHttpUrl` first.
* at most. With `width`, it comes as a WebP copy scaled down to it, or as it
* is when no copy can be made (`scalableImage`, and ffmpeg). The route checks
* `url` with `isValidHttpUrl` first.
*
* @throws {Error} When the host can't be reached, as
* {@link fetchWithTimeout} reports it.
*/
export async function fetchProxiedImage(
url: string,
width?: ImageWidth | null,
): Promise<ProxiedImage | null> {
if (!width) return await fetchOriginal(url);
const key = `${url}#w${width}`;
const cached = readImageCache(key);
if (cached) return cached;
const original = await fetchOriginal(url);
if (!original) return null;
const how = scalableImage(original.bytes);
const bytes = how &&
await scaleImageBytes(original.bytes, width, how.orientation);
if (!bytes) return original;
writeImageCache(key, bytes, "image/webp");
return { bytes, contentType: "image/webp" };
}
/** Returns the image at `url` as its host sends it: see above. */
async function fetchOriginal(url: string): Promise<ProxiedImage | null> {
const cached = readImageCache(url);
if (cached) return cached;

7
api/routes/attachments.ts
View File

@@ -4,9 +4,10 @@
*
* - `POST /api/attachments` — signed in. Stores an image (multipart `file`, at
* most 5 MB) and answers its id.
* - `GET /api/attachments/:attachmentId?token=` — public, but an image in a
* private dump or playlist, or in a comment on a private dump, needs its
* owner's media `token`. Serves the image.
* - `GET /api/attachments/:attachmentId?w=&token=` — public, but an image in
* a private dump or playlist, or in a comment on a private dump, needs its
* owner's media `token`. Serves the image, scaled down to `w`
* (`shared/image.ts`) when it names one.
*
* @module
*/

8
api/routes/avatars.ts
View File

@@ -1,14 +1,14 @@
/**
* Stores and serves the users' avatars. The profile page uploads the avatar,
* and the client's `Avatar` component loads it, with the user's `updatedAt` as
* a cache-busting version.
* and the client's `Avatar` component loads it at the size it shows, with the
* user's `updatedAt` as a cache-busting version.
*
* - `PUT /api/me/avatar` — signed in. Replaces the caller's avatar (multipart
* `file`, at most 5 MB), sends `user_updated` to everyone, and answers the
* caller's user. 400 for an account from another instance, whose avatar
* comes from there. The instances the caller signed in to are told.
* - `GET /api/users/:key/avatar` — public. Serves a user's avatar; 404 without
* one.
* - `GET /api/users/:key/avatar?w=` — public. Serves a user's avatar, scaled
* down to `w` (`shared/image.ts`) when it names one; 404 without one.
*
* @module
*/

6
api/routes/playlists.ts
View File

@@ -26,9 +26,9 @@
* or with `playlist:moderate`. Removes a dump.
* - `PUT /api/playlists/:playlistId/cover` — signed in, as its owner or with
* `playlist:moderate`. Replaces its cover (multipart `file`, at most 5 MB).
* - `GET /api/playlists/:playlistId/cover?token=` — public, but a private
* playlist's cover needs its owner's media `token`. Serves its cover, or 404s
* without one.
* - `GET /api/playlists/:playlistId/cover?w=&token=` — public, but a private
* playlist's cover needs its owner's media `token`. Serves its cover, scaled
* down to `w` (`shared/image.ts`) when it names one, or 404s without one.
* - `PUT /api/playlists/:playlistId/order` — signed in, as its owner or with
* `playlist:moderate`. Reorders the dumps the caller can see in it.
*

12
api/routes/remote.ts
View File

@@ -6,9 +6,9 @@
*
* - `GET /api/remote/preview?url=` — signed in. Answers the preview of a link,
* and whether its page could be reached.
* - `GET /api/remote/image?url=` — public. Serves a remote raster image
* through this site; 400 when it isn't one, 502 when its host can't be
* reached.
* - `GET /api/remote/image?url=&w=` — public. Serves a remote raster image
* through this site, scaled down to `w` (`shared/image.ts`) when it names
* one; 400 when it isn't an image, 502 when its host can't be reached.
* - `GET /api/remote/bandcamp?url=&force=` — public. Answers a Bandcamp
* page's tracks and their stream URLs, for the native player; `force=1`
* skips the server's cache.
@@ -19,6 +19,7 @@
*/
import { Router } from "@oak/oak";
import { parseImageWidth } from "../../shared/image.ts";
import { isValidHttpUrl } from "../remote/fetch.ts";
import { fetchProxiedImage } from "../remote/image-proxy.ts";
import { resolveBandcamp } from "../remote/bandcamp-streams.ts";
@@ -88,7 +89,10 @@ router.get("/image", async (ctx) => {
}
try {
const image = await fetchProxiedImage(url);
const image = await fetchProxiedImage(
url,
parseImageWidth(ctx.request.url.searchParams.get("w")),
);
if (!image) {
ctx.response.status = 400;
return;

51
api/routes/thumbnails.ts
View File

@@ -5,11 +5,12 @@
* from an upload are generated on first request by lib/ffmpeg.ts, which keeps
* them.
*
* - `GET /api/dumps/:dumpId/thumbnail?v=&token=` — public; a private dump's
* thumbnail needs its poster's media `token`. Serves the dump's custom
* thumbnail, else one made from its upload: a still of a video, or a
* scaled-down copy of an image (the image itself when none can be made); 404
* when there is neither.
* - `GET /api/dumps/:dumpId/thumbnail?v=&w=&token=` — public; a private
* dump's thumbnail needs its poster's media `token`. Serves the dump's
* custom thumbnail, else one made from its upload: a still of a video, or
* its image. An image comes scaled down to `w` (`shared/image.ts`), or to
* the widest of those widths without one, and as it is when no copy can be
* made; 404 when there is neither.
* - `PUT /api/dumps/:dumpId/thumbnail` — signed in, as its poster or with
* `dump:moderate`. Replaces its custom thumbnail (multipart `file`, at most 5
* MB).
@@ -32,18 +33,14 @@ import {
import { serveFile } from "../lib/serve-file.ts";
import {
replaceUploadedFile,
serveImage,
serveUploadedFile,
validateImageUpload,
} from "../lib/upload.ts";
import { DUMPS_DIR } from "../config.ts";
import { MAX_IMAGE_WIDTH } from "../../shared/image.ts";
import { resolveMediaDump } from "../lib/dump-media.ts";
import {
ensureGenerated,
generatedPaths,
imagePreview,
PREVIEWABLE_IMAGES,
videoStill,
} from "../lib/ffmpeg.ts";
import { ensureGenerated, generatedPaths, videoStill } from "../lib/ffmpeg.ts";
import { can } from "../lib/permissions.ts";
import { type AuthContext, authMiddleware } from "../middleware/auth.ts";
@@ -69,6 +66,7 @@ router.get("/:dumpId/thumbnail", async (ctx) => {
dump.thumbnailMime,
0,
dump.isPrivate,
MAX_IMAGE_WIDTH,
);
if (served) return;
}
@@ -92,29 +90,12 @@ router.get("/:dumpId/thumbnail", async (ctx) => {
) return;
}
if (mime.startsWith("image/")) {
// A scaled-down copy for the feed; the original when there's none to make
// (a GIF, no ffmpeg, an image ffmpeg can't read).
ctx.response.headers.set("X-Content-Type-Options", "nosniff");
const ext = PREVIEWABLE_IMAGES[mime];
if (ext) {
const previewPath = generated.preview[ext];
if (
await ensureGenerated(
previewPath,
() => imagePreview(srcPath, previewPath),
) &&
await serveFile(ctx, previewPath, {
mime: ext === "png" ? "image/png" : "image/jpeg",
cacheControl: generatedCache,
})
) return;
}
if (
/^image\/(png|jpeg|gif|webp|avif|bmp)$/.test(mime) &&
await serveFile(ctx, srcPath, { mime, cacheControl: generatedCache })
) return;
}
// A scaled-down copy for the feed; the original when there's none to make
// (no ffmpeg, an image it can't read).
if (
/^image\/(png|jpeg|gif|webp|avif|bmp)$/.test(mime) &&
await serveImage(ctx, srcPath, mime, generatedCache, MAX_IMAGE_WIDTH)
) return;
throw new APIException(404, "No thumbnail for this dump");
});

15
api/services/digest-service.ts
View File

@@ -25,6 +25,7 @@ import {
} from "../../shared/settings.ts";
import type { CommunityStats } from "../../shared/community.ts";
import { type Dump, dumpFileVersion } from "../../shared/dump.ts";
import type { ImageWidth } from "../../shared/image.ts";
import { siteHost } from "../../shared/url.ts";
import { markdownToPlainText } from "../../shared/markdown.ts";
import {
@@ -101,11 +102,19 @@ function quoteOf(markdown: string | undefined | null): string | undefined {
: flat;
}
/**
* The width the emails' thumbnails are asked at: the widest an email shows
* one, on a dense screen.
*/
const EMAIL_IMAGE_WIDTH: ImageWidth = 640;
/**
* Returns the image a mail client can load for `dump`, which must be public:
* the thumbnails route for a custom thumbnail or an image or video upload,
* else the link preview's image, through the image proxy when it is plain
* http, which a mail client may refuse.
* scaled to the email's width as a JPEG or PNG (`f=compat`), since not every
* mail client reads the WebP a browser gets; else the link preview's image,
* through the image proxy when it is plain http, which a mail client may
* refuse.
*/
function thumbnailOf(dump: Dump): string | null {
const isMedia = dump.kind === "file" &&
@@ -113,7 +122,7 @@ function thumbnailOf(dump: Dump): string | null {
if (dump.thumbnailMime || isMedia) {
return `${PUBLIC_URL}/api/dumps/${dump.id}/thumbnail?v=${
dumpFileVersion(dump)
}`;
}&w=${EMAIL_IMAGE_WIDTH}&f=compat`;
}
const thumbnailUrl = dump.richContent?.thumbnailUrl;
if (typeof thumbnailUrl !== "string") return null;

15
api/services/dump-service.ts
View File

@@ -45,7 +45,7 @@ import {
} from "./notification-service.ts";
import { changeSlug, newSlug, resolveSlug } from "./slug-service.ts";
import { canonicalizeUrl } from "../lib/canonical-url.ts";
import { removeGeneratedMedia } from "../lib/ffmpeg.ts";
import { removeGeneratedMedia, removeScaledImages } from "../lib/ffmpeg.ts";
import {
DUMP_ALLOWED_MIME_PREFIXES,
DUMP_ALLOWED_MIME_TYPES,
@@ -854,6 +854,13 @@ export function setDumpThumbnail(dumpId: string, mime: string): Dump {
return updatedDump;
}
/** Deletes the file of dump `dumpId`'s custom thumbnail, and its copies. */
async function removeCustomThumbnail(dumpId: string): Promise<void> {
const path = customThumbnailPath(dumpId);
await Deno.remove(path).catch(() => {});
await removeScaledImages(path);
}
/**
* Removes a dump's custom thumbnail and its file, for
* `DELETE /api/dumps/:dumpId/thumbnail`.
@@ -864,7 +871,7 @@ export async function removeDumpThumbnail(dumpId: string): Promise<Dump> {
const dump = fetchDump(dumpId);
db.prepare(`UPDATE dumps SET custom_thumbnail_mime = NULL WHERE id = ?;`)
.run(dump.id);
await Deno.remove(customThumbnailPath(dump.id)).catch(() => {});
await removeCustomThumbnail(dump.id);
const updatedDump: Dump = { ...dump, thumbnailMime: undefined };
announceDumpChange(dump, updatedDump);
return updatedDump;
@@ -891,9 +898,7 @@ export async function deleteDump(dumpId: string): Promise<void> {
await Deno.remove(`${DUMPS_DIR}/${dump.id}`).catch(() => {});
await removeGeneratedMedia(dump.id);
}
if (dump.thumbnailMime) {
await Deno.remove(customThumbnailPath(dump.id)).catch(() => {});
}
if (dump.thumbnailMime) await removeCustomThumbnail(dump.id);
announceDumpChange(dump, null, holders);
}

54
api/test/helpers.ts
View File

@@ -388,6 +388,60 @@ export const PNG: Uint8Array<ArrayBuffer> = Uint8Array.from(
(c) => c.charCodeAt(0),
);
/** Whether ffmpeg runs here; the tests of what it makes are skipped without. */
export async function hasFfmpeg(): Promise<boolean> {
try {
return (await new Deno.Command("ffmpeg", {
args: ["-version"],
stdout: "null",
stderr: "null",
}).output()).success;
} catch {
return false;
}
}
/**
* Makes a real image with ffmpeg, a `width`×`height` field of `color`, as a
* PNG, or as a GIF of three frames.
*/
export async function makeImage(
width: number,
height: number,
color = "red",
format: "png" | "gif" = "png",
): Promise<Uint8Array<ArrayBuffer>> {
const { stdout } = await new Deno.Command("ffmpeg", {
args: [
"-f",
"lavfi",
"-i",
`color=c=${color}:size=${width}x${height}:rate=3`,
"-frames:v",
format === "gif" ? "3" : "1",
"-f",
format === "gif" ? "gif" : "image2pipe",
...(format === "gif" ? [] : ["-c:v", "png"]),
"-",
],
stdout: "piped",
stderr: "null",
}).output();
return new Uint8Array(stdout);
}
/** Returns the width, in pixels, a WebP file's header states. */
export function webpWidth(bytes: Uint8Array): number {
const kind = new TextDecoder().decode(bytes.subarray(12, 16));
// Lossy: the frame's start code, then 14 bits of width.
if (kind === "VP8 ") return (bytes[26] | (bytes[27] << 8)) & 0x3FFF;
// Lossless: a signature byte, then 14 bits of width minus one.
if (kind === "VP8L") return ((bytes[21] | (bytes[22] << 8)) & 0x3FFF) + 1;
// Extended, as an animation or an image with transparency: the canvas's
// width minus one, on 24 bits.
return (bytes[24] | (bytes[25] << 8) | (bytes[26] << 16)) + 1;
}
export function formWithFile(
bytes: Uint8Array<ArrayBuffer>,
name = "file.png",

62
api/test/lib/ffmpeg.test.ts Normal file
View File

@@ -0,0 +1,62 @@
import { TEST_DIR } from "../setup.ts";
import { assertEquals } from "@std/assert";
import { sweepGeneratedMedia } from "../../lib/ffmpeg.ts";
const UPLOADS = `${TEST_DIR}/uploads`;
async function write(path: string, ageMs = 0): Promise<void> {
const full = `${UPLOADS}/${path}`;
await Deno.mkdir(full.slice(0, full.lastIndexOf("/")), { recursive: true });
await Deno.writeTextFile(full, "x");
if (ageMs > 0) {
const at = new Date(Date.now() - ageMs);
await Deno.utime(full, at, at);
}
}
const exists = (path: string) =>
Deno.stat(`${UPLOADS}/${path}`).then(() => true, () => false);
Deno.test("sweepGeneratedMedia: takes the copies of gone uploads, the old previews and abandoned temporary files", async () => {
const HOURS = 60 * 60_000;
const kept = [
// An upload and its copies, in each format.
"avatars/u1",
"scaled/avatars/u1.w128.webp",
"scaled/avatars/u1.w640.png",
// A custom thumbnail is an upload; a still and a waveform go with their
// dump, not here.
"thumbnails/d1-custom",
"scaled/thumbnails/d1-custom.w960.webp",
"thumbnails/d1.jpg",
"thumbnails/d1.waveform.json",
// Being written right now.
"scaled/avatars/u1.w320.webp.1234.tmp",
"scaled/tmp/fresh",
"thumbnails/d1.jpg.1234.tmp",
];
const swept = [
// Their upload is gone.
"scaled/avatars/u2.w128.webp",
"scaled/dumps/d2.w960.jpg",
// The name the previews of image dumps had.
"thumbnails/d1.preview.jpg",
"thumbnails/d3.preview.png",
];
const abandoned = [
"scaled/avatars/u1.w960.webp.5678.tmp",
"scaled/tmp/stale",
"scaled/tmp/stale.webp",
"thumbnails/d1.waveform.json.5678.tmp",
];
for (const path of [...kept, ...swept]) await write(path);
for (const path of abandoned) await write(path, 2 * HOURS);
assertEquals(await sweepGeneratedMedia(), swept.length + abandoned.length);
for (const path of kept) assertEquals(await exists(path), true, path);
for (const path of [...swept, ...abandoned]) {
assertEquals(await exists(path), false, path);
}
// Nothing left to take.
assertEquals(await sweepGeneratedMedia(), 0);
});

70
api/test/lib/upload.test.ts
View File

@@ -1,7 +1,11 @@
import "../setup.ts";
import { PNG } from "../helpers.ts";
import { assertEquals, assertThrows } from "@std/assert";
import { detectImageMime, validateImageUpload } from "../../lib/upload.ts";
import {
detectImageMime,
scalableImage,
validateImageUpload,
} from "../../lib/upload.ts";
import { APIException } from "../../lib/api-exception.ts";
Deno.test("detectImageMime: sniffs magic bytes, not names", () => {
@@ -40,3 +44,67 @@ Deno.test("validateImageUpload: rejects non-images and anything over 5 MB", () =
big.set(PNG);
assertThrows(() => validateImageUpload(big), APIException, "too large");
});
/** A JPEG's first bytes: SOI, then an EXIF segment stating `orientation`. */
function jpegWithOrientation(orientation: number, little: boolean): Uint8Array {
const u16 = (n: number) => little ? [n & 0xff, n >> 8] : [n >> 8, n & 0xff];
const u32 = (n: number) => little ? [...u16(n), 0, 0] : [0, 0, ...u16(n)];
const tiff = [
...(little ? [0x49, 0x49] : [0x4d, 0x4d]),
...u16(42),
...u32(8),
...u16(1),
...u16(0x0112),
...u16(3),
...u32(1),
...u16(orientation),
...[0, 0],
...u32(0),
];
const exif = [0x45, 0x78, 0x69, 0x66, 0, 0, ...tiff];
const length = exif.length + 2;
return new Uint8Array([
...[0xff, 0xd8],
// A JFIF segment first, as a camera's file may have.
...[0xff, 0xe0, 0x00, 0x04, 0x00, 0x00],
...[0xff, 0xe1, length >> 8, length & 0xff],
...exif,
...[0xff, 0xda],
]);
}
Deno.test("scalableImage: a JPEG's EXIF orientation, upright without one", () => {
const turn = (data: Uint8Array) => scalableImage(data)?.orientation;
assertEquals(turn(jpegWithOrientation(6, false)), 6);
assertEquals(turn(jpegWithOrientation(8, true)), 8);
assertEquals(turn(jpegWithOrientation(42, true)), 1);
assertEquals(turn(new Uint8Array([0xff, 0xd8, 0xff, 0xda, 0, 2])), 1);
// Cut short of what it announces: upright, not an error.
assertEquals(turn(jpegWithOrientation(6, false).subarray(0, 20)), 1);
assertEquals(turn(PNG), 1);
});
Deno.test("scalableImage: an animated WebP and what isn't an image are served as they are", () => {
const webp = (flags: number) =>
new Uint8Array([
...new TextEncoder().encode("RIFF\0\0\0\0WEBPVP8X"),
...[10, 0, 0, 0],
flags,
]);
assertEquals(scalableImage(webp(0x02)), null);
// A reader without WebP gets a PNG of what may be transparent, a JPEG of
// the rest.
assertEquals(scalableImage(webp(0x10)), { orientation: 1, compat: "png" });
assertEquals(
scalableImage(new TextEncoder().encode("RIFF\0\0\0\0WEBPVP8 ")),
{ orientation: 1, compat: "png" },
);
assertEquals(scalableImage(PNG), { orientation: 1, compat: "png" });
assertEquals(scalableImage(jpegWithOrientation(1, true))?.compat, "jpg");
assertEquals(scalableImage(new TextEncoder().encode("BM-a-bitmap")), {
orientation: 1,
compat: "jpg",
});
assertEquals(scalableImage(new TextEncoder().encode("<svg/>")), null);
assertEquals(scalableImage(new Uint8Array()), null);
});

44
api/test/routes/avatars.test.ts
View File

@@ -1,4 +1,14 @@
import { createUser, formWithFile, get, PNG, put, test } from "../helpers.ts";
import {
createUser,
formWithFile,
get,
hasFfmpeg,
makeImage,
PNG,
put,
test,
webpWidth,
} from "../helpers.ts";
import type { SelfUser } from "../../../shared/user.ts";
import { assertEquals, assertFalse } from "@std/assert";
@@ -27,3 +37,35 @@ test("avatars: upload, serve, and 404 when there is none", async () => {
400,
);
});
test("avatars: `?w=` answers a scaled copy, made again from a new avatar", async () => {
if (!await hasFfmpeg()) return;
const user = await createUser();
const red = await makeImage(600, 400, "red");
await put("/api/me/avatar", { auth: user.auth, form: formWithFile(red) });
const small = await get(`/api/users/${user.id}/avatar?w=128`);
assertEquals(small.status, 200);
assertEquals(small.headers.get("content-type"), "image/webp");
assertEquals(small.headers.get("cache-control"), "public, max-age=3600");
assertEquals(webpWidth(small.bytes), 128);
// Never wider than the upload itself.
assertEquals(
webpWidth((await get(`/api/users/${user.id}/avatar?w=960`)).bytes),
600,
);
// Without a width, or with one that isn't a number, the upload goes out.
assertEquals((await get(`/api/users/${user.id}/avatar`)).bytes, red);
assertEquals((await get(`/api/users/${user.id}/avatar?w=big`)).bytes, red);
await put("/api/me/avatar", {
auth: user.auth,
form: formWithFile(await makeImage(600, 400, "blue")),
});
const again = await get(`/api/users/${user.id}/avatar?w=128`);
assertEquals(webpWidth(again.bytes), 128);
assertFalse(
again.bytes.length === small.bytes.length &&
again.bytes.every((b, i) => b === small.bytes[i]),
);
});

16
api/test/routes/dumps.test.ts
View File

@@ -5,6 +5,7 @@ import {
del,
formWithFile,
get,
hasFfmpeg,
html,
insertCategory,
insertComment,
@@ -767,16 +768,27 @@ test("custom thumbnails: set, served, removed", async () => {
});
assertEquals(set.status, 200);
assertEquals(set.data.thumbnailMime, "image/png");
// A scaled copy by default, as for an upload's own thumbnail; the image
// itself where no copy can be made.
const thumb = await get(`/api/dumps/${id}/thumbnail`);
assertEquals(thumb.status, 200);
assertEquals(thumb.headers.get("content-type"), "image/png");
assertEquals(thumb.bytes, PNG);
if (await hasFfmpeg()) {
assertEquals(thumb.headers.get("content-type"), "image/webp");
assertEquals(
readFile(`scaled/thumbnails/${id}-custom.w960.webp`) !== null,
true,
);
} else {
assertEquals(thumb.headers.get("content-type"), "image/png");
assertEquals(thumb.bytes, PNG);
}
const removed = await del<Dump>(`/api/dumps/${id}/thumbnail`, {
auth: owner.auth,
});
assertEquals(removed.data.thumbnailMime, undefined);
assertEquals(readFile(`thumbnails/${id}-custom`), null);
assertEquals(readFile(`scaled/thumbnails/${id}-custom.w960.webp`), null);
// A URL dump without its own thumbnail has none to serve.
assertEquals((await get(`/api/dumps/${id}/thumbnail`)).status, 404);
});

29
api/test/routes/remote.test.ts
View File

@@ -1,4 +1,12 @@
import { get, html, PNG, stubNetwork } from "../helpers.ts";
import {
get,
hasFfmpeg,
html,
makeImage,
PNG,
stubNetwork,
webpWidth,
} from "../helpers.ts";
import { assertEquals, assertFalse } from "@std/assert";
import type { Tralbum, YouTubePlaylist } from "../../../shared/player.ts";
@@ -29,6 +37,25 @@ Deno.test("GET /api/remote/image: rasters only, fetched once for everyone", asyn
assertEquals(svg.status, 400);
});
Deno.test("GET /api/remote/image: `?w=` scales the image, fetched once for every width", async () => {
if (!await hasFfmpeg()) return;
const wide = await makeImage(800, 200);
using net = stubNetwork(() =>
new Response(wide, { headers: { "content-type": "image/png" } })
);
const path = `/api/remote/image?url=${
encodeURIComponent("https://cdn.example.com/wide.png")
}`;
const small = await get(`${path}&w=128`);
assertEquals(small.status, 200);
assertEquals(small.headers.get("content-type"), "image/webp");
assertEquals(webpWidth(small.bytes), 128);
assertEquals(webpWidth((await get(`${path}&w=320`)).bytes), 320);
assertEquals((await get(`${path}&w=128`)).bytes, small.bytes);
assertEquals((await get(path)).bytes, wide);
assertEquals(net.calls.length, 1);
});
Deno.test("GET /api/remote/image: an error answer is never served nor cached", async () => {
let failing = true;
using net = stubNetwork(() =>

115
api/test/routes/thumbnails.test.ts
View File

@@ -1,34 +1,24 @@
import {
createUser,
db,
del,
get,
hasFfmpeg,
insertDump,
makeImage,
PNG,
test,
TEST_DIR,
webpWidth,
} from "../helpers.ts";
import { assertEquals } from "@std/assert";
const BYTES = new TextEncoder().encode("0123456789");
function createdAtOf(id: string): string {
return (db.prepare(`SELECT created_at FROM dumps WHERE id = ?;`).get(id) as {
created_at: string;
}).created_at;
}
async function hasFfmpeg(): Promise<boolean> {
try {
return (await new Deno.Command("ffmpeg", {
args: ["-version"],
stdout: "null",
stderr: "null",
}).output()).success;
} catch {
return false;
}
}
function readDir(path: string): string[] {
try {
return [...Deno.readDirSync(path)].map((e) => e.name);
@@ -81,54 +71,87 @@ test("GET /api/dumps/:id/thumbnail: serves a cached video still, with privacy",
assertEquals((await get(`/api/dumps/nope/thumbnail`)).status, 400);
});
test("GET /api/dumps/:id/thumbnail: an image dump gets a preview, a GIF itself", async () => {
test("GET /api/dumps/:id/thumbnail: an image dump gets a scaled copy, or itself when none can be made", async () => {
const owner = await createUser();
const scaled = (id: string) =>
readDir(`${TEST_DIR}/uploads/scaled/dumps`).filter((n) => n.startsWith(id))
.sort();
// Not an image ffmpeg can read (or no ffmpeg at all): the original goes out.
const gif = new TextEncoder().encode("GIF89a-not-really");
const animated = insertDump({
const broken = insertDump({
userId: owner.id,
fileName: "a.gif",
fileMime: "image/gif",
fileData: gif,
});
const res = await get(`/api/dumps/${animated.id}/thumbnail`);
assertEquals(res.status, 200);
assertEquals(res.headers.get("content-type"), "image/gif");
assertEquals(res.bytes, gif);
const fallback = await get(`/api/dumps/${broken.id}/thumbnail`);
assertEquals(fallback.status, 200);
assertEquals(fallback.headers.get("content-type"), "image/gif");
assertEquals(fallback.bytes, gif);
assertEquals(scaled(broken.id), []);
// A real PNG is scaled into a preview, kept beside the video stills.
if (!await hasFfmpeg()) {
const photo = insertDump({
userId: owner.id,
fileName: "p.png",
fileMime: "image/png",
fileData: PNG,
});
const itself = await get(`/api/dumps/${photo.id}/thumbnail`);
assertEquals(itself.headers.get("content-type"), "image/png");
assertEquals(itself.bytes, PNG);
return;
}
// A real image is scaled to the widest width without `w`, and to the width
// asked for with it, each copy kept under the upload's own path.
const photo = insertDump({
userId: owner.id,
fileName: "p.png",
fileMime: "image/png",
fileData: PNG,
fileData: await makeImage(1400, 300),
});
const preview = await get(`/api/dumps/${photo.id}/thumbnail`);
assertEquals(preview.status, 200);
assertEquals(preview.headers.get("content-type"), "image/png");
if (await hasFfmpeg()) {
assertEquals(
readDir(`${TEST_DIR}/uploads/thumbnails`).filter((n) =>
n.startsWith(photo.id)
),
[`${photo.id}.preview.png`],
);
}
const widest = await get(`/api/dumps/${photo.id}/thumbnail`);
assertEquals(widest.status, 200);
assertEquals(widest.headers.get("content-type"), "image/webp");
assertEquals(webpWidth(widest.bytes), 960);
const small = await get(`/api/dumps/${photo.id}/thumbnail?w=128`);
assertEquals(small.headers.get("content-type"), "image/webp");
assertEquals(webpWidth(small.bytes), 128);
// A width between two of the list's takes the one above.
const between = await get(`/api/dumps/${photo.id}/thumbnail?w=200`);
assertEquals(webpWidth(between.bytes), 320);
assertEquals(scaled(photo.id), [
`${photo.id}.w128.webp`,
`${photo.id}.w320.webp`,
`${photo.id}.w960.webp`,
]);
// Not an image ffmpeg can read (or no ffmpeg at all): the original goes out.
const broken = insertDump({
// A reader without WebP, as a mail client, asks for a PNG or a JPEG.
const compat = await get(`/api/dumps/${photo.id}/thumbnail?w=640&f=compat`);
assertEquals(compat.status, 200);
assertEquals(compat.headers.get("content-type"), "image/png");
// Its header states the width, on four bytes from the sixteenth.
assertEquals(new DataView(compat.bytes.buffer).getUint32(16), 640);
assertEquals(scaled(photo.id).includes(`${photo.id}.w640.png`), true);
// An animation keeps its frames.
const animated = insertDump({
userId: owner.id,
fileName: "b.png",
fileMime: "image/png",
fileData: BYTES,
fileName: "anim.gif",
fileMime: "image/gif",
fileData: await makeImage(400, 300, "blue", "gif"),
});
const fallback = await get(`/api/dumps/${broken.id}/thumbnail`);
assertEquals(fallback.status, 200);
assertEquals(fallback.headers.get("content-type"), "image/png");
assertEquals(fallback.bytes, BYTES);
const moving = await get(`/api/dumps/${animated.id}/thumbnail?w=128`);
assertEquals(moving.headers.get("content-type"), "image/webp");
assertEquals(webpWidth(moving.bytes), 128);
assertEquals(new TextDecoder().decode(moving.bytes).includes("ANMF"), true);
// The copies go with the dump.
assertEquals(
readDir(`${TEST_DIR}/uploads/thumbnails`).filter((n) =>
n.startsWith(broken.id)
),
[],
(await del(`/api/dumps/${photo.id}`, { auth: owner.auth })).status,
200,
);
assertEquals(scaled(photo.id), []);
});

42
shared/image.ts Normal file
View File

@@ -0,0 +1,42 @@
/**
* Names the widths the site's own images come in. Every image the API serves
* (avatars, covers, thumbnails, attachments, proxied images) takes `?w=` and
* answers a copy scaled down to it, so a card or an avatar doesn't load the
* upload at full size. The frontend builds those URLs from this list
* (`imageSrcSet` in `src/utils/shared/urls.ts`) and the server makes no other
* width (`serveImage` in `api/lib/upload.ts`), which bounds what it keeps.
*
* @module
*/
/**
* The widths, in pixels, smallest first: an avatar on a dense screen, a small
* cover, a card on a phone, and the widest feed card on a dense screen.
*/
export const IMAGE_WIDTHS = [128, 320, 640, 960] as const;
/** One of {@link IMAGE_WIDTHS}. */
export type ImageWidth = typeof IMAGE_WIDTHS[number];
/** The largest of {@link IMAGE_WIDTHS}, which a thumbnail never exceeds. */
export const MAX_IMAGE_WIDTH: ImageWidth =
IMAGE_WIDTHS[IMAGE_WIDTHS.length - 1];
/**
* Returns the narrowest of {@link IMAGE_WIDTHS} that covers `width` pixels, or
* the widest when none does.
*/
export function snapImageWidth(width: number): ImageWidth {
return IMAGE_WIDTHS.find((w) => w >= width) ?? MAX_IMAGE_WIDTH;
}
/**
* Returns the width a `?w=` value asks for, snapped to {@link IMAGE_WIDTHS},
* or null when it is absent or not a positive number, which asks for the image
* as it is.
*/
export function parseImageWidth(raw: string | null): ImageWidth | null {
if (raw === null || !/^\d{1,5}$/.test(raw)) return null;
const width = Number(raw);
return width > 0 ? snapImageWidth(width) : null;
}

22
shared/test/image.test.ts Normal file
View File

@@ -0,0 +1,22 @@
import { assertEquals } from "@std/assert";
import { parseImageWidth, snapImageWidth } from "../image.ts";
Deno.test("snapImageWidth: the narrowest width that covers, the widest past them all", () => {
assertEquals(snapImageWidth(1), 128);
assertEquals(snapImageWidth(128), 128);
assertEquals(snapImageWidth(129), 320);
assertEquals(snapImageWidth(700), 960);
assertEquals(snapImageWidth(5000), 960);
});
Deno.test("parseImageWidth: a positive number, or no width at all", () => {
assertEquals(parseImageWidth("320"), 320);
assertEquals(parseImageWidth("200"), 320);
assertEquals(parseImageWidth(null), null);
assertEquals(parseImageWidth(""), null);
assertEquals(parseImageWidth("0"), null);
assertEquals(parseImageWidth("-64"), null);
assertEquals(parseImageWidth("12.5"), null);
assertEquals(parseImageWidth("wide"), null);
assertEquals(parseImageWidth("999999999"), null);
});

9
src/components/dump/FilePreview.tsx
View File

@@ -12,7 +12,9 @@ import {
dumpFileUrl,
dumpThumbnailUrl,
dumpWaveformUrl,
imageProps,
} from "../../utils/shared/urls.ts";
import { IMAGE_SIZES } from "../../config/imageSizes.ts";
import { useAuth } from "../../hooks/auth/useAuth.ts";
import {
fileArtworkUrl,
@@ -72,6 +74,7 @@ function FilePlayButton({ dump }: { dump: Dump }) {
<Thumbnail
src={fileArtworkUrl(dump, token)}
className="rich-content-compact-thumbnail"
sizes={IMAGE_SIZES.compactThumbnail}
placeholder={{ seed: dump.fileName ?? dump.id, glyph: mimeIcon(mime) }}
placeholderClassName="rich-content-compact-thumbnail"
/>
@@ -141,7 +144,10 @@ export function FilePreview(
// A card needs the server's scaled-down copy, not the full upload.
return (
<img
src={dumpThumbnailUrl(dump, token)}
{...imageProps(
dumpThumbnailUrl(dump, token),
IMAGE_SIZES.compactThumbnail,
)}
alt={dump.fileName}
className="rich-content-compact-thumbnail"
loading="lazy"
@@ -158,6 +164,7 @@ export function FilePreview(
<Thumbnail
src={artworkUrl}
className="rich-content-compact-thumbnail"
sizes={IMAGE_SIZES.compactThumbnail}
placeholder={{
seed: dump.fileName ?? dump.id,
glyph: mimeIcon(mime),

2
src/components/dump/JournalCard.tsx
View File

@@ -10,6 +10,7 @@ import { Plural, Trans } from "@lingui/react/macro";
import type { Dump } from "../../../shared/dump.ts";
import { relativeTime } from "../../utils/shared/relativeTime.ts";
import { dumpUrl } from "../../utils/shared/urls.ts";
import { IMAGE_SIZES } from "../../config/imageSizes.ts";
import { useAuth } from "../../hooks/auth/useAuth.ts";
import { useDumpCardLink } from "../../hooks/dump/useDumpCardLink.ts";
import {
@@ -195,6 +196,7 @@ export const JournalCard: React.MemoExoticComponent<
<div className="journal-card-image">
<Thumbnail
src={thumbnailUrl}
sizes={IMAGE_SIZES.journal[shape]}
placeholder={dump.kind === "file"
// A video still fails without ffmpeg on the server, and the MIME
// glyph says more than an initial taken from the file name.

3
src/components/dump/RichContentCard.tsx
View File

@@ -14,6 +14,7 @@ import {
usePlayRichContent,
} from "../../hooks/player/usePlayRichContent.ts";
import { Thumbnail } from "../shared/Thumbnail.tsx";
import { IMAGE_SIZES } from "../../config/imageSizes.ts";
/**
* Passed to {@link RichContentCard} by `DumpCard` (`compact`), `Dump`,
@@ -68,6 +69,7 @@ export function RichContentCard(
src={thumbnailSrc}
alt={richContent.title ?? ""}
className="rich-content-compact-thumbnail"
sizes={IMAGE_SIZES.compactThumbnail}
placeholder={placeholder}
placeholderClassName="rich-content-compact-thumbnail"
/>
@@ -135,6 +137,7 @@ export function RichContentCard(
src={thumbnailSrc}
alt={richContent.title ?? ""}
className="rich-content-thumbnail"
sizes={IMAGE_SIZES.richThumbnail}
placeholder={placeholder}
placeholderClassName="rich-content-thumbnail"
/>

2
src/components/player/GlobalPlayer.tsx
View File

@@ -17,6 +17,7 @@ import {
import { MediaPlayer } from "./MediaPlayer.tsx";
import { fmt } from "../../utils/player/duration.ts";
import { Thumbnail } from "../shared/Thumbnail.tsx";
import { IMAGE_SIZES } from "../../config/imageSizes.ts";
import { useYouTubePlaylist } from "../../hooks/player/useYouTubePlaylist.ts";
import { embedListId, withJsApi } from "../../utils/player/youtube.ts";
@@ -201,6 +202,7 @@ export function GlobalPlayer(): React.JSX.Element | null {
<Thumbnail
src={current.artworkUrl}
className="global-player-artwork"
sizes={IMAGE_SIZES.playerArtwork}
placeholder={{ siteName: current.subtitle }}
placeholderClassName="global-player-artwork"
loading="eager"

16
src/components/playlist/PlaylistCard.tsx
View File

@@ -10,7 +10,12 @@ import { t } from "@lingui/core/macro";
import { Plural, Trans } from "@lingui/react/macro";
import { type Playlist, playlistActivity } from "../../../shared/playlist.ts";
import { relativeTime } from "../../utils/shared/relativeTime.ts";
import { playlistImageUrl, playlistUrl } from "../../utils/shared/urls.ts";
import {
imageProps,
playlistImageUrl,
playlistUrl,
} from "../../utils/shared/urls.ts";
import { IMAGE_SIZES } from "../../config/imageSizes.ts";
import { useAuth } from "../../hooks/auth/useAuth.ts";
import { useCardLink } from "../../hooks/shared/useCardLink.ts";
import {
@@ -40,6 +45,8 @@ export interface PlaylistCardProps {
* public badges on every card there would say nothing.
*/
privateBadgeOnly?: boolean;
/** Set for a tile of the playlists page's grid, whose cover is larger. */
grid?: boolean;
}
/**
@@ -51,7 +58,7 @@ export interface PlaylistCardProps {
* class lays it out: a row in a `.dump-feed`, a tile in a `.playlist-grid`.
*/
export function PlaylistCard(
{ playlist, onDelete, isOwner, privateBadgeOnly }: PlaylistCardProps,
{ playlist, onDelete, isOwner, privateBadgeOnly, grid }: PlaylistCardProps,
): React.JSX.Element {
const { mediaToken } = useAuth();
const { cardProps } = useCardLink(
@@ -75,7 +82,10 @@ export function PlaylistCard(
{playlist.imageMime
? (
<img
src={playlistImageUrl(playlist, mediaToken)}
{...imageProps(
playlistImageUrl(playlist, mediaToken),
grid ? IMAGE_SIZES.playlistTile : IMAGE_SIZES.playlistThumb,
)}
alt=""
className="playlist-card-img"
loading="lazy"

1
src/components/playlist/PlaylistCardList.tsx
View File

@@ -40,6 +40,7 @@ export function PlaylistCardList(
isOwner={!!user && user.id === p.userId}
onDelete={onDelete && (() => onDelete(p))}
privateBadgeOnly={privateBadgeOnly}
grid={grid}
/>
))}
</ul>

8
src/components/playlist/PlaylistHeader.tsx
View File

@@ -9,7 +9,8 @@ import { Link } from "react-router";
import { t } from "@lingui/core/macro";
import { Trans } from "@lingui/react/macro";
import type { PlaylistWithDumps } from "../../../shared/playlist.ts";
import { playlistImageUrl } from "../../utils/shared/urls.ts";
import { imageProps, playlistImageUrl } from "../../utils/shared/urls.ts";
import { IMAGE_SIZES } from "../../config/imageSizes.ts";
import { relativeTime } from "../../utils/shared/relativeTime.ts";
import { useAuth } from "../../hooks/auth/useAuth.ts";
import { Markdown } from "../editor/Markdown.tsx";
@@ -37,7 +38,10 @@ export function PlaylistHeader(
<>
{playlist.imageMime && (
<img
src={playlistImageUrl(playlist, mediaToken)}
{...imageProps(
playlistImageUrl(playlist, mediaToken),
IMAGE_SIZES.playlistBanner,
)}
alt=""
className="playlist-detail-img"
/>

9
src/components/shared/Avatar.tsx
View File

@@ -6,7 +6,7 @@
*/
import { useContext, useState } from "react";
import { userApiUrl } from "../../utils/shared/urls.ts";
import { imageProps, userApiUrl } from "../../utils/shared/urls.ts";
import { AvatarOverrideContext } from "../../contexts/ws/AvatarOverrideContext.ts";
/**
@@ -25,8 +25,9 @@ export interface AvatarProps {
}
/**
* Renders the image from `GET /api/users/:userId/avatar`, or the user's initial
* when they have none or it fails to load. An avatar added, changed or removed
* Renders the image from `GET /api/users/:userId/avatar`, scaled by the server
* to the size it is shown at, or the user's initial when they have none or it
* fails to load. An avatar added, changed or removed
* since the caller's data was fetched shows at once, through the
* `user_updated` messages that `WSProvider` puts in `AvatarOverrideContext`.
*/
@@ -55,7 +56,7 @@ export function Avatar(
: userApiUrl({ id: userId }, "avatar");
return (
<img
src={src}
{...imageProps(src, size)}
alt={username}
title={username}
style={sizeStyle}

13
src/components/shared/Thumbnail.tsx
View File

@@ -7,6 +7,7 @@
import { useState } from "react";
import { canProxyImage, proxyImageUrl } from "../../utils/shared/proxyImage.ts";
import { imageProps, isApiImageUrl } from "../../utils/shared/urls.ts";
import {
ThumbnailPlaceholder,
type ThumbnailPlaceholderProps,
@@ -23,6 +24,12 @@ export interface ThumbnailProps {
/** Applies to the image only. */
className?: string;
loading?: "lazy" | "eager";
/**
* The width the image is laid out at, as `imageProps` takes it. An image
* the API serves, the site's own or a proxied one, then loads no larger
* than that; another site's loads as that site sends it.
*/
sizes: number | string;
/** Shown when there's no `src` or it fails to load. */
placeholder: Omit<ThumbnailPlaceholderProps, "className">;
placeholderClassName?: string;
@@ -42,6 +49,7 @@ export function Thumbnail(
alt = "",
className,
loading = "lazy",
sizes,
placeholder,
placeholderClassName,
}: ThumbnailProps,
@@ -64,11 +72,10 @@ export function Thumbnail(
);
}
const url = proxyImageUrl(src, { force: attempt === "proxied" });
return (
<img
src={attempt === "proxied"
? proxyImageUrl(src, { force: true })
: proxyImageUrl(src)}
{...(isApiImageUrl(url) ? imageProps(url, sizes) : { src: url })}
alt={alt}
className={className}
loading={loading}

44
src/config/imageSizes.ts Normal file
View File

@@ -0,0 +1,44 @@
/**
* States the width each of the site's images is laid out at, which App.css
* sets, so that `imageProps` asks the server for a copy no larger. A number
* is a fixed width in CSS pixels, a string a `sizes` attribute for a width
* that follows the viewport's. The components that show a thumbnail, a cover
* or a player's artwork take theirs from here; an avatar's is its `size`.
*
* @module
*/
import type { JournalShape } from "../utils/feed/journalLayout.ts";
/** A tile of the journal's grid spanning one of its columns. */
const JOURNAL_COLUMN =
"(max-width: 460px) 100vw, (max-width: 680px) 50vw, (max-width: 980px) 33vw, 290px";
/** A tile of the journal's grid spanning two of its columns. */
const JOURNAL_TWO_COLUMNS =
"(max-width: 680px) 100vw, (max-width: 980px) 66vw, 590px";
/** The width of each image, by the place it is shown. */
export const IMAGE_SIZES = {
/**
* A compact thumbnail at its widest, in a dump card's preview; the rows
* that show it smaller take the same copy.
*/
compactThumbnail: 128,
/** A link's thumbnail beside its text, above it on a narrow screen. */
richThumbnail: "(max-width: 512px) 100vw, 180px",
/** A journal tile's image, by the tile's shape. */
journal: {
feature: JOURNAL_TWO_COLUMNS,
wide: JOURNAL_TWO_COLUMNS,
tall: JOURNAL_COLUMN,
square: JOURNAL_COLUMN,
} satisfies Record<JournalShape, string>,
/** A playlist's cover in a row of a list. */
playlistThumb: 80,
/** A playlist's cover as a tile of the grid, two to a row when narrow. */
playlistTile: "(max-width: 512px) 50vw, 250px",
/** A playlist's cover as the banner of its page. */
playlistBanner: "(max-width: 860px) 100vw, 860px",
/** The artwork in the global player's header. */
playerArtwork: 36,
} as const;

9
src/test/components/dump/FilePreview.test.tsx
View File

@@ -35,7 +35,12 @@ describe("FilePreview", () => {
const d = file("image/png");
renderWithContexts(<FilePreview dump={d} compact />, { session });
const img = screen.getByRole("img", { name: "thing" });
expect(img).toHaveAttribute("src", dumpThumbnailUrl(d, session.mediaToken));
// The copy scaled to a card's preview, on a dense screen.
expect(img).toHaveAttribute(
"src",
`${dumpThumbnailUrl(d, session.mediaToken)}&w=320`,
);
expect(img).toHaveAttribute("sizes", "128px");
expect(img).toHaveAttribute("loading", "lazy");
});
@@ -101,7 +106,7 @@ describe("FilePreview", () => {
);
expect(container.querySelector("button img")).toHaveAttribute(
"src",
dumpThumbnailUrl(d),
`${dumpThumbnailUrl(d)}&w=320`,
);
});

7
src/test/components/dump/JournalCard.test.tsx
View File

@@ -123,8 +123,9 @@ describe("JournalCard", () => {
<JournalCard dump={dump} shape="tall" {...base} />
</ul>,
);
expect(container.querySelector("img")!.getAttribute("src")).toBe(
dumpThumbnailUrl(dump),
);
const img = container.querySelector("img")!;
expect(img.getAttribute("src")).toBe(`${dumpThumbnailUrl(dump)}&w=960`);
// A tile of one column: the browser takes a narrower copy for it.
expect(img.getAttribute("sizes")).toContain("290px");
});
});

2
src/test/components/dump/RichContentCard.test.tsx
View File

@@ -75,7 +75,7 @@ describe("RichContentCard", () => {
);
expect(screen.getByRole("img", { name: "A video" })).toHaveAttribute(
"src",
"http://localhost:8000/api/dumps/d/thumbnail",
"http://localhost:8000/api/dumps/d/thumbnail?w=960",
);
});

11
src/test/components/shared/Avatar.test.tsx
View File

@@ -5,14 +5,17 @@ import { AvatarOverrideContext } from "../../../contexts/ws/AvatarOverrideContex
import { renderWithContexts } from "../../render.tsx";
describe("Avatar", () => {
test("an image when the user has one, versioned", () => {
test("an image when the user has one, versioned, at the size shown", () => {
renderWithContexts(
<Avatar userId="u1" username="ann" hasAvatar version={7} />,
<Avatar userId="u1" username="ann" hasAvatar version={7} size={32} />,
);
expect(screen.getByRole("img", { name: "ann" })).toHaveAttribute(
const img = screen.getByRole("img", { name: "ann" });
expect(img).toHaveAttribute(
"src",
"http://localhost:8000/api/users/u1/avatar?v=7",
"http://localhost:8000/api/users/u1/avatar?v=7&w=128",
);
expect(img).toHaveAttribute("sizes", "32px");
expect(img.getAttribute("srcset")).toContain("avatar?v=7&w=320 320w");
});
test("initials otherwise, and after the image fails", () => {

24
src/test/components/shared/Thumbnail.test.tsx
View File

@@ -11,17 +11,20 @@ describe("Thumbnail", () => {
<Thumbnail
src="https://img.test/a.png"
alt="art"
sizes={128}
placeholder={placeholder}
/>,
);
const img = screen.getByRole("img", { name: "art" });
expect(img).toHaveAttribute("src", "https://img.test/a.png");
expect(img).toHaveAttribute("referrerpolicy", "no-referrer");
// Another site's image comes as that site sends it.
expect(img).not.toHaveAttribute("srcset");
fireEvent.error(img);
expect(screen.getByRole("img", { name: "art" }).getAttribute("src"))
.toContain(
"/api/remote/image?url=",
);
const proxied = screen.getByRole("img", { name: "art" });
expect(proxied.getAttribute("src")).toContain("/api/remote/image?url=");
// Through the proxy, it is scaled as the site's own are.
expect(proxied.getAttribute("srcset")).toContain("a.png&w=128 128w");
fireEvent.error(screen.getByRole("img", { name: "art" }));
expect(screen.queryByRole("img", { name: "art" })).not.toBeInTheDocument();
expect(screen.getByText("S")).toBeInTheDocument();
@@ -32,16 +35,24 @@ describe("Thumbnail", () => {
<Thumbnail
src="http://localhost:8000/api/dumps/d/thumbnail"
alt="t"
sizes="(max-width: 512px) 100vw, 180px"
placeholder={placeholder}
/>,
);
fireEvent.error(screen.getByRole("img", { name: "t" }));
const img = screen.getByRole("img", { name: "t" });
expect(img).toHaveAttribute(
"src",
"http://localhost:8000/api/dumps/d/thumbnail?w=960",
);
expect(img.getAttribute("srcset")).toContain("thumbnail?w=320 320w");
expect(img).toHaveAttribute("sizes", "(max-width: 512px) 100vw, 180px");
fireEvent.error(img);
expect(screen.queryByRole("img", { name: "t" })).not.toBeInTheDocument();
});
test("no src: straight to the placeholder; a new src tries again", () => {
const { rerender } = renderWithContexts(
<Thumbnail placeholder={placeholder} alt="t" />,
<Thumbnail placeholder={placeholder} alt="t" sizes={128} />,
);
expect(screen.queryByRole("img", { name: "t" })).not.toBeInTheDocument();
rerender(
@@ -49,6 +60,7 @@ describe("Thumbnail", () => {
src="https://img.test/b.png"
placeholder={placeholder}
alt="t"
sizes={128}
/>,
);
expect(screen.getByRole("img", { name: "t" })).toBeInTheDocument();

4
src/test/pages/PlaylistDetail.test.tsx
View File

@@ -126,7 +126,7 @@ describe("PlaylistDetail", () => {
await screen.findByRole("heading", { name: "Mix" });
expect(document.querySelector(".playlist-detail-img")).toHaveAttribute(
"src",
"http://localhost:8000/api/playlists/p1/cover?v=5000",
"http://localhost:8000/api/playlists/p1/cover?v=5000&w=960",
);
});
@@ -324,7 +324,7 @@ describe("PlaylistDetail", () => {
await waitFor(() =>
expect(document.querySelector(".playlist-detail-img")).toHaveAttribute(
"src",
"http://localhost:8000/api/playlists/p1/cover?v=9000",
"http://localhost:8000/api/playlists/p1/cover?v=9000&w=960",
)
);
});

2
src/test/pages/UserPublicProfile.test.tsx
View File

@@ -523,7 +523,7 @@ describe("UserPublicProfile", () => {
expect(document.querySelector(".profile-avatar-wrapper img"))
.toHaveAttribute(
"src",
"http://localhost:8000/api/users/ann/avatar?v=7000",
"http://localhost:8000/api/users/ann/avatar?v=7000&w=320",
)
);
});

25
src/test/utils/shared/urls.test.ts
View File

@@ -3,6 +3,8 @@ import {
dumpFileUrl,
dumpThumbnailUrl,
dumpUrl,
imageProps,
isApiImageUrl,
isAt,
normalizeUrl,
playlistImageUrl,
@@ -65,4 +67,27 @@ describe("urls", () => {
expect(playlistImageUrl({ id: "p", isPublic: false }, null))
.toBe(`${API_URL}/api/playlists/p/cover`);
});
test("imageProps asks for the copies the server makes, by the width shown", () => {
const avatar = `${API_URL}/api/users/u1/avatar`;
expect(imageProps(avatar, 32)).toEqual({
src: `${avatar}?w=128`,
srcSet: [128, 320, 640, 960].map((w) => `${avatar}?w=${w} ${w}w`)
.join(", "),
sizes: "32px",
});
// Twice the width shown, for a dense screen that reads only `src`.
expect(imageProps(avatar, 128).src).toBe(`${avatar}?w=320`);
// A URL with a query keeps it, and a width that follows the viewport's
// takes the widest copy as its `src`.
const cover = `${API_URL}/api/playlists/p1/cover?v=7`;
const fluid = imageProps(cover, "(max-width: 512px) 50vw, 250px");
expect(fluid.src).toBe(`${cover}&w=960`);
expect(fluid.srcSet).toContain(`${cover}&w=640 640w`);
expect(fluid.sizes).toBe("(max-width: 512px) 50vw, 250px");
});
test("isApiImageUrl tells the API's images from another site's", () => {
expect(isApiImageUrl(`${API_URL}/api/dumps/d1/thumbnail`)).toBe(true);
expect(isApiImageUrl(`${API_URL}/api/remote/image?url=x`)).toBe(true);
expect(isApiImageUrl("https://img.test/a.png")).toBe(false);
});
});

34
src/utils/shared/urls.ts
View File

@@ -14,6 +14,7 @@ import {
playlistPath,
} from "../../../shared/paths.ts";
import { dumpFileVersion } from "../../../shared/dump.ts";
import { IMAGE_WIDTHS, snapImageWidth } from "../../../shared/image.ts";
/**
* Passed to {@link dumpFileUrl} and {@link dumpThumbnailUrl}, usually as a
@@ -166,3 +167,36 @@ export function playlistImageUrl(
params.length ? `?${params.join("&")}` : ""
}`;
}
/**
* Checks whether `url` is an image the API serves, its own or one it proxies,
* which all take `?w=` and answer a scaled-down copy. Another site's image,
* loaded from its own host, doesn't.
*/
export function isApiImageUrl(url: string): boolean {
return url.startsWith(`${API_URL}/api/`);
}
/**
* Returns what an `<img>` needs to load `url`, one of the API's images, no
* larger than it is shown: a `srcset` of the copies the server makes
* (`shared/image.ts`), from which the browser takes the narrowest that fills
* `sizes` on its screen, and a `src` for whatever reads only that. `sizes` is
* the width the image is laid out at, in CSS pixels, or a `sizes` attribute
* when that width follows the viewport's. `Avatar`, `Thumbnail` and the
* playlists' covers spread it on their image.
*/
export function imageProps(
url: string,
sizes: number | string,
): { src: string; srcSet: string; sizes: string } {
const at = (width: number) =>
`${url}${url.includes("?") ? "&" : "?"}w=${width}`;
return {
// Twice the laid-out width, as most screens want; the widest when that
// width isn't known here.
src: at(snapImageWidth(typeof sizes === "number" ? sizes * 2 : Infinity)),
srcSet: IMAGE_WIDTHS.map((width) => `${at(width)} ${width}w`).join(", "),
sizes: typeof sizes === "number" ? `${sizes}px` : sizes,
};
}