v3: the backend is documented for deno doc, one way in every file, and the docs build and publish themselves — comments were in three shapes: file headers as // lines, as an untagged /** */ (which deno doc drops, only an @module block counts) or missing; 43 exports described by a // deno doc can't see, two JSDoc blocks sitting on the wrong declaration and one duplicated; endpoints commented in 5 of 18 route files, three ways, on router.get() calls no doc tool reads. The rule, applied to every non-test module of api/, migrations included: each opens with a /** … @module */ block; every export has JSDoc, and so does every field of an exported interface; a comment describing a declaration is /** */, // only notes code; functions open with a verb, everything else with a noun phrase; parameters are named in the prose, {@link} only for symbols in scope; whatever throws to its caller says so with @throws (for an APIException, the statuses and when); a router's JSDoc lists its endpoints, method and path, who may call it, what it does. deno doc --lint goes from 528 errors to none in api/ (shared/ keeps its 385; it is next). The lint also wanted explicit types on config.ts's settings and a few return types, JsonLdResult exported, and the routers typed Router. deno task doc builds the HTML reference into docs/ from every module of api/ and shared/ but the tests, deno task doc:lint lints it; the index page is the module doc of a root mod.ts (deno.json's exports), which introduces the backend, shared/ and the frontend to come and links every module, and mod.test.ts fails when one is missing. The docs workflow builds it on every push to main that touches them and publishes it to the gh-pages branch, like publish-assistant's. Two defects found along the way are fixed: a private playlist's cover was served to anyone who had its id or slug, and now needs its owner's media token, as private dump files do (playlistImageUrl adds it for private playlists only, so public cover URLs stay the same for everyone); og.ts and static.ts each filled index.html's placeholders, and now share lib/index-html.ts, so the OG middleware picks up a rebuilt index.html instead of keeping the first it read. The README states the documentation rule. 205 backend and 628 frontend tests pass.
All checks were successful
Build and Publish Docker Image / build-and-push (push) Successful in 47s
Build and deploy docs / build-and-push (push) Successful in 26s

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
khannurien
2026-09-27 21:07:34 +00:00
parent 1702370447
commit d62e4a9883
119 changed files with 2967 additions and 611 deletions

View File

@@ -1,3 +1,11 @@
/**
* The guarded fetch every request to another site goes through: each hop must
* resolve to a public address, the whole exchange has a deadline, and pages are
* read up to a cap.
*
* @module
*/
import { isPrivateHostname, resolvePublicAddress } from "./public-address.ts";
import { isSelfUrl } from "../lib/self-url.ts";
@@ -12,7 +20,8 @@ const FETCH_HEADERS = {
const MAX_REDIRECTS = 5;
/**
* Fetch `url` via a `curl --insecure` subprocess and return a minimal Response.
* Fetches `url` through a `curl --insecure` subprocess and returns a minimal
* Response.
* Used as a last resort when Deno's rustls rejects the server's TLS certificate
* due to an unsupported algorithm (e.g. UnsupportedSignatureAlgorithm).
* Returns null if curl is unavailable or exits non-zero.
@@ -91,7 +100,7 @@ async function fetchViaCurl(
}
/**
* Most bytes of a fetched page that are read. Metadata lives in the <head>,
* The most bytes of a fetched page that are read. Metadata lives in the <head>,
* and a "page" past this is a download, not something to scrape.
*/
export const MAX_PAGE_BYTES = 4 * 1024 * 1024;
@@ -161,7 +170,7 @@ export async function readTextCapped(
return new TextDecoder().decode(bytes);
}
/** One request, no redirects followed. The timeout covers the body too. */
/** Makes one request, following no redirects. The timeout covers the body too. */
async function fetchOnce(
url: URL,
address: string,
@@ -203,9 +212,14 @@ async function fetchOnce(
}
/**
* Fetch a user-supplied URL. Every hop — the URL itself and each redirect —
* must resolve to a public address (see resolvePublicAddress), so a link or a
* redirect can't point the server at its own network.
* Fetches a user-supplied URL, following up to five redirects. Every hop — the
* URL itself and each redirect — must resolve to a public address (see
* {@link resolvePublicAddress}), so a link or a redirect can't point the
* server at its own network.
*
* @throws {TypeError} When a hop isn't http(s), there are too many redirects,
* or the request fails.
* @throws {Error} When a hop's host is private or can't be resolved.
*/
export async function fetchWithTimeout(
url: string,
@@ -227,6 +241,10 @@ export async function fetchWithTimeout(
}
}
/**
* Checks whether `raw` is an http(s) URL a dump may link to: not on a private
* host, unless it is one of this site's own pages.
*/
export function isValidHttpUrl(raw: string): boolean {
try {
const u = new URL(raw);