Files
gerbeur/api/remote
khannurien 90fc281e90 v4: every image the API serves comes scaled to the size it is shown at — a first visit to the feed loaded 4.8 MB of images out of 5.1 MB in all: an avatar shown at 32 pixels was the upload itself, a 1.4 MB GIF among them, and so were a cover, a custom thumbnail, an attachment and a proxied image, only an image dump having a preview, 960 pixels wide. Every route that answers an image now takes ?w= and sends a WebP copy at most that wide, of the widths shared/image.ts lists, 128, 320, 640 and 960, a width between two taking the one above, and never wider than the image; without it the upload goes out as before, a dump's thumbnail alone being scaled to the widest by default, the custom ones included. serveImage (lib/upload.ts) is where they all end, and the image proxy scales what it fetched the same way, keeping each width in its cache. lib/ffmpeg.ts makes the copies as it made the previews, which they replace: on first request, under uploads/scaled/ at the upload's own path, each given its upload's date and made again when the two differ, so a new avatar or cover needs nobody to delete the old ones. WebP keeps a PNG's transparency and a GIF's animation. A JPEG is turned by its EXIF orientation here, ffmpeg's own rotation of images depending on its version. An animated WebP, which ffmpeg can't read, and an image it fails on go out as they are, a failure being remembered until the upload changes. ?f=compat asks for the copy as a JPEG, or a PNG for what may be transparent: the digest's emails take their thumbnails that way, at 640 pixels, not every mail client reading WebP. On the client, imageProps (utils/shared/urls.ts) gives an image its srcset of those copies and the width it is laid out at, which config/imageSizes.ts states for each place from App.css: Avatar, Thumbnail, which scales an API image and leaves another site's as that site sends it, and the playlists' covers use it. The images of notes and comments still load as uploaded. The copies go with the dump, the custom thumbnail or the attachment they were made from, and generatedMediaSweepJob, a minute after the start then daily, takes those whose upload is gone, the previews under their former name and the temporary files a generation left unfinished.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 19:01:36 +00:00
..
v4: link previews are read once, in one place, and no longer take an error page, a relative image or a mask icon for the page's — each provider fetched its page and scanned it with regular expressions of its own, the generic one some fifteen times, and Bandcamp and SoundCloud read only OG tags. fetchPage (remote/fetch.ts) now fetches every page, follows redirects to the address it was served from, and refuses an error status, so a 404 or a bot challenge is unreachable rather than a preview titled "Just a moment...", and a refresh no longer overwrites good metadata with it. parsePage (remote/html.ts) reads it in one linear pass with a real attribute reader: unquoted values, a > inside quotes, og: tags under name as well as property, and nothing from comments or scripts. Every URL a page names is resolved against where it was served from, so a relative og:image works; entities are decoded in one pass, &eacute; and &rsquo; included, and one that names no character no longer fails the preview (bandcamp-tralbum.ts shares the decoder). The mask icon is no longer the favicon, JSON-LD prefers the page's node over the site's, tracking pixels and lazy placeholders aren't thumbnails, an og:url with a slug keeps the OG tags, and a <title> loses its site suffix. genericPreview is the base of the Bandcamp, SoundCloud and YouTube channel previews; SoundCloud takes www., m. and on.soundcloud.com short links, embedding the page they lead to, and Bandcamp keeps only its own player. The service tidies every preview: text on one line, titles and descriptions capped, only http(s) images and players. RichContent.type is now RichContentType, and the unused videoId is gone.
2026-10-09 20:46:32 +00:00
v4: what the server and the client both state is declared once — several rules were written out in two or more files, some already drifting apart. A link's site name (its host without www.) was computed in five places, now shared/url.ts's siteHost; dumpFileVersion, which the frontend and the server had to keep equal by hand, is one function in shared/dump.ts; the Markdown flattening of the chat's reply previews and the digest's quotes is shared/markdown.ts's markdownToPlainText, so the digest also keeps an image's alt text; an attachment's URL and how to find one in a text are shared/attachment.ts, which the editor, Markdown and attachment-service read, its id pattern the new UUID_RE that the file and thumbnail routes check ids with. On the server, the Bandcamp, YouTube playlist and link preview caches use one TtlCache (api/lib/ttl-cache.ts) instead of three copies of the same expiry and eviction; the link previews' theme-color and the emails' theme tokens read colours with one parser (api/lib/css-color.ts), and a transparent theme-color now gives no tint; the WebSocket route sends through ws-service's send, the welcome included, so it is type-checked; fetch.ts strips an IPv6 literal's brackets with public-address's bareHostname; and a category's slug is checked by the same validateSlug as a dump's, with its own length. The migrations keep their frozen copies, as they document. 272 backend tests and 677 frontend tests pass, and deno check, tsc, eslint and doc:lint are clean.
2026-10-09 13:16:52 +00:00
v4: link previews are read once, in one place, and no longer take an error page, a relative image or a mask icon for the page's — each provider fetched its page and scanned it with regular expressions of its own, the generic one some fifteen times, and Bandcamp and SoundCloud read only OG tags. fetchPage (remote/fetch.ts) now fetches every page, follows redirects to the address it was served from, and refuses an error status, so a 404 or a bot challenge is unreachable rather than a preview titled "Just a moment...", and a refresh no longer overwrites good metadata with it. parsePage (remote/html.ts) reads it in one linear pass with a real attribute reader: unquoted values, a > inside quotes, og: tags under name as well as property, and nothing from comments or scripts. Every URL a page names is resolved against where it was served from, so a relative og:image works; entities are decoded in one pass, &eacute; and &rsquo; included, and one that names no character no longer fails the preview (bandcamp-tralbum.ts shares the decoder). The mask icon is no longer the favicon, JSON-LD prefers the page's node over the site's, tracking pixels and lazy placeholders aren't thumbnails, an og:url with a slug keeps the OG tags, and a <title> loses its site suffix. genericPreview is the base of the Bandcamp, SoundCloud and YouTube channel previews; SoundCloud takes www., m. and on.soundcloud.com short links, embedding the page they lead to, and Bandcamp keeps only its own player. The service tidies every preview: text on one line, titles and descriptions capped, only http(s) images and players. RichContent.type is now RichContentType, and the unused videoId is gone.
2026-10-09 20:46:32 +00:00
v4: link previews are read once, in one place, and no longer take an error page, a relative image or a mask icon for the page's — each provider fetched its page and scanned it with regular expressions of its own, the generic one some fifteen times, and Bandcamp and SoundCloud read only OG tags. fetchPage (remote/fetch.ts) now fetches every page, follows redirects to the address it was served from, and refuses an error status, so a 404 or a bot challenge is unreachable rather than a preview titled "Just a moment...", and a refresh no longer overwrites good metadata with it. parsePage (remote/html.ts) reads it in one linear pass with a real attribute reader: unquoted values, a > inside quotes, og: tags under name as well as property, and nothing from comments or scripts. Every URL a page names is resolved against where it was served from, so a relative og:image works; entities are decoded in one pass, &eacute; and &rsquo; included, and one that names no character no longer fails the preview (bandcamp-tralbum.ts shares the decoder). The mask icon is no longer the favicon, JSON-LD prefers the page's node over the site's, tracking pixels and lazy placeholders aren't thumbnails, an og:url with a slug keeps the OG tags, and a <title> loses its site suffix. genericPreview is the base of the Bandcamp, SoundCloud and YouTube channel previews; SoundCloud takes www., m. and on.soundcloud.com short links, embedding the page they lead to, and Bandcamp keeps only its own player. The service tidies every preview: text on one line, titles and descriptions capped, only http(s) images and players. RichContent.type is now RichContentType, and the unused videoId is gone.
2026-10-09 20:46:32 +00:00
v4: every image the API serves comes scaled to the size it is shown at — a first visit to the feed loaded 4.8 MB of images out of 5.1 MB in all: an avatar shown at 32 pixels was the upload itself, a 1.4 MB GIF among them, and so were a cover, a custom thumbnail, an attachment and a proxied image, only an image dump having a preview, 960 pixels wide. Every route that answers an image now takes ?w= and sends a WebP copy at most that wide, of the widths shared/image.ts lists, 128, 320, 640 and 960, a width between two taking the one above, and never wider than the image; without it the upload goes out as before, a dump's thumbnail alone being scaled to the widest by default, the custom ones included. serveImage (lib/upload.ts) is where they all end, and the image proxy scales what it fetched the same way, keeping each width in its cache. lib/ffmpeg.ts makes the copies as it made the previews, which they replace: on first request, under uploads/scaled/ at the upload's own path, each given its upload's date and made again when the two differ, so a new avatar or cover needs nobody to delete the old ones. WebP keeps a PNG's transparency and a GIF's animation. A JPEG is turned by its EXIF orientation here, ffmpeg's own rotation of images depending on its version. An animated WebP, which ffmpeg can't read, and an image it fails on go out as they are, a failure being remembered until the upload changes. ?f=compat asks for the copy as a JPEG, or a PNG for what may be transparent: the digest's emails take their thumbnails that way, at 640 pixels, not every mail client reading WebP. On the client, imageProps (utils/shared/urls.ts) gives an image its srcset of those copies and the width it is laid out at, which config/imageSizes.ts states for each place from App.css: Avatar, Thumbnail, which scales an API image and leaves another site's as that site sends it, and the playlists' covers use it. The images of notes and comments still load as uploaded. The copies go with the dump, the custom thumbnail or the attachment they were made from, and generatedMediaSweepJob, a minute after the start then daily, takes those whose upload is gone, the previews under their former name and the temporary files a generation left unfinished.
2026-10-10 19:01:36 +00:00
v4: what the server and the client both state is declared once — several rules were written out in two or more files, some already drifting apart. A link's site name (its host without www.) was computed in five places, now shared/url.ts's siteHost; dumpFileVersion, which the frontend and the server had to keep equal by hand, is one function in shared/dump.ts; the Markdown flattening of the chat's reply previews and the digest's quotes is shared/markdown.ts's markdownToPlainText, so the digest also keeps an image's alt text; an attachment's URL and how to find one in a text are shared/attachment.ts, which the editor, Markdown and attachment-service read, its id pattern the new UUID_RE that the file and thumbnail routes check ids with. On the server, the Bandcamp, YouTube playlist and link preview caches use one TtlCache (api/lib/ttl-cache.ts) instead of three copies of the same expiry and eviction; the link previews' theme-color and the emails' theme tokens read colours with one parser (api/lib/css-color.ts), and a transparent theme-color now gives no tint; the WebSocket route sends through ws-service's send, the welcome included, so it is type-checked; fetch.ts strips an IPv6 literal's brackets with public-address's bareHostname; and a category's slug is checked by the same validateSlug as a dump's, with its own length. The migrations keep their frozen copies, as they document. 272 backend tests and 677 frontend tests pass, and deno check, tsc, eslint and doc:lint are clean.
2026-10-09 13:16:52 +00:00
v3: fix the backend's correctness findings — each by a rule shared across the code rather than one more special case. Link previews: readTextCapped decoded every page as UTF-8, so a Latin-1 page came out garbled unless it went through the curl fallback, which re-encoded it on its own; pages are now decoded in the charset their Content-Type or <meta> declares, and the curl fallback hands the bytes over as they are. The image proxy served and cached for a day an error answer that claimed an image type, and read a body without Content-Length whole before checking its size; it now refuses an error status and reads through the same capped reader. The icons task's Twemoji fetch gives up after 15 s instead of holding the start up, and extractLinkHref, which only its tests used, is gone. Backlinks matched a linked URL against the raw url column, so https://x.com/ missed a dump posted as https://x.com; they now compare canonical URLs, as reposts do. The four LIKE searches put the query in raw, so % and _ were wildcards and the mention search for "a_b" found "axb"; lib/like.ts escapes them. Every list query now treats a malformed row the same way, a 500 through model/rows.ts's parseRows, where ten of them silently dropped it; the playlist dumps query uses the model's column list rather than a hand-written copy, and the aliased list is derived from the plain one. The vote and like services commit through transaction() instead of their own BEGIN and ROLLBACK, which would have rolled back a caller's transaction.
2026-09-29 19:35:53 +00:00
v4: what the server and the client both state is declared once — several rules were written out in two or more files, some already drifting apart. A link's site name (its host without www.) was computed in five places, now shared/url.ts's siteHost; dumpFileVersion, which the frontend and the server had to keep equal by hand, is one function in shared/dump.ts; the Markdown flattening of the chat's reply previews and the digest's quotes is shared/markdown.ts's markdownToPlainText, so the digest also keeps an image's alt text; an attachment's URL and how to find one in a text are shared/attachment.ts, which the editor, Markdown and attachment-service read, its id pattern the new UUID_RE that the file and thumbnail routes check ids with. On the server, the Bandcamp, YouTube playlist and link preview caches use one TtlCache (api/lib/ttl-cache.ts) instead of three copies of the same expiry and eviction; the link previews' theme-color and the emails' theme tokens read colours with one parser (api/lib/css-color.ts), and a transparent theme-color now gives no tint; the WebSocket route sends through ws-service's send, the welcome included, so it is type-checked; fetch.ts strips an IPv6 literal's brackets with public-address's bareHostname; and a category's slug is checked by the same validateSlug as a dump's, with its own length. The migrations keep their frozen copies, as they document. 272 backend tests and 677 frontend tests pass, and deno check, tsc, eslint and doc:lint are clean.
2026-10-09 13:16:52 +00:00