v3: reworked posting — three panels, drafts, duplicate detection, upload progress
All checks were successful
Build and Publish Docker Image / build-and-push (push) Successful in 42s

Posting a dump was a single form where the important choices were the easiest
to miss. It is now three panels: link or file, why & where, playlists.

Composition:
- No more URL/File toggle. An empty panel offers both ways in at once and the
  kind follows what you actually did; a file dropped anywhere in the modal is
  accepted, not just on the zone.
- Categories and visibility get their own panel instead of a disclosure that
  read as optional, and the primary button stays "Next" until they've been
  seen. Visibility carries a real label now.
- The draft (link, title, why, categories, visibility) is mirrored to
  localStorage on every change and restored on reopen, so Escape or a stray
  backdrop click costs nothing. Only an attached file can't be restored, so
  that is the one case that asks before closing.
- URL dumps can carry a poster-supplied title instead of being stuck with
  whatever the page scraped, editable right under the preview.
- Multipart uploads go through XHR so there is a real progress bar and a
  percentage on the button, rather than 50 MB of silence.

Duplicates:
- New dumps.url_canonical column (+ index, backfilled by 0013) holding a lossy
  key that ignores scheme, www., trailing slashes, tracking parameters and
  YouTube share shapes. GET /api/dumps/by-url reads it, and the create form
  warns "already dumped by X" while it fetches the preview. Never blocking.

Fixes:
- /api/preview now reports whether the page was actually reached: a failed
  fetch still yields a hostname-only stub, so a dead link and a page without
  metadata used to render identically.
- The Web Share Target never worked. The manifest posts to "/", but the index
  redirect dropped the query string, so every Android share landed on the feed
  with nothing pre-filled.
- File dumps no longer take the extension into their title.
- The link field no longer autofocuses on touch, where it raised a keyboard
  over the modal.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TiAPtJZeCYYk8rKehtLUQU
This commit is contained in:
khannurien
2026-09-08 14:40:37 +00:00
parent d76154d15d
commit fb8364e24d
21 changed files with 1688 additions and 304 deletions

104
api/lib/canonical-url.ts Normal file
View File

@@ -0,0 +1,104 @@
/**
* Canonical form of a URL, used *only* to recognise that two links point at the
* same thing — the duplicate check the create form runs while it fetches a
* preview. It is a lookup key, never something we display or fetch: `dumps.url`
* keeps the exact string the poster submitted.
*
* Because it is only ever compared against other canonical forms, it may be
* lossy in ways a real URL never could be — it forces `https`, drops `www.`,
* and throws away share/tracking parameters and timestamps, so
* `http://www.example.com/a/?utm_source=x` and `https://example.com/a` collapse
* to one key.
*
* Stored in `dumps.url_canonical`. Migration 0013 carries a frozen copy of this
* logic to backfill existing rows; changing the rules here therefore needs a
* new backfill migration, or old rows keep keys computed under the old rules.
*/
const TRACKING_PARAMS = new Set([
"fbclid",
"gclid",
"dclid",
"msclkid",
"twclid",
"yclid",
"mc_cid",
"mc_eid",
"igshid",
"igsh",
"si",
"spm",
"ref_src",
"ref_url",
"_ga",
"_gl",
"__twitter_impression",
]);
function isTrackingParam(key: string): boolean {
const k = key.toLowerCase();
return k.startsWith("utm_") || TRACKING_PARAMS.has(k);
}
// Hosts already stripped of a leading "www.".
const YOUTUBE_HOSTS = new Set([
"youtube.com",
"m.youtube.com",
"music.youtube.com",
"youtube-nocookie.com",
]);
/**
* The video a YouTube URL points at, in any of the shapes people paste
* (`youtu.be/ID`, `/watch?v=ID`, `/embed/ID`, `/shorts/ID`, `/live/ID`).
* Timestamps are deliberately ignored: the same video linked at 2:30 is still
* the same video for duplicate purposes.
*/
function youtubeVideoId(
host: string,
pathname: string,
params: URLSearchParams,
): string | null {
if (host === "youtu.be") return pathname.split("/")[1] || null;
if (!YOUTUBE_HOSTS.has(host)) return null;
if (pathname === "/watch") return params.get("v");
if (/^\/(embed|shorts|live)\//.test(pathname)) {
return pathname.split("/")[2] || null;
}
return null;
}
export function canonicalizeUrl(raw: string): string | null {
let u: URL;
try {
u = new URL(raw);
} catch {
return null;
}
if (u.protocol !== "http:" && u.protocol !== "https:") return null;
const host = u.hostname.toLowerCase().replace(/^www\./, "");
if (!host) return null;
const videoId = youtubeVideoId(host, u.pathname, u.searchParams);
if (videoId) return `https://youtube.com/watch?v=${videoId}`;
const listId = YOUTUBE_HOSTS.has(host) && u.pathname === "/playlist"
? u.searchParams.get("list")
: null;
if (listId) return `https://youtube.com/playlist?list=${listId}`;
// A single trailing slash is never meaningful; "/" itself becomes "".
const path = u.pathname.replace(/\/+$/, "");
const params = [...u.searchParams.entries()]
.filter(([key]) => !isTrackingParam(key))
.sort(([a, av], [b, bv]) => a.localeCompare(b) || av.localeCompare(bv));
const query = new URLSearchParams(params).toString();
// A bare "#section" anchor points into the same page, so it is dropped —
// but "#/route" and "#!/route" address distinct pages of a hash-routed app.
const hash = /^#!?\//.test(u.hash) ? u.hash : "";
return `https://${host}${path}${query ? `?${query}` : ""}${hash}`;
}