vincent/publish-assistant
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Surface stale deadlines instead of showing them as upcoming
All checks were successful
Build and deploy static pages / build-and-push (push) Successful in 14s
Details

Browse Source 
The calendar's deadline table rendered every recorded submission cycle, so
elapsed rounds sorted to the top and read as the next thing due — 13 of its 17
rows were already in the past. Underneath that, the fetcher preserves
`source: manual` entries forever without ever checking whether they still
describe a future deadline, and 11 of 13 cloud-edge venues are manual.

Preserving them is correct: WikiCFP cannot express multi-cycle venues and does
not carry most systems conferences at all, so re-fetching would replace
researched rounds with worse data or nothing. What was missing is a signal.

- Extract the cycle helpers into cycles.py so the fetcher and the generator
  share one definition of which round a reader should act on. cycles_of()
  reads an entry without folding flat fields into a cycles list, which would
  otherwise rewrite every single-round entry the fetcher writes back.
- The calendar table lists open cycles only; a venue whose every round has
  elapsed moves to an "Awaiting the next call" table instead of vanishing.
- pa-fetch-deadlines records those venues under a new stale: key and probes
  the year-bumped CFP URL to say which ones already have a next edition
  online. A candidate must return 200, mention the target year, and be newer
  than any year the entry records — venues.yaml still points SC at sc24.
- README: Task 7 and a matching agent prompt for keeping deadlines current,
  since nothing in this repo refreshes them on its own.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Vincent Lannurien
2026-08-17 22:25:24 +02:00
parent 827c432dc0
commit 5b06f484f5
6 changed files with 431 additions and 122 deletions
Download Patch File Download Diff File Expand all files Collapse all files

82
README.md
View File

@@ -157,7 +157,9 @@ A venue whose `deadlines.yaml` entry says `source: manual` is skipped by the fet
**Conferences not on WikiCFP** (for systems/networking): OSDI, NSDI, USENIX ATC, SC, MobiSys, SEC. Use `wikicfp_id: false` for all of them.
**Manual deadline entries**: Add entries with `source: manual` to `site/data/<topic>/deadlines.yaml`. The fetcher preserves all `source: manual` entries across runs. Format:
**Manual deadline entries**: Add entries with `source: manual` to `site/data/<topic>/deadlines.yaml`. The fetcher preserves all `source: manual` entries across runs — indefinitely, and by design: WikiCFP cannot express multi-cycle venues and does not carry most systems conferences at all, so re-fetching a manual entry would replace researched data with worse data or nothing. Nothing refreshes these but a person or an agent.
Because of that, a manual entry can sit unchanged long after its deadline passes. The fetcher does not fix this, but it does **report** it: any entry whose recorded rounds have all elapsed is listed under `stale:` in `deadlines.yaml` and printed at the end of the run. See Task 7. Format:
```yaml
ATC:
source: manual
@@ -202,7 +204,7 @@ What the generator does with cycles:
| --- | --- |
| Venue page | Multi-cycle venues get a **Submission Cycles** table with every round; the next open one is bolded and marked `(next)`. Single-cycle venues keep the **Upcoming Deadline** block unchanged. |
| Venue index | One row per venue, showing the next open cycle with its name appended to the paper deadline. |
| Calendar table | One row per cycle, with a `Cycle` column, sorted by paper deadline. |
| Calendar table | One row per **still-open** cycle, with a `Cycle` column, sorted by paper deadline. A venue whose every round has elapsed drops out of this table into an **Awaiting the next call** table below it, showing the last round that ran — an elapsed round sorts to the top by date and would otherwise read as the next thing due. |
| Calendar grid | One event set per cycle; titles read `EuroSys Fall — paper deadline`. Elapsed cycles are kept — the grid is time-indexed, so past rounds only show when you navigate back to their month. |
| Front matter | `next_deadline` / `abstract_deadline` / `notification` / `camera_ready` reflect the **next** cycle. Multi-cycle venues additionally carry `cycles` and `next_cycle`. |
@@ -259,7 +261,7 @@ Every field must come from a published CFP. There is no fetcher for these, so no
Special issue deadlines appear in three places:
- **Journal page** — a "Special Issues" table below the journal metadata
- **Calendar page** — a "Journal Special Issues" table below the conference deadlines. The heading always renders; with no entries recorded it carries a line saying no call is open, so the page doesn't read as if journals went untracked
- **Calendar page** — a "Journal Special Issues" table below the conference deadlines (which themselves split into open cycles and "Awaiting the next call"). The heading always renders; with no entries recorded it carries a line saying no call is open, so the page doesn't read as if journals went untracked
- **FullCalendar grid** — events colored purple (`#7b2cbf`), titled `ACR SI — Title — deadline type`, carrying `cfp_url` in `extendedProps`
Dates use the same `MMM D, YYYY` format as conference deadlines. `notification` additionally accepts month-only precision (`Jan 2027`), which lands on the first of that month in the calendar grid; the tables always print the raw string.
@@ -308,9 +310,22 @@ Fetches submission deadlines from WikiCFP for a specific topic. Preserves `sourc
```bash
uv run pa-fetch-deadlines \
--venues site/data/cloud-edge/venues.yaml \
--output site/data/cloud-edge/deadlines.yaml
--output site/data/cloud-edge/deadlines.yaml \
[--no-probe]
```
After running: check `deadlines.yaml` for the `missing:` list, then do Task 2.
Writes three report keys alongside `deadlines:`:
| Key | Meaning |
| --- | --- |
| `missing:` | No CFP found at all — acronyms only. Fill in by hand (Task 2). |
| `stale:` | The entry exists but every recorded round has elapsed. One map per venue: `acronym`, `last_deadline`, `source`, `cfp_url`, and `next_edition_url` when the probe found one. Refresh by hand (Task 7). |
**Next-edition probe**: for stale venues only, the script bumps the year in the recorded `cfp_url` (and, failing that, the venue's `url`) and checks whether that page exists — `mobisys/2026/` → `mobisys/2027/`, `sc26.supercomputing.org` → `sc27.…`, `ipdps2027/2027-call-for-papers.html` → `ipdps2028/2028-…`. A candidate counts only if it returns 200, mentions the target year (guarding against hosts that serve a landing page for any path), and is newer than any year the entry already records (`venues.yaml` URLs lag — SC's still points at sc24). This does not read deadlines; it only tells you which venues are worth researching *now*. `--no-probe` skips it.
The run ends with a summary naming every missing and stale venue. It always exits 0 — `build.sh` runs under `set -euo pipefail`, so a non-zero exit would abort the build before content is generated.
After running: check the summary, then do Task 2 (missing) and Task 7 (stale).
### `pa-fetch-best-papers`
Scrapes [jeffhuang.com/best_paper_awards/](https://jeffhuang.com/best_paper_awards/) and writes `best_papers.yaml`. Run once per year.
@@ -579,6 +594,44 @@ Checklist:
---
### Keep deadlines current
```
Refresh the deadlines for the publish-assistant topic "<slug>".
1. Run: ./build.sh --skip-rankings --topic <slug>
Read the summary the deadline fetch prints at the end — it lists every venue
under missing: (no entry at all) and stale: (entry frozen on an edition that
already ran), and for stale ones whether the next edition's page is online.
2. For each missing: venue, visit the official CFP page and add a source: manual
entry to site/data/<slug>/deadlines.yaml (README Task 2).
3. For each stale: venue:
- If the report printed "NEW EDITION: <url>", open it and record the new
round(s): abstract deadline, paper deadline, notification, camera-ready,
event dates, location, cfp_url.
- If it printed "next edition not announced yet", still check the venue site
once — the probe only tries a year-bumped URL, so it misses venues that
move host between editions (HPDC) or use an unrelated page name.
- If the new call genuinely is not out, leave the entry untouched. It shows
under "Awaiting the next call" on the calendar, which is correct.
- Multi-round venues keep source: manual and a cycles: list. Do not hand them
back to WikiCFP — it records only one round and would drop the others.
- Update the venue's url in venues.yaml if it still points at the old edition.
4. Re-run: ./build.sh --skip-rankings --topic <slug>
Checklist:
- [ ] Every stale: venue either updated or confirmed to have no published CFP
- [ ] missing: list empty, or each entry explained
- [ ] No source: manual entry lost its cycles
- [ ] site/data/<slug>/ and the regenerated site/content/<slug>/ both committed
(CI only runs hugo — it never regenerates content)
```
---
## Agent Tasks (Reference)
### Task 1 — Bootstrap a topic
@@ -586,7 +639,7 @@ See the "Bootstrap a new topic" prompt above. The one-shot checklist is embedded
### Task 2 — Fill in missing deadlines
**Trigger:** `pa-fetch-deadlines` lists conferences under `missing:`, or a deadline looks wrong.
**Trigger:** `pa-fetch-deadlines` lists conferences under `missing:`, or a deadline looks wrong. For venues that *have* an entry whose rounds have all elapsed, see Task 7 instead — same research, different starting point.
1. For each missing conference, visit the official website's "Call for Papers" / "Important Dates" page.
2. Also check WikiCFP manually — if you find the right event ID, add `wikicfp_id` to `venues.yaml` so future runs fetch it automatically.
@@ -631,6 +684,21 @@ See the "Add a venue to an existing topic" prompt above.
### Task 6 — Annual cycle refresh
See the "Annual cycle refresh for a topic" prompt above.
### Task 7 — Keep deadlines current
**Trigger:** routine maintenance. Nothing in this repo refreshes deadlines on its own — no cron, no CI job, and manual entries are preserved forever by design (see "Manual deadline entries"). An agent has to run this, and should run it whenever picking the repo up.
1. Run `./build.sh --skip-rankings` (add `--topic <slug>` to scope it). Read the summary the deadline fetch prints at the end.
2. **`missing:`** — no entry at all. Do Task 2.
3. **`stale:`** — the entry is frozen on an edition that already ran. For each one:
- If the report says `NEW EDITION: <url>`, open it: the next call is already published, so the dates are there to be recorded.
- Otherwise check the venue's site anyway before moving on — the probe only tries a year-bumped URL, so it misses venues that change host between editions (HPDC) or announce on a page with an unrelated name.
- When the new CFP exists, replace that venue's rounds in `deadlines.yaml`, update `event_dates`, `location`, `cfp_url`, and the venue's `url` in `venues.yaml` if it points at the old edition. Keep `source: manual` on multi-cycle venues — see "WikiCFP cannot express this".
- When it does not exist yet, leave the entry alone. It stays under "Awaiting the next call" on the calendar, which is the honest state.
4. Re-run `./build.sh --skip-rankings`, then commit **both** `site/data/<topic>/` and the regenerated `site/content/<topic>/` — CI only runs `hugo` and will not regenerate anything.
Worth re-running even when no deadline changed: the calendar's open/elapsed split is computed against the date of the run, so the committed page drifts as rounds pass.
---
## Known Gotchas (for agents picking this up)
@@ -649,6 +717,8 @@ See the "Annual cycle refresh for a topic" prompt above.
- **WikiCFP `<th>` labels**: deadline detail pages use `<th>` for label cells, not `<td>`. The parser looks for `<th>+<td>` pairs.
- **Calendar events**: `pa-generate` embeds events as YAML in the `events:` front matter field. The layout at `site/layouts/_default/calendar.html` reads `.Params.events` and initializes FullCalendar. Do not remove the `layout: calendar` front matter field.
- **Deadline preservation**: never strip deadline fields from `deadlines.yaml` just because the submission window has closed. Remove an entry only once the conference has taken place.
- **Nothing refreshes deadlines on its own**: `source: manual` entries are preserved across every fetcher run by design, and there is no cron or CI job that would notice. A frozen entry is reported under `stale:`, never fixed. Run Task 7.
- **The calendar's open/elapsed split is a build-time decision**: `pa-generate` compares each cycle against the date it runs, so a round that lapses after the last build stays in the "Conference Deadlines" table until content is regenerated *and committed*. Re-running `./build.sh` periodically is worthwhile even when no deadline data changed.
- **WikiCFP IDs are edition-specific**: each year's event gets a new ID. IDs verified at one point in time will be wrong once a new cycle begins. See Task 6 for the annual refresh checklist.
---