updated README

2026-08-17 16:50:16 +00:00
parent 30e7430820
commit 7b7eb88d5b
1 changed files with 89 additions and 14 deletions
--- a/README.md
+++ b/README.md
@@ -21,12 +21,18 @@ Work is split between **automated scripts** (data fetching, Hugo content generat

 ```
 publish-assistant/
+├── .gitea/workflows/gh-pages.yaml        # CI: renders the site, deploys to the gh-pages branch
 ├── site/                                 # Hugo project
 │   ├── hugo.toml
-│   ├── themes/PaperMod/                  # PaperMod theme (git submodule)
+│   ├── package.json                      # FullCalendar deps (resolved by Hugo's js.Build)
+│   ├── themes/PaperMod/                  # PaperMod theme (git submodule — never edit in place)
+│   ├── assets/
+│   │   ├── js/calendar.js                # FullCalendar bootstrap, bundled by js.Build
+│   │   └── css/extended/custom.css       # Theme overrides (auto-appended by PaperMod's head)
 │   ├── layouts/
 │   │   ├── _default/calendar.html        # FullCalendar layout
 │   │   ├── _default/digests.html         # Digest list layout
+│   │   ├── _default/_markup/render-link.html  # Rewrites root-relative links for subpath baseURL
 │   │   └── partials/
 │   │       ├── header.html               # Section-aware nav override
 │   │       └── extend_head.html          # Calendar CSS injection
@@ -41,7 +47,7 @@ publish-assistant/
 │       ├── topics.yaml                   # Topic registry (nav + build loop)
 │       ├── rankings/                     # Shared across all topics
 │       │   ├── icore.csv                 # ICORE conference rankings
-│       │   └── scimago.csv               # SCImago journal rankings (optional)
+│       │   └── scimago.csv               # SCImago journal rankings (manual download, committed)
 │       └── <topic>/                      # One subdir per topic, e.g. cloud-edge/
 │           ├── venues.yaml               # Master venue list ← primary edit target
 │           ├── deadlines.yaml            # Deadline cache; manual entries preserved
@@ -70,6 +76,7 @@ publish-assistant/

 ```bash
 uv sync                        # installs all dependencies + registers CLI tools
+npm --prefix site ci           # FullCalendar packages; required for any hugo build

 # Available commands after sync:
 uv run pa-fetch-icore
@@ -85,6 +92,32 @@ uv run pa-generate
 ./build.sh --skip-rankings --topic cloud-edge --dev
 ```

+`site/assets/js/calendar.js` imports `@fullcalendar/*`, and Hugo's `js.Build` resolves
+those from `site/node_modules/` at render time. That directory is gitignored, so a fresh
+clone needs `npm --prefix site ci` before `hugo` (or `build.sh`) will succeed.
+
+Cloning: the theme is a submodule — `git clone --recurse-submodules`, or
+`git submodule update --init --recursive` after the fact.
+
+---
+
+## Deployment
+
+`.gitea/workflows/gh-pages.yaml` runs on every push to `main` (and on manual dispatch):
+checkout with submodules → install Hugo (pinned via `HUGO_VERSION`) → `npm ci` in `site/`
+→ `hugo --source site --minify --gc --baseURL "$SITE_BASE_URL"` → push `site/public/` to
+the orphan `gh-pages` branch with `peaceiris/actions-gh-pages`.
+
+**CI does not fetch data and does not run `build.sh`.** Everything under `site/content/`
+is generated locally by `./build.sh` and committed; CI only renders it. If generated
+content is not committed, the deployed site is stale.
+
+**Base URL**: the site is served from a subdirectory (`https://pub.sqrt.fr/<owner>/<repo>/`
+by default). Override with the repo variable `SITE_BASE_URL` (Settings → Actions →
+Variables) if the site moves. Because generated Markdown stores links as root-relative
+paths (`/cloud-edge/venues/…`), three layouts run them through `relURL` so they survive the
+subpath — see "Subpath-safe links" under Hugo Content Structure.
+
 ---

 ## Data Sources
@@ -92,7 +125,7 @@ uv run pa-generate
 | Source | What it provides | Automatable? | Known issues |
 | --- | --- | --- | --- |
 | [ICORE](https://portal.core.edu.au/conf-ranks/) | Conference rankings (A*, A, B, C) | Yes | Pagination uses `javascript:jumpPage('N')` — handled in `fetch_icore.py` |
-| [SCImago](https://www.scimagojr.com/) | Journal quartiles, SJR, H-index | Blocked | Anti-bot returns HTML; add data manually to `venues.yaml` under `scimago_quartile` etc. |
+| [SCImago](https://www.scimagojr.com/) | Journal quartiles, SJR, H-index | Blocked | Anti-bot returns HTML. A manually downloaded CSV is committed at `site/data/rankings/scimago.csv`; `pa-generate --scimago` matches journals by title. Inline `venues.yaml` values still win. |
 | [DBLP](https://dblp.org/) | Paper metadata by venue | Yes | Use `dblp_key` field in `venues.yaml` |
 | [OpenAlex](https://openalex.org/) | Papers, open-access links | Yes | Fallback when DBLP is thin |
 | [WikiCFP](http://wikicfp.com/) | Submission deadlines | Partially | See detailed notes below |
@@ -155,7 +188,7 @@ journals:
    url: "https://www.computer.org/csdl/journal/td"
    dblp_key: "journals/tpds"
    submission_model: rolling
-    scimago_quartile: Q1      # add manually — SCImago CSV download is blocked
+    scimago_quartile: Q1      # optional — falls back to rankings/scimago.csv by title
    scimago_sjr: "1.560"
    scimago_h_index: "131"
 ```
@@ -192,11 +225,12 @@ uv run pa-fetch-icore --query "distributed systems"
 ```

 ### `pa-fetch-scimago`
-Downloads SCImago CSV. **Currently blocked by anti-bot.** Will raise a descriptive error if it receives HTML instead of CSV. Add journal data manually to `venues.yaml` instead.
+Downloads SCImago CSV. **Currently blocked by anti-bot.** Will raise a descriptive error if it receives HTML instead of CSV (`build.sh` treats this as a warning and continues on cached data).
 ```bash
 uv run pa-fetch-scimago --list-areas    # show area codes
 uv run pa-fetch-scimago --area 1705     # networks
 ```
+The working substitute is a manual download: scimagojr.com → Journal Rankings → pick an area → Download, saved as `site/data/rankings/scimago.csv` (semicolon-separated; the loader sniffs the delimiter). That file is committed and `pa-generate` reads it, so journals without inline `scimago_*` fields still get a quartile, SJR, and H-index. Matching is by `full_name` against the CSV `Title` column (exact, then substring), so a journal whose name differs from SCImago's still needs inline values.

 ### `pa-fetch-deadlines`
 Fetches submission deadlines from WikiCFP for a specific topic. Preserves `source: manual` entries.
@@ -262,6 +296,34 @@ Hugo treats `site/content/<topic>/` as a section. All pages inside it are served

 To add a topic: populate `site/data/topics.yaml` + `site/data/<topic>/venues.yaml`, then run `./build.sh --topic <slug>`. The new section appears in the global nav automatically.

+### Subpath-safe links
+
+`pa-generate` writes root-relative links (`/cloud-edge/venues/…`). Those are correct when
+the site is served from a domain root, but they ignore the subdirectory in `baseURL` and
+404 under `https://pub.sqrt.fr/<owner>/<repo>/`. Rather than teaching the generator about
+deployment, three layouts normalize at render time — each strips the leading slash so
+`relURL` will prepend the baseURL subpath (`relURL` leaves already-root-relative input
+alone):
+
+- **`layouts/_default/_markup/render-link.html`** — Hugo render hook applied to every
+  Markdown link in generated bodies. Protocol-relative `//host/…` links are left as-is.
+- **`layouts/_default/calendar.html`** — rewrites the `url` of each event before
+  `jsonify`, so calendar clicks land on the right page.
+- **`layouts/partials/header.html`** — nav links go through `relURL`, and the active-item
+  check compares against `.RelPermalink`, which carries the same prefix.
+
+All of this is a no-op when the site is served from the domain root (the local dev server).
+
+### Theme and styling
+
+`themes/PaperMod` is a pristine git submodule pinned to an upstream commit. **Never edit
+files inside it**: those changes cannot be committed from this repo, and CI checks the
+submodule out fresh, so they would silently vanish from the deployed site. Project CSS
+lives in `site/assets/css/extended/custom.css`, which PaperMod's `head.html` appends after
+its own stylesheet — currently a shorter `.first-entry` hero, since PaperMod reserves
+320px/260px for a lead image the site does not use. Calendar-only CSS stays in
+`layouts/partials/extend_head.html`, gated on `layout == "calendar"`.
+
 ### Venues

 **`site/content/<topic>/venues/_index.md`** — overview, links to conferences and journals. Generated.
@@ -301,9 +363,10 @@ Steps:
   website URL for the upcoming edition, DBLP stream key (conf/<key>), and
   WikiCFP event ID if you can find it (set wikicfp_id: false for short or
   ambiguous acronyms).
-3. For each journal: record acronym, full name, ISSN, SCImago quartile + SJR
-   + H-index (add inline to venues.yaml — the CSV download is blocked), DBLP
-   stream key (journals/<key>), submission model (rolling / special issues).
+3. For each journal: record acronym, full name, ISSN, DBLP stream key
+   (journals/<key>), submission model (rolling / special issues). Quartile,
+   SJR, and H-index come from site/data/rankings/scimago.csv when full_name
+   matches the CSV Title; add them inline to venues.yaml only when it doesn't.
 4. Create site/data/<slug>/venues.yaml following the schema in the README.
 5. Create site/data/<slug>/papers/ (empty directory).
 6. Add the topic to site/data/topics.yaml:
@@ -323,15 +386,18 @@ Steps:
 10. Run: ./build.sh --skip-rankings --topic <slug>
    Then: hugo --source site --minify
    Confirm the site builds cleanly and /<slug>/venues/ loads correctly.
+11. Commit the generated site/content/<slug>/ tree along with the data files —
+    CI renders committed content and never regenerates it.

 Checklist before finishing:
 - [ ] site/data/<slug>/venues.yaml has all venues with correct dblp_key
 - [ ] wikicfp_id set or false on every conference
- [ ] SCImago data added inline for every journal
+- [ ] Every journal shows a quartile (from scimago.csv, or inline if unmatched)
 - [ ] site/data/topics.yaml updated
 - [ ] site/content/<slug>/_index.md created
 - [ ] ./build.sh --skip-rankings --topic <slug> runs without errors
 - [ ] hugo --source site --minify succeeds
+- [ ] Generated content under site/content/<slug>/ committed
 ```

 ---
@@ -355,7 +421,11 @@ Add <ACRONYM> to the publish-assistant topic "<slug>".
 5. Run: uv run pa-generate (defaults to cloud-edge) or with explicit --venues /
   --content / --base-path flags for the target topic.
 6. Confirm the new venue page appears at /<slug>/venues/conferences/<acronym>/
-   (or journals/) with correct metadata.
+   (or journals/) with correct metadata. For a journal, check that the quartile
+   resolved from site/data/rankings/scimago.csv; if it came out blank, the
+   full_name doesn't match the CSV Title — add scimago_* fields inline.
+7. Commit the regenerated site/content/<slug>/ files together with the data
+   changes; CI deploys committed content and does not regenerate it.
 ```

 ---
@@ -433,6 +503,7 @@ Checklist:
 - [ ] All wikicfp_id values verified (or set to false + manual entry)
 - [ ] missing: list in deadlines.yaml is empty
 - [ ] Build succeeds, no broken links
+- [ ] Regenerated site/content/<slug>/ committed so CI deploys the new deadlines
 ```

 ---
@@ -479,8 +550,8 @@ selected:
 ### Task 4 — Refresh rankings

 1. Run `uv run pa-fetch-icore` for fresh ICORE data.
-2. For SCImago: download the CSV manually from [scimagojr.com](https://www.scimagojr.com/journalrank.php) and place at `site/data/rankings/scimago.csv`. The automated fetch is blocked.
-3. For each journal in `venues.yaml`, compare `scimago_quartile` / `scimago_sjr` against new CSV. Update inline values if changed.
+2. For SCImago: download the CSV manually from [scimagojr.com](https://www.scimagojr.com/journalrank.php) and replace `site/data/rankings/scimago.csv` (commit it — CI does not fetch). The automated fetch is blocked.
+3. For each journal in `venues.yaml` that carries inline `scimago_quartile` / `scimago_sjr`, compare against the new CSV and update if changed. Journals without inline values pick the new numbers up automatically on the next `pa-generate`.
 4. Run `./build.sh --skip-deadlines` (or per-topic with `--topic <slug>`).

 ### Task 5 — Add a new venue mid-cycle
@@ -499,7 +570,11 @@ See the "Annual cycle refresh for a topic" prompt above.
 - **`_index.md` vs `index.md`**: section pages use `_index.md` (list template), leaf pages use `index.md` (single template). Digest pages are `index.md` — using `_index.md` makes them section pages and breaks pagination.
 - **Rankings CSVs are in `site/data/rankings/`**: Hugo's data loader is configured to ignore `data/rankings/*.csv` via `ignoreFiles` in `hugo.toml`. If you move these files or add new CSVs, update `ignoreFiles` accordingly — Hugo cannot parse arbitrary CSV as a data map and will error on build.
 - **ICORE pagination**: the ICORE portal uses `javascript:jumpPage('N')` links, not standard `?page=N` URLs. `fetch_icore.py` handles this. If you get only 50 results instead of ~900, pagination is broken.
- **SCImago blocked**: `pa-fetch-scimago` will raise a clear error if anti-bot HTML is returned. Add data inline to `venues.yaml` instead.
+- **SCImago blocked**: `pa-fetch-scimago` will raise a clear error if anti-bot HTML is returned. Refresh `site/data/rankings/scimago.csv` by hand, or add data inline to `venues.yaml`.
+- **Never edit `site/themes/PaperMod/`**: it is a submodule pinned to upstream, and CI checks it out fresh — edits there are not committed and disappear on deploy. Put CSS in `site/assets/css/extended/custom.css` and template overrides in `site/layouts/`.
+- **Generated content is committed, and CI never regenerates it**: the Gitea workflow only runs `hugo`. After editing any `site/data/` file, run `./build.sh` (or `pa-generate`) and commit the resulting `site/content/` diff, or the deployed site will not change.
+- **`npm ci` before `hugo`**: `js.Build` resolves `@fullcalendar/*` from `site/node_modules/`, which is gitignored. A fresh clone that skips it gets a build error on the calendar page.
+- **Root-relative links vs. subpath `baseURL`**: generated links start with `/`, which Hugo leaves untouched — they break when the site is served from a subdirectory. The render hook, calendar layout, and header partial each strip the leading slash and call `relURL`. If you add a layout that emits links from data, do the same; do not "fix" it by changing `--base-path`, which sets the topic prefix, not the deployment prefix.
 - **WikiCFP `<th>` labels**: deadline detail pages use `<th>` for label cells, not `<td>`. The parser looks for `<th>+<td>` pairs.
 - **Calendar events**: `pa-generate` embeds events as YAML in the `events:` front matter field. The layout at `site/layouts/_default/calendar.html` reads `.Params.events` and initializes FullCalendar. Do not remove the `layout: calendar` front matter field.
 - **Deadline preservation**: never strip deadline fields from `deadlines.yaml` just because the submission window has closed. Remove an entry only once the conference has taken place.