Extract the store engines' shared core
A store engine reads a WarpEngine catalog and writes some host's own game library format. Only that last step is about the host: the catalog, the release selection, the machine matching, the state and the HTTP were the same code in every engine, and until now there was one engine, so they lived inside it. A second engine — RetroArch — made the seam visible, so the shared part moves here as a single file both engines fetch at install time. An adapter now supplies three things: a DEFAULT_CONFIG describing its host, an accept() callback deciding which catalog entries that host can run, and the code that writes the host's library format. Two generalisations came out of having two hosts rather than one: - `resolve_for_arch` becomes `resolve_for_host`. Batocera only ever answered "linux", so architecture was the only variable; a RetroArch host can be Windows, macOS or Android, and that decides such things as what a libretro core file is called. Keys are now os-arch, arch, os, *. - state records are keyed by `scope`, not by ES `system`. For Batocera the two are the same string, so an installed box keeps working without a migration — `record_scope()` falls back to `system` for records written before this existed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,112 @@
|
||||
# warpstore — the shared core of the WarpEngine store engines
|
||||
|
||||
A **store engine** reads a [WarpEngine](https://git.teletypegames.org/engines/warp_engine)
|
||||
catalog and writes some host platform's own game library format. Only that last
|
||||
step differs between hosts. Everything before it — talking to the API, picking
|
||||
which release to install, matching the machine, remembering what was put where —
|
||||
is this module.
|
||||
|
||||
```
|
||||
┌──────────────────────────────┐
|
||||
│ warpstore.py │
|
||||
│ catalog · releases · host │
|
||||
│ state · config · HTTP │
|
||||
└──────────────┬───────────────┘
|
||||
┌──────────────┴───────────────┐
|
||||
▼ ▼
|
||||
warp-engine-batocera-store warp-engine-retroarch-store
|
||||
ROM folders + gamelist.xml .lpl playlists + thumbnails
|
||||
│ │
|
||||
▼ ▼
|
||||
EmulationStation RetroArch
|
||||
```
|
||||
|
||||
Single file, Python 3 standard library only. The hosts range from a Batocera box
|
||||
(`python3`, no `pip`) to a desktop, and neither may be asked to install anything.
|
||||
An engine's installer drops `warpstore.py` next to the adapter script and the
|
||||
adapter does `import warpstore`.
|
||||
|
||||
## What an adapter has to supply
|
||||
|
||||
Three things, and nothing else:
|
||||
|
||||
1. **A `DEFAULT_CONFIG`** describing its host — where things live, which catalog
|
||||
platform maps to what.
|
||||
2. **An `accept(spec, sw)` callback** deciding which catalog entries that host can
|
||||
run, and how they are grouped.
|
||||
3. **The code that writes the host's library format.**
|
||||
|
||||
```python
|
||||
import warpstore as ws
|
||||
|
||||
def accept(spec, sw):
|
||||
"""Return (scope, extras), or raise ws.Skip(reason) to reject the title."""
|
||||
if spec["system"] not in available_systems:
|
||||
raise ws.Skip(f"no '{spec['system']}' ROM folder on this box")
|
||||
return spec["system"], {"system": spec["system"]}
|
||||
|
||||
cfg = ws.load_config(path, DEFAULT_CONFIG, required=("paths.subfolder",))
|
||||
ws.init(path)
|
||||
games, skipped = ws.select_games(cfg, ws.fetch_catalog(cfg, use_cache=True), accept)
|
||||
```
|
||||
|
||||
`scope` is how the host groups its library — a Batocera system, a RetroArch
|
||||
playlist. It is what `state.json` is keyed by (`<scope>:<name>`), so two hosts
|
||||
can carry a game of the same name without colliding.
|
||||
|
||||
## What is in here
|
||||
|
||||
| Area | Functions |
|
||||
|---|---|
|
||||
| Logging | `set_tag`, `set_verbose`, `log`, `debug`, `die` |
|
||||
| Files | `load_json`, `write_json`, `write_atomic`, `within` |
|
||||
| Config | `deep_merge`, `load_config` |
|
||||
| Store home | `init`, then `HOME`, `CONFIG_PATH`, `STATE_PATH`, `CATALOG_CACHE` |
|
||||
| HTTP | `http_get`, `http_download`, `api_url`, `download_url`, `user_agent` |
|
||||
| Host | `machine`, `host_os`, `host`, `resolve_for_host` |
|
||||
| Catalog | `fetch_catalog`, `select_games`, `pick_release`, `asset_basename`, `download_image` |
|
||||
| State | `load_state`, `save_state`, `game_key`, `record_scope`, `records_by_scope`, `match_keys`, `limit_to_names` |
|
||||
|
||||
Three of them carry decisions worth knowing about.
|
||||
|
||||
**`resolve_for_host(value, host)`** — a config value that may depend on the
|
||||
machine. A plain string is the same everywhere (that is what a `cartridge` is:
|
||||
data for an emulator). Where it differs, the value is a map and the most specific
|
||||
key wins: `{"linux-aarch64": …, "aarch64": …, "linux": …, "*": …}`. With no
|
||||
matching key and no `*` the answer is `None`, and the caller reports the title as
|
||||
skipped rather than installing something that cannot run.
|
||||
|
||||
**`within(path, root)`** — every delete a store performs is guarded by this. A
|
||||
store may only remove files from the subtree it owns, never from the user's own
|
||||
library and never from another store's.
|
||||
|
||||
**`write_atomic`** — nothing is ever written in place. A store interrupted
|
||||
mid-sync would otherwise leave a half-written playlist or gamelist, which is
|
||||
worse than an old one.
|
||||
|
||||
## State
|
||||
|
||||
`state.json`, version 2, keyed `<scope>:<name>`:
|
||||
|
||||
```json
|
||||
{
|
||||
"version": 2,
|
||||
"installed": {
|
||||
"c64:blessingofra": { "name": "blessingofra", "scope": "c64", "asset": "blessingofra-2.0.0.prg", "…": "…" }
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`record_scope()` falls back to a record's `system` field, which is what the
|
||||
Batocera store wrote before this module existed — and there the two were the same
|
||||
string, so an installed box keeps working with no migration. A `version: 1` file,
|
||||
keyed by bare software name, is re-keyed on first run.
|
||||
|
||||
## Users
|
||||
|
||||
- [`warp-engine-batocera-store`](https://git.teletypegames.org/tools/warp-engine-batocera-store) — EmulationStation ROM folders, `gamelist.xml`, Ports launchers
|
||||
- [`warp-engine-retroarch-store`](https://git.teletypegames.org/tools/warp-engine-retroarch-store) — RetroArch `.lpl` playlists and thumbnail folders
|
||||
|
||||
Both pin nothing: they fetch `warpstore.py` from this repository's `master` at
|
||||
install time. A change here therefore reaches every store on its next install —
|
||||
which is the point, and also the reason to keep the surface above small.
|
||||
Reference in New Issue
Block a user