warpstore — the shared core of the WarpEngine store engines

A store engine reads a WarpEngine catalog and writes some host platform's own game library format. Only that last step differs between hosts. Everything before it — talking to the API, picking which release to install, matching the machine, remembering what was put where — is this module.

                    ┌──────────────────────────────┐
                    │  warpstore.py                │
                    │  catalog · releases · host   │
                    │  state · config · HTTP       │
                    └──────────────┬───────────────┘
                    ┌──────────────┴───────────────┐
                    ▼                              ▼
    warp-engine-batocera-store        warp-engine-retroarch-store
    ROM folders + gamelist.xml        .lpl playlists + thumbnails
                    │                              │
                    ▼                              ▼
             EmulationStation                  RetroArch

Single file, Python 3 standard library only. The hosts range from a Batocera box (python3, no pip) to a desktop, and neither may be asked to install anything. An engine's installer drops warpstore.py next to the adapter script and the adapter does import warpstore.

What an adapter has to supply

Three things, and nothing else:

1. A DEFAULT_CONFIG describing its host — where things live, which catalog platform maps to what.
2. An accept(spec, sw) callback deciding which catalog entries that host can run, and how they are grouped.
3. The code that writes the host's library format.

import warpstore as ws

def accept(spec, sw):
    """Return (scope, extras), or raise ws.Skip(reason) to reject the title."""
    if spec["system"] not in available_systems:
        raise ws.Skip(f"no '{spec['system']}' ROM folder on this box")
    return spec["system"], {"system": spec["system"]}

cfg = ws.load_config(path, DEFAULT_CONFIG, required=("paths.subfolder",))
ws.init(path)
games, skipped = ws.select_games(cfg, ws.fetch_catalog(cfg, use_cache=True), accept)

scope is how the host groups its library — a Batocera system, a RetroArch playlist. It is what state.json is keyed by (<scope>:<name>), so two hosts can carry a game of the same name without colliding.

What is in here

Area	Functions
Logging	set_tag, set_verbose, log, debug, die
Files	load_json, write_json, write_atomic, within
Config	deep_merge, load_config
Store home	init, then HOME, CONFIG_PATH, STATE_PATH, CATALOG_CACHE
HTTP	http_get, http_download, api_url, download_url, user_agent
Host	machine, host_os, host, resolve_for_host
Catalog	fetch_catalog, select_games, pick_release, asset_basename, download_image
State	load_state, save_state, game_key, record_scope, records_by_scope, match_keys, limit_to_names

Three of them carry decisions worth knowing about.

resolve_for_host(value, host) — a config value that may depend on the machine. A plain string is the same everywhere (that is what a cartridge is: data for an emulator). Where it differs, the value is a map and the most specific key wins: {"linux-aarch64": …, "aarch64": …, "linux": …, "*": …}. With no matching key and no * the answer is None, and the caller reports the title as skipped rather than installing something that cannot run.

within(path, root) — every delete a store performs is guarded by this. A store may only remove files from the subtree it owns, never from the user's own library and never from another store's.

write_atomic — nothing is ever written in place. A store interrupted mid-sync would otherwise leave a half-written playlist or gamelist, which is worse than an old one.

State

state.json, version 2, keyed <scope>:<name>:

{
  "version": 2,
  "installed": {
    "c64:blessingofra": { "name": "blessingofra", "scope": "c64", "asset": "blessingofra-2.0.0.prg", "…": "…" }
  }
}

record_scope() falls back to a record's system field, which is what the Batocera store wrote before this module existed — and there the two were the same string, so an installed box keeps working with no migration. A version: 1 file, keyed by bare software name, is re-keyed on first run.

Users

• warp-engine-batocera-store — EmulationStation ROM folders, gamelist.xml, Ports launchers
• warp-engine-retroarch-store — RetroArch .lpl playlists and thumbnail folders

Both pin nothing: they fetch warpstore.py from this repository's master at install time. A change here therefore reaches every store on its next install — which is the point, and also the reason to keep the surface above small.