stores/warp-engine-batocera-store
4
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Install native games as Ports, not ROMs

Browse Source 
The store could only offer what an emulator boots, which meant cartridges
and nothing else — a little under half the catalog. Batocera runs Linux
and its Ports system takes .sh launchers, so a native build can be
installed too; it just is not a ROM.

`install: "port"` unpacks the zip, finds the executable and writes a
launcher that cd's into its directory first, because a game loads assets
relative to the working directory. Python's extraction drops the
executable bit, so it is restored from the mode the archive records.

The payload lives under ports/.data/<subfolder>/, whose leading dot is
what hides it from EmulationStation, while the launchers sit in a normal
subfolder that gets scanned like any other system's. Both carry the store
subfolder, so an uninstall or prune cannot reach another store's files —
verified by pointing a state entry at a foreign namespace and watching it
be refused.

Verified end to end on aarch64 against the live catalog: bevydemo
installs as a port with an executable launcher and binary, re-syncs
without re-downloading 29 MB, and uninstalls taking its 106 MB payload
with it. `install` defaults to rom, so existing configs are unaffected.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
mr.zero
2026-08-17 08:12:42 +02:00
co-authored by Claude Opus 5
parent b6d71aea1c
commit 4be089f9c3
3 changed files with 195 additions and 20 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+40 -1
View File
@@ -149,7 +149,7 @@ counts, favourites and scraped media, and two stores can share one gamelist.
| `catalog.statuses` | `["released", "archived"]` | catalog `status` values to install |
| `catalog.owner_id` | `null` | restrict to one publisher (`/api/software?owner_id=`) |
| `catalog.only` / `catalog.exclude` | `[]` | software-name allow / deny lists |
| `platforms` | c64, tic80 | platform → system, asset kind, extension, enabled; the kind may be per-architecture |
| `platforms` | c64, tic80 | platform → system, asset kind, extension, enabled; `install` picks ROM or Port, and the kind may be per-architecture |
| `behavior.prune` | `true` | remove games that left the catalog or the filters |
| `behavior.timeout` | `30` | HTTP timeout, seconds |
| `behavior.insecure` | `false` | skip TLS verification (self-hosted test instances) |
@@ -172,6 +172,45 @@ launches, so they are not mapped by default. Adding one is a config edit:
"platforms": { "godot": { "system": "godot", "kind": "linux_x64", "ext": ".zip", "enabled": true } }
```
### Native games as Ports
A cartridge is booted by an emulator, so it goes into that system's ROM folder.
A native build is a program: Batocera runs Linux, and its **Ports** system takes
`.sh` launchers. Set `install` to `port` and point `system` at `ports`:
```json
"platforms": {
"bevy": {
"system": "ports",
"install": "port",
"kind": { "x86_64": "linux_x64", "aarch64": "linux_arm64" },
"ext": ".zip",
"enabled": true
}
}
```
The zip is unpacked and a launcher written:
```
/userdata/roms/ports/<subfolder>/<name>.sh the launcher ES lists
/userdata/roms/ports/<subfolder>/images/<name>.png box art
/userdata/roms/ports/.data/<subfolder>/<name>/ the unpacked program
```
The payload sits under `.data`, whose leading dot is what hides it from
EmulationStation — the launchers themselves are in a normal subfolder, scanned
like any other system's. Both paths carry the store's subfolder, so an uninstall
or a prune can only ever reach this store's own files.
The launcher `cd`s into the directory holding the executable before running it:
a game loads its `assets/` relative to the working directory. Python's zip
extraction drops the executable bit, so it is restored from the archive's
recorded mode.
`install` defaults to `rom`, so a config that does not mention it behaves exactly
as before.
### Per-architecture assets
Batocera runs on x86_64 mini-PCs, on the Raspberry Pi and on the ARM handhelds
config.example.json
+26 -8
View File
@@ -8,29 +8,47 @@
"download": "/api/download"
}
},
"paths": {
"roms_root": "/userdata/roms",
"subfolder": "example"
},
"emulationstation": {
"folder_name": "Example Store",
"restart": true
},
"catalog": {
"statuses": ["released", "archived"],
"statuses": [
"released",
"archived"
],
"owner_id": null,
"only": [],
"exclude": []
},
"platforms": {
"c64": { "system": "c64", "kind": "cartridge", "ext": ".prg", "enabled": true },
"tic80": { "system": "tic80", "kind": "cartridge", "ext": ".tic", "enabled": true }
"c64": {
"system": "c64",
"kind": "cartridge",
"ext": ".prg",
"enabled": true
},
"tic80": {
"system": "tic80",
"kind": "cartridge",
"ext": ".tic",
"enabled": true
},
"bevy": {
"system": "ports",
"install": "port",
"kind": {
"x86_64": "linux_x64",
"aarch64": "linux_arm64"
},
"ext": ".zip",
"enabled": false
}
},
"behavior": {
"prune": true,
"timeout": 30,
store.py
+129 -11
View File
@@ -17,6 +17,7 @@ import argparse
import json
import os
import platform
import shlex
import shutil
import subprocess
import sys
@@ -26,6 +27,7 @@ import urllib.error
import urllib.parse
import urllib.request
import xml.etree.ElementTree as ET
import zipfile
VERSION = "2.0.0"
STATE_VERSION = 2
@@ -398,6 +400,7 @@ def select_games(cfg, catalog, available_systems, arch=None):
"title": sw.get("title") or name,
"desc": (sw.get("desc") or "").strip(),
"author": (sw.get("author") or "").strip(),
"install": spec.get("install", "rom"),
"image_url": sw.get("imageUrl"),
"releasedate": es_date(release.get("createdAt")),
}
@@ -437,23 +440,123 @@ def rel_image_path(cfg, game, ext):
return f"{subfolder(cfg)}/images/{game['name']}{ext}"
def rel_port_launcher(cfg, game):
return f"{subfolder(cfg)}/{game['name']}.sh"
def rel_port_data(cfg, game):
# A `.data` pont-prefixe rejti el az ES elol; a store almappaja alatta
# tartja kulon az egyes store-ok payloadjat.
return f".data/{subfolder(cfg)}/{game['name']}"
def find_executable(root, name):
"""A kicsomagolt fa fo binarisa.
Az unix zip megorzi a futtathato bitet, tehat elsore azt keressuk. Ha
tobb jelolt van, a szoftver nevevel egyezo nyer; ha egy sem futtathato,
a nevegyezes az utolso esely.
"""
execs, by_name = [], []
for base, _, files in os.walk(root):
for f in files:
full = os.path.join(base, f)
if os.access(full, os.X_OK) and not os.path.isdir(full):
execs.append(full)
if f == name:
by_name.append(full)
for candidate in (execs, by_name):
if len(candidate) == 1:
return candidate[0]
exact = [c for c in candidate if os.path.basename(c) == name]
if len(exact) == 1:
return exact[0]
return None
def install_port(cfg, game, dry_run=False):
"""A zip asset kicsomagolasa + Ports indito. Visszaad (launcher, data) relativ utakat."""
base = system_dir(cfg, game["system"])
launcher_rel = rel_port_launcher(cfg, game)
data_rel = rel_port_data(cfg, game)
launcher_abs = os.path.join(base, launcher_rel)
data_abs = os.path.join(base, data_rel)
if dry_run:
log(f"would install {game['name']} {game['version']} as a port -> {launcher_rel}")
return launcher_rel, data_rel
url = api_url(cfg, "download", {"path": game["asset"]})
os.makedirs(os.path.dirname(launcher_abs), exist_ok=True)
tmp_zip = os.path.join(os.path.dirname(launcher_abs), f".{TAG}-{game['name']}.zip")
size = http_download(cfg, url, tmp_zip)
try:
shutil.rmtree(data_abs, ignore_errors=True)
os.makedirs(data_abs, exist_ok=True)
with zipfile.ZipFile(tmp_zip) as zf:
zf.extractall(data_abs)
# A zip futtathato bitjeit a Python nem allitja vissza.
for info in zf.infolist():
mode = info.external_attr >> 16
if mode & 0o111:
target = os.path.join(data_abs, info.filename)
if os.path.isfile(target):
os.chmod(target, os.stat(target).st_mode | 0o111)
finally:
if os.path.exists(tmp_zip):
os.unlink(tmp_zip)
exe = find_executable(data_abs, game["name"])
if not exe:
shutil.rmtree(data_abs, ignore_errors=True)
raise IOError(f"no executable found in {game['asset']}")
# A jatek az assetjeit a munkakonyvtarhoz kepest tolti be, ezert a
# binaris konyvtarabol kell inditani.
exe_dir = os.path.dirname(os.path.relpath(exe, data_abs))
run_dir = os.path.join(data_abs, exe_dir) if exe_dir else data_abs
write_atomic(launcher_abs, (
"#!/bin/bash\n"
f"# {game['title']} — {cfg['store']['name']}\n"
f"cd {shlex.quote(run_dir)} || exit 1\n"
f"exec ./{os.path.basename(exe)}\n"
).encode("utf-8"))
os.chmod(launcher_abs, 0o755)
log(f"installed {game['name']} {game['version']} ({size} bytes) as a port -> {launcher_rel}")
return launcher_rel, data_rel
def install_game(cfg, game, state_entry, dry_run=False):
"""Download the asset + box art if missing. Returns (record, changed)."""
base = system_dir(cfg, game["system"])
rom_rel = rel_rom_path(cfg, game)
rom_abs = os.path.join(base, rom_rel)
changed = False
data_rel = None
if os.path.isfile(rom_abs) and os.path.getsize(rom_abs) > 0:
debug(f"{game['name']}: {game['asset']} already present")
else:
url = api_url(cfg, "download", {"path": game["asset"]})
if dry_run:
log(f"would download {game['name']} {game['version']} -> {rom_abs}")
if game.get("install") == "port":
# A port ket dologbol all: egy indito .sh es a kicsomagolt payload.
rom_rel = rel_port_launcher(cfg, game)
data_rel = rel_port_data(cfg, game)
installed = (os.path.isfile(os.path.join(base, rom_rel))
and os.path.isdir(os.path.join(base, data_rel))
and state_entry and state_entry.get("asset") == game["asset"])
if installed:
debug(f"{game['name']}: port already installed")
else:
size = http_download(cfg, url, rom_abs)
log(f"installed {game['name']} {game['version']} ({size} bytes) -> {rom_rel}")
changed = True
rom_rel, data_rel = install_port(cfg, game, dry_run=dry_run)
changed = True
else:
rom_rel = rel_rom_path(cfg, game)
rom_abs = os.path.join(base, rom_rel)
if os.path.isfile(rom_abs) and os.path.getsize(rom_abs) > 0:
debug(f"{game['name']}: {game['asset']} already present")
else:
url = api_url(cfg, "download", {"path": game["asset"]})
if dry_run:
log(f"would download {game['name']} {game['version']} -> {rom_abs}")
else:
size = http_download(cfg, url, rom_abs)
log(f"installed {game['name']} {game['version']} ({size} bytes) -> {rom_rel}")
changed = True
image_rel = state_entry.get("image") if state_entry else None
if game.get("image_url"):
@@ -477,6 +580,8 @@ def install_game(cfg, game, state_entry, dry_run=False):
record = dict(game)
record["rom"] = rom_rel
record["image"] = image_rel
if data_rel:
record["data"] = data_rel
return record, changed
@@ -518,6 +623,19 @@ def remove_game(cfg, record, dry_run=False):
os.unlink(path)
debug(f"removed {path}")
# A port payloadja egy konyvtar a rejtett .data alatt.
data_rel = record.get("data")
if data_rel:
data_abs = os.path.join(base, data_rel)
if not data_rel.startswith(f".data/{subfolder(cfg)}/"):
log(f"warning: refusing to delete {data_abs} (outside .data/{subfolder(cfg)}/)")
elif os.path.isdir(data_abs):
if dry_run:
log(f"would remove {data_abs}")
else:
shutil.rmtree(data_abs, ignore_errors=True)
debug(f"removed {data_abs}")
# --------------------------------------------------------------------------
# gamelist.xml