A checklist of the release assets that exist and the ones the CI could still build

Nothing said whether a release was finished. A pipeline step can fail, a
platform can gain a target, a title can be published from a laptop — and the
catalog happily serves a release with three of its seven assets. The only way
to find out was to open `/api/softwares/<name>/builds` one name at a time.

    $ bin/rails warp_engine:builds:check

    rabbitroller  ebitengine  v1.1.1  1/7
      [x] html
      [ ] win_x86
      [ ] win_x64
      [ ] linux_x64
      [ ] linux_arm64
      [-] mac_x64
      [-] mac_arm64

    The CI can build these and the release does not have them:
      rabbitroller v1.1.1 (ebitengine): win_x86, win_x64, linux_x64, linux_arm64

    5 softwares, 5 releases, 14/32 assets present, 16 missing from CI-built kinds

The third marker is the point. Two different questions were being conflated:
what the **updater** can ingest (`Platform#expected_kinds`) and what the
**pipeline** actually produces. For six of the seven platforms they are the
same list; for ebitengine they are not — the updater accepts macOS builds, the
Woodpecker builder cannot cross-compile them (osxcross), so `mac_x64` and
`mac_arm64` would be reported as gaps on every ebitengine release, for ever.
`[ ]` is now a real gap and `[-]` is "upload it by hand or not at all", so the
list stays worth reading.

That knowledge is data, next to the templates that produce it:
`PipelineConfig::BUILT_KINDS`, reachable through the adapter as
`built_kinds(platform)`. An adapter that cannot answer — no CI configured, a
platform with no builder — returns nil, and then nothing is excused: every
missing kind counts as a gap.

`WarpEngine::AssetCoverage` computes it (`#rows`, `#missing_rows`, `#totals`)
and `AssetCoverageChecklist` renders it, so the numbers are testable without a
terminal. Options are env vars: NAME, PLATFORM, RELEASES=all, ONLY=missing, and
STRICT=1 for a non-zero exit in CI. The default is the newest release per
software, using the same non-`dev-` rule the catalog API uses for "latest".

33 new examples, including two that keep BUILT_KINDS honest: every claimed kind
has to be a kind the updater knows, and has to appear in the platform's rendered
pipeline. Adding a platform without teaching this list about it fails the suite.

Ships in 0.8.0.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-08-23 09:34:23 +02:00
co-authored by Claude Opus 5
parent e6ef4f4191
commit a3e688a432
10 changed files with 557 additions and 0 deletions
+66
View File
@@ -17,6 +17,9 @@ Repository: `https://git.teletypegames.org/engines/warp_engine`
value object that knows its `label`, its `expected_kinds` and its updater
`service`. Reference it instead of writing a platform list of your own —
`PlatformLink::SUPPORTED_PLATFORMS` is kept as an alias of `Platform::NAMES`.
- **Asset checklist**: `rake warp_engine:builds:check` prints, per software and
release, which release assets exist and which the platform's pipeline could
still deliver. See *Which assets are missing*.
- **CI-callable updater**: your build pipeline uploads artifacts over HTTP
and calls one endpoint — WarpEngine extracts archives, parses metadata
and upserts the catalog records. Supported platforms out of the
@@ -310,6 +313,69 @@ accepts both:
When switching to `:database`, create the tokens and move your pipelines to
them first — the flip invalidates the shared secret immediately.
## Which assets are missing (rake)
A release is complete when it carries every asset kind its platform can
produce. Nothing enforces that — a pipeline step can fail, a platform can gain
a target, a title can be published from a laptop — so the engine ships a
checklist:
```
$ bin/rails warp_engine:builds:check
WarpEngine release asset coverage — latest release per software
expected: the kinds a platform's updater registers · [ ] the CI builds it · [-] the CI does not, upload it by hand
rabbitroller ebitengine v1.1.1 3/7
[x] html
[x] win_x64
[x] linux_x64
[ ] win_x86
[ ] linux_arm64
[-] mac_x64
[-] mac_arm64
The CI can build these and the release does not have them:
rabbitroller v1.1.1 (ebitengine): win_x86, linux_arm64
12 softwares, 12 releases, 61/78 assets present, 11 missing from CI-built kinds
4 softwares are missing an asset the CI builds
2 assets are expected but this CI does not build them
```
Three markers, three different facts:
| Marker | Meaning |
| --- | --- |
| `[x]` | the release has this asset |
| `[ ]` | the platform expects it **and** the CI pipeline builds it — a real gap |
| `[-]` | the platform expects it but this CI cannot build it (ebitengine's macOS targets need osxcross) — a manual upload, or nothing |
| `[+]` | the release has an asset the platform does not expect |
The expectations come from two places, and the difference is the point:
`WarpEngine::Platform#expected_kinds` is what the **updater** can ingest, and
the CI adapter's `built_kinds(platform)` is what the **pipeline** produces. An
adapter that cannot answer (no CI configured, or a platform with no builder)
returns nil, and then nothing is excused: every missing kind is reported as a
gap.
Options are environment variables, so the task composes in a shell:
| Variable | Effect |
| --- | --- |
| `NAME=rabbitroller` | one software |
| `PLATFORM=ebitengine` | one platform |
| `RELEASES=all` | every release, not just the newest one per software |
| `ONLY=missing` | only the releases that are short of a CI-built asset |
| `STRICT=1` | exit non-zero when anything is missing (for CI) |
The newest release is the newest non-`dev-` version, the same rule the catalog
API uses when it names a title's latest release.
The same numbers are available as objects: `WarpEngine::AssetCoverage.new(...)`
answers `#rows`, `#missing_rows` and `#totals`, and
`WarpEngine::AssetCoverageChecklist` is what renders them.
## CI
Every CI feature — serving pipeline configs, syncing repositories, triggering
@@ -0,0 +1,83 @@
module WarpEngine
class AssetCoverageChecklist
MARKERS = {
present: "[x]",
missing: "[ ]",
manual: "[-]",
unexpected: "[+]"
}.freeze
def initialize(coverage, only_missing: false)
@coverage = coverage
@only_missing = only_missing
end
attr_reader :coverage, :only_missing
def lines
[ *header, *body, *footer ]
end
def to_s = lines.join("\n")
private
def header
[
"WarpEngine release asset coverage — #{coverage.releases == :all ? 'every release' : 'latest release per software'}",
"expected: the kinds a platform's updater registers · #{MARKERS[:missing]} the CI builds it · " \
"#{MARKERS[:manual]} the CI does not, upload it by hand",
""
]
end
def body
shown = only_missing ? coverage.missing_rows : coverage.rows
return [ "Nothing to report.", "" ] if shown.empty?
shown.flat_map { |row| [ title(row), *checklist(row), "" ] }
end
def title(row)
parts = [ row.software.name, row.platform&.name || "unknown platform" ]
parts << (row.released? ? "v#{row.version}" : "no release")
parts << ratio(row) if row.released? && row.expected.any?
parts.join(" ")
end
def ratio(row) = "#{(row.expected & row.present).size}/#{row.expected.size}"
def checklist(row)
return [ " (this platform is not in the registry, so nothing is expected)" ] if row.unknown_platform?
return [ " (no release to check)" ] unless row.released?
return [ " (this platform expects no assets)" ] if row.expected.empty? && row.present.empty?
row.expected.map { |kind| " #{MARKERS[row.state_of(kind)]} #{kind}" } +
row.unexpected.map { |kind| " #{MARKERS[:unexpected]} #{kind} (not expected on #{row.platform&.name})" }
end
def footer
totals = coverage.totals
lines = []
if coverage.missing_rows.any? && !only_missing
lines << "The CI can build these and the release does not have them:"
lines += coverage.missing_rows.map do |row|
" #{row.software.name} v#{row.version} (#{row.platform&.name}): #{row.buildable_missing.join(', ')}"
end
lines << ""
end
lines << format(
"%d softwares, %d releases, %d/%d assets present, %d missing from CI-built kinds",
totals[:softwares], totals[:releases], totals[:present], totals[:expected], totals[:missing]
)
lines << "#{totals[:incomplete_softwares]} softwares are missing an asset the CI builds" if totals[:missing].positive?
lines << "#{totals[:manual_missing]} assets are expected but this CI does not build them" if totals[:manual_missing].positive?
lines << "#{totals[:without_release]} softwares have no release yet" if totals[:without_release].positive?
lines << "#{totals[:unknown_platform]} softwares are on a platform the registry does not know" if totals[:unknown_platform].positive?
lines << "#{totals[:unexpected]} assets are not expected on their platform" if totals[:unexpected].positive?
lines
end
end
end
@@ -0,0 +1,121 @@
module WarpEngine
class AssetCoverage
Row = Struct.new(:software, :release, :platform, :expected, :present, :ci_kinds, keyword_init: true) do
def missing = expected - present
def buildable_missing
return missing if ci_kinds.nil?
missing & ci_kinds
end
def manual_missing = missing - buildable_missing
def unexpected = present - expected
def complete? = release.present? && missing.empty?
def actionable? = release.present? && buildable_missing.any?
def unknown_platform? = platform.nil?
def released? = release.present?
def version = release&.version
def state_of(kind)
return :present if present.include?(kind)
return :manual if ci_kinds && !ci_kinds.include?(kind)
:missing
end
end
def initialize(name: nil, platform: nil, releases: :latest)
@name = name.presence
@platform = platform.presence
@releases = releases == :all ? :all : :latest
end
attr_reader :name, :platform, :releases
def rows
@rows ||= scope.flat_map { |software| rows_for(software) }
end
def missing_rows = rows.select(&:actionable?)
def incomplete_rows = rows.select { |row| row.released? && !row.complete? }
def totals
counted = rows.select(&:released?)
{
softwares: rows.map { |row| row.software.id }.uniq.size,
releases: counted.size,
expected: counted.sum { |row| row.expected.size },
present: counted.sum { |row| (row.expected & row.present).size },
missing: counted.sum { |row| row.buildable_missing.size },
manual_missing: counted.sum { |row| row.manual_missing.size },
unexpected: counted.sum { |row| row.unexpected.size },
incomplete_softwares: missing_rows.map { |row| row.software.id }.uniq.size,
without_release: rows.count { |row| !row.released? },
unknown_platform: rows.count(&:unknown_platform?)
}
end
private
def scope
relation = WarpEngine::Software.order(:name).includes(releases: :release_assets)
relation = relation.where(name: name) if name
relation = relation.where(platform: platform) if platform
relation
end
def rows_for(software)
expected = expected_kinds(software)
selected = select_releases(software.releases.to_a)
return [ row(software, nil, expected) ] if selected.empty?
selected.map { |release| row(software, release, expected) }
end
def row(software, release, expected)
Row.new(
software: software,
release: release,
platform: WarpEngine::Platform.find(software.platform),
expected: expected,
present: release ? release.release_assets.map(&:kind).uniq : [],
ci_kinds: ci_kinds(software.platform)
)
end
def ci_kinds(platform)
@ci_kinds ||= {}
return @ci_kinds[platform] if @ci_kinds.key?(platform)
ci = WarpEngine.ci
@ci_kinds[platform] = ci.respond_to?(:built_kinds) ? ci.built_kinds(platform) : nil
end
def expected_kinds(software)
WarpEngine::Platform.find(software.platform)&.expected_kinds || []
end
def select_releases(all)
return sorted(all) if releases == :all
[ latest(all) ].compact
end
def sorted(all) = all.sort_by { |release| [ release.created_at || Time.at(0), release.id ] }.reverse
def latest(all)
candidates = all.reject { |release| release.version.to_s.start_with?("dev-") }
candidates = all if candidates.empty?
sorted(candidates).first
end
end
end
@@ -0,0 +1,22 @@
namespace :warp_engine do
namespace :builds do
desc "Checklist of which release assets exist and which the platform expects " \
"(NAME=slug PLATFORM=godot RELEASES=all ONLY=missing STRICT=1)"
task check: :environment do
coverage = WarpEngine::AssetCoverage.new(
name: ENV["NAME"],
platform: ENV["PLATFORM"],
releases: ENV["RELEASES"] == "all" ? :all : :latest
)
checklist = WarpEngine::AssetCoverageChecklist.new(
coverage, only_missing: ENV["ONLY"] == "missing"
)
puts checklist.to_s
if ENV["STRICT"] == "1" && coverage.totals[:missing].positive?
abort "\nSTRICT: #{coverage.totals[:missing]} expected assets are missing."
end
end
end
end
@@ -43,6 +43,7 @@ module WarpEngine
def name = "none"
def configured? = false
def platforms = {}
def built_kinds(_platform) = nil
def update_server = nil
def repos = raise(NotConfigured, MESSAGE)
@@ -28,6 +28,8 @@ module WarpEngine
def platforms = @config.platforms
def built_kinds(platform) = @config.built_kinds(platform)
def client
@client ||= Client.new(url: @url, token: @api_token)
end
@@ -7,6 +7,16 @@ module WarpEngine
class PipelineConfig
PLATFORM_FORMAT = /\A[a-z0-9_-]+\z/
BUILT_KINDS = {
"tic80" => %w[cartridge source html docs win_x64 linux_x64 mac_x64],
"ebitengine" => %w[html win_x86 win_x64 linux_x64 linux_arm64],
"godot" => %w[html win_x86 win_x64 linux_x64 mac_universal],
"love" => %w[html win_x64 linux_x64 mac_universal],
"bevy" => %w[html win_x64 linux_x64 linux_arm64],
"c64" => %w[cartridge],
"phaser" => %w[html]
}.freeze
def initialize(platforms: {})
@platforms = platforms.to_h { |key, spec| [ key.to_s, spec.to_h.symbolize_keys ] }
end
@@ -33,6 +43,13 @@ module WarpEngine
)
end
def built_kinds(platform)
platform = platform.to_s
return nil unless @platforms.key?(platform)
BUILT_KINDS[platform]
end
def self.templates_dir
Pathname.new(__dir__).join("platforms")
end
@@ -0,0 +1,54 @@
require "rails_helper"
RSpec.describe WarpEngine::CI::Woodpecker::PipelineConfig do
BUILT_KINDS = described_class::BUILT_KINDS
def self.fragment_for(kind)
case kind
when "html" then ".html.zip"
when "docs" then "-docs.zip"
when "cartridge", "source" then nil
else kind.tr("_", "-")
end
end
def config_for(platform)
described_class.new(platforms: { platform => { builder: "builder:latest", exporter: "exporter:latest" } })
end
it "knows every platform in the registry" do
expect(BUILT_KINDS.keys).to match_array(WarpEngine::Platform.names)
end
WarpEngine::Platform::NAMES.each do |platform|
context platform do
let(:kinds) { BUILT_KINDS.fetch(platform) }
let(:rendered) do
config_for(platform).render(platform: platform, name: "example", update_server: "https://example.test")
end
it "claims only kinds the updater knows how to ingest" do
expect(kinds - WarpEngine::Platform.find(platform).expected_kinds).to be_empty
end
it "claims only kinds the pipeline actually packages" do
kinds.each do |kind|
fragment = self.class.fragment_for(kind)
next if fragment.nil?
expect(rendered).to include(fragment), "#{platform}: nothing in the pipeline produces #{kind}"
end
end
end
end
describe "#built_kinds" do
it "answers with the kinds of a platform it has a builder for" do
expect(config_for("tic80").built_kinds("tic80")).to include("cartridge", "html", "win_x64")
end
it "answers nil for a platform with no builder configured, so nothing is excused" do
expect(config_for("tic80").built_kinds("godot")).to be_nil
end
end
end
@@ -0,0 +1,52 @@
require "rails_helper"
RSpec.describe WarpEngine::AssetCoverageChecklist do
let(:software) { create(:software, name: "rabbitroller", platform: "ebitengine") }
before do
release = create(:release, software: software, version: "1.1.1")
WarpEngine::ReleaseAsset.create!(release: release, kind: "html", path: "/s/rabbitroller-1.1.1")
ci = instance_double(WarpEngine::CI::Woodpecker::Adapter)
allow(ci).to receive(:built_kinds) { |platform| WarpEngine::CI::Woodpecker::PipelineConfig::BUILT_KINDS[platform] }
allow(WarpEngine).to receive(:ci).and_return(ci)
end
def output(**options) = described_class.new(WarpEngine::AssetCoverage.new(**options)).to_s
it "checks off what is there and leaves the rest open" do
lines = output.lines.map(&:chomp)
expect(lines).to include("rabbitroller ebitengine v1.1.1 1/7")
expect(lines).to include(" [x] html")
expect(lines).to include(" [ ] linux_arm64")
expect(lines).to include(" [-] mac_arm64")
end
it "lists what the CI could build and the release does not have" do
expect(output).to include("The CI can build these and the release does not have them:")
expect(output).to include("rabbitroller v1.1.1 (ebitengine): win_x86, win_x64, linux_x64, linux_arm64")
end
it "counts the assets and says how many the CI does not build" do
expect(output).to include("1 softwares, 1 releases, 1/7 assets present, 4 missing from CI-built kinds")
expect(output).to include("2 assets are expected but this CI does not build them")
end
it "shows only the incomplete releases when asked" do
complete = create(:software, name: "pong", platform: "c64")
release = create(:release, software: complete, version: "0.1")
WarpEngine::ReleaseAsset.create!(release: release, kind: "cartridge", path: "/s/pong-0.1.prg")
text = described_class.new(WarpEngine::AssetCoverage.new, only_missing: true).to_s
expect(text).to include("rabbitroller")
expect(text).not_to include("pong")
end
it "says so when there is nothing to report" do
text = described_class.new(WarpEngine::AssetCoverage.new(name: "nothing-here")).to_s
expect(text).to include("Nothing to report.")
end
end
@@ -0,0 +1,139 @@
require "rails_helper"
RSpec.describe WarpEngine::AssetCoverage do
def release_with(software:, version:, kinds:, created_at: Time.current)
release = create(:release, software: software, version: version, created_at: created_at)
kinds.each { |kind| WarpEngine::ReleaseAsset.create!(release: release, kind: kind, path: "/s/#{version}-#{kind}") }
release
end
describe "#rows" do
it "says which expected kinds are present and which are missing" do
software = create(:software, name: "impostor", platform: "tic80")
release_with(software: software, version: "1.0", kinds: %w[cartridge source html])
row = described_class.new.rows.first
expect(row.software).to eq(software)
expect(row.expected).to eq(WarpEngine::Platform.find("tic80").expected_kinds)
expect(row.present).to contain_exactly("cartridge", "source", "html")
expect(row.missing).to contain_exactly("docs", "win_x64", "linux_x64", "mac_x64")
expect(row).not_to be_complete
end
it "calls a release with every expected asset complete" do
software = create(:software, name: "pong", platform: "c64")
release_with(software: software, version: "0.1.0", kinds: %w[cartridge])
row = described_class.new.rows.first
expect(row).to be_complete
expect(row.missing).to be_empty
expect(row).not_to be_actionable
end
it "reports an asset the platform does not expect" do
software = create(:software, name: "phasergame", platform: "phaser")
release_with(software: software, version: "1.0", kinds: %w[html win_x64])
row = described_class.new.rows.first
expect(row.unexpected).to eq([ "win_x64" ])
end
it "reports a software with no release at all" do
create(:software, name: "empty", platform: "godot")
row = described_class.new.rows.first
expect(row).not_to be_released
expect(row).not_to be_actionable
end
it "expects nothing from a platform the registry does not know" do
software = create(:software, name: "amigagame", platform: "tic80")
software.update_column(:platform, "amiga")
release_with(software: software, version: "1.0", kinds: [])
row = described_class.new.rows.first
expect(row).to be_unknown_platform
expect(row.expected).to be_empty
end
it "checks the newest release by default, and every release on demand" do
software = create(:software, name: "rabbitroller", platform: "ebitengine")
release_with(software: software, version: "1.0.0", kinds: %w[html], created_at: 2.days.ago)
release_with(software: software, version: "1.1.1", kinds: %w[html win_x64], created_at: 1.day.ago)
expect(described_class.new.rows.map(&:version)).to eq([ "1.1.1" ])
expect(described_class.new(releases: :all).rows.map(&:version)).to eq([ "1.1.1", "1.0.0" ])
end
it "does not call a dev build the latest release" do
software = create(:software, name: "rabbitroller", platform: "ebitengine")
release_with(software: software, version: "1.1.1", kinds: %w[html], created_at: 2.days.ago)
release_with(software: software, version: "dev-abc123-master", kinds: %w[html], created_at: 1.hour.ago)
expect(described_class.new.rows.map(&:version)).to eq([ "1.1.1" ])
end
it "narrows to one software or one platform" do
create(:software, name: "impostor", platform: "tic80")
create(:software, name: "rabbitroller", platform: "ebitengine")
expect(described_class.new(name: "impostor").rows.map { |r| r.software.name }).to eq([ "impostor" ])
expect(described_class.new(platform: "ebitengine").rows.map { |r| r.software.name }).to eq([ "rabbitroller" ])
end
end
describe "what the CI can build" do
let(:software) { create(:software, name: "rabbitroller", platform: "ebitengine") }
before { release_with(software: software, version: "1.1.1", kinds: %w[html]) }
it "separates the kinds the pipeline builds from the ones it cannot" do
ci = instance_double(WarpEngine::CI::Woodpecker::Adapter)
allow(ci).to receive(:built_kinds).with("ebitengine")
.and_return(%w[html win_x86 win_x64 linux_x64 linux_arm64])
allow(WarpEngine).to receive(:ci).and_return(ci)
row = described_class.new.rows.first
expect(row.buildable_missing).to contain_exactly("win_x86", "win_x64", "linux_x64", "linux_arm64")
expect(row.manual_missing).to contain_exactly("mac_x64", "mac_arm64")
expect(row.state_of("linux_arm64")).to eq(:missing)
expect(row.state_of("mac_x64")).to eq(:manual)
expect(row.state_of("html")).to eq(:present)
end
it "treats every missing kind as buildable when the CI cannot say" do
ci = instance_double(WarpEngine::CI::Woodpecker::Adapter)
allow(ci).to receive(:built_kinds).and_return(nil)
allow(WarpEngine).to receive(:ci).and_return(ci)
row = described_class.new.rows.first
expect(row.buildable_missing).to eq(row.missing)
expect(row.manual_missing).to be_empty
end
end
describe "#totals" do
it "counts releases, assets and the softwares that are short of one" do
tic80 = create(:software, name: "impostor", platform: "tic80")
release_with(software: tic80, version: "1.0", kinds: %w[cartridge source html docs win_x64 linux_x64 mac_x64])
c64 = create(:software, name: "pong", platform: "c64")
release_with(software: c64, version: "0.1", kinds: [])
totals = described_class.new.totals
expect(totals[:softwares]).to eq(2)
expect(totals[:releases]).to eq(2)
expect(totals[:expected]).to eq(8)
expect(totals[:present]).to eq(7)
expect(totals[:missing]).to eq(1)
expect(totals[:incomplete_softwares]).to eq(1)
end
end
end