WarpEngine 0.7.0: a képtár és a CI is a hoszté, adapteren keresztül

Két dolog volt beépítve az engine-be, ami nem az övé.

A **képtár** eddig `WarpEngine::Image` volt, pedig a modell teljesen általános:
a teletypegames-ben a tagok arcképét is ez hordozza, nem csak a katalógus
borítóit. Az `Image` modell, a feltöltött fájlok, az `/api/image/:id` végpont
és az admin oldal ezért átkerült a hosztba, az engine pedig adapteren szól
hozzá (`WarpEngine::Images`): `url_for` adja a katalógus JSON `imageUrl`-jét,
`select_options` a software-form képválasztóját, `build_from_upload` a
"tölts fel új képet" ágat. Az alapértelmezés az `Image` osztály, tehát a
default útvonal bitre a régi. A `SoftwareImage` (a katalógus-kapcsolat)
maradt az engine-ben, és **az `images` tábla nem mozdult**: az engine csak
abbahagyta a létrehozását, a generátor írja meg hoszt-kódként.

A **CI** eddig végig Woodpecker volt: kliens, aláírás-ellenőrzés,
pipeline-receptek, repo-szinkron, secret-kiosztás. Mindez egy adapter mögé
került (`WarpEngine.ci`), a Woodpecker-implementáció pedig az engine-ben
maradt `WarpEngine::CI::Woodpecker` néven — kliens, adapter, httpsig-ellenőrző
és a platformonkénti pipeline-receptek, mert a YAML-dialektus a szolgáltatóé.
Az engine saját kódja már nem nevez szolgáltatót: `CI::Repo` és `CI::Run`
értékeket kap, `CI::ConnectionError`/`ApiError`/`NotConfigured` hibákat dob, a
`Pipeline` pedig `remote_repo_id`-t ad a történelmi `woodpecker_repo_id`
kolumna fölött (a tábla itt sem mozdult). `c.ci_adapter = :none` azt jelenti,
hogy ez a hoszt nem buildel: az `/api/ci/*` 503, a `/build/config` elutasít,
az admin akciók elbújnak.

Mindkét seam a hoszt initializerében van kimondva, nem alapértelmezésre
hagyva — a hoszt megnevezi, mi a képtára és mi a CI-ja.

Törés a 0.6-hoz képest: `image_container_path`, `image_owners`,
`ci_platforms`, `ci_update_server`, `ci_extension_public_key(_url)`,
`woodpecker_url`, `woodpecker_api_token`, `woodpecker_repo_owner` és a
`WarpEngine.woodpecker_configured?` megszűnt; a helyük `c.image_class_name` /
`c.image_adapter` és `c.ci_adapter`. Az `/api/ci/*` `latest_run`/`trigger`
válasza a normalizált `CI::Run` alakot adja (number, status, branch, message,
createdAt, url), a `pipelines` lista pedig `repo_id`-t is közöl a megtartott
`woodpecker_repo_id` mellett.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-08-23 00:56:01 +02:00
co-authored by Claude Opus 5
parent a0fbf1e2b4
commit 19d142ef64
62 changed files with 1391 additions and 793 deletions
+139 -44
View File
@@ -10,7 +10,7 @@ Repository: `https://git.teletypegames.org/engines/warp_engine`
## Features ## Features
- **Catalog domain**: `Software`, `Release`, `ReleaseAsset`, `ExternalLink`, - **Catalog domain**: `Software`, `Release`, `ReleaseAsset`, `ExternalLink`,
`PlatformLink`, `Image`, `SoftwareImage`, `Download` models with soft-delete `PlatformLink`, `SoftwareImage`, `Download` models with soft-delete
semantics and download statistics. semantics and download statistics.
- **CI-callable updater**: your build pipeline uploads artifacts over HTTP - **CI-callable updater**: your build pipeline uploads artifacts over HTTP
and calls one endpoint — WarpEngine extracts archives, parses metadata and calls one endpoint — WarpEngine extracts archives, parses metadata
@@ -24,6 +24,10 @@ Repository: `https://git.teletypegames.org/engines/warp_engine`
is the catalog as it always was. is the catalog as it always was.
- **Client sign-in**: an RFC 8628 device authorization grant for clients with no - **Client sign-in**: an RFC 8628 device authorization grant for clients with no
browser of their own, over the host's own user model. Off unless configured. browser of their own, over the host's own user model. Off unless configured.
- **Host-owned image library**: the catalog links a software to its images and
publishes their URLs, but the image model, its files, its endpoint and its
admin page belong to the host — the same model can serve avatars or anything
else the site has (see *Images*).
- **Pluggable storage**: artifacts are served through a storage adapter - **Pluggable storage**: artifacts are served through a storage adapter
(`:local` by default); a host can serve them from an object store without (`:local` by default); a host can serve them from an object store without
patching the engine. patching the engine.
@@ -31,17 +35,15 @@ Repository: `https://git.teletypegames.org/engines/warp_engine`
`ActiveSupport::Notifications` (`warp_engine.publish`), so hosts can react `ActiveSupport::Notifications` (`warp_engine.publish`), so hosts can react
to new builds without model callbacks. to new builds without model callbacks.
- **Public JSON API**: catalog listing, highlighted title, per-platform build - **Public JSON API**: catalog listing, highlighted title, per-platform build
matrix, image serving, download tracking, and a static file server for matrix, download tracking, and a static file server for web-playable builds.
web-playable builds.
- **Admin (optional)**: if the host runs ActiveAdmin, WarpEngine contributes - **Admin (optional)**: if the host runs ActiveAdmin, WarpEngine contributes
ready-made resources — a catalog editor with nested release/asset forms, an ready-made resources — a catalog editor with nested release/asset forms, a
image library with orphan cleanup, a file manager with a picker mode, and file manager with a picker mode, and download statistics. Without ActiveAdmin the engine runs headless
download statistics. Without ActiveAdmin the engine runs headless
(API + updater only). (API + updater only).
- **Woodpecker CI management (optional)**: with a Woodpecker API token - **Pluggable CI**: pipeline configs, repo sync, build triggers and secret
configured, the admin also gains repo sync, per-repo pipeline history provisioning all go through one adapter. Woodpecker ships with the engine
with manual triggers, and automatic provisioning of application tokens (`WarpEngine::CI::Woodpecker`) and is the default; `:none` turns CI off, and
as Woodpecker secrets. another server is a host-supplied object.
## Requirements ## Requirements
@@ -177,12 +179,18 @@ cannot say which gem came from where.
Then: Then:
```sh ```sh
rails g warp_engine:install # initializer + create_warp_engine_tables migration rails g warp_engine:install # initializer, image library (model + endpoint), migrations
rails db:migrate rails db:migrate
``` ```
```ruby ```ruby
# config/routes.rb — keep it the last entry so your own routes win # config/routes.rb
namespace :api do
# The image library is yours; the engine only publishes these URLs.
get "image/:id", to: "images#show"
end
# keep the mount the last entry so your own routes win
mount WarpEngine::Engine => "/" mount WarpEngine::Engine => "/"
``` ```
@@ -192,9 +200,12 @@ mount WarpEngine::Engine => "/"
# config/initializers/warp_engine.rb # config/initializers/warp_engine.rb
Rails.application.config.to_prepare do Rails.application.config.to_prepare do
WarpEngine.configure do |c| WarpEngine.configure do |c|
# Where CI drops build artifacts and where images are stored # Where CI drops build artifacts
c.file_container_path = ENV.fetch("FILE_CONTAINER_PATH", "/softwares") c.file_container_path = ENV.fetch("FILE_CONTAINER_PATH", "/softwares")
c.image_container_path = ENV.fetch("IMAGE_CONTAINER_PATH", "/images")
# Your image model (see "Images" below). "Image" is the default.
# c.image_class_name = "Media::Picture"
# c.image_adapter = MyImageLibrary.new
# Shared secret for the /build/* endpoints. # Shared secret for the /build/* endpoints.
# nil => the endpoints reject every request. # nil => the endpoints reject every request.
@@ -217,6 +228,15 @@ Rails.application.config.to_prepare do
# after backfilling owners — ownerless softwares are claimable by anyone. # after backfilling owners — ownerless softwares are claimable by anyone.
# c.enforce_software_ownership = true # c.enforce_software_ownership = true
# Which CI server builds the games (see "CI" below). :woodpecker (default)
# reads WOODPECKER_URL / WOODPECKER_API_TOKEN from the environment; naming
# the adapter is better, because then the settings are here in the open.
# c.ci_adapter = WarpEngine::CI::Woodpecker::Adapter.new(
# url: ENV["WOODPECKER_URL"], api_token: ENV["WOODPECKER_API_TOKEN"],
# platforms: { "godot" => { builder: "registry.example/godot-builder:4.6" } }
# )
# c.ci_adapter = :none # this host builds nothing
# Who may see a title and who may download it (see "Access" below). # Who may see a title and who may download it (see "Access" below).
# :open (default) lists everything and serves everything. # :open (default) lists everything and serves everything.
# c.access_policy = MyStore::AccessPolicy.new # c.access_policy = MyStore::AccessPolicy.new
@@ -228,16 +248,6 @@ Rails.application.config.to_prepare do
# #
# How to recognise a caller with a session instead of a bearer token. # How to recognise a caller with a session instead of a bearer token.
# c.subject_resolver = ->(request) { request.env["warden"]&.user } # c.subject_resolver = ->(request) { request.env["warden"]&.user }
# If your app's own models reference catalog images, register them so the
# admin Images page counts them as "in use":
# c.image_owners = [
# {
# label: "member",
# image_ids: -> { Member.where.not(image_id: nil).distinct.pluck(:image_id) },
# usage_label: ->(image) { "member" if Member.where(image_id: image.id).exists? }
# }
# ]
end end
end end
``` ```
@@ -291,7 +301,56 @@ accepts both:
When switching to `:database`, create the tokens and move your pipelines to When switching to `:database`, create the tokens and move your pipelines to
them first — the flip invalidates the shared secret immediately. them first — the flip invalidates the shared secret immediately.
## CI pipeline configs (Woodpecker) ## CI
Every CI feature — serving pipeline configs, syncing repositories, triggering
builds, provisioning secrets — goes through **one adapter**. Woodpecker ships
with the engine as `WarpEngine::CI::Woodpecker`, and it is the default:
```ruby
# nothing set: Woodpecker, configured from ENV, exactly as before
c.ci_adapter = :woodpecker
# named explicitly (recommended — the host says what its CI is)
c.ci_adapter = WarpEngine::CI::Woodpecker::Adapter.new(
url: ENV["WOODPECKER_URL"],
api_token: ENV["WOODPECKER_API_TOKEN"],
repo_owner: ENV["WOODPECKER_REPO_OWNER"],
public_key_url: "https://ci.example.org/api/signature/public-key",
update_server: "https://catalog.example.org",
platforms: { "godot" => { builder: "registry.example/godot-builder:4.6" } }
)
# this host builds nothing: /api/ci/* answers 503, /build/config refuses,
# and the admin pipeline actions hide themselves
c.ci_adapter = :none
```
The engine's own code never names a provider: `WarpEngine.ci` is the adapter,
`WarpEngine::CI::Repo` and `WarpEngine::CI::Run` are what it hands back, and
`WarpEngine::CI::ConnectionError` / `ApiError` / `NotConfigured` are what it
raises. A host with a different CI server implements this contract instead:
| Method | Purpose |
| --- | --- |
| `name` | provider label, shown in the admin |
| `configured?` | false keeps every CI feature inactive |
| `platforms` | which platforms `/build/config` can serve |
| `update_server` | server URL written into the generated pipeline; nil = the request's base_url |
| `repos`, `repo(id)` | `CI::Repo` values for the repo sync |
| `activate_repo(id)`, `deactivate_repo(id)` | enable/disable a repository |
| `runs(repo_id, page:)`, `run(repo_id, number)` | `CI::Run` values (`number` may be `"latest"`) |
| `trigger(repo_id, branch:)` | start a build, returns a `CI::Run` |
| `secret_names(repo_id)`, `secret_set(repo_id, name:, value:)`, `secret_delete(repo_id, name)` | token provisioning |
| `verify_config_request(request)` | is this really the CI server calling |
| `config_marker(params)` | `{ platform:, name: }` from the request, or nil for "not ours" |
| `pipeline_config(platform:, name:, update_server:)` | the pipeline definition, or nil for an unknown platform |
| `config_response(platform:, config:)` | the body the CI server expects |
`Pipeline` records keep the provider-neutral `remote_repo_id` reader over the
historical `woodpecker_repo_id` column, so the table did not have to move.
### Pipeline configs (the Woodpecker adapter)
WarpEngine can act as a [Woodpecker configuration extension](https://woodpecker-ci.org/docs/usage/extensions/configuration-extension): WarpEngine can act as a [Woodpecker configuration extension](https://woodpecker-ci.org/docs/usage/extensions/configuration-extension):
instead of a copy-pasted `.woodpecker.yaml` in every game repo, the repo holds a instead of a copy-pasted `.woodpecker.yaml` in every game repo, the repo holds a
@@ -305,35 +364,30 @@ platform: godot
``` ```
- `POST /build/config` — the extension endpoint Woodpecker calls on every - `POST /build/config` — the extension endpoint Woodpecker calls on every
pipeline start (httpsig/ed25519-signed request, verified against pipeline start (httpsig/ed25519-signed request, verified against the
`ci_extension_public_key(_url)`). Non-marker configs get a `204` so the adapter's `public_key` / `public_key_url`). Non-marker configs get a `204` so the
repo's own YAML keeps running — opt-in migration, and putting a full repo's own YAML keeps running — opt-in migration, and putting a full
pipeline back into the repo is the opt-out. pipeline back into the repo is the opt-out.
- `GET /build/config?platform=godot` — renders the same pipeline as a preview. - `GET /build/config?platform=godot` — renders the same pipeline as a preview.
Configuration: `ci_platforms` maps platform names to builder images The adapter's `platforms` maps platform names to builder images
(`{ "godot" => { builder: "..." }, "tic80" => { builder: ..., exporter: ... } }`); (`{ "godot" => { builder: "..." }, "tic80" => { builder: ..., exporter: ... } }`);
an empty map (default) disables the feature. Set the Woodpecker side with an empty map (default) disables the feature. Set the Woodpecker side with
`WOODPECKER_CONFIG_EXTENSION_ENDPOINT=https://your-host/build/config` (or `WOODPECKER_CONFIG_EXTENSION_ENDPOINT=https://your-host/build/config` (or
per-repo in Settings → Extensions). Templates live in per-repo in Settings → Extensions). Templates live with the adapter, in
`app/services/warp_engine/platforms/<platform>/pipeline.yaml.erb`. `lib/warp_engine/ci/woodpecker/platforms/<platform>/pipeline.yaml.erb` — the
YAML dialect belongs to the provider, so a second adapter brings its own.
## Woodpecker CI management ### CI management (the Woodpecker adapter)
Beyond serving pipeline configs, WarpEngine can drive the Woodpecker REST API Beyond serving pipeline configs, the adapter drives the Woodpecker REST API
itself. Set `woodpecker_url` and `woodpecker_api_token` — while either is nil itself. Give it a `url` and an `api_token` — with either missing, `configured?`
(the default), every management feature stays inactive and the admin pages is false, every management feature stays inactive and the admin pages hide
hide themselves: themselves.
```ruby
c.woodpecker_url = ENV["WOODPECKER_URL"] # e.g. "https://ci.example.org"
c.woodpecker_api_token = ENV["WOODPECKER_API_TOKEN"] # PAT of a Woodpecker *instance admin*
c.woodpecker_repo_owner = ENV["WOODPECKER_REPO_OWNER"] # forge org the game repos live under
```
What it unlocks (all surfaced in the admin): What it unlocks (all surfaced in the admin):
- **Repo sync** (*Pipelines → Sync from Woodpecker*): mirrors the Woodpecker - **Repo sync** (*Pipelines → Sync from *): mirrors the provider's
repo list into `Pipeline` records, auto-matching each repo to a catalog repo list into `Pipeline` records, auto-matching each repo to a catalog
`Software` by name; repos that disappear from Woodpecker are deactivated. `Software` by name; repos that disappear from Woodpecker are deactivated.
Platform and software links are editable by hand afterwards. Platform and software links are editable by hand afterwards.
@@ -342,7 +396,7 @@ What it unlocks (all surfaced in the admin):
last-pipeline status shown on the Pipelines index. The software's admin last-pipeline status shown on the Pipelines index. The software's admin
page links to its pipelines from the Quick Links sidebar. page links to its pipelines from the Quick Links sidebar.
- **Secret provisioning**: database application tokens are pushed to the - **Secret provisioning**: database application tokens are pushed to the
repos as the `application_token` Woodpecker secret — creating a token repos as the `application_token` CI secret — creating a token
provisions it to its owner's repos (unrestricted tokens to all active provisions it to its owner's repos (unrestricted tokens to all active
repos), deleting a token removes the secret, and *Rotate* creates a repos), deleting a token removes the secret, and *Rotate* creates a
replacement token, provisions it everywhere and revokes the old one in a replacement token, provisions it everywhere and revokes the old one in a
@@ -392,6 +446,47 @@ signing adapter turns them into redirects without any further change.
the admin file manager — still writes to the local disk. A remote adapter the admin file manager — still writes to the local disk. A remote adapter
needs its own upload path today. needs its own upload path today.
## Images
An image is not a catalog concept: the same picture library serves covers,
screenshots, avatars and whatever else a site has. So the engine owns only the
link — `SoftwareImage`, ordered, with one default — while the `Image` model, the
uploaded files and the admin page live in the host. `rails g warp_engine:install`
writes a working one (`app/models/image.rb`, `app/controllers/api/images_controller.rb`,
a `create_images` migration); the engine talks to it through an adapter:
```ruby
c.image_class_name = "Image" # default
```
The default adapter expects that model to answer `original_filename`,
`content_type`, `file_path` and `file_upload=` (the generated one does), and
publishes every image as `/api/image/<id>` — the address clients have always
used, which is why the generated route serves exactly that. A host that keeps
its pictures somewhere else replaces the adapter instead:
```ruby
class MyImageLibrary
def model_name = "Media::Picture"
def model = Media::Picture
def url_for(image_id) = "https://cdn.example.com/#{image_id}.webp"
def build_from_upload(upload) = Media::Picture.new(file_upload: upload)
def label_for(record) = record.title
def available?(record) = record.stored?
def select_options = Media::Picture.order(:title).pluck(:title, :id)
end
c.image_adapter = MyImageLibrary.new
```
`url_for` is what lands in the catalog JSON (`imageUrl`, `images[].url`) and in
the admin previews; `select_options` fills the image picker on the software
form; `build_from_upload` is what "upload a new image" on that form calls.
**The `images` table stays where it is.** Moving the model out of the engine did
not touch it: the engine no longer creates it (the install generator does, as
host code), and `software_images.image_id` still points at the same rows.
## Access ## Access
Who may see a title, and who may download it. The default answers "everyone" to Who may see a title, and who may download it. The default answers "everyone" to
@@ -523,7 +618,7 @@ its own account of who fetched what without reaching into `DownloadService`.
| `GET /api/software/highlighted` | The currently highlighted title | | `GET /api/software/highlighted` | The currently highlighted title |
| `GET /api/builds` | Expected asset kinds per platform (build matrix) | | `GET /api/builds` | Expected asset kinds per platform (build matrix) |
| `GET /api/softwares/:name/builds` | Actual vs. missing build assets per release | | `GET /api/softwares/:name/builds` | Actual vs. missing build assets per release |
| `GET /api/image/:id` | Serves catalog images | | `GET /api/image/:id` | Serves images — **the host's endpoint**, see *Images* |
| `GET /api/download?path=` | Serves an artifact and logs a download record | | `GET /api/download?path=` | Serves an artifact and logs a download record |
| `GET /file/*path` | Serves static build output (web-playable games, docs) | | `GET /file/*path` | Serves static build output (web-playable games, docs) |
| `POST /api/auth/device` | Starts a device sign-in; returns the code pair (404 without a client identity) | | `POST /api/auth/device` | Starts a device sign-in; returns the code pair (404 without a client identity) |
+3 -3
View File
@@ -57,7 +57,7 @@ ActiveAdmin.register WarpEngine::ApplicationToken, as: "Application Token" do
end end
action_item :rotate, only: :show do action_item :rotate, only: :show do
if WarpEngine.woodpecker_configured? if WarpEngine.ci.configured?
link_to "Rotate Token", rotate_admin_application_token_path(resource), link_to "Rotate Token", rotate_admin_application_token_path(resource),
method: :post, data: { confirm: "This will revoke the current token, create a new one, and push it to Woodpecker. Continue?" } method: :post, data: { confirm: "This will revoke the current token, create a new one, and push it to Woodpecker. Continue?" }
end end
@@ -101,7 +101,7 @@ ActiveAdmin.register WarpEngine::ApplicationToken, as: "Application Token" do
success.html do success.html do
session[:warp_engine_plain_token] = resource.plain_token session[:warp_engine_plain_token] = resource.plain_token
if WarpEngine.woodpecker_configured? if WarpEngine.ci.configured?
service = WarpEngine::SecretSyncService.new service = WarpEngine::SecretSyncService.new
pipelines = service.pipelines_for_token(resource) pipelines = service.pipelines_for_token(resource)
if pipelines.any? if pipelines.any?
@@ -124,7 +124,7 @@ ActiveAdmin.register WarpEngine::ApplicationToken, as: "Application Token" do
end end
def destroy def destroy
if WarpEngine.woodpecker_configured? if WarpEngine.ci.configured?
WarpEngine::SecretSyncService.new.deprovision(resource) WarpEngine::SecretSyncService.new.deprovision(resource)
end end
resource.revoke! resource.revoke!
-77
View File
@@ -1,77 +0,0 @@
ActiveAdmin.register WarpEngine::Image, as: "Image" do
permit_params :file_upload
menu parent: "🌀 WarpEngine", priority: 5, label: "🖼️ Images"
used_ids = -> {
WarpEngine::SoftwareImage.unscope(:order).distinct.pluck(:image_id) +
WarpEngine.config.image_owners.flat_map { |o| o[:image_ids].call }
}
scope :all, default: true
scope("In use") { |scope| scope.where(id: used_ids.call) }
scope("Orphan") { |scope| scope.where.not(id: used_ids.call) }
batch_action :delete_orphans, confirm: "Delete all selected orphan images and their files?" do |ids|
orphan_ids = ids.map(&:to_i) - used_ids.call
WarpEngine::Image.where(id: orphan_ids).find_each do |img|
File.delete(img.file_path) if File.exist?(img.file_path)
img.destroy
end
redirect_to admin_images_path, notice: "Deleted #{orphan_ids.size} orphan image(s)."
end
index do
selectable_column
id_column
column :original_filename
column :content_type
column(:preview) do |img|
if File.exist?(img.file_path)
image_tag("/api/image/#{img.id}", style: "max-height:60px;max-width:120px;object-fit:contain;")
end
end
column(:usage) do |img|
uses = []
uses << "#{img.software_images.size} software(s)" if img.software_images.any?
uses.concat(WarpEngine.config.image_owners.filter_map { |o| o[:usage_label].call(img) })
uses.any? ? uses.join(", ") : status_tag("orphan", class: "warning")
end
column :created_at
actions
end
filter :original_filename
filter :content_type
show do
attributes_table do
row :id
row :original_filename
row :filename
row :content_type
row(:preview) do |img|
if File.exist?(img.file_path)
image_tag("/api/image/#{img.id}", style: "max-height:300px;max-width:100%;object-fit:contain;")
else
"File not found on disk"
end
end
row :created_at
row :updated_at
end
end
form html: { multipart: true } do |f|
f.inputs do
f.input :file_upload, as: :file, label: "Image File"
end
f.actions
end
controller do
def scoped_collection
super.includes(:software_images)
end
end
end
+14 -15
View File
@@ -33,7 +33,7 @@ ActiveAdmin.register WarpEngine::Pipeline, as: "Pipeline" do
} }
column :last_pipeline_at column :last_pipeline_at
actions defaults: true do |pipeline| actions defaults: true do |pipeline|
if pipeline.active && WarpEngine.woodpecker_configured? if pipeline.active && WarpEngine.ci.configured?
item "Trigger", trigger_admin_pipeline_path(pipeline), method: :post, class: "member_link" item "Trigger", trigger_admin_pipeline_path(pipeline), method: :post, class: "member_link"
end end
end end
@@ -59,7 +59,7 @@ ActiveAdmin.register WarpEngine::Pipeline, as: "Pipeline" do
sidebar "Details", only: :show do sidebar "Details", only: :show do
attributes_table_for resource do attributes_table_for resource do
row :id row :id
row :woodpecker_repo_id row("Repo id") { |r| r.remote_repo_id }
row :repo_owner row :repo_owner
row :repo_name row :repo_name
row :platform row :platform
@@ -74,22 +74,21 @@ ActiveAdmin.register WarpEngine::Pipeline, as: "Pipeline" do
show do show do
panel "Pipelines" do panel "Pipelines" do
if !WarpEngine.woodpecker_configured? if !WarpEngine.ci.configured?
para "Woodpecker is not configured. Set woodpecker_url and woodpecker_api_token in the WarpEngine initializer.", para "No CI provider is configured. Set c.ci_adapter in the WarpEngine initializer.",
style: "color:#999;" style: "color:#999;"
elsif !resource.active elsif !resource.active
para "This repository is inactive.", style: "color:#999;" para "This repository is inactive.", style: "color:#999;"
else else
begin begin
pipelines = WarpEngine::PipelineService.new.list_pipelines(resource, page: 1) runs = WarpEngine::PipelineService.new.runs(resource, page: 1)
if pipelines.is_a?(Array) && pipelines.any? if runs.any?
table_for pipelines.first(10) do table_for runs.first(10) do
column("Number") { |p| p["number"] } column("Number") { |r| r.number }
column("Status") { |p| status_tag p["status"], class: p["status"] == "success" ? "yes" : "no" } column("Status") { |r| status_tag r.status, class: r.success? ? "yes" : "no" }
column("Branch") { |p| p["branch"] } column("Branch") { |r| r.branch }
column("Message") { |p| p["message"]&.truncate(60) } column("Message") { |r| r.message&.truncate(60) }
column("Created") { |r| r.created_at ? r.created_at.strftime("%Y-%m-%d %H:%M") : "-" }
column("Created") { |p| p["created"] ? Time.zone.at(p["created"]).strftime("%Y-%m-%d %H:%M") : "-" }
end end
else else
para "No pipelines found.", style: "color:#999;" para "No pipelines found.", style: "color:#999;"
@@ -118,8 +117,8 @@ ActiveAdmin.register WarpEngine::Pipeline, as: "Pipeline" do
end end
action_item :sync_repos, only: :index do action_item :sync_repos, only: :index do
if WarpEngine.woodpecker_configured? if WarpEngine.ci.configured?
link_to "Sync from Woodpecker", sync_admin_pipelines_path, method: :post link_to "Sync from #{WarpEngine.ci.name}", sync_admin_pipelines_path, method: :post
end end
end end
+4 -4
View File
@@ -27,8 +27,8 @@ ActiveAdmin.register WarpEngine::Software, as: "Software" do
column(:releases) { |sw| sw.releases.size } column(:releases) { |sw| sw.releases.size }
column(:image) do |sw| column(:image) do |sw|
si = sw.software_images.detect(&:is_default?) || sw.software_images.first si = sw.software_images.detect(&:is_default?) || sw.software_images.first
if si&.image && File.exist?(si.image.file_path) if si&.image && WarpEngine::Images.available?(si.image)
image_tag "/api/image/#{si.image_id}", style: "max-height:40px;max-width:80px;object-fit:contain;" image_tag WarpEngine::Images.url_for(si.image_id), style: "max-height:40px;max-width:80px;object-fit:contain;"
end end
end end
column :created_at column :created_at
@@ -114,10 +114,10 @@ ActiveAdmin.register WarpEngine::Software, as: "Software" do
f.inputs "Images" do f.inputs "Images" do
f.has_many :software_images, allow_destroy: true, new_record: true do |si| f.has_many :software_images, allow_destroy: true, new_record: true do |si|
img_hint = if si.object.image_id.present? img_hint = if si.object.image_id.present?
si.template.image_tag("/api/image/#{si.object.image_id}", style: "max-height:80px;max-width:160px;object-fit:contain;margin-top:6px;").html_safe si.template.image_tag(WarpEngine::Images.url_for(si.object.image_id), style: "max-height:80px;max-width:160px;object-fit:contain;margin-top:6px;").html_safe
end end
si.input :image_id, as: :select, label: "Image", si.input :image_id, as: :select, label: "Image",
collection: WarpEngine::Image.order(:original_filename).map { |img| [ img.original_filename, img.id ] }, collection: WarpEngine::Images.select_options,
include_blank: "— select —", include_blank: "— select —",
hint: img_hint hint: img_hint
si.input :file_upload, as: :file, label: "Upload new image" si.input :file_upload, as: :file, label: "Upload new image"
@@ -3,7 +3,7 @@ module WarpEngine
class CiController < ApiController class CiController < ApiController
include UpdateAuthentication include UpdateAuthentication
before_action :require_woodpecker! before_action :require_ci!
resource_description do resource_description do
short "CI pipeline management" short "CI pipeline management"
@@ -11,7 +11,7 @@ module WarpEngine
api :GET, "/api/ci/pipelines", "List active pipelines" api :GET, "/api/ci/pipelines", "List active pipelines"
returns code: 200, desc: "JSON array of tracked pipelines" returns code: 200, desc: "JSON array of tracked pipelines"
error code: 503, desc: "Woodpecker not configured" error code: 503, desc: "No CI provider configured"
def pipelines def pipelines
records = Pipeline.active.includes(:software) records = Pipeline.active.includes(:software)
render json: records.map { |p| pipeline_json(p) } render json: records.map { |p| pipeline_json(p) }
@@ -20,12 +20,12 @@ module WarpEngine
api :GET, "/api/ci/pipelines/:id/status", "Get a pipeline with its latest run" api :GET, "/api/ci/pipelines/:id/status", "Get a pipeline with its latest run"
param :id, :number, required: true, desc: "Pipeline id" param :id, :number, required: true, desc: "Pipeline id"
returns code: 200, desc: "JSON with pipeline and latest run data" returns code: 200, desc: "JSON with pipeline and latest run data"
error code: 503, desc: "Woodpecker not configured" error code: 503, desc: "No CI provider configured"
def status def status
pipeline = Pipeline.find(params[:id]) pipeline = Pipeline.find(params[:id])
latest_run = begin latest_run = begin
PipelineService.new.pipeline_detail(pipeline, "latest") PipelineService.new.run(pipeline, "latest")
rescue WoodpeckerClient::ApiError rescue CI::Error
nil nil
end end
render json: { pipeline: pipeline_json(pipeline), latest_run: latest_run } render json: { pipeline: pipeline_json(pipeline), latest_run: latest_run }
@@ -37,7 +37,7 @@ module WarpEngine
param :branch, String, required: false, desc: "Branch to build (default: main)" param :branch, String, required: false, desc: "Branch to build (default: main)"
returns code: 200, desc: "JSON with triggered run data" returns code: 200, desc: "JSON with triggered run data"
error code: 401, desc: "Invalid secret" error code: 401, desc: "Invalid secret"
error code: 503, desc: "Woodpecker not configured" error code: 503, desc: "No CI provider configured"
def trigger def trigger
unless update_authorized?(required_scope: ApplicationToken::UPDATE_SCOPE) unless update_authorized?(required_scope: ApplicationToken::UPDATE_SCOPE)
return render json: { error: "Unauthorized" }, status: :unauthorized return render json: { error: "Unauthorized" }, status: :unauthorized
@@ -50,16 +50,17 @@ module WarpEngine
private private
def require_woodpecker! def require_ci!
return if WarpEngine.woodpecker_configured? return if WarpEngine.ci.configured?
render json: { error: "Woodpecker not configured" }, status: :service_unavailable render json: { error: "CI not configured" }, status: :service_unavailable
end end
def pipeline_json(pipeline) def pipeline_json(pipeline)
{ {
id: pipeline.id, id: pipeline.id,
woodpecker_repo_id: pipeline.woodpecker_repo_id, repo_id: pipeline.remote_repo_id,
woodpecker_repo_id: pipeline.remote_repo_id,
repo_owner: pipeline.repo_owner, repo_owner: pipeline.repo_owner,
repo_name: pipeline.repo_name, repo_name: pipeline.repo_name,
platform: pipeline.platform, platform: pipeline.platform,
@@ -1,17 +0,0 @@
module WarpEngine
class Api::ImagesController < ApiController
resource_description do
short "Images"
formats [ "binary" ]
end
api :GET, "/api/image/:id", "Get image by ID"
param :id, :number, required: true, desc: "Image ID"
returns code: 200, desc: "Image binary data"
error code: 404, desc: "Image not found"
def show
image = WarpEngine::ImageService.new.show(WarpEngine::ImageShowInputDto.new(id: params[:id]))
send_file image.file_path, type: image.content_type, disposition: "inline"
end
end
end
@@ -3,76 +3,60 @@ module WarpEngine
class ConfigsController < ApiController class ConfigsController < ApiController
resource_description do resource_description do
short "Woodpecker CI pipeline configs" short "CI pipeline configs"
end end
api :GET, "/build/config", "Preview the generated pipeline config for a platform" api :GET, "/build/config", "Preview the generated pipeline config for a platform"
param :platform, String, required: true, desc: "Platform (a configured ci_platforms key, e.g. tic80)" param :platform, String, required: true, desc: "Platform (a platform the CI adapter serves, e.g. tic80)"
param :name, String, required: false, desc: "Software name substituted into the pipeline (default: example)" param :name, String, required: false, desc: "Software name substituted into the pipeline (default: example)"
returns code: 200, desc: "Pipeline YAML (text/yaml)" returns code: 200, desc: "Pipeline YAML (text/yaml)"
error code: 404, desc: "Unknown platform" error code: 404, desc: "Unknown platform"
def show def show
yaml = render_config(platform: params[:platform], name: params[:name].presence || "example") config = render_config(platform: params[:platform], name: params[:name].presence || "example")
return render json: { error: "Unknown platform" }, status: :not_found if yaml.nil? return render json: { error: "Unknown platform" }, status: :not_found if config.nil?
render plain: yaml, content_type: "text/yaml" render plain: config, content_type: "text/yaml"
end end
api :POST, "/build/config", "Woodpecker configuration extension endpoint" api :POST, "/build/config", "CI configuration extension endpoint"
description <<~DESC description <<~DESC
Called by the Woodpecker server on every pipeline start (httpsig-signed request). Called by the CI server on every pipeline start (a signed request the CI adapter
If the repo's .woodpecker.yaml is a marker (has a `platform:` key), responds with verifies). If the repo's config is a marker (has a `platform:` key), responds with
the generated pipeline; otherwise responds 204 so the repo's own config runs. the generated pipeline; otherwise responds 204 so the repo's own config runs.
DESC DESC
returns code: 200, desc: %(JSON: {"configs": [{"name": ..., "data": "<pipeline YAML>"}]}) returns code: 200, desc: %(JSON: the adapter's config response — for Woodpecker {"configs": [{"name": ..., "data": "<pipeline YAML>"}]})
returns code: 204, desc: "Not a marker config — keep the repo's own configuration" returns code: 204, desc: "Not a marker config — keep the repo's own configuration"
error code: 403, desc: "Missing or invalid request signature" error code: 403, desc: "Missing or invalid request signature"
error code: 422, desc: "Marker requests an unknown platform" error code: 422, desc: "Marker requests an unknown platform"
def create def create
unless WarpEngine::CiSignatureVerifier.new(request).valid? unless ci.verify_config_request(request)
return render json: { error: "Invalid signature" }, status: :forbidden return render json: { error: "Invalid signature" }, status: :forbidden
end end
marker = find_marker marker = ci.config_marker(params)
return head :no_content if marker.nil? return head :no_content if marker.nil?
platform = marker["platform"].to_s config = render_config(platform: marker[:platform], name: marker[:name])
name = marker["name"].presence || repo_name if config.nil?
yaml = render_config(platform: platform, name: name) return render json: { error: "Unknown platform: #{marker[:platform]}" }, status: :unprocessable_entity
if yaml.nil?
return render json: { error: "Unknown platform: #{platform}" }, status: :unprocessable_entity
end end
render json: { configs: [ { name: platform, data: yaml } ] } render json: ci.config_response(platform: marker[:platform], config: config)
end end
private private
def ci
WarpEngine.ci
end
def render_config(platform:, name:) def render_config(platform:, name:)
WarpEngine::CiConfigService.new.render( ci.pipeline_config(
platform: platform, platform: platform,
name: name, name: name,
update_server: WarpEngine.config.ci_update_server.presence || request.base_url update_server: ci.update_server.presence || request.base_url
) )
end end
def find_marker
configs = params[:configuration].presence || params[:configs].presence || []
configs.each do |config|
data = config[:data].to_s
parsed = begin
YAML.safe_load(data)
rescue Psych::Exception
nil
end
return parsed if parsed.is_a?(Hash) && parsed.key?("platform")
end
nil
end
def repo_name
params.dig(:repo, :name).to_s
end
end end
end end
end end
@@ -1,3 +0,0 @@
module WarpEngine
ImageShowInputDto = Struct.new(:id, keyword_init: true)
end
+2 -2
View File
@@ -1,7 +1,7 @@
module WarpEngine module WarpEngine
class ApplicationJob < ActiveJob::Base class ApplicationJob < ActiveJob::Base
retry_on WoodpeckerClient::ConnectionError, wait: 30.seconds, attempts: 3 retry_on CI::ConnectionError, wait: 30.seconds, attempts: 3
discard_on WoodpeckerClient::ApiError do |job, error| discard_on CI::ApiError, CI::NotConfigured do |job, error|
Rails.logger.error("[#{job.class.name}] discarded: #{error.message}") Rails.logger.error("[#{job.class.name}] discarded: #{error.message}")
end end
end end
-36
View File
@@ -1,36 +0,0 @@
module WarpEngine
class Image < ApplicationRecord
def self.upload_path
WarpEngine.config.image_container_path
end
has_many :software_images, dependent: :restrict_with_error
default_scope { where(deleted_at: nil) }
attr_accessor :file_upload
before_save :process_upload, if: -> { file_upload.present? }
def self.ransackable_attributes(auth_object = nil)
%w[content_type created_at deleted_at filename id original_filename updated_at]
end
def file_path
File.join(self.class.upload_path, filename.to_s)
end
ActiveSupport.run_load_hooks(:warp_engine_image, self)
private
def process_upload
FileUtils.mkdir_p(self.class.upload_path)
self.original_filename = file_upload.original_filename
self.content_type = file_upload.content_type.presence || "application/octet-stream"
ext = File.extname(file_upload.original_filename)
self.filename = "#{SecureRandom.uuid}#{ext}"
IO.copy_stream(file_upload.to_io, file_path)
end
end
end
+8
View File
@@ -24,6 +24,14 @@ module WarpEngine
"#{repo_owner}/#{repo_name}" "#{repo_owner}/#{repo_name}"
end end
def remote_repo_id
woodpecker_repo_id
end
def remote_repo_id=(value)
self.woodpecker_repo_id = value
end
def self.ransackable_attributes(auth_object = nil) def self.ransackable_attributes(auth_object = nil)
%w[active created_at deleted_at id last_pipeline_at last_pipeline_status %w[active created_at deleted_at id last_pipeline_at last_pipeline_status
platform repo_name repo_owner software_id woodpecker_repo_id] platform repo_name repo_owner software_id woodpecker_repo_id]
+2 -2
View File
@@ -1,7 +1,7 @@
module WarpEngine module WarpEngine
class SoftwareImage < ApplicationRecord class SoftwareImage < ApplicationRecord
belongs_to :software belongs_to :software
belongs_to :image, optional: true belongs_to :image, optional: true, class_name: WarpEngine::Images.model_name
attr_accessor :file_upload attr_accessor :file_upload
@@ -28,7 +28,7 @@ module WarpEngine
private private
def create_image_from_upload def create_image_from_upload
img = Image.new(file_upload: file_upload) img = WarpEngine::Images.build_from_upload(file_upload)
if img.save if img.save
self.image = img self.image = img
else else
@@ -21,11 +21,11 @@ module WarpEngine
field(:platformLinks) { |sw| PlatformLinkSerializer.render_as_hash(WarpEngine::PlatformLink.for_platform(sw.platform)) } field(:platformLinks) { |sw| PlatformLinkSerializer.render_as_hash(WarpEngine::PlatformLink.for_platform(sw.platform)) }
field(:imageUrl) { |sw| field(:imageUrl) { |sw|
si = sw.software_images.detect(&:is_default?) || sw.software_images.first si = sw.software_images.detect(&:is_default?) || sw.software_images.first
si ? "/api/image/#{si.image_id}" : nil si ? WarpEngine::Images.url_for(si.image_id) : nil
} }
field(:images) { |sw| field(:images) { |sw|
sw.software_images.map { |si| sw.software_images.map { |si|
{ url: "/api/image/#{si.image_id}", isDefault: si.is_default?, position: si.position } { url: WarpEngine::Images.url_for(si.image_id), isDefault: si.is_default?, position: si.position }
} }
} }
end end
@@ -1,37 +0,0 @@
require "erb"
module WarpEngine
class CiConfigService
PLATFORM_FORMAT = /\A[a-z0-9_-]+\z/
def render(platform:, name:, update_server:)
platform = platform.to_s
return nil unless platform.match?(PLATFORM_FORMAT)
spec = platform_spec(platform)
return nil if spec.nil?
path = templates_dir.join(platform, "pipeline.yaml.erb")
return nil unless path.exist?
ERB.new(path.read, trim_mode: "-").result_with_hash(
name: name.to_s,
update_server: update_server.to_s,
builder: spec[:builder],
exporter: spec[:exporter]
)
end
private
def platform_spec(platform)
spec = WarpEngine.config.ci_platforms.stringify_keys[platform]
spec&.symbolize_keys
end
def templates_dir
WarpEngine::Engine.root.join("app", "services", "warp_engine", "platforms")
end
end
end
@@ -1,142 +0,0 @@
require "openssl"
require "base64"
require "net/http"
require "digest"
module WarpEngine
class CiSignatureVerifier
CAVAGE_PARAM = /(\w+)="([^"]*)"/
@key_cache = {}
@key_mutex = Mutex.new
class << self
def fetch_public_key(url)
@key_mutex.synchronize do
@key_cache[url] ||= Net::HTTP.get(URI.parse(url))
end
end
def reset_key_cache!
@key_mutex.synchronize { @key_cache = {} }
end
end
def initialize(request)
@request = request
end
def valid?
pem = public_key_pem
if pem.blank?
Rails.logger.error("[CiSignatureVerifier] no ci_extension_public_key(_url) configured — rejecting request")
return false
end
key = OpenSSL::PKey.read(pem)
if @request.headers["Signature-Input"].present?
rfc9421_valid?(key)
else
cavage_valid?(key)
end
rescue OpenSSL::PKey::PKeyError, ArgumentError => e
Rails.logger.error("[CiSignatureVerifier] #{e.class}: #{e.message}")
false
end
private
def public_key_pem
config = WarpEngine.config
return config.ci_extension_public_key if config.ci_extension_public_key.present?
return nil if config.ci_extension_public_key_url.blank?
self.class.fetch_public_key(config.ci_extension_public_key_url)
rescue StandardError => e
Rails.logger.error("[CiSignatureVerifier] public key fetch failed: #{e.class}: #{e.message}")
nil
end
def rfc9421_valid?(key)
input = @request.headers["Signature-Input"].to_s
match = input.match(/\A\s*([\w.-]+)=(\(.*)\z/m)
return false if match.nil?
label, inner = match[1], match[2]
components = inner[/\((.*?)\)/m, 1].to_s.scan(/"([^"]*)"/).flatten
return false if components.empty?
signature = @request.headers["Signature"].to_s[/#{Regexp.escape(label)}=:([A-Za-z0-9+\/=]+):/, 1]
return false if signature.blank?
return false unless content_digest_valid?(components)
lines = components.map do |component|
value = component_value(component)
return false if value.nil?
%("#{component}": #{value})
end
lines << %("@signature-params": #{inner})
key.verify(nil, Base64.decode64(signature), lines.join("\n"))
end
def component_value(name)
case name
when "@request-target" then @request.fullpath
when "@method" then @request.request_method
when "@target-uri" then @request.original_url
when "@authority" then @request.host_with_port
when "@path" then @request.path
when "@query" then "?#{@request.query_string}"
when /\A@/ then nil
else @request.headers[name]
end
end
def content_digest_valid?(components)
return true unless components.include?("content-digest")
digest = @request.headers["Content-Digest"].to_s[/sha-256=:([A-Za-z0-9+\/=]+):/, 1]
return false if digest.blank?
expected = Digest::SHA256.base64digest(@request.raw_post)
ActiveSupport::SecurityUtils.secure_compare(digest, expected)
end
def cavage_valid?(key)
params = cavage_params
return false if params.nil? || params["signature"].blank?
signing_string = cavage_signing_string(params.fetch("headers", "date"))
return false if signing_string.nil?
key.verify(nil, Base64.decode64(params["signature"]), signing_string)
end
def cavage_params
header = @request.headers["Signature"].presence
if header.nil?
auth = @request.headers["Authorization"].to_s
header = auth.delete_prefix("Signature ") if auth.start_with?("Signature ")
end
return nil if header.blank?
header.scan(CAVAGE_PARAM).to_h
end
def cavage_signing_string(headers_list)
lines = headers_list.split(" ").map do |name|
if name == "(request-target)"
"(request-target): #{@request.request_method.downcase} #{@request.fullpath}"
else
value = @request.headers[name]
return nil if value.nil?
"#{name.downcase}: #{value}"
end
end
lines.join("\n")
end
end
end
@@ -1,7 +0,0 @@
module WarpEngine
class ImageService
def show(input)
WarpEngine::Image.find(input.id)
end
end
end
+11 -11
View File
@@ -1,31 +1,31 @@
module WarpEngine module WarpEngine
class PipelineService class PipelineService
def initialize(client: WoodpeckerClient.new) def initialize(ci: WarpEngine.ci)
@client = client @ci = ci
end end
def trigger(pipeline, branch: "main") def trigger(pipeline, branch: "main")
@client.trigger_pipeline(pipeline.woodpecker_repo_id, branch: branch) @ci.trigger(pipeline.remote_repo_id, branch: branch)
end end
def list_pipelines(pipeline, page: 1) def runs(pipeline, page: 1)
runs = @client.list_pipelines(pipeline.woodpecker_repo_id, page: page) runs = @ci.runs(pipeline.remote_repo_id, page: page)
refresh_last_pipeline(pipeline, runs.first) if page == 1 && runs.is_a?(Array) refresh_last_run(pipeline, runs.first) if page == 1
runs runs
end end
def pipeline_detail(pipeline, number) def run(pipeline, number)
@client.get_pipeline(pipeline.woodpecker_repo_id, number) @ci.run(pipeline.remote_repo_id, number)
end end
private private
def refresh_last_pipeline(pipeline, run) def refresh_last_run(pipeline, run)
return unless run && pipeline.persisted? return unless run && pipeline.persisted?
pipeline.update_columns( pipeline.update_columns(
last_pipeline_status: run["status"], last_pipeline_status: run.status,
last_pipeline_at: run["created"] ? Time.zone.at(run["created"]) : nil last_pipeline_at: run.created_at
) )
end end
end end
@@ -1,32 +1,26 @@
module WarpEngine module WarpEngine
class PipelineSyncService class PipelineSyncService
def initialize(client: WoodpeckerClient.new) def initialize(ci: WarpEngine.ci)
@client = client @ci = ci
end end
def sync_all def sync_all
remote_repos = @client.list_repos remote_repos = @ci.repos
results = { created: [], updated: [], deactivated: [] } results = { created: [], updated: [], deactivated: [] }
remote_ids = remote_repos.map { |r| r["id"] } remote_ids = remote_repos.map(&:id)
remote_repos.each do |remote| remote_repos.each do |remote|
record = Pipeline.unscoped.find_or_initialize_by( record = Pipeline.unscoped.find_or_initialize_by(woodpecker_repo_id: remote.id)
woodpecker_repo_id: remote["id"]
)
was_new = record.new_record? was_new = record.new_record?
record.assign_attributes( record.assign_attributes(
repo_name: remote["name"], repo_name: remote.name,
repo_owner: remote["owner"], repo_owner: remote.owner,
active: remote["active"], active: remote.active?,
deleted_at: nil deleted_at: nil
) )
if record.platform.blank? || record.platform == Pipeline::UNKNOWN_PLATFORM assign_platform(record, remote.name)
sw = Software.find_by(name: remote["name"])
record.platform = sw&.platform || Pipeline::UNKNOWN_PLATFORM
record.software = sw if sw
end
next unless record.save next unless record.save
@@ -42,12 +36,12 @@ module WarpEngine
end end
def activate(repo_id) def activate(repo_id)
@client.activate_repo(repo_id) @ci.activate_repo(repo_id)
sync_single(repo_id) sync_single(repo_id)
end end
def deactivate(repo_id) def deactivate(repo_id)
@client.deactivate_repo(repo_id) @ci.deactivate_repo(repo_id)
record = Pipeline.find_by!(woodpecker_repo_id: repo_id) record = Pipeline.find_by!(woodpecker_repo_id: repo_id)
record.update!(active: false) record.update!(active: false)
end end
@@ -55,19 +49,23 @@ module WarpEngine
private private
def sync_single(repo_id) def sync_single(repo_id)
remote = @client.get_repo(repo_id) remote = @ci.repo(repo_id)
record = Pipeline.unscoped.find_or_initialize_by(woodpecker_repo_id: repo_id) record = Pipeline.unscoped.find_or_initialize_by(woodpecker_repo_id: repo_id)
record.assign_attributes( record.assign_attributes(
repo_name: remote["name"], repo_owner: remote["owner"], repo_name: remote.name, repo_owner: remote.owner,
active: remote["active"], deleted_at: nil active: remote.active?, deleted_at: nil
) )
if record.platform.blank? assign_platform(record, remote.name)
sw = Software.find_by(name: remote["name"])
record.platform = sw&.platform || Pipeline::UNKNOWN_PLATFORM
record.software = sw if sw
end
record.save! record.save!
record record
end end
def assign_platform(record, repo_name)
return unless record.platform.blank? || record.platform == Pipeline::UNKNOWN_PLATFORM
sw = Software.find_by(name: repo_name)
record.platform = sw&.platform || Pipeline::UNKNOWN_PLATFORM
record.software = sw if sw
end
end end
end end
+13 -22
View File
@@ -2,21 +2,17 @@ module WarpEngine
class SecretSyncService class SecretSyncService
SECRET_NAME = "application_token".freeze SECRET_NAME = "application_token".freeze
def initialize(client: WoodpeckerClient.new) def initialize(ci: WarpEngine.ci)
@client = client @ci = ci
end end
def provision(plain_token, pipelines:) def provision(plain_token, pipelines:)
results = { synced: [], failed: [] } results = { synced: [], failed: [] }
pipelines.each do |pipeline| Array(pipelines).each do |pipeline|
if secret_exists?(pipeline.woodpecker_repo_id) @ci.secret_set(pipeline.remote_repo_id, name: SECRET_NAME, value: plain_token)
@client.update_secret(pipeline.woodpecker_repo_id, SECRET_NAME, value: plain_token)
else
@client.create_secret(pipeline.woodpecker_repo_id, name: SECRET_NAME, value: plain_token)
end
results[:synced] << pipeline results[:synced] << pipeline
rescue WoodpeckerClient::ApiError, WoodpeckerClient::ConnectionError => e rescue CI::Error => e
Rails.logger.error("[SecretSyncService] failed for #{pipeline.full_name}: #{e.message}") Rails.logger.error("[SecretSyncService] failed for #{pipeline.full_name}: #{e.message}")
results[:failed] << { pipeline: pipeline, error: e.message } results[:failed] << { pipeline: pipeline, error: e.message }
end end
@@ -24,14 +20,18 @@ module WarpEngine
results results
end end
def deprovision(application_token) def remove(pipelines:)
pipelines_for_token(application_token).each do |pipeline| Array(pipelines).each do |pipeline|
@client.delete_secret(pipeline.woodpecker_repo_id, SECRET_NAME) @ci.secret_delete(pipeline.remote_repo_id, SECRET_NAME)
rescue WoodpeckerClient::ApiError => e rescue CI::Error => e
Rails.logger.warn("[SecretSyncService] delete failed for #{pipeline.full_name}: #{e.message}") Rails.logger.warn("[SecretSyncService] delete failed for #{pipeline.full_name}: #{e.message}")
end end
end end
def deprovision(application_token)
remove(pipelines: pipelines_for_token(application_token))
end
def rotate(application_token) def rotate(application_token)
pipelines = pipelines_for_token(application_token) pipelines = pipelines_for_token(application_token)
return { rotated: false, reason: "no pipelines" } if pipelines.empty? return { rotated: false, reason: "no pipelines" } if pipelines.empty?
@@ -67,14 +67,5 @@ module WarpEngine
Pipeline.active.where(software_id: software_ids).to_a Pipeline.active.where(software_id: software_ids).to_a
end end
end end
private
def secret_exists?(repo_id)
secrets = @client.list_secrets(repo_id)
secrets.any? { |s| s["name"] == SECRET_NAME }
rescue WoodpeckerClient::ApiError
false
end
end end
end end
@@ -1,153 +0,0 @@
require "net/http"
require "json"
require "uri"
module WarpEngine
class WoodpeckerClient
class ApiError < StandardError
attr_reader :status, :body
def initialize(message, status:, body: nil)
super(message)
@status = status
@body = body
end
end
class ConnectionError < StandardError; end
def initialize(base_url: nil, token: nil)
@base_url = (base_url || WarpEngine.config.woodpecker_url).to_s.chomp("/")
@token = token || WarpEngine.config.woodpecker_api_token
end
def list_repos
get("/api/repos")
end
def get_repo(repo_id)
get("/api/repos/#{repo_id}")
end
def activate_repo(repo_id)
post("/api/repos", body: { id: repo_id })
end
def deactivate_repo(repo_id)
delete("/api/repos/#{repo_id}")
end
def list_secrets(repo_id)
get("/api/repos/#{repo_id}/secrets")
end
def create_secret(repo_id, name:, value:, events: %w[push tag deployment])
post("/api/repos/#{repo_id}/secrets",
body: { name: name, value: value, events: events })
end
def update_secret(repo_id, secret_name, value:)
patch("/api/repos/#{repo_id}/secrets/#{secret_name}",
body: { value: value })
end
def delete_secret(repo_id, secret_name)
delete("/api/repos/#{repo_id}/secrets/#{secret_name}")
end
def list_pipelines(repo_id, page: 1, per_page: 25)
get("/api/repos/#{repo_id}/pipelines",
params: { page: page, perPage: per_page })
end
def latest_pipeline(repo_id)
get("/api/repos/#{repo_id}/pipelines/latest")
end
def get_pipeline(repo_id, number)
get("/api/repos/#{repo_id}/pipelines/#{number}")
end
def trigger_pipeline(repo_id, branch: "main")
post("/api/repos/#{repo_id}/pipelines",
body: { branch: branch })
end
private
def get(path, params: {})
uri = build_uri(path, params)
request = Net::HTTP::Get.new(uri)
execute(uri, request)
end
def post(path, body: {})
uri = build_uri(path)
request = Net::HTTP::Post.new(uri)
request.body = body.to_json
request.content_type = "application/json"
execute(uri, request)
end
def patch(path, body: {})
uri = build_uri(path)
request = Net::HTTP::Patch.new(uri)
request.body = body.to_json
request.content_type = "application/json"
execute(uri, request)
end
def delete(path)
uri = build_uri(path)
request = Net::HTTP::Delete.new(uri)
execute(uri, request)
end
def build_uri(path, params = {})
uri = URI.parse("#{@base_url}#{path}")
uri.query = URI.encode_www_form(params) if params.any?
uri
end
def execute(uri, request)
request["Authorization"] = "Bearer #{@token}"
request["Accept"] = "application/json"
response = Net::HTTP.start(uri.hostname, uri.port,
use_ssl: uri.scheme == "https",
open_timeout: 10,
read_timeout: 30) do |http|
http.request(request)
end
handle_response(uri, response)
rescue Errno::ECONNREFUSED, Errno::EHOSTUNREACH, Net::OpenTimeout,
Net::ReadTimeout, SocketError => e
raise ConnectionError, "Cannot reach Woodpecker at #{@base_url}: #{e.message}"
end
def handle_response(uri, response)
case response
when Net::HTTPSuccess, Net::HTTPNoContent
return nil if response.body.blank?
begin
JSON.parse(response.body)
rescue JSON::ParserError
raise ApiError.new(
"Expected JSON from #{uri.path} but got: #{response.body.truncate(80)}",
status: response.code.to_i, body: response.body
)
end
when Net::HTTPNotFound
raise ApiError.new("Not found: #{uri.path}", status: 404, body: response.body)
else
raise ApiError.new(
"Woodpecker API error #{response.code}: #{response.body&.truncate(200)}",
status: response.code.to_i,
body: response.body
)
end
end
end
end
-1
View File
@@ -10,7 +10,6 @@ WarpEngine::Engine.routes.draw do
get "software", to: "software#index" get "software", to: "software#index"
get "software/highlighted", to: "software_highlighted#index" get "software/highlighted", to: "software_highlighted#index"
get "image/:id", to: "images#show"
get "download", to: "downloads#show" get "download", to: "downloads#show"
get "builds", to: "builds#index" get "builds", to: "builds#index"
get "softwares/:name/builds", to: "software_builds#show" get "softwares/:name/builds", to: "software_builds#show"
@@ -0,0 +1,16 @@
module Api
class ImagesController < ActionController::API
rescue_from ActiveRecord::RecordNotFound do
render json: { error: "Not found" }, status: :not_found
end
rescue_from Errno::ENOENT do
render json: { error: "Not found" }, status: :not_found
end
def show
image = Image.find(params[:id])
send_file image.file_path, type: image.content_type, disposition: "inline"
end
end
end
@@ -0,0 +1,32 @@
class Image < ApplicationRecord
def self.upload_path
ENV.fetch("IMAGE_CONTAINER_PATH", "/images")
end
has_many :software_images, class_name: "WarpEngine::SoftwareImage", dependent: :restrict_with_error
default_scope { where(deleted_at: nil) }
attr_accessor :file_upload
before_save :process_upload, if: -> { file_upload.present? }
def self.ransackable_attributes(auth_object = nil)
%w[content_type created_at deleted_at filename id original_filename updated_at]
end
def file_path
File.join(self.class.upload_path, filename.to_s)
end
private
def process_upload
FileUtils.mkdir_p(self.class.upload_path)
self.original_filename = file_upload.original_filename
self.content_type = file_upload.content_type.presence || "application/octet-stream"
ext = File.extname(file_upload.original_filename)
self.filename = "#{SecureRandom.uuid}#{ext}"
IO.copy_stream(file_upload.to_io, file_path)
end
end
@@ -1,8 +1,15 @@
Rails.application.config.to_prepare do Rails.application.config.to_prepare do
WarpEngine.configure do |c| WarpEngine.configure do |c|
c.file_container_path = ENV.fetch("FILE_CONTAINER_PATH", "/softwares") c.file_container_path = ENV.fetch("FILE_CONTAINER_PATH", "/softwares")
c.image_container_path = ENV.fetch("IMAGE_CONTAINER_PATH", "/images")
c.update_secret = ENV["UPDATE_SECRET"] c.update_secret = ENV["UPDATE_SECRET"]
c.image_class_name = "Image"
c.ci_adapter = WarpEngine::CI::Woodpecker::Adapter.new(
url: ENV["WOODPECKER_URL"],
api_token: ENV["WOODPECKER_API_TOKEN"],
repo_owner: ENV["WOODPECKER_REPO_OWNER"]
)
end end
end end
@@ -1,5 +1,9 @@
Rails.application.routes.draw do Rails.application.routes.draw do
apipie apipie
namespace :api do
get "image/:id", to: "images#show"
end
mount WarpEngine::Engine => "/" mount WarpEngine::Engine => "/"
end end
@@ -17,6 +17,12 @@ module WarpEngine
template "initializer.rb", "config/initializers/warp_engine.rb" template "initializer.rb", "config/initializers/warp_engine.rb"
end end
def copy_image_library
template "image.rb", "app/models/image.rb"
template "images_controller.rb", "app/controllers/api/images_controller.rb"
migration_template "create_images.rb", "db/migrate/create_images.rb"
end
def copy_migration def copy_migration
migration_template "create_warp_engine_tables.rb", "db/migrate/create_warp_engine_tables.rb" migration_template "create_warp_engine_tables.rb", "db/migrate/create_warp_engine_tables.rb"
end end
@@ -27,7 +33,11 @@ module WarpEngine
WarpEngine telepítve. Következő lépések: WarpEngine telepítve. Következő lépések:
1. rails db:migrate 1. rails db:migrate
2. mount WarpEngine::Engine => "/" a config/routes.rb végére 2. mount WarpEngine::Engine => "/" a config/routes.rb végére
3. állítsd be a config/initializers/warp_engine.rb-t 3. a képkönyvtár a hosztod: a routes.rb-be
namespace :api do
get "image/:id", to: "images#show"
end
4. állítsd be a config/initializers/warp_engine.rb-t
MSG MSG
end end
end end
@@ -0,0 +1,12 @@
class CreateImages < ActiveRecord::Migration[8.0]
def change
create_table :images do |t|
t.string :filename, null: false
t.string :original_filename, null: false
t.string :content_type, default: "application/octet-stream", null: false
t.datetime :deleted_at, precision: 3
t.timestamps
t.index :deleted_at
end
end
end
@@ -60,21 +60,13 @@ class CreateWarpEngineTables < ActiveRecord::Migration[8.0]
t.index :deleted_at t.index :deleted_at
end end
create_table :images do |t|
t.string :filename, null: false
t.string :original_filename, null: false
t.string :content_type, default: "application/octet-stream", null: false
t.datetime :deleted_at, precision: 3
t.timestamps
t.index :deleted_at
end
create_table :software_images do |t| create_table :software_images do |t|
t.references :software, null: false, foreign_key: { on_delete: :cascade } t.references :software, null: false, foreign_key: { on_delete: :cascade }
t.references :image, null: false, foreign_key: true, index: false t.bigint :image_id, null: false
t.boolean :is_default, default: false, null: false t.boolean :is_default, default: false, null: false
t.integer :position, default: 0, null: false t.integer :position, default: 0, null: false
t.timestamps t.timestamps
t.index :image_id
t.index [ :software_id, :image_id ], unique: true t.index [ :software_id, :image_id ], unique: true
t.index [ :software_id, :position ] t.index [ :software_id, :position ]
end end
@@ -0,0 +1,32 @@
class Image < ApplicationRecord
def self.upload_path
ENV.fetch("IMAGE_CONTAINER_PATH", "/images")
end
has_many :software_images, class_name: "WarpEngine::SoftwareImage", dependent: :restrict_with_error
default_scope { where(deleted_at: nil) }
attr_accessor :file_upload
before_save :process_upload, if: -> { file_upload.present? }
def self.ransackable_attributes(auth_object = nil)
%w[content_type created_at deleted_at filename id original_filename updated_at]
end
def file_path
File.join(self.class.upload_path, filename.to_s)
end
private
def process_upload
FileUtils.mkdir_p(self.class.upload_path)
self.original_filename = file_upload.original_filename
self.content_type = file_upload.content_type.presence || "application/octet-stream"
ext = File.extname(file_upload.original_filename)
self.filename = "#{SecureRandom.uuid}#{ext}"
IO.copy_stream(file_upload.to_io, file_path)
end
end
@@ -0,0 +1,16 @@
module Api
class ImagesController < ActionController::API
rescue_from ActiveRecord::RecordNotFound do
render json: { error: "Not found" }, status: :not_found
end
rescue_from Errno::ENOENT do
render json: { error: "Not found" }, status: :not_found
end
def show
image = Image.find(params[:id])
send_file image.file_path, type: image.content_type, disposition: "inline"
end
end
end
@@ -1,4 +1,17 @@
Rails.application.config.to_prepare do Rails.application.config.to_prepare do
WarpEngine.configure do |c| WarpEngine.configure do |c|
c.file_container_path = ENV.fetch("FILE_CONTAINER_PATH", "/softwares")
c.update_secret = ENV["UPDATE_SECRET"]
c.image_class_name = "Image"
c.ci_adapter = WarpEngine::CI::Woodpecker::Adapter.new(
url: ENV["WOODPECKER_URL"],
api_token: ENV["WOODPECKER_API_TOKEN"],
repo_owner: ENV["WOODPECKER_REPO_OWNER"],
public_key_url: ENV["WOODPECKER_PUBLIC_KEY_URL"],
platforms: {}
)
end end
end end
+10 -4
View File
@@ -7,6 +7,8 @@ require "warp_engine/version"
require "warp_engine/configuration" require "warp_engine/configuration"
require "warp_engine/storage" require "warp_engine/storage"
require "warp_engine/access" require "warp_engine/access"
require "warp_engine/images"
require "warp_engine/ci"
module WarpEngine module WarpEngine
@@ -22,10 +24,6 @@ module WarpEngine
yield(config) yield(config)
end end
def self.woodpecker_configured?
config.woodpecker_url.present? && config.woodpecker_api_token.present?
end
def self.instruments_publish? def self.instruments_publish?
true true
end end
@@ -34,6 +32,14 @@ module WarpEngine
Storage.adapter Storage.adapter
end end
def self.images
Images.adapter
end
def self.ci
CI.adapter
end
def self.access_policy def self.access_policy
AccessPolicy.current AccessPolicy.current
end end
+94
View File
@@ -0,0 +1,94 @@
module WarpEngine
module CI
class Error < StandardError; end
class ConnectionError < Error; end
class NotConfigured < Error; end
class ApiError < Error
attr_reader :status, :body
def initialize(message, status: nil, body: nil)
super(message)
@status = status
@body = body
end
end
Repo = Struct.new(:id, :name, :owner, :active, :raw, keyword_init: true) do
def active? = active ? true : false
def full_name = "#{owner}/#{name}"
end
Run = Struct.new(:number, :status, :branch, :message, :created_at, :url, :raw, keyword_init: true) do
def success? = status.to_s == "success"
def as_json(*)
{
number: number,
status: status,
branch: branch,
message: message,
createdAt: created_at&.utc&.iso8601,
url: url
}
end
end
class Null
MESSAGE = "no CI adapter is configured (WarpEngine.config.ci_adapter)".freeze
def name = "none"
def configured? = false
def platforms = {}
def update_server = nil
def repos = raise(NotConfigured, MESSAGE)
def repo(_id) = raise(NotConfigured, MESSAGE)
def activate_repo(_id) = raise(NotConfigured, MESSAGE)
def deactivate_repo(_id) = raise(NotConfigured, MESSAGE)
def runs(_repo_id, page: 1) = raise(NotConfigured, MESSAGE)
def run(_repo_id, _number) = raise(NotConfigured, MESSAGE)
def trigger(_repo_id, branch: nil) = raise(NotConfigured, MESSAGE)
def secret_names(_repo_id) = raise(NotConfigured, MESSAGE)
def secret_set(_repo_id, name:, value:) = raise(NotConfigured, MESSAGE)
def secret_delete(_repo_id, _name) = raise(NotConfigured, MESSAGE)
def verify_config_request(_request) = false
def config_marker(_params) = nil
def pipeline_config(platform:, name:, update_server:) = nil
def config_response(platform:, config:) = {}
end
class << self
def adapter
configured = WarpEngine.config.ci_adapter
case configured
when nil, :woodpecker, "woodpecker" then woodpecker_adapter
when :none, "none", :null then null_adapter
else configured
end
end
def woodpecker? = adapter.is_a?(Woodpecker::Adapter)
def woodpecker_adapter
@woodpecker_adapter ||= Woodpecker::Adapter.new
end
def null_adapter
@null_adapter ||= Null.new
end
def reset!
@woodpecker_adapter = nil
@null_adapter = nil
end
end
end
end
require "warp_engine/ci/woodpecker"
+4
View File
@@ -0,0 +1,4 @@
require "warp_engine/ci/woodpecker/client"
require "warp_engine/ci/woodpecker/signature_verifier"
require "warp_engine/ci/woodpecker/pipeline_config"
require "warp_engine/ci/woodpecker/adapter"
+146
View File
@@ -0,0 +1,146 @@
module WarpEngine
module CI
module Woodpecker
class Adapter
SECRET_EVENTS = %w[push tag deployment].freeze
attr_reader :url, :repo_owner, :update_server
def initialize(url: ENV["WOODPECKER_URL"],
api_token: ENV["WOODPECKER_API_TOKEN"],
repo_owner: ENV["WOODPECKER_REPO_OWNER"],
public_key: nil,
public_key_url: nil,
platforms: {},
update_server: nil)
@url = url.presence
@api_token = api_token.presence
@repo_owner = repo_owner.presence
@public_key = public_key.presence
@public_key_url = public_key_url.presence
@update_server = update_server.presence
@config = PipelineConfig.new(platforms: platforms || {})
end
def name = "Woodpecker"
def configured? = @url.present? && @api_token.present?
def platforms = @config.platforms
def client
@client ||= Client.new(url: @url, token: @api_token)
end
def repos
Array(client.list_repos).map { |remote| to_repo(remote) }
end
def repo(repo_id)
to_repo(client.get_repo(repo_id))
end
def activate_repo(repo_id)
client.activate_repo(repo_id)
nil
end
def deactivate_repo(repo_id)
client.deactivate_repo(repo_id)
nil
end
def runs(repo_id, page: 1)
Array(client.list_pipelines(repo_id, page: page)).map { |run| to_run(run) }
end
def run(repo_id, number)
remote = if number.to_s == "latest"
client.latest_pipeline(repo_id)
else
client.get_pipeline(repo_id, number)
end
remote && to_run(remote)
end
def trigger(repo_id, branch: "main")
to_run(client.trigger_pipeline(repo_id, branch: branch || "main"))
end
def secret_names(repo_id)
Array(client.list_secrets(repo_id)).map { |secret| secret["name"] }
end
def secret_set(repo_id, name:, value:)
if secret_names(repo_id).include?(name)
client.update_secret(repo_id, name, value: value)
else
client.create_secret(repo_id, name: name, value: value, events: SECRET_EVENTS)
end
nil
end
def secret_delete(repo_id, name)
client.delete_secret(repo_id, name)
nil
end
def verify_config_request(request)
SignatureVerifier.new(request, public_key: @public_key, public_key_url: @public_key_url).valid?
end
def config_marker(params)
configs = params[:configuration].presence || params[:configs].presence || []
Array(configs).each do |config|
parsed = parse_marker(config[:data].to_s)
next unless parsed
return { platform: parsed["platform"].to_s,
name: parsed["name"].presence || params.dig(:repo, :name).to_s }
end
nil
end
def pipeline_config(platform:, name:, update_server:)
@config.render(platform: platform, name: name, update_server: update_server)
end
def config_response(platform:, config:)
{ configs: [ { name: platform, data: config } ] }
end
private
def parse_marker(data)
parsed = begin
YAML.safe_load(data)
rescue Psych::Exception
nil
end
parsed if parsed.is_a?(Hash) && parsed.key?("platform")
end
def to_repo(remote)
return nil if remote.nil?
Repo.new(id: remote["id"], name: remote["name"], owner: remote["owner"],
active: remote["active"], raw: remote)
end
def to_run(remote)
return nil if remote.nil?
Run.new(
number: remote["number"],
status: remote["status"],
branch: remote["branch"],
message: remote["message"],
created_at: remote["created"] ? Time.zone.at(remote["created"]) : nil,
url: remote["forge_url"] || remote["link_url"],
raw: remote
)
end
end
end
end
end
+143
View File
@@ -0,0 +1,143 @@
require "net/http"
require "json"
require "uri"
module WarpEngine
module CI
module Woodpecker
class Client
def initialize(url:, token:)
@base_url = url.to_s.chomp("/")
@token = token
end
def list_repos
get("/api/repos")
end
def get_repo(repo_id)
get("/api/repos/#{repo_id}")
end
def activate_repo(repo_id)
post("/api/repos", body: { id: repo_id })
end
def deactivate_repo(repo_id)
delete("/api/repos/#{repo_id}")
end
def list_secrets(repo_id)
get("/api/repos/#{repo_id}/secrets")
end
def create_secret(repo_id, name:, value:, events: %w[push tag deployment])
post("/api/repos/#{repo_id}/secrets",
body: { name: name, value: value, events: events })
end
def update_secret(repo_id, secret_name, value:)
patch("/api/repos/#{repo_id}/secrets/#{secret_name}",
body: { value: value })
end
def delete_secret(repo_id, secret_name)
delete("/api/repos/#{repo_id}/secrets/#{secret_name}")
end
def list_pipelines(repo_id, page: 1, per_page: 25)
get("/api/repos/#{repo_id}/pipelines",
params: { page: page, perPage: per_page })
end
def latest_pipeline(repo_id)
get("/api/repos/#{repo_id}/pipelines/latest")
end
def get_pipeline(repo_id, number)
get("/api/repos/#{repo_id}/pipelines/#{number}")
end
def trigger_pipeline(repo_id, branch: "main")
post("/api/repos/#{repo_id}/pipelines",
body: { branch: branch })
end
private
def get(path, params: {})
uri = build_uri(path, params)
execute(uri, Net::HTTP::Get.new(uri))
end
def post(path, body: {})
uri = build_uri(path)
request = Net::HTTP::Post.new(uri)
request.body = body.to_json
request.content_type = "application/json"
execute(uri, request)
end
def patch(path, body: {})
uri = build_uri(path)
request = Net::HTTP::Patch.new(uri)
request.body = body.to_json
request.content_type = "application/json"
execute(uri, request)
end
def delete(path)
uri = build_uri(path)
execute(uri, Net::HTTP::Delete.new(uri))
end
def build_uri(path, params = {})
uri = URI.parse("#{@base_url}#{path}")
uri.query = URI.encode_www_form(params) if params.any?
uri
end
def execute(uri, request)
request["Authorization"] = "Bearer #{@token}"
request["Accept"] = "application/json"
response = Net::HTTP.start(uri.hostname, uri.port,
use_ssl: uri.scheme == "https",
open_timeout: 10,
read_timeout: 30) do |http|
http.request(request)
end
handle_response(uri, response)
rescue Errno::ECONNREFUSED, Errno::EHOSTUNREACH, Net::OpenTimeout,
Net::ReadTimeout, SocketError => e
raise CI::ConnectionError, "Cannot reach Woodpecker at #{@base_url}: #{e.message}"
end
def handle_response(uri, response)
case response
when Net::HTTPSuccess, Net::HTTPNoContent
return nil if response.body.blank?
begin
JSON.parse(response.body)
rescue JSON::ParserError
raise CI::ApiError.new(
"Expected JSON from #{uri.path} but got: #{response.body.truncate(80)}",
status: response.code.to_i, body: response.body
)
end
when Net::HTTPNotFound
raise CI::ApiError.new("Not found: #{uri.path}", status: 404, body: response.body)
else
raise CI::ApiError.new(
"Woodpecker API error #{response.code}: #{response.body&.truncate(200)}",
status: response.code.to_i,
body: response.body
)
end
end
end
end
end
end
@@ -0,0 +1,42 @@
require "erb"
require "pathname"
module WarpEngine
module CI
module Woodpecker
class PipelineConfig
PLATFORM_FORMAT = /\A[a-z0-9_-]+\z/
def initialize(platforms: {})
@platforms = platforms.to_h { |key, spec| [ key.to_s, spec.to_h.symbolize_keys ] }
end
def platforms
@platforms
end
def render(platform:, name:, update_server:)
platform = platform.to_s
return nil unless platform.match?(PLATFORM_FORMAT)
spec = @platforms[platform]
return nil if spec.nil?
path = self.class.templates_dir.join(platform, "pipeline.yaml.erb")
return nil unless path.exist?
ERB.new(path.read, trim_mode: "-").result_with_hash(
name: name.to_s,
update_server: update_server.to_s,
builder: spec[:builder],
exporter: spec[:exporter]
)
end
def self.templates_dir
Pathname.new(__dir__).join("platforms")
end
end
end
end
end
@@ -0,0 +1,145 @@
require "openssl"
require "base64"
require "net/http"
require "digest"
module WarpEngine
module CI
module Woodpecker
class SignatureVerifier
CAVAGE_PARAM = /(\w+)="([^"]*)"/
@key_cache = {}
@key_mutex = Mutex.new
class << self
def fetch_public_key(url)
@key_mutex.synchronize do
@key_cache[url] ||= Net::HTTP.get(URI.parse(url))
end
end
def reset_key_cache!
@key_mutex.synchronize { @key_cache = {} }
end
end
def initialize(request, public_key: nil, public_key_url: nil)
@request = request
@public_key = public_key
@public_key_url = public_key_url
end
def valid?
pem = public_key_pem
if pem.blank?
Rails.logger.error("[CI::Woodpecker] no public key configured — rejecting request")
return false
end
key = OpenSSL::PKey.read(pem)
if @request.headers["Signature-Input"].present?
rfc9421_valid?(key)
else
cavage_valid?(key)
end
rescue OpenSSL::PKey::PKeyError, ArgumentError => e
Rails.logger.error("[CI::Woodpecker] #{e.class}: #{e.message}")
false
end
private
def public_key_pem
return @public_key if @public_key.present?
return nil if @public_key_url.blank?
self.class.fetch_public_key(@public_key_url)
rescue StandardError => e
Rails.logger.error("[CI::Woodpecker] public key fetch failed: #{e.class}: #{e.message}")
nil
end
def rfc9421_valid?(key)
input = @request.headers["Signature-Input"].to_s
match = input.match(/\A\s*([\w.-]+)=(\(.*)\z/m)
return false if match.nil?
label, inner = match[1], match[2]
components = inner[/\((.*?)\)/m, 1].to_s.scan(/"([^"]*)"/).flatten
return false if components.empty?
signature = @request.headers["Signature"].to_s[/#{Regexp.escape(label)}=:([A-Za-z0-9+\/=]+):/, 1]
return false if signature.blank?
return false unless content_digest_valid?(components)
lines = components.map do |component|
value = component_value(component)
return false if value.nil?
%("#{component}": #{value})
end
lines << %("@signature-params": #{inner})
key.verify(nil, Base64.decode64(signature), lines.join("\n"))
end
def component_value(name)
case name
when "@request-target" then @request.fullpath
when "@method" then @request.request_method
when "@target-uri" then @request.original_url
when "@authority" then @request.host_with_port
when "@path" then @request.path
when "@query" then "?#{@request.query_string}"
when /\A@/ then nil
else @request.headers[name]
end
end
def content_digest_valid?(components)
return true unless components.include?("content-digest")
digest = @request.headers["Content-Digest"].to_s[/sha-256=:([A-Za-z0-9+\/=]+):/, 1]
return false if digest.blank?
expected = Digest::SHA256.base64digest(@request.raw_post)
ActiveSupport::SecurityUtils.secure_compare(digest, expected)
end
def cavage_valid?(key)
params = cavage_params
return false if params.nil? || params["signature"].blank?
signing_string = cavage_signing_string(params.fetch("headers", "date"))
return false if signing_string.nil?
key.verify(nil, Base64.decode64(params["signature"]), signing_string)
end
def cavage_params
header = @request.headers["Signature"].presence
if header.nil?
auth = @request.headers["Authorization"].to_s
header = auth.delete_prefix("Signature ") if auth.start_with?("Signature ")
end
return nil if header.blank?
header.scan(CAVAGE_PARAM).to_h
end
def cavage_signing_string(headers_list)
lines = headers_list.split(" ").map do |name|
if name == "(request-target)"
"(request-target): #{@request.request_method.downcase} #{@request.fullpath}"
else
value = @request.headers[name]
return nil if value.nil?
"#{name.downcase}: #{value}"
end
end
lines.join("\n")
end
end
end
end
end
+6 -18
View File
@@ -2,20 +2,14 @@ module WarpEngine
class Configuration class Configuration
attr_accessor :file_container_path, attr_accessor :file_container_path,
:image_container_path,
:update_secret, :update_secret,
:application_token_source, :application_token_source,
:application_token_owner_class, :application_token_owner_class,
:max_upload_size, :max_upload_size,
:enforce_software_ownership, :enforce_software_ownership,
:ci_platforms, :ci_adapter,
:ci_extension_public_key, :image_class_name,
:ci_extension_public_key_url, :image_adapter,
:ci_update_server,
:woodpecker_url,
:woodpecker_api_token,
:woodpecker_repo_owner,
:image_owners,
:storage_adapter, :storage_adapter,
:access_policy, :access_policy,
:access_token_owner_class, :access_token_owner_class,
@@ -26,20 +20,14 @@ module WarpEngine
def initialize def initialize
@file_container_path = ENV.fetch("FILE_CONTAINER_PATH", "/softwares") @file_container_path = ENV.fetch("FILE_CONTAINER_PATH", "/softwares")
@image_container_path = ENV.fetch("IMAGE_CONTAINER_PATH", "/images")
@update_secret = ENV["UPDATE_SECRET"] @update_secret = ENV["UPDATE_SECRET"]
@application_token_source = :env @application_token_source = :env
@application_token_owner_class = nil @application_token_owner_class = nil
@max_upload_size = 500 * 1024 * 1024 @max_upload_size = 500 * 1024 * 1024
@enforce_software_ownership = false @enforce_software_ownership = false
@ci_platforms = {} @ci_adapter = :woodpecker
@ci_extension_public_key = nil @image_class_name = "Image"
@ci_extension_public_key_url = nil @image_adapter = nil
@ci_update_server = nil
@woodpecker_url = ENV["WOODPECKER_URL"]
@woodpecker_api_token = ENV["WOODPECKER_API_TOKEN"]
@woodpecker_repo_owner = ENV["WOODPECKER_REPO_OWNER"]
@image_owners = []
@storage_adapter = :local @storage_adapter = :local
@access_policy = :open @access_policy = :open
@access_token_owner_class = nil @access_token_owner_class = nil
+68
View File
@@ -0,0 +1,68 @@
module WarpEngine
module Images
class HostModel
URL_PREFIX = "/api/image".freeze
def model_name
WarpEngine.config.image_class_name
end
def model
model_name.to_s.constantize
end
def url_for(image_id)
image_id.present? ? "#{URL_PREFIX}/#{image_id}" : nil
end
def build_from_upload(upload)
model.new(file_upload: upload)
end
def label_for(record)
record.original_filename
end
def available?(record)
return true unless record.respond_to?(:file_path)
File.exist?(record.file_path.to_s)
end
def select_options
model.order(:original_filename).map { |record| [ label_for(record), record.id ] }
end
end
class << self
def adapter
configured = WarpEngine.config.image_adapter
case configured
when nil, :host_model, "host_model" then host_model_adapter
else configured
end
end
def host_model? = adapter.is_a?(HostModel)
def host_model_adapter
@host_model_adapter ||= HostModel.new
end
def reset!
@host_model_adapter = nil
end
def model_name = adapter.model_name
def model = adapter.model
def url_for(image_id) = adapter.url_for(image_id)
def build_from_upload(upload) = adapter.build_from_upload(upload)
def label_for(record) = adapter.label_for(record)
def available?(record) = adapter.available?(record)
def select_options = adapter.select_options
end
end
end
+1 -1
View File
@@ -1,5 +1,5 @@
module WarpEngine module WarpEngine
VERSION = "0.5.2" VERSION = "0.7.0"
VERSION_HEADER = "WarpEngine-Version".freeze VERSION_HEADER = "WarpEngine-Version".freeze
end end
@@ -1,10 +1,10 @@
require "rails_helper" require "rails_helper"
require "webmock/rspec" require "webmock/rspec"
RSpec.describe WarpEngine::WoodpeckerClient do RSpec.describe WarpEngine::CI::Woodpecker::Client do
let(:base_url) { "https://ci.example.test" } let(:base_url) { "https://ci.example.test" }
let(:token) { "wp-test-token" } let(:token) { "wp-test-token" }
let(:client) { described_class.new(base_url: base_url, token: token) } let(:client) { described_class.new(url: base_url, token: token) }
def stub_wp(method, path, status: 200, body: nil, request_body: nil) def stub_wp(method, path, status: 200, body: nil, request_body: nil)
stub = stub_request(method, "#{base_url}#{path}") stub = stub_request(method, "#{base_url}#{path}")
@@ -91,7 +91,7 @@ RSpec.describe WarpEngine::WoodpeckerClient do
it "raises ApiError on 404" do it "raises ApiError on 404" do
stub_wp(:get, "/api/repos/999", status: 404, body: { "error" => "not found" }) stub_wp(:get, "/api/repos/999", status: 404, body: { "error" => "not found" })
expect { client.get_repo(999) }.to raise_error(WarpEngine::WoodpeckerClient::ApiError) { |e| expect { client.get_repo(999) }.to raise_error(WarpEngine::CI::ApiError) { |e|
expect(e.status).to eq(404) expect(e.status).to eq(404)
} }
end end
@@ -99,7 +99,7 @@ RSpec.describe WarpEngine::WoodpeckerClient do
it "raises ApiError on 500" do it "raises ApiError on 500" do
stub_wp(:get, "/api/repos", status: 500, body: { "error" => "internal" }) stub_wp(:get, "/api/repos", status: 500, body: { "error" => "internal" })
expect { client.list_repos }.to raise_error(WarpEngine::WoodpeckerClient::ApiError) { |e| expect { client.list_repos }.to raise_error(WarpEngine::CI::ApiError) { |e|
expect(e.status).to eq(500) expect(e.status).to eq(500)
} }
end end
@@ -107,13 +107,13 @@ RSpec.describe WarpEngine::WoodpeckerClient do
it "raises ConnectionError on connection refused" do it "raises ConnectionError on connection refused" do
stub_request(:get, "#{base_url}/api/repos").to_raise(Errno::ECONNREFUSED) stub_request(:get, "#{base_url}/api/repos").to_raise(Errno::ECONNREFUSED)
expect { client.list_repos }.to raise_error(WarpEngine::WoodpeckerClient::ConnectionError) expect { client.list_repos }.to raise_error(WarpEngine::CI::ConnectionError)
end end
it "raises ConnectionError on timeout" do it "raises ConnectionError on timeout" do
stub_request(:get, "#{base_url}/api/repos").to_timeout stub_request(:get, "#{base_url}/api/repos").to_timeout
expect { client.list_repos }.to raise_error(WarpEngine::WoodpeckerClient::ConnectionError) expect { client.list_repos }.to raise_error(WarpEngine::CI::ConnectionError)
end end
it "raises ApiError when a 200 response is not JSON" do it "raises ApiError when a 200 response is not JSON" do
@@ -121,7 +121,7 @@ RSpec.describe WarpEngine::WoodpeckerClient do
.to_return(status: 200, body: "<!doctype html><html></html>", .to_return(status: 200, body: "<!doctype html><html></html>",
headers: { "Content-Type" => "text/html" }) headers: { "Content-Type" => "text/html" })
expect { client.list_repos }.to raise_error(WarpEngine::WoodpeckerClient::ApiError, /Expected JSON/) expect { client.list_repos }.to raise_error(WarpEngine::CI::ApiError, /Expected JSON/)
end end
end end
end end
+136
View File
@@ -0,0 +1,136 @@
require "rails_helper"
RSpec.describe WarpEngine::CI do
after do
WarpEngine.config.ci_adapter = :woodpecker
WarpEngine::CI.reset!
end
describe ".adapter" do
it "drives Woodpecker unless the host says otherwise" do
expect(WarpEngine.ci).to be_a(WarpEngine::CI::Woodpecker::Adapter)
expect(WarpEngine.ci.name).to eq("Woodpecker")
end
it "answers with the null adapter for a host that runs no CI" do
WarpEngine.config.ci_adapter = :none
expect(WarpEngine.ci).to be_a(WarpEngine::CI::Null)
expect(WarpEngine.ci).not_to be_configured
end
it "hands back whatever object the host named" do
own = Class.new { def name = "Forge Runner" }.new
WarpEngine.config.ci_adapter = own
expect(WarpEngine.ci).to be(own)
end
end
describe WarpEngine::CI::Null do
let(:adapter) { described_class.new }
it "serves no platform and verifies no request" do
expect(adapter.platforms).to be_empty
expect(adapter.verify_config_request(nil)).to be(false)
expect(adapter.config_marker({})).to be_nil
expect(adapter.pipeline_config(platform: "godot", name: "x", update_server: "y")).to be_nil
end
it "raises rather than pretending to manage repositories" do
expect { adapter.repos }.to raise_error(WarpEngine::CI::NotConfigured)
expect { adapter.trigger(1) }.to raise_error(WarpEngine::CI::NotConfigured)
end
end
describe WarpEngine::CI::Woodpecker::Adapter do
let(:client) { instance_double(WarpEngine::CI::Woodpecker::Client) }
let(:adapter) do
described_class.new(url: "https://ci.test", api_token: "tok",
platforms: { "godot" => { builder: "registry.test/godot:1" } },
update_server: "https://games.test")
end
before { allow(adapter).to receive(:client).and_return(client) }
it "is inactive without a server and a token" do
expect(described_class.new(url: nil, api_token: nil)).not_to be_configured
expect(adapter).to be_configured
end
it "normalizes a repository" do
allow(client).to receive(:list_repos).and_return([
{ "id" => 7, "name" => "game", "owner" => "org", "active" => true }
])
repo = adapter.repos.first
expect(repo.id).to eq(7)
expect(repo.full_name).to eq("org/game")
expect(repo).to be_active
end
it "normalizes a run, timestamp included" do
allow(client).to receive(:trigger_pipeline).and_return(
{ "number" => 4, "status" => "success", "branch" => "main",
"message" => "ship it", "created" => 1_754_500_000 }
)
run = adapter.trigger(7, branch: "main")
expect(run.number).to eq(4)
expect(run).to be_success
expect(run.created_at).to eq(Time.zone.at(1_754_500_000))
expect(run.as_json[:createdAt]).to be_present
end
it "creates a secret the repository does not have yet" do
allow(client).to receive(:list_secrets).and_return([])
allow(client).to receive(:create_secret)
adapter.secret_set(7, name: "application_token", value: "plain")
expect(client).to have_received(:create_secret).with(
7, name: "application_token", value: "plain", events: described_class::SECRET_EVENTS
)
end
it "updates a secret the repository already has" do
allow(client).to receive(:list_secrets).and_return([ { "name" => "application_token" } ])
allow(client).to receive(:update_secret)
adapter.secret_set(7, name: "application_token", value: "plain")
expect(client).to have_received(:update_secret).with(7, "application_token", value: "plain")
end
it "reads the platform marker out of a config request" do
params = ActionController::Parameters.new(
repo: { name: "mygame" },
configuration: [ { name: ".woodpecker.yaml", data: "platform: godot\n" } ]
)
expect(adapter.config_marker(params)).to eq({ platform: "godot", name: "mygame" })
end
it "ignores a config request that is not a marker" do
params = ActionController::Parameters.new(
configuration: [ { name: ".woodpecker.yaml", data: "steps:\n - name: build\n" } ]
)
expect(adapter.config_marker(params)).to be_nil
end
it "renders only the platforms it was given" do
expect(adapter.pipeline_config(platform: "godot", name: "mygame",
update_server: "https://games.test")).to include("registry.test/godot:1")
expect(adapter.pipeline_config(platform: "amiga", name: "mygame",
update_server: "https://games.test")).to be_nil
end
it "shapes the response the way the CI server expects" do
expect(adapter.config_response(platform: "godot", config: "steps: []"))
.to eq({ configs: [ { name: "godot", data: "steps: []" } ] })
end
end
end
+27
View File
@@ -0,0 +1,27 @@
class Image < ActiveRecord::Base
def self.upload_path
Rails.root.join("tmp/images").to_s
end
has_many :software_images, class_name: "WarpEngine::SoftwareImage", dependent: :restrict_with_error
default_scope { where(deleted_at: nil) }
attr_accessor :file_upload
before_save :process_upload, if: -> { file_upload.present? }
def file_path
File.join(self.class.upload_path, filename.to_s)
end
private
def process_upload
FileUtils.mkdir_p(self.class.upload_path)
self.original_filename = file_upload.original_filename
self.content_type = file_upload.content_type.presence || "application/octet-stream"
self.filename = "#{SecureRandom.uuid}#{File.extname(file_upload.original_filename)}"
IO.copy_stream(file_upload.to_io, file_path)
end
end
+7
View File
@@ -0,0 +1,7 @@
FactoryBot.define do
factory :image, class: "Image" do
sequence(:filename) { |n| "image-#{n}.png" }
original_filename { "image.png" }
content_type { "image/png" }
end
end
+51
View File
@@ -0,0 +1,51 @@
require "rails_helper"
RSpec.describe WarpEngine::Images do
after { WarpEngine.config.image_adapter = nil }
describe "the default adapter" do
it "is the host model named in the configuration" do
expect(described_class.model_name).to eq("Image")
expect(described_class.model).to eq(Image)
end
it "serves images from the address the clients have always used" do
expect(described_class.url_for(7)).to eq("/api/image/7")
expect(described_class.url_for(nil)).to be_nil
end
it "offers the host's images to an admin select" do
image = create(:image, original_filename: "cover.png")
expect(described_class.select_options).to include([ "cover.png", image.id ])
end
it "builds a host image from an upload" do
expect(described_class.build_from_upload(nil)).to be_a(Image)
end
end
describe "a host adapter" do
it "decides where an image is served from" do
WarpEngine.config.image_adapter = Class.new do
def url_for(image_id) = "https://cdn.example.com/#{image_id}.png"
end.new
expect(described_class.url_for(9)).to eq("https://cdn.example.com/9.png")
end
end
it "links the catalog to the host's image model" do
expect(WarpEngine::SoftwareImage.reflect_on_association(:image).klass).to eq(Image)
end
it "puts the adapter URL in the catalog payload" do
software = create(:software)
image = create(:image)
WarpEngine::SoftwareImage.create!(software: software, image_id: image.id, is_default: true)
payload = WarpEngine::SoftwareSerializer.render_as_hash(software.reload)
expect(payload[:imageUrl]).to eq("/api/image/#{image.id}")
end
end
+3 -3
View File
@@ -33,14 +33,14 @@ RSpec.describe "Engine migrations" do
end end
it "does not create a table the install generator also creates" do it "does not create a table the install generator also creates" do
template = WarpEngine::Engine.root.join( templates = Dir.glob(
"lib/generators/warp_engine/install/templates/create_warp_engine_tables.rb" WarpEngine::Engine.root.join("lib/generators/warp_engine/install/templates/create_*.rb")
) )
engine_tables = versions.flat_map do |version| engine_tables = versions.flat_map do |version|
file = Dir.glob(WarpEngine::Engine.root.join("db/migrate/#{version}_*.rb")).first file = Dir.glob(WarpEngine::Engine.root.join("db/migrate/#{version}_*.rb")).first
File.read(file).scan(/create_table :(\w+)/).flatten File.read(file).scan(/create_table :(\w+)/).flatten
end end
template_tables = File.read(template).scan(/create_table :(\w+)/).flatten template_tables = templates.flat_map { |t| File.read(t).scan(/create_table :(\w+)/).flatten }
duplicated = (engine_tables & template_tables) - [ "application_tokens" ] duplicated = (engine_tables & template_tables) - [ "application_tokens" ]
expect(duplicated).to be_empty, expect(duplicated).to be_empty,
+21 -11
View File
@@ -10,11 +10,21 @@ RSpec.describe "Build configs endpoint", type: :request do
} }
end end
before do def woodpecker(platforms: nil, public_key: :default)
allow(WarpEngine.config).to receive(:ci_platforms).and_return(ci_platforms) WarpEngine::CI::Woodpecker::Adapter.new(
allow(WarpEngine.config).to receive(:ci_extension_public_key).and_return(signing_key.public_to_pem) url: "https://ci.example.test",
api_token: "wp-token",
platforms: platforms.nil? ? ci_platforms : platforms,
public_key: public_key == :default ? signing_key.public_to_pem : public_key
)
end end
def serving(adapter)
allow(WarpEngine).to receive(:ci).and_return(adapter)
end
before { serving(woodpecker) }
def signed_headers(body, path: "/build/config", digest_body: nil) def signed_headers(body, path: "/build/config", digest_body: nil)
digest = "sha-256=:#{Digest::SHA256.base64digest(digest_body || body)}:" digest = "sha-256=:#{Digest::SHA256.base64digest(digest_body || body)}:"
inner = %{("@request-target" "content-digest");created=#{Time.now.to_i};alg="ed25519"} inner = %{("@request-target" "content-digest");created=#{Time.now.to_i};alg="ed25519"}
@@ -69,7 +79,7 @@ RSpec.describe "Build configs endpoint", type: :request do
end end
it "returns 404 when the feature is not configured" do it "returns 404 when the feature is not configured" do
allow(WarpEngine.config).to receive(:ci_platforms).and_return({}) serving(woodpecker(platforms: {}))
get "/build/config", params: { platform: "godot" } get "/build/config", params: { platform: "godot" }
@@ -146,7 +156,7 @@ RSpec.describe "Build configs endpoint", type: :request do
payload = extension_payload("platform: godot\n") payload = extension_payload("platform: godot\n")
headers = signed_headers(payload) headers = signed_headers(payload)
other_key = OpenSSL::PKey.generate_key("ed25519") other_key = OpenSSL::PKey.generate_key("ed25519")
allow(WarpEngine.config).to receive(:ci_extension_public_key).and_return(other_key.public_to_pem) serving(woodpecker(public_key: other_key.public_to_pem))
post "/build/config", params: payload, headers: headers post "/build/config", params: payload, headers: headers
@@ -170,8 +180,7 @@ RSpec.describe "Build configs endpoint", type: :request do
end end
it "rejects every request when no public key is configured" do it "rejects every request when no public key is configured" do
allow(WarpEngine.config).to receive(:ci_extension_public_key).and_return(nil) serving(woodpecker(public_key: nil))
allow(WarpEngine.config).to receive(:ci_extension_public_key_url).and_return(nil)
payload = extension_payload("platform: godot\n") payload = extension_payload("platform: godot\n")
post "/build/config", params: payload, headers: signed_headers(payload) post "/build/config", params: payload, headers: signed_headers(payload)
@@ -182,16 +191,17 @@ RSpec.describe "Build configs endpoint", type: :request do
describe "shipped templates" do describe "shipped templates" do
it "renders every template to valid YAML with non-empty steps" do it "renders every template to valid YAML with non-empty steps" do
templates = Dir[WarpEngine::Engine.root.join("app/services/warp_engine/platforms/*/pipeline.yaml.erb")] templates = Dir[WarpEngine::CI::Woodpecker::PipelineConfig.templates_dir.join("*/pipeline.yaml.erb")]
expect(templates).not_to be_empty expect(templates).not_to be_empty
templates.each do |path| templates.each do |path|
platform = File.basename(File.dirname(path)) platform = File.basename(File.dirname(path))
allow(WarpEngine.config).to receive(:ci_platforms).and_return( config = WarpEngine::CI::Woodpecker::PipelineConfig.new(
platform => { builder: "registry.example/builder:1", exporter: "registry.example/exporter:1" } platforms: { platform => { builder: "registry.example/builder:1",
exporter: "registry.example/exporter:1" } }
) )
yaml = WarpEngine::CiConfigService.new.render( yaml = config.render(
platform: platform, name: "example", update_server: "https://games.example" platform: platform, name: "example", update_server: "https://games.example"
) )
+25 -15
View File
@@ -1,15 +1,16 @@
require "rails_helper" require "rails_helper"
RSpec.describe "CI API", type: :request do RSpec.describe "CI API", type: :request do
let(:ci) { instance_double(WarpEngine::CI::Woodpecker::Adapter, configured?: true, name: "Woodpecker") }
before do before do
allow(WarpEngine.config).to receive(:woodpecker_url).and_return("https://ci.test") allow(WarpEngine).to receive(:ci).and_return(ci)
allow(WarpEngine.config).to receive(:woodpecker_api_token).and_return("wp-token")
allow(WarpEngine.config).to receive(:update_secret).and_return("s3cret") allow(WarpEngine.config).to receive(:update_secret).and_return("s3cret")
end end
describe "GET /api/ci/pipelines" do describe "GET /api/ci/pipelines" do
it "returns active repos" do it "returns active repos" do
repo = create(:pipeline, repo_name: "mygame", platform: "tic80") create(:pipeline, repo_name: "mygame", platform: "tic80")
get "/api/ci/pipelines" get "/api/ci/pipelines"
@@ -17,10 +18,11 @@ RSpec.describe "CI API", type: :request do
json = JSON.parse(response.body) json = JSON.parse(response.body)
expect(json.size).to eq(1) expect(json.size).to eq(1)
expect(json.first["repo_name"]).to eq("mygame") expect(json.first["repo_name"]).to eq("mygame")
expect(json.first["repo_id"]).to eq(json.first["woodpecker_repo_id"])
end end
it "returns 503 when woodpecker not configured" do it "returns 503 when no CI provider is configured" do
allow(WarpEngine.config).to receive(:woodpecker_url).and_return(nil) allow(ci).to receive(:configured?).and_return(false)
get "/api/ci/pipelines" get "/api/ci/pipelines"
@@ -29,18 +31,27 @@ RSpec.describe "CI API", type: :request do
end end
describe "GET /api/ci/pipelines/:id/status" do describe "GET /api/ci/pipelines/:id/status" do
it "returns repo with pipeline status" do it "returns the repo with its latest run" do
repo = create(:pipeline, repo_owner: "org", repo_name: "game") repo = create(:pipeline, repo_owner: "org", repo_name: "game")
client = instance_double(WarpEngine::WoodpeckerClient) allow(ci).to receive(:run).with(repo.remote_repo_id, "latest")
allow(WarpEngine::WoodpeckerClient).to receive(:new).and_return(client) .and_return(WarpEngine::CI::Run.new(number: 1, status: "success"))
allow(client).to receive(:get_pipeline)
.and_return({ "number" => 1, "status" => "success" })
get "/api/ci/pipelines/#{repo.id}/status" get "/api/ci/pipelines/#{repo.id}/status"
expect(response).to have_http_status(:ok) expect(response).to have_http_status(:ok)
json = JSON.parse(response.body) json = JSON.parse(response.body)
expect(json["pipeline"]["repo_name"]).to eq("game") expect(json["pipeline"]["repo_name"]).to eq("game")
expect(json["latest_run"]["status"]).to eq("success")
end
it "answers without a run when the provider is unreachable" do
repo = create(:pipeline)
allow(ci).to receive(:run).and_raise(WarpEngine::CI::ConnectionError, "down")
get "/api/ci/pipelines/#{repo.id}/status"
expect(response).to have_http_status(:ok)
expect(JSON.parse(response.body)["latest_run"]).to be_nil
end end
end end
@@ -53,12 +64,10 @@ RSpec.describe "CI API", type: :request do
expect(response).to have_http_status(:unauthorized) expect(response).to have_http_status(:unauthorized)
end end
it "triggers a pipeline with valid secret" do it "triggers a pipeline with a valid secret" do
repo = create(:pipeline, repo_owner: "org", repo_name: "game") repo = create(:pipeline, repo_owner: "org", repo_name: "game")
client = instance_double(WarpEngine::WoodpeckerClient) allow(ci).to receive(:trigger).with(repo.remote_repo_id, branch: "main")
allow(WarpEngine::WoodpeckerClient).to receive(:new).and_return(client) .and_return(WarpEngine::CI::Run.new(number: 7, status: "pending"))
allow(client).to receive(:trigger_pipeline)
.and_return({ "number" => 7, "status" => "pending" })
post "/api/ci/pipelines/#{repo.id}/trigger", post "/api/ci/pipelines/#{repo.id}/trigger",
headers: { "X-Update-Secret" => "s3cret" } headers: { "X-Update-Secret" => "s3cret" }
@@ -66,6 +75,7 @@ RSpec.describe "CI API", type: :request do
expect(response).to have_http_status(:ok) expect(response).to have_http_status(:ok)
json = JSON.parse(response.body) json = JSON.parse(response.body)
expect(json["triggered"]).to be true expect(json["triggered"]).to be true
expect(json["pipeline"]["number"]).to eq(7)
end end
end end
end end
+1 -1
View File
@@ -15,7 +15,7 @@ RSpec.describe "the WarpEngine-Version header", type: :request do
end end
it "is on an error response" do it "is on an error response" do
get "/api/image/999999" get "/api/softwares/no-such-software/builds"
expect(response).to have_http_status(:not_found) expect(response).to have_http_status(:not_found)
expect(response.headers["WarpEngine-Version"]).to eq(WarpEngine::VERSION) expect(response.headers["WarpEngine-Version"]).to eq(WarpEngine::VERSION)
+22 -18
View File
@@ -1,31 +1,35 @@
require "rails_helper" require "rails_helper"
RSpec.describe WarpEngine::PipelineService do RSpec.describe WarpEngine::PipelineService do
let(:client) { instance_double(WarpEngine::WoodpeckerClient) } let(:ci) { instance_double(WarpEngine::CI::Woodpecker::Adapter) }
let(:service) { described_class.new(client: client) } let(:service) { described_class.new(ci: ci) }
def ci_run(number:, status: "success", created_at: nil)
WarpEngine::CI::Run.new(number: number, status: status, created_at: created_at)
end
describe "#trigger" do describe "#trigger" do
it "delegates to client" do it "delegates to client" do
repo = build(:pipeline, repo_owner: "org", repo_name: "game") repo = build(:pipeline, repo_owner: "org", repo_name: "game")
allow(client).to receive(:trigger_pipeline).and_return({ "number" => 6 }) allow(ci).to receive(:trigger).and_return(ci_run(number: 6, status: "pending"))
result = service.trigger(repo, branch: "main") result = service.trigger(repo, branch: "main")
expect(result["number"]).to eq(6) expect(result.number).to eq(6)
expect(client).to have_received(:trigger_pipeline).with(repo.woodpecker_repo_id, branch: "main") expect(ci).to have_received(:trigger).with(repo.remote_repo_id, branch: "main")
end end
end end
describe "#list_pipelines" do describe "#runs" do
it "returns paginated pipelines and refreshes the repo's cached last pipeline" do it "returns the runs and refreshes the repo's cached last run" do
repo = create(:pipeline, repo_owner: "org", repo_name: "game") repo = create(:pipeline, repo_owner: "org", repo_name: "game")
pipelines = [ runs = [
{ "number" => 2, "status" => "failure", "created" => 1_754_500_000 }, ci_run(number: 2, status: "failure", created_at: Time.zone.at(1_754_500_000)),
{ "number" => 1, "status" => "success", "created" => 1_754_400_000 } ci_run(number: 1, status: "success", created_at: Time.zone.at(1_754_400_000))
] ]
allow(client).to receive(:list_pipelines).with(repo.woodpecker_repo_id, page: 1).and_return(pipelines) allow(ci).to receive(:runs).with(repo.remote_repo_id, page: 1).and_return(runs)
expect(service.list_pipelines(repo)).to eq(pipelines) expect(service.runs(repo)).to eq(runs)
repo.reload repo.reload
expect(repo.last_pipeline_status).to eq("failure") expect(repo.last_pipeline_status).to eq("failure")
@@ -35,20 +39,20 @@ RSpec.describe WarpEngine::PipelineService do
it "does not touch the cache on later pages" do it "does not touch the cache on later pages" do
repo = create(:pipeline, repo_owner: "org", repo_name: "game", repo = create(:pipeline, repo_owner: "org", repo_name: "game",
last_pipeline_status: "success") last_pipeline_status: "success")
allow(client).to receive(:list_pipelines).with(repo.woodpecker_repo_id, page: 2) allow(ci).to receive(:runs).with(repo.remote_repo_id, page: 2)
.and_return([{ "number" => 1, "status" => "failure", "created" => 1_754_400_000 }]) .and_return([ ci_run(number: 1, status: "failure", created_at: Time.zone.at(1_754_400_000)) ])
service.list_pipelines(repo, page: 2) service.runs(repo, page: 2)
expect(repo.reload.last_pipeline_status).to eq("success") expect(repo.reload.last_pipeline_status).to eq("success")
end end
it "leaves the cache alone when the repo has no pipelines" do it "leaves the cache alone when the repo has no runs" do
repo = create(:pipeline, repo_owner: "org", repo_name: "game", repo = create(:pipeline, repo_owner: "org", repo_name: "game",
last_pipeline_status: "success") last_pipeline_status: "success")
allow(client).to receive(:list_pipelines).and_return([]) allow(ci).to receive(:runs).and_return([])
expect(service.list_pipelines(repo)).to eq([]) expect(service.runs(repo)).to eq([])
expect(repo.reload.last_pipeline_status).to eq("success") expect(repo.reload.last_pipeline_status).to eq("success")
end end
end end
+17 -21
View File
@@ -1,14 +1,16 @@
require "rails_helper" require "rails_helper"
RSpec.describe WarpEngine::PipelineSyncService do RSpec.describe WarpEngine::PipelineSyncService do
let(:client) { instance_double(WarpEngine::WoodpeckerClient) } let(:ci) { instance_double(WarpEngine::CI::Woodpecker::Adapter) }
let(:service) { described_class.new(client: client) } let(:service) { described_class.new(ci: ci) }
def ci_repo(id:, name: "mygame", owner: "org", active: true)
WarpEngine::CI::Repo.new(id: id, name: name, owner: owner, active: active)
end
describe "#sync_all" do describe "#sync_all" do
it "creates new Pipeline records from Woodpecker" do it "creates new Pipeline records from the CI provider" do
allow(client).to receive(:list_repos).and_return([ allow(ci).to receive(:repos).and_return([ ci_repo(id: 1) ])
{ "id" => 1, "name" => "mygame", "owner" => "org", "active" => true }
])
result = service.sync_all result = service.sync_all
@@ -21,9 +23,7 @@ RSpec.describe WarpEngine::PipelineSyncService do
it "updates existing records" do it "updates existing records" do
existing = create(:pipeline, woodpecker_repo_id: 1, repo_name: "old", platform: "tic80") existing = create(:pipeline, woodpecker_repo_id: 1, repo_name: "old", platform: "tic80")
allow(client).to receive(:list_repos).and_return([ allow(ci).to receive(:repos).and_return([ ci_repo(id: 1, name: "newname") ])
{ "id" => 1, "name" => "newname", "owner" => "org", "active" => true }
])
result = service.sync_all result = service.sync_all
@@ -31,9 +31,9 @@ RSpec.describe WarpEngine::PipelineSyncService do
expect(existing.reload.repo_name).to eq("newname") expect(existing.reload.repo_name).to eq("newname")
end end
it "deactivates repos missing from Woodpecker" do it "deactivates repos the provider no longer has" do
orphan = create(:pipeline, woodpecker_repo_id: 99, active: true) orphan = create(:pipeline, woodpecker_repo_id: 99, active: true)
allow(client).to receive(:list_repos).and_return([]) allow(ci).to receive(:repos).and_return([])
result = service.sync_all result = service.sync_all
@@ -43,9 +43,7 @@ RSpec.describe WarpEngine::PipelineSyncService do
it "auto-detects platform from matching Software" do it "auto-detects platform from matching Software" do
create(:software, name: "mygame", platform: "godot") create(:software, name: "mygame", platform: "godot")
allow(client).to receive(:list_repos).and_return([ allow(ci).to receive(:repos).and_return([ ci_repo(id: 1) ])
{ "id" => 1, "name" => "mygame", "owner" => "org", "active" => true }
])
result = service.sync_all result = service.sync_all
@@ -55,11 +53,9 @@ RSpec.describe WarpEngine::PipelineSyncService do
end end
describe "#activate" do describe "#activate" do
it "calls client and syncs the repo" do it "calls the provider and syncs the repo" do
allow(client).to receive(:activate_repo).with(42) allow(ci).to receive(:activate_repo).with(42)
allow(client).to receive(:get_repo).with(42).and_return( allow(ci).to receive(:repo).with(42).and_return(ci_repo(id: 42, name: "game"))
{ "id" => 42, "name" => "game", "owner" => "org", "active" => true }
)
repo = service.activate(42) repo = service.activate(42)
@@ -69,9 +65,9 @@ RSpec.describe WarpEngine::PipelineSyncService do
end end
describe "#deactivate" do describe "#deactivate" do
it "calls client and marks repo inactive" do it "calls the provider and marks the repo inactive" do
repo = create(:pipeline, woodpecker_repo_id: 42, active: true) repo = create(:pipeline, woodpecker_repo_id: 42, active: true)
allow(client).to receive(:deactivate_repo).with(42) allow(ci).to receive(:deactivate_repo).with(42)
service.deactivate(42) service.deactivate(42)
+13 -29
View File
@@ -1,8 +1,8 @@
require "rails_helper" require "rails_helper"
RSpec.describe WarpEngine::SecretSyncService do RSpec.describe WarpEngine::SecretSyncService do
let(:client) { instance_double(WarpEngine::WoodpeckerClient) } let(:ci) { instance_double(WarpEngine::CI::Woodpecker::Adapter) }
let(:service) { described_class.new(client: client) } let(:service) { described_class.new(ci: ci) }
before do before do
allow(WarpEngine.config).to receive(:application_token_source).and_return(:database) allow(WarpEngine.config).to receive(:application_token_source).and_return(:database)
@@ -10,37 +10,22 @@ RSpec.describe WarpEngine::SecretSyncService do
end end
describe "#provision" do describe "#provision" do
it "creates secrets on repos that don't have one" do it "hands the token to the provider for every pipeline" do
repo = create(:pipeline) repo = create(:pipeline)
allow(client).to receive(:list_secrets).with(repo.woodpecker_repo_id).and_return([]) allow(ci).to receive(:secret_set)
allow(client).to receive(:create_secret)
result = service.provision("plaintoken", pipelines: [ repo ]) result = service.provision("plaintoken", pipelines: [ repo ])
expect(result[:synced]).to eq([ repo ]) expect(result[:synced]).to eq([ repo ])
expect(client).to have_received(:create_secret).with( expect(ci).to have_received(:secret_set).with(
repo.woodpecker_repo_id, name: "application_token", value: "plaintoken" repo.remote_repo_id, name: "application_token", value: "plaintoken"
)
end
it "updates secrets on repos that already have one" do
repo = create(:pipeline)
allow(client).to receive(:list_secrets).with(repo.woodpecker_repo_id)
.and_return([ { "name" => "application_token" } ])
allow(client).to receive(:update_secret)
result = service.provision("newtoken", pipelines: [ repo ])
expect(result[:synced]).to eq([ repo ])
expect(client).to have_received(:update_secret).with(
repo.woodpecker_repo_id, "application_token", value: "newtoken"
) )
end end
it "records failed repos without raising" do it "records failed repos without raising" do
repo = create(:pipeline) repo = create(:pipeline)
allow(client).to receive(:list_secrets).and_raise( allow(ci).to receive(:secret_set).and_raise(
WarpEngine::WoodpeckerClient::ConnectionError, "unreachable" WarpEngine::CI::ConnectionError, "unreachable"
) )
result = service.provision("tok", pipelines: [ repo ]) result = service.provision("tok", pipelines: [ repo ])
@@ -56,11 +41,11 @@ RSpec.describe WarpEngine::SecretSyncService do
sw = create(:software, name: "game1", owner: token.owner) sw = create(:software, name: "game1", owner: token.owner)
repo = create(:pipeline, :with_software, software: sw) repo = create(:pipeline, :with_software, software: sw)
allow(client).to receive(:delete_secret) allow(ci).to receive(:secret_delete)
service.deprovision(token) service.deprovision(token)
expect(client).to have_received(:delete_secret).with(repo.woodpecker_repo_id, "application_token") expect(ci).to have_received(:secret_delete).with(repo.remote_repo_id, "application_token")
end end
end end
@@ -70,8 +55,7 @@ RSpec.describe WarpEngine::SecretSyncService do
sw = create(:software, name: "game1", owner: token.owner) sw = create(:software, name: "game1", owner: token.owner)
repo = create(:pipeline, :with_software, software: sw) repo = create(:pipeline, :with_software, software: sw)
allow(client).to receive(:list_secrets).and_return([]) allow(ci).to receive(:secret_set)
allow(client).to receive(:create_secret)
result = service.rotate(token) result = service.rotate(token)
@@ -85,8 +69,8 @@ RSpec.describe WarpEngine::SecretSyncService do
sw = create(:software, name: "game1", owner: token.owner) sw = create(:software, name: "game1", owner: token.owner)
create(:pipeline, :with_software, software: sw) create(:pipeline, :with_software, software: sw)
allow(client).to receive(:list_secrets).and_raise( allow(ci).to receive(:secret_set).and_raise(
WarpEngine::WoodpeckerClient::ConnectionError, "down" WarpEngine::CI::ConnectionError, "down"
) )
result = service.rotate(token) result = service.rotate(token)
+1 -1
View File
@@ -6,7 +6,7 @@ Gem::Specification.new do |spec|
spec.authors = [ "Teletype Games" ] spec.authors = [ "Teletype Games" ]
spec.summary = "Retro software catalog engine with a CI-callable release updater" spec.summary = "Retro software catalog engine with a CI-callable release updater"
spec.description = "Mountable Rails engine providing a software catalog (softwares, releases, " \ spec.description = "Mountable Rails engine providing a software catalog (softwares, releases, " \
"release assets, images, platform links), a pipeline-callable update endpoint, " \ "release assets, image links, platform links), a pipeline-callable update endpoint, " \
"a public read-only JSON API and ActiveAdmin resources for a host-owned admin." "a public read-only JSON API and ActiveAdmin resources for a host-owned admin."
spec.license = "MIT" spec.license = "MIT"
spec.homepage = "https://teletypegames.org" spec.homepage = "https://teletypegames.org"