From 19d142ef6487a9256ab70daf0d5ecb2800490380 Mon Sep 17 00:00:00 2001 From: Zsolt Tasnadi Date: Sun, 23 Aug 2026 00:56:01 +0200 Subject: [PATCH] =?UTF-8?q?WarpEngine=200.7.0:=20a=20k=C3=A9pt=C3=A1r=20?= =?UTF-8?q?=C3=A9s=20a=20CI=20is=20a=20hoszt=C3=A9,=20adapteren=20kereszt?= =?UTF-8?q?=C3=BCl?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Két dolog volt beépítve az engine-be, ami nem az övé. A **képtár** eddig `WarpEngine::Image` volt, pedig a modell teljesen általános: a teletypegames-ben a tagok arcképét is ez hordozza, nem csak a katalógus borítóit. Az `Image` modell, a feltöltött fájlok, az `/api/image/:id` végpont és az admin oldal ezért átkerült a hosztba, az engine pedig adapteren szól hozzá (`WarpEngine::Images`): `url_for` adja a katalógus JSON `imageUrl`-jét, `select_options` a software-form képválasztóját, `build_from_upload` a "tölts fel új képet" ágat. Az alapértelmezés az `Image` osztály, tehát a default útvonal bitre a régi. A `SoftwareImage` (a katalógus-kapcsolat) maradt az engine-ben, és **az `images` tábla nem mozdult**: az engine csak abbahagyta a létrehozását, a generátor írja meg hoszt-kódként. A **CI** eddig végig Woodpecker volt: kliens, aláírás-ellenőrzés, pipeline-receptek, repo-szinkron, secret-kiosztás. Mindez egy adapter mögé került (`WarpEngine.ci`), a Woodpecker-implementáció pedig az engine-ben maradt `WarpEngine::CI::Woodpecker` néven — kliens, adapter, httpsig-ellenőrző és a platformonkénti pipeline-receptek, mert a YAML-dialektus a szolgáltatóé. Az engine saját kódja már nem nevez szolgáltatót: `CI::Repo` és `CI::Run` értékeket kap, `CI::ConnectionError`/`ApiError`/`NotConfigured` hibákat dob, a `Pipeline` pedig `remote_repo_id`-t ad a történelmi `woodpecker_repo_id` kolumna fölött (a tábla itt sem mozdult). `c.ci_adapter = :none` azt jelenti, hogy ez a hoszt nem buildel: az `/api/ci/*` 503, a `/build/config` elutasít, az admin akciók elbújnak. Mindkét seam a hoszt initializerében van kimondva, nem alapértelmezésre hagyva — a hoszt megnevezi, mi a képtára és mi a CI-ja. Törés a 0.6-hoz képest: `image_container_path`, `image_owners`, `ci_platforms`, `ci_update_server`, `ci_extension_public_key(_url)`, `woodpecker_url`, `woodpecker_api_token`, `woodpecker_repo_owner` és a `WarpEngine.woodpecker_configured?` megszűnt; a helyük `c.image_class_name` / `c.image_adapter` és `c.ci_adapter`. Az `/api/ci/*` `latest_run`/`trigger` válasza a normalizált `CI::Run` alakot adja (number, status, branch, message, createdAt, url), a `pipelines` lista pedig `repo_id`-t is közöl a megtartott `woodpecker_repo_id` mellett. Co-Authored-By: Claude Opus 5 (1M context) --- README.md | 185 +++++++++++++----- app/admin/application_tokens.rb | 6 +- app/admin/images.rb | 77 -------- app/admin/pipelines.rb | 29 ++- app/admin/softwares.rb | 8 +- .../warp_engine/api/ci_controller.rb | 21 +- .../warp_engine/api/images_controller.rb | 17 -- .../warp_engine/build/configs_controller.rb | 58 ++---- app/dtos/warp_engine/image_show_input_dto.rb | 3 - app/jobs/warp_engine/application_job.rb | 4 +- app/models/warp_engine/image.rb | 36 ---- app/models/warp_engine/pipeline.rb | 8 + app/models/warp_engine/software_image.rb | 4 +- .../warp_engine/software_serializer.rb | 4 +- app/services/warp_engine/ci_config_service.rb | 37 ---- .../warp_engine/ci_signature_verifier.rb | 142 -------------- app/services/warp_engine/image_service.rb | 7 - app/services/warp_engine/pipeline_service.rb | 22 +-- .../warp_engine/pipeline_sync_service.rb | 48 +++-- .../warp_engine/secret_sync_service.rb | 35 ++-- app/services/warp_engine/woodpecker_client.rb | 153 --------------- config/routes.rb | 1 - .../app/controllers/api/images_controller.rb | 16 ++ examples/compose/host_app/app/models/image.rb | 32 +++ .../config/initializers/warp_engine.rb | 11 +- examples/compose/host_app/config/routes.rb | 4 + .../warp_engine/install/install_generator.rb | 12 +- .../install/templates/create_images.rb | 12 ++ .../templates/create_warp_engine_tables.rb | 12 +- .../warp_engine/install/templates/image.rb | 32 +++ .../install/templates/images_controller.rb | 16 ++ .../install/templates/initializer.rb | 13 ++ lib/warp_engine.rb | 14 +- lib/warp_engine/ci.rb | 94 +++++++++ lib/warp_engine/ci/woodpecker.rb | 4 + lib/warp_engine/ci/woodpecker/adapter.rb | 146 ++++++++++++++ lib/warp_engine/ci/woodpecker/client.rb | 143 ++++++++++++++ .../ci/woodpecker/pipeline_config.rb | 42 ++++ .../platforms/bevy/pipeline.yaml.erb | 0 .../platforms/c64/pipeline.yaml.erb | 0 .../platforms/ebitengine/pipeline.yaml.erb | 0 .../platforms/godot/pipeline.yaml.erb | 0 .../platforms/love/pipeline.yaml.erb | 0 .../platforms/phaser/pipeline.yaml.erb | 0 .../platforms/tic80/pipeline.yaml.erb | 0 .../ci/woodpecker/signature_verifier.rb | 145 ++++++++++++++ lib/warp_engine/configuration.rb | 24 +-- lib/warp_engine/images.rb | 68 +++++++ lib/warp_engine/version.rb | 2 +- .../woodpecker_client_spec.rb | 14 +- spec/ci_spec.rb | 136 +++++++++++++ spec/dummy/app/models/image.rb | 27 +++ spec/factories/images.rb | 7 + spec/images_spec.rb | 51 +++++ spec/migrations_spec.rb | 6 +- .../requests/build_configs_controller_spec.rb | 32 +-- spec/requests/ci_controller_spec.rb | 40 ++-- spec/requests/version_header_spec.rb | 2 +- spec/services/pipeline_service_spec.rb | 40 ++-- spec/services/pipeline_sync_service_spec.rb | 38 ++-- spec/services/secret_sync_service_spec.rb | 42 ++-- warp_engine.gemspec | 2 +- 62 files changed, 1391 insertions(+), 793 deletions(-) delete mode 100644 app/admin/images.rb delete mode 100644 app/controllers/warp_engine/api/images_controller.rb delete mode 100644 app/dtos/warp_engine/image_show_input_dto.rb delete mode 100644 app/models/warp_engine/image.rb delete mode 100644 app/services/warp_engine/ci_config_service.rb delete mode 100644 app/services/warp_engine/ci_signature_verifier.rb delete mode 100644 app/services/warp_engine/image_service.rb delete mode 100644 app/services/warp_engine/woodpecker_client.rb create mode 100644 examples/compose/host_app/app/controllers/api/images_controller.rb create mode 100644 examples/compose/host_app/app/models/image.rb create mode 100644 lib/generators/warp_engine/install/templates/create_images.rb create mode 100644 lib/generators/warp_engine/install/templates/image.rb create mode 100644 lib/generators/warp_engine/install/templates/images_controller.rb create mode 100644 lib/warp_engine/ci.rb create mode 100644 lib/warp_engine/ci/woodpecker.rb create mode 100644 lib/warp_engine/ci/woodpecker/adapter.rb create mode 100644 lib/warp_engine/ci/woodpecker/client.rb create mode 100644 lib/warp_engine/ci/woodpecker/pipeline_config.rb rename {app/services/warp_engine => lib/warp_engine/ci/woodpecker}/platforms/bevy/pipeline.yaml.erb (100%) rename {app/services/warp_engine => lib/warp_engine/ci/woodpecker}/platforms/c64/pipeline.yaml.erb (100%) rename {app/services/warp_engine => lib/warp_engine/ci/woodpecker}/platforms/ebitengine/pipeline.yaml.erb (100%) rename {app/services/warp_engine => lib/warp_engine/ci/woodpecker}/platforms/godot/pipeline.yaml.erb (100%) rename {app/services/warp_engine => lib/warp_engine/ci/woodpecker}/platforms/love/pipeline.yaml.erb (100%) rename {app/services/warp_engine => lib/warp_engine/ci/woodpecker}/platforms/phaser/pipeline.yaml.erb (100%) rename {app/services/warp_engine => lib/warp_engine/ci/woodpecker}/platforms/tic80/pipeline.yaml.erb (100%) create mode 100644 lib/warp_engine/ci/woodpecker/signature_verifier.rb create mode 100644 lib/warp_engine/images.rb rename spec/{services => ci}/woodpecker_client_spec.rb (87%) create mode 100644 spec/ci_spec.rb create mode 100644 spec/dummy/app/models/image.rb create mode 100644 spec/factories/images.rb create mode 100644 spec/images_spec.rb diff --git a/README.md b/README.md index c10cb02..948ec27 100644 --- a/README.md +++ b/README.md @@ -10,7 +10,7 @@ Repository: `https://git.teletypegames.org/engines/warp_engine` ## Features - **Catalog domain**: `Software`, `Release`, `ReleaseAsset`, `ExternalLink`, - `PlatformLink`, `Image`, `SoftwareImage`, `Download` models with soft-delete + `PlatformLink`, `SoftwareImage`, `Download` models with soft-delete semantics and download statistics. - **CI-callable updater**: your build pipeline uploads artifacts over HTTP and calls one endpoint — WarpEngine extracts archives, parses metadata @@ -24,6 +24,10 @@ Repository: `https://git.teletypegames.org/engines/warp_engine` is the catalog as it always was. - **Client sign-in**: an RFC 8628 device authorization grant for clients with no browser of their own, over the host's own user model. Off unless configured. +- **Host-owned image library**: the catalog links a software to its images and + publishes their URLs, but the image model, its files, its endpoint and its + admin page belong to the host — the same model can serve avatars or anything + else the site has (see *Images*). - **Pluggable storage**: artifacts are served through a storage adapter (`:local` by default); a host can serve them from an object store without patching the engine. @@ -31,17 +35,15 @@ Repository: `https://git.teletypegames.org/engines/warp_engine` `ActiveSupport::Notifications` (`warp_engine.publish`), so hosts can react to new builds without model callbacks. - **Public JSON API**: catalog listing, highlighted title, per-platform build - matrix, image serving, download tracking, and a static file server for - web-playable builds. + matrix, download tracking, and a static file server for web-playable builds. - **Admin (optional)**: if the host runs ActiveAdmin, WarpEngine contributes - ready-made resources — a catalog editor with nested release/asset forms, an - image library with orphan cleanup, a file manager with a picker mode, and - download statistics. Without ActiveAdmin the engine runs headless + ready-made resources — a catalog editor with nested release/asset forms, a + file manager with a picker mode, and download statistics. Without ActiveAdmin the engine runs headless (API + updater only). -- **Woodpecker CI management (optional)**: with a Woodpecker API token - configured, the admin also gains repo sync, per-repo pipeline history - with manual triggers, and automatic provisioning of application tokens - as Woodpecker secrets. +- **Pluggable CI**: pipeline configs, repo sync, build triggers and secret + provisioning all go through one adapter. Woodpecker ships with the engine + (`WarpEngine::CI::Woodpecker`) and is the default; `:none` turns CI off, and + another server is a host-supplied object. ## Requirements @@ -177,12 +179,18 @@ cannot say which gem came from where. Then: ```sh -rails g warp_engine:install # initializer + create_warp_engine_tables migration +rails g warp_engine:install # initializer, image library (model + endpoint), migrations rails db:migrate ``` ```ruby -# config/routes.rb — keep it the last entry so your own routes win +# config/routes.rb +namespace :api do + # The image library is yours; the engine only publishes these URLs. + get "image/:id", to: "images#show" +end + +# keep the mount the last entry so your own routes win mount WarpEngine::Engine => "/" ``` @@ -192,9 +200,12 @@ mount WarpEngine::Engine => "/" # config/initializers/warp_engine.rb Rails.application.config.to_prepare do WarpEngine.configure do |c| - # Where CI drops build artifacts and where images are stored - c.file_container_path = ENV.fetch("FILE_CONTAINER_PATH", "/softwares") - c.image_container_path = ENV.fetch("IMAGE_CONTAINER_PATH", "/images") + # Where CI drops build artifacts + c.file_container_path = ENV.fetch("FILE_CONTAINER_PATH", "/softwares") + + # Your image model (see "Images" below). "Image" is the default. + # c.image_class_name = "Media::Picture" + # c.image_adapter = MyImageLibrary.new # Shared secret for the /build/* endpoints. # nil => the endpoints reject every request. @@ -217,6 +228,15 @@ Rails.application.config.to_prepare do # after backfilling owners — ownerless softwares are claimable by anyone. # c.enforce_software_ownership = true + # Which CI server builds the games (see "CI" below). :woodpecker (default) + # reads WOODPECKER_URL / WOODPECKER_API_TOKEN from the environment; naming + # the adapter is better, because then the settings are here in the open. + # c.ci_adapter = WarpEngine::CI::Woodpecker::Adapter.new( + # url: ENV["WOODPECKER_URL"], api_token: ENV["WOODPECKER_API_TOKEN"], + # platforms: { "godot" => { builder: "registry.example/godot-builder:4.6" } } + # ) + # c.ci_adapter = :none # this host builds nothing + # Who may see a title and who may download it (see "Access" below). # :open (default) lists everything and serves everything. # c.access_policy = MyStore::AccessPolicy.new @@ -228,16 +248,6 @@ Rails.application.config.to_prepare do # # How to recognise a caller with a session instead of a bearer token. # c.subject_resolver = ->(request) { request.env["warden"]&.user } - - # If your app's own models reference catalog images, register them so the - # admin Images page counts them as "in use": - # c.image_owners = [ - # { - # label: "member", - # image_ids: -> { Member.where.not(image_id: nil).distinct.pluck(:image_id) }, - # usage_label: ->(image) { "member" if Member.where(image_id: image.id).exists? } - # } - # ] end end ``` @@ -291,7 +301,56 @@ accepts both: When switching to `:database`, create the tokens and move your pipelines to them first — the flip invalidates the shared secret immediately. -## CI pipeline configs (Woodpecker) +## CI + +Every CI feature — serving pipeline configs, syncing repositories, triggering +builds, provisioning secrets — goes through **one adapter**. Woodpecker ships +with the engine as `WarpEngine::CI::Woodpecker`, and it is the default: + +```ruby +# nothing set: Woodpecker, configured from ENV, exactly as before +c.ci_adapter = :woodpecker + +# named explicitly (recommended — the host says what its CI is) +c.ci_adapter = WarpEngine::CI::Woodpecker::Adapter.new( + url: ENV["WOODPECKER_URL"], + api_token: ENV["WOODPECKER_API_TOKEN"], + repo_owner: ENV["WOODPECKER_REPO_OWNER"], + public_key_url: "https://ci.example.org/api/signature/public-key", + update_server: "https://catalog.example.org", + platforms: { "godot" => { builder: "registry.example/godot-builder:4.6" } } +) + +# this host builds nothing: /api/ci/* answers 503, /build/config refuses, +# and the admin pipeline actions hide themselves +c.ci_adapter = :none +``` + +The engine's own code never names a provider: `WarpEngine.ci` is the adapter, +`WarpEngine::CI::Repo` and `WarpEngine::CI::Run` are what it hands back, and +`WarpEngine::CI::ConnectionError` / `ApiError` / `NotConfigured` are what it +raises. A host with a different CI server implements this contract instead: + +| Method | Purpose | +| --- | --- | +| `name` | provider label, shown in the admin | +| `configured?` | false keeps every CI feature inactive | +| `platforms` | which platforms `/build/config` can serve | +| `update_server` | server URL written into the generated pipeline; nil = the request's base_url | +| `repos`, `repo(id)` | `CI::Repo` values for the repo sync | +| `activate_repo(id)`, `deactivate_repo(id)` | enable/disable a repository | +| `runs(repo_id, page:)`, `run(repo_id, number)` | `CI::Run` values (`number` may be `"latest"`) | +| `trigger(repo_id, branch:)` | start a build, returns a `CI::Run` | +| `secret_names(repo_id)`, `secret_set(repo_id, name:, value:)`, `secret_delete(repo_id, name)` | token provisioning | +| `verify_config_request(request)` | is this really the CI server calling | +| `config_marker(params)` | `{ platform:, name: }` from the request, or nil for "not ours" | +| `pipeline_config(platform:, name:, update_server:)` | the pipeline definition, or nil for an unknown platform | +| `config_response(platform:, config:)` | the body the CI server expects | + +`Pipeline` records keep the provider-neutral `remote_repo_id` reader over the +historical `woodpecker_repo_id` column, so the table did not have to move. + +### Pipeline configs (the Woodpecker adapter) WarpEngine can act as a [Woodpecker configuration extension](https://woodpecker-ci.org/docs/usage/extensions/configuration-extension): instead of a copy-pasted `.woodpecker.yaml` in every game repo, the repo holds a @@ -305,35 +364,30 @@ platform: godot ``` - `POST /build/config` — the extension endpoint Woodpecker calls on every - pipeline start (httpsig/ed25519-signed request, verified against - `ci_extension_public_key(_url)`). Non-marker configs get a `204` so the + pipeline start (httpsig/ed25519-signed request, verified against the + adapter's `public_key` / `public_key_url`). Non-marker configs get a `204` so the repo's own YAML keeps running — opt-in migration, and putting a full pipeline back into the repo is the opt-out. - `GET /build/config?platform=godot` — renders the same pipeline as a preview. -Configuration: `ci_platforms` maps platform names to builder images +The adapter's `platforms` maps platform names to builder images (`{ "godot" => { builder: "..." }, "tic80" => { builder: ..., exporter: ... } }`); an empty map (default) disables the feature. Set the Woodpecker side with `WOODPECKER_CONFIG_EXTENSION_ENDPOINT=https://your-host/build/config` (or -per-repo in Settings → Extensions). Templates live in -`app/services/warp_engine/platforms//pipeline.yaml.erb`. +per-repo in Settings → Extensions). Templates live with the adapter, in +`lib/warp_engine/ci/woodpecker/platforms//pipeline.yaml.erb` — the +YAML dialect belongs to the provider, so a second adapter brings its own. -## Woodpecker CI management +### CI management (the Woodpecker adapter) -Beyond serving pipeline configs, WarpEngine can drive the Woodpecker REST API -itself. Set `woodpecker_url` and `woodpecker_api_token` — while either is nil -(the default), every management feature stays inactive and the admin pages -hide themselves: - -```ruby -c.woodpecker_url = ENV["WOODPECKER_URL"] # e.g. "https://ci.example.org" -c.woodpecker_api_token = ENV["WOODPECKER_API_TOKEN"] # PAT of a Woodpecker *instance admin* -c.woodpecker_repo_owner = ENV["WOODPECKER_REPO_OWNER"] # forge org the game repos live under -``` +Beyond serving pipeline configs, the adapter drives the Woodpecker REST API +itself. Give it a `url` and an `api_token` — with either missing, `configured?` +is false, every management feature stays inactive and the admin pages hide +themselves. What it unlocks (all surfaced in the admin): -- **Repo sync** (*Pipelines → Sync from Woodpecker*): mirrors the Woodpecker +- **Repo sync** (*Pipelines → Sync from …*): mirrors the provider's repo list into `Pipeline` records, auto-matching each repo to a catalog `Software` by name; repos that disappear from Woodpecker are deactivated. Platform and software links are editable by hand afterwards. @@ -342,7 +396,7 @@ What it unlocks (all surfaced in the admin): last-pipeline status shown on the Pipelines index. The software's admin page links to its pipelines from the Quick Links sidebar. - **Secret provisioning**: database application tokens are pushed to the - repos as the `application_token` Woodpecker secret — creating a token + repos as the `application_token` CI secret — creating a token provisions it to its owner's repos (unrestricted tokens to all active repos), deleting a token removes the secret, and *Rotate* creates a replacement token, provisions it everywhere and revokes the old one in a @@ -392,6 +446,47 @@ signing adapter turns them into redirects without any further change. the admin file manager — still writes to the local disk. A remote adapter needs its own upload path today. +## Images + +An image is not a catalog concept: the same picture library serves covers, +screenshots, avatars and whatever else a site has. So the engine owns only the +link — `SoftwareImage`, ordered, with one default — while the `Image` model, the +uploaded files and the admin page live in the host. `rails g warp_engine:install` +writes a working one (`app/models/image.rb`, `app/controllers/api/images_controller.rb`, +a `create_images` migration); the engine talks to it through an adapter: + +```ruby +c.image_class_name = "Image" # default +``` + +The default adapter expects that model to answer `original_filename`, +`content_type`, `file_path` and `file_upload=` (the generated one does), and +publishes every image as `/api/image/` — the address clients have always +used, which is why the generated route serves exactly that. A host that keeps +its pictures somewhere else replaces the adapter instead: + +```ruby +class MyImageLibrary + def model_name = "Media::Picture" + def model = Media::Picture + def url_for(image_id) = "https://cdn.example.com/#{image_id}.webp" + def build_from_upload(upload) = Media::Picture.new(file_upload: upload) + def label_for(record) = record.title + def available?(record) = record.stored? + def select_options = Media::Picture.order(:title).pluck(:title, :id) +end + +c.image_adapter = MyImageLibrary.new +``` + +`url_for` is what lands in the catalog JSON (`imageUrl`, `images[].url`) and in +the admin previews; `select_options` fills the image picker on the software +form; `build_from_upload` is what "upload a new image" on that form calls. + +**The `images` table stays where it is.** Moving the model out of the engine did +not touch it: the engine no longer creates it (the install generator does, as +host code), and `software_images.image_id` still points at the same rows. + ## Access Who may see a title, and who may download it. The default answers "everyone" to @@ -523,7 +618,7 @@ its own account of who fetched what without reaching into `DownloadService`. | `GET /api/software/highlighted` | The currently highlighted title | | `GET /api/builds` | Expected asset kinds per platform (build matrix) | | `GET /api/softwares/:name/builds` | Actual vs. missing build assets per release | -| `GET /api/image/:id` | Serves catalog images | +| `GET /api/image/:id` | Serves images — **the host's endpoint**, see *Images* | | `GET /api/download?path=` | Serves an artifact and logs a download record | | `GET /file/*path` | Serves static build output (web-playable games, docs) | | `POST /api/auth/device` | Starts a device sign-in; returns the code pair (404 without a client identity) | diff --git a/app/admin/application_tokens.rb b/app/admin/application_tokens.rb index 1b607d5..217a624 100644 --- a/app/admin/application_tokens.rb +++ b/app/admin/application_tokens.rb @@ -57,7 +57,7 @@ ActiveAdmin.register WarpEngine::ApplicationToken, as: "Application Token" do end action_item :rotate, only: :show do - if WarpEngine.woodpecker_configured? + if WarpEngine.ci.configured? link_to "Rotate Token", rotate_admin_application_token_path(resource), method: :post, data: { confirm: "This will revoke the current token, create a new one, and push it to Woodpecker. Continue?" } end @@ -101,7 +101,7 @@ ActiveAdmin.register WarpEngine::ApplicationToken, as: "Application Token" do success.html do session[:warp_engine_plain_token] = resource.plain_token - if WarpEngine.woodpecker_configured? + if WarpEngine.ci.configured? service = WarpEngine::SecretSyncService.new pipelines = service.pipelines_for_token(resource) if pipelines.any? @@ -124,7 +124,7 @@ ActiveAdmin.register WarpEngine::ApplicationToken, as: "Application Token" do end def destroy - if WarpEngine.woodpecker_configured? + if WarpEngine.ci.configured? WarpEngine::SecretSyncService.new.deprovision(resource) end resource.revoke! diff --git a/app/admin/images.rb b/app/admin/images.rb deleted file mode 100644 index 2f9dc77..0000000 --- a/app/admin/images.rb +++ /dev/null @@ -1,77 +0,0 @@ -ActiveAdmin.register WarpEngine::Image, as: "Image" do - permit_params :file_upload - - menu parent: "🌀 WarpEngine", priority: 5, label: "🖼️ Images" - - used_ids = -> { - WarpEngine::SoftwareImage.unscope(:order).distinct.pluck(:image_id) + - WarpEngine.config.image_owners.flat_map { |o| o[:image_ids].call } - } - - scope :all, default: true - scope("In use") { |scope| scope.where(id: used_ids.call) } - scope("Orphan") { |scope| scope.where.not(id: used_ids.call) } - - batch_action :delete_orphans, confirm: "Delete all selected orphan images and their files?" do |ids| - orphan_ids = ids.map(&:to_i) - used_ids.call - WarpEngine::Image.where(id: orphan_ids).find_each do |img| - File.delete(img.file_path) if File.exist?(img.file_path) - img.destroy - end - redirect_to admin_images_path, notice: "Deleted #{orphan_ids.size} orphan image(s)." - end - - index do - selectable_column - id_column - column :original_filename - column :content_type - column(:preview) do |img| - if File.exist?(img.file_path) - image_tag("/api/image/#{img.id}", style: "max-height:60px;max-width:120px;object-fit:contain;") - end - end - column(:usage) do |img| - uses = [] - uses << "#{img.software_images.size} software(s)" if img.software_images.any? - uses.concat(WarpEngine.config.image_owners.filter_map { |o| o[:usage_label].call(img) }) - uses.any? ? uses.join(", ") : status_tag("orphan", class: "warning") - end - column :created_at - actions - end - - filter :original_filename - filter :content_type - - show do - attributes_table do - row :id - row :original_filename - row :filename - row :content_type - row(:preview) do |img| - if File.exist?(img.file_path) - image_tag("/api/image/#{img.id}", style: "max-height:300px;max-width:100%;object-fit:contain;") - else - "File not found on disk" - end - end - row :created_at - row :updated_at - end - end - - form html: { multipart: true } do |f| - f.inputs do - f.input :file_upload, as: :file, label: "Image File" - end - f.actions - end - - controller do - def scoped_collection - super.includes(:software_images) - end - end -end diff --git a/app/admin/pipelines.rb b/app/admin/pipelines.rb index ac94d6b..ce603d5 100644 --- a/app/admin/pipelines.rb +++ b/app/admin/pipelines.rb @@ -33,7 +33,7 @@ ActiveAdmin.register WarpEngine::Pipeline, as: "Pipeline" do } column :last_pipeline_at actions defaults: true do |pipeline| - if pipeline.active && WarpEngine.woodpecker_configured? + if pipeline.active && WarpEngine.ci.configured? item "Trigger", trigger_admin_pipeline_path(pipeline), method: :post, class: "member_link" end end @@ -59,7 +59,7 @@ ActiveAdmin.register WarpEngine::Pipeline, as: "Pipeline" do sidebar "Details", only: :show do attributes_table_for resource do row :id - row :woodpecker_repo_id + row("Repo id") { |r| r.remote_repo_id } row :repo_owner row :repo_name row :platform @@ -74,22 +74,21 @@ ActiveAdmin.register WarpEngine::Pipeline, as: "Pipeline" do show do panel "Pipelines" do - if !WarpEngine.woodpecker_configured? - para "Woodpecker is not configured. Set woodpecker_url and woodpecker_api_token in the WarpEngine initializer.", + if !WarpEngine.ci.configured? + para "No CI provider is configured. Set c.ci_adapter in the WarpEngine initializer.", style: "color:#999;" elsif !resource.active para "This repository is inactive.", style: "color:#999;" else begin - pipelines = WarpEngine::PipelineService.new.list_pipelines(resource, page: 1) - if pipelines.is_a?(Array) && pipelines.any? - table_for pipelines.first(10) do - column("Number") { |p| p["number"] } - column("Status") { |p| status_tag p["status"], class: p["status"] == "success" ? "yes" : "no" } - column("Branch") { |p| p["branch"] } - column("Message") { |p| p["message"]&.truncate(60) } - - column("Created") { |p| p["created"] ? Time.zone.at(p["created"]).strftime("%Y-%m-%d %H:%M") : "-" } + runs = WarpEngine::PipelineService.new.runs(resource, page: 1) + if runs.any? + table_for runs.first(10) do + column("Number") { |r| r.number } + column("Status") { |r| status_tag r.status, class: r.success? ? "yes" : "no" } + column("Branch") { |r| r.branch } + column("Message") { |r| r.message&.truncate(60) } + column("Created") { |r| r.created_at ? r.created_at.strftime("%Y-%m-%d %H:%M") : "-" } end else para "No pipelines found.", style: "color:#999;" @@ -118,8 +117,8 @@ ActiveAdmin.register WarpEngine::Pipeline, as: "Pipeline" do end action_item :sync_repos, only: :index do - if WarpEngine.woodpecker_configured? - link_to "Sync from Woodpecker", sync_admin_pipelines_path, method: :post + if WarpEngine.ci.configured? + link_to "Sync from #{WarpEngine.ci.name}", sync_admin_pipelines_path, method: :post end end diff --git a/app/admin/softwares.rb b/app/admin/softwares.rb index 1e511a2..21ff3b6 100644 --- a/app/admin/softwares.rb +++ b/app/admin/softwares.rb @@ -27,8 +27,8 @@ ActiveAdmin.register WarpEngine::Software, as: "Software" do column(:releases) { |sw| sw.releases.size } column(:image) do |sw| si = sw.software_images.detect(&:is_default?) || sw.software_images.first - if si&.image && File.exist?(si.image.file_path) - image_tag "/api/image/#{si.image_id}", style: "max-height:40px;max-width:80px;object-fit:contain;" + if si&.image && WarpEngine::Images.available?(si.image) + image_tag WarpEngine::Images.url_for(si.image_id), style: "max-height:40px;max-width:80px;object-fit:contain;" end end column :created_at @@ -114,10 +114,10 @@ ActiveAdmin.register WarpEngine::Software, as: "Software" do f.inputs "Images" do f.has_many :software_images, allow_destroy: true, new_record: true do |si| img_hint = if si.object.image_id.present? - si.template.image_tag("/api/image/#{si.object.image_id}", style: "max-height:80px;max-width:160px;object-fit:contain;margin-top:6px;").html_safe + si.template.image_tag(WarpEngine::Images.url_for(si.object.image_id), style: "max-height:80px;max-width:160px;object-fit:contain;margin-top:6px;").html_safe end si.input :image_id, as: :select, label: "Image", - collection: WarpEngine::Image.order(:original_filename).map { |img| [ img.original_filename, img.id ] }, + collection: WarpEngine::Images.select_options, include_blank: "— select —", hint: img_hint si.input :file_upload, as: :file, label: "Upload new image" diff --git a/app/controllers/warp_engine/api/ci_controller.rb b/app/controllers/warp_engine/api/ci_controller.rb index 09884d9..c495010 100644 --- a/app/controllers/warp_engine/api/ci_controller.rb +++ b/app/controllers/warp_engine/api/ci_controller.rb @@ -3,7 +3,7 @@ module WarpEngine class CiController < ApiController include UpdateAuthentication - before_action :require_woodpecker! + before_action :require_ci! resource_description do short "CI pipeline management" @@ -11,7 +11,7 @@ module WarpEngine api :GET, "/api/ci/pipelines", "List active pipelines" returns code: 200, desc: "JSON array of tracked pipelines" - error code: 503, desc: "Woodpecker not configured" + error code: 503, desc: "No CI provider configured" def pipelines records = Pipeline.active.includes(:software) render json: records.map { |p| pipeline_json(p) } @@ -20,12 +20,12 @@ module WarpEngine api :GET, "/api/ci/pipelines/:id/status", "Get a pipeline with its latest run" param :id, :number, required: true, desc: "Pipeline id" returns code: 200, desc: "JSON with pipeline and latest run data" - error code: 503, desc: "Woodpecker not configured" + error code: 503, desc: "No CI provider configured" def status pipeline = Pipeline.find(params[:id]) latest_run = begin - PipelineService.new.pipeline_detail(pipeline, "latest") - rescue WoodpeckerClient::ApiError + PipelineService.new.run(pipeline, "latest") + rescue CI::Error nil end render json: { pipeline: pipeline_json(pipeline), latest_run: latest_run } @@ -37,7 +37,7 @@ module WarpEngine param :branch, String, required: false, desc: "Branch to build (default: main)" returns code: 200, desc: "JSON with triggered run data" error code: 401, desc: "Invalid secret" - error code: 503, desc: "Woodpecker not configured" + error code: 503, desc: "No CI provider configured" def trigger unless update_authorized?(required_scope: ApplicationToken::UPDATE_SCOPE) return render json: { error: "Unauthorized" }, status: :unauthorized @@ -50,16 +50,17 @@ module WarpEngine private - def require_woodpecker! - return if WarpEngine.woodpecker_configured? + def require_ci! + return if WarpEngine.ci.configured? - render json: { error: "Woodpecker not configured" }, status: :service_unavailable + render json: { error: "CI not configured" }, status: :service_unavailable end def pipeline_json(pipeline) { id: pipeline.id, - woodpecker_repo_id: pipeline.woodpecker_repo_id, + repo_id: pipeline.remote_repo_id, + woodpecker_repo_id: pipeline.remote_repo_id, repo_owner: pipeline.repo_owner, repo_name: pipeline.repo_name, platform: pipeline.platform, diff --git a/app/controllers/warp_engine/api/images_controller.rb b/app/controllers/warp_engine/api/images_controller.rb deleted file mode 100644 index 230e933..0000000 --- a/app/controllers/warp_engine/api/images_controller.rb +++ /dev/null @@ -1,17 +0,0 @@ -module WarpEngine - class Api::ImagesController < ApiController - resource_description do - short "Images" - formats [ "binary" ] - end - - api :GET, "/api/image/:id", "Get image by ID" - param :id, :number, required: true, desc: "Image ID" - returns code: 200, desc: "Image binary data" - error code: 404, desc: "Image not found" - def show - image = WarpEngine::ImageService.new.show(WarpEngine::ImageShowInputDto.new(id: params[:id])) - send_file image.file_path, type: image.content_type, disposition: "inline" - end - end -end diff --git a/app/controllers/warp_engine/build/configs_controller.rb b/app/controllers/warp_engine/build/configs_controller.rb index 01d457f..d7834eb 100644 --- a/app/controllers/warp_engine/build/configs_controller.rb +++ b/app/controllers/warp_engine/build/configs_controller.rb @@ -3,76 +3,60 @@ module WarpEngine class ConfigsController < ApiController resource_description do - short "Woodpecker CI pipeline configs" + short "CI pipeline configs" end api :GET, "/build/config", "Preview the generated pipeline config for a platform" - param :platform, String, required: true, desc: "Platform (a configured ci_platforms key, e.g. tic80)" + param :platform, String, required: true, desc: "Platform (a platform the CI adapter serves, e.g. tic80)" param :name, String, required: false, desc: "Software name substituted into the pipeline (default: example)" returns code: 200, desc: "Pipeline YAML (text/yaml)" error code: 404, desc: "Unknown platform" def show - yaml = render_config(platform: params[:platform], name: params[:name].presence || "example") - return render json: { error: "Unknown platform" }, status: :not_found if yaml.nil? + config = render_config(platform: params[:platform], name: params[:name].presence || "example") + return render json: { error: "Unknown platform" }, status: :not_found if config.nil? - render plain: yaml, content_type: "text/yaml" + render plain: config, content_type: "text/yaml" end - api :POST, "/build/config", "Woodpecker configuration extension endpoint" + api :POST, "/build/config", "CI configuration extension endpoint" description <<~DESC - Called by the Woodpecker server on every pipeline start (httpsig-signed request). - If the repo's .woodpecker.yaml is a marker (has a `platform:` key), responds with + Called by the CI server on every pipeline start (a signed request the CI adapter + verifies). If the repo's config is a marker (has a `platform:` key), responds with the generated pipeline; otherwise responds 204 so the repo's own config runs. DESC - returns code: 200, desc: %(JSON: {"configs": [{"name": ..., "data": ""}]}) + returns code: 200, desc: %(JSON: the adapter's config response — for Woodpecker {"configs": [{"name": ..., "data": ""}]}) returns code: 204, desc: "Not a marker config — keep the repo's own configuration" error code: 403, desc: "Missing or invalid request signature" error code: 422, desc: "Marker requests an unknown platform" def create - unless WarpEngine::CiSignatureVerifier.new(request).valid? + unless ci.verify_config_request(request) return render json: { error: "Invalid signature" }, status: :forbidden end - marker = find_marker + marker = ci.config_marker(params) return head :no_content if marker.nil? - platform = marker["platform"].to_s - name = marker["name"].presence || repo_name - yaml = render_config(platform: platform, name: name) - if yaml.nil? - return render json: { error: "Unknown platform: #{platform}" }, status: :unprocessable_entity + config = render_config(platform: marker[:platform], name: marker[:name]) + if config.nil? + return render json: { error: "Unknown platform: #{marker[:platform]}" }, status: :unprocessable_entity end - render json: { configs: [ { name: platform, data: yaml } ] } + render json: ci.config_response(platform: marker[:platform], config: config) end private + def ci + WarpEngine.ci + end + def render_config(platform:, name:) - WarpEngine::CiConfigService.new.render( + ci.pipeline_config( platform: platform, name: name, - update_server: WarpEngine.config.ci_update_server.presence || request.base_url + update_server: ci.update_server.presence || request.base_url ) end - - def find_marker - configs = params[:configuration].presence || params[:configs].presence || [] - configs.each do |config| - data = config[:data].to_s - parsed = begin - YAML.safe_load(data) - rescue Psych::Exception - nil - end - return parsed if parsed.is_a?(Hash) && parsed.key?("platform") - end - nil - end - - def repo_name - params.dig(:repo, :name).to_s - end end end end diff --git a/app/dtos/warp_engine/image_show_input_dto.rb b/app/dtos/warp_engine/image_show_input_dto.rb deleted file mode 100644 index 65af195..0000000 --- a/app/dtos/warp_engine/image_show_input_dto.rb +++ /dev/null @@ -1,3 +0,0 @@ -module WarpEngine - ImageShowInputDto = Struct.new(:id, keyword_init: true) -end diff --git a/app/jobs/warp_engine/application_job.rb b/app/jobs/warp_engine/application_job.rb index 70e313c..b127ad2 100644 --- a/app/jobs/warp_engine/application_job.rb +++ b/app/jobs/warp_engine/application_job.rb @@ -1,7 +1,7 @@ module WarpEngine class ApplicationJob < ActiveJob::Base - retry_on WoodpeckerClient::ConnectionError, wait: 30.seconds, attempts: 3 - discard_on WoodpeckerClient::ApiError do |job, error| + retry_on CI::ConnectionError, wait: 30.seconds, attempts: 3 + discard_on CI::ApiError, CI::NotConfigured do |job, error| Rails.logger.error("[#{job.class.name}] discarded: #{error.message}") end end diff --git a/app/models/warp_engine/image.rb b/app/models/warp_engine/image.rb deleted file mode 100644 index 5b2212f..0000000 --- a/app/models/warp_engine/image.rb +++ /dev/null @@ -1,36 +0,0 @@ -module WarpEngine - class Image < ApplicationRecord - def self.upload_path - WarpEngine.config.image_container_path - end - - has_many :software_images, dependent: :restrict_with_error - - default_scope { where(deleted_at: nil) } - - attr_accessor :file_upload - - before_save :process_upload, if: -> { file_upload.present? } - - def self.ransackable_attributes(auth_object = nil) - %w[content_type created_at deleted_at filename id original_filename updated_at] - end - - def file_path - File.join(self.class.upload_path, filename.to_s) - end - - ActiveSupport.run_load_hooks(:warp_engine_image, self) - - private - - def process_upload - FileUtils.mkdir_p(self.class.upload_path) - self.original_filename = file_upload.original_filename - self.content_type = file_upload.content_type.presence || "application/octet-stream" - ext = File.extname(file_upload.original_filename) - self.filename = "#{SecureRandom.uuid}#{ext}" - IO.copy_stream(file_upload.to_io, file_path) - end - end -end diff --git a/app/models/warp_engine/pipeline.rb b/app/models/warp_engine/pipeline.rb index 9500434..71c25be 100644 --- a/app/models/warp_engine/pipeline.rb +++ b/app/models/warp_engine/pipeline.rb @@ -24,6 +24,14 @@ module WarpEngine "#{repo_owner}/#{repo_name}" end + def remote_repo_id + woodpecker_repo_id + end + + def remote_repo_id=(value) + self.woodpecker_repo_id = value + end + def self.ransackable_attributes(auth_object = nil) %w[active created_at deleted_at id last_pipeline_at last_pipeline_status platform repo_name repo_owner software_id woodpecker_repo_id] diff --git a/app/models/warp_engine/software_image.rb b/app/models/warp_engine/software_image.rb index 3a0165c..3a396fc 100644 --- a/app/models/warp_engine/software_image.rb +++ b/app/models/warp_engine/software_image.rb @@ -1,7 +1,7 @@ module WarpEngine class SoftwareImage < ApplicationRecord belongs_to :software - belongs_to :image, optional: true + belongs_to :image, optional: true, class_name: WarpEngine::Images.model_name attr_accessor :file_upload @@ -28,7 +28,7 @@ module WarpEngine private def create_image_from_upload - img = Image.new(file_upload: file_upload) + img = WarpEngine::Images.build_from_upload(file_upload) if img.save self.image = img else diff --git a/app/serializers/warp_engine/software_serializer.rb b/app/serializers/warp_engine/software_serializer.rb index 68f4965..ac2d048 100644 --- a/app/serializers/warp_engine/software_serializer.rb +++ b/app/serializers/warp_engine/software_serializer.rb @@ -21,11 +21,11 @@ module WarpEngine field(:platformLinks) { |sw| PlatformLinkSerializer.render_as_hash(WarpEngine::PlatformLink.for_platform(sw.platform)) } field(:imageUrl) { |sw| si = sw.software_images.detect(&:is_default?) || sw.software_images.first - si ? "/api/image/#{si.image_id}" : nil + si ? WarpEngine::Images.url_for(si.image_id) : nil } field(:images) { |sw| sw.software_images.map { |si| - { url: "/api/image/#{si.image_id}", isDefault: si.is_default?, position: si.position } + { url: WarpEngine::Images.url_for(si.image_id), isDefault: si.is_default?, position: si.position } } } end diff --git a/app/services/warp_engine/ci_config_service.rb b/app/services/warp_engine/ci_config_service.rb deleted file mode 100644 index b82fbc6..0000000 --- a/app/services/warp_engine/ci_config_service.rb +++ /dev/null @@ -1,37 +0,0 @@ -require "erb" - -module WarpEngine - - class CiConfigService - PLATFORM_FORMAT = /\A[a-z0-9_-]+\z/ - - def render(platform:, name:, update_server:) - platform = platform.to_s - return nil unless platform.match?(PLATFORM_FORMAT) - - spec = platform_spec(platform) - return nil if spec.nil? - - path = templates_dir.join(platform, "pipeline.yaml.erb") - return nil unless path.exist? - - ERB.new(path.read, trim_mode: "-").result_with_hash( - name: name.to_s, - update_server: update_server.to_s, - builder: spec[:builder], - exporter: spec[:exporter] - ) - end - - private - - def platform_spec(platform) - spec = WarpEngine.config.ci_platforms.stringify_keys[platform] - spec&.symbolize_keys - end - - def templates_dir - WarpEngine::Engine.root.join("app", "services", "warp_engine", "platforms") - end - end -end diff --git a/app/services/warp_engine/ci_signature_verifier.rb b/app/services/warp_engine/ci_signature_verifier.rb deleted file mode 100644 index ef18360..0000000 --- a/app/services/warp_engine/ci_signature_verifier.rb +++ /dev/null @@ -1,142 +0,0 @@ -require "openssl" -require "base64" -require "net/http" -require "digest" - -module WarpEngine - - class CiSignatureVerifier - CAVAGE_PARAM = /(\w+)="([^"]*)"/ - - @key_cache = {} - @key_mutex = Mutex.new - - class << self - - def fetch_public_key(url) - @key_mutex.synchronize do - @key_cache[url] ||= Net::HTTP.get(URI.parse(url)) - end - end - - def reset_key_cache! - @key_mutex.synchronize { @key_cache = {} } - end - end - - def initialize(request) - @request = request - end - - def valid? - pem = public_key_pem - if pem.blank? - Rails.logger.error("[CiSignatureVerifier] no ci_extension_public_key(_url) configured — rejecting request") - return false - end - - key = OpenSSL::PKey.read(pem) - if @request.headers["Signature-Input"].present? - rfc9421_valid?(key) - else - cavage_valid?(key) - end - rescue OpenSSL::PKey::PKeyError, ArgumentError => e - Rails.logger.error("[CiSignatureVerifier] #{e.class}: #{e.message}") - false - end - - private - - def public_key_pem - config = WarpEngine.config - return config.ci_extension_public_key if config.ci_extension_public_key.present? - return nil if config.ci_extension_public_key_url.blank? - - self.class.fetch_public_key(config.ci_extension_public_key_url) - rescue StandardError => e - Rails.logger.error("[CiSignatureVerifier] public key fetch failed: #{e.class}: #{e.message}") - nil - end - - def rfc9421_valid?(key) - input = @request.headers["Signature-Input"].to_s - match = input.match(/\A\s*([\w.-]+)=(\(.*)\z/m) - return false if match.nil? - - label, inner = match[1], match[2] - components = inner[/\((.*?)\)/m, 1].to_s.scan(/"([^"]*)"/).flatten - return false if components.empty? - - signature = @request.headers["Signature"].to_s[/#{Regexp.escape(label)}=:([A-Za-z0-9+\/=]+):/, 1] - return false if signature.blank? - return false unless content_digest_valid?(components) - - lines = components.map do |component| - value = component_value(component) - return false if value.nil? - %("#{component}": #{value}) - end - lines << %("@signature-params": #{inner}) - - key.verify(nil, Base64.decode64(signature), lines.join("\n")) - end - - def component_value(name) - case name - when "@request-target" then @request.fullpath - when "@method" then @request.request_method - when "@target-uri" then @request.original_url - when "@authority" then @request.host_with_port - when "@path" then @request.path - when "@query" then "?#{@request.query_string}" - when /\A@/ then nil - else @request.headers[name] - end - end - - def content_digest_valid?(components) - return true unless components.include?("content-digest") - - digest = @request.headers["Content-Digest"].to_s[/sha-256=:([A-Za-z0-9+\/=]+):/, 1] - return false if digest.blank? - - expected = Digest::SHA256.base64digest(@request.raw_post) - ActiveSupport::SecurityUtils.secure_compare(digest, expected) - end - - def cavage_valid?(key) - params = cavage_params - return false if params.nil? || params["signature"].blank? - - signing_string = cavage_signing_string(params.fetch("headers", "date")) - return false if signing_string.nil? - - key.verify(nil, Base64.decode64(params["signature"]), signing_string) - end - - def cavage_params - header = @request.headers["Signature"].presence - if header.nil? - auth = @request.headers["Authorization"].to_s - header = auth.delete_prefix("Signature ") if auth.start_with?("Signature ") - end - return nil if header.blank? - - header.scan(CAVAGE_PARAM).to_h - end - - def cavage_signing_string(headers_list) - lines = headers_list.split(" ").map do |name| - if name == "(request-target)" - "(request-target): #{@request.request_method.downcase} #{@request.fullpath}" - else - value = @request.headers[name] - return nil if value.nil? - "#{name.downcase}: #{value}" - end - end - lines.join("\n") - end - end -end diff --git a/app/services/warp_engine/image_service.rb b/app/services/warp_engine/image_service.rb deleted file mode 100644 index 69be38d..0000000 --- a/app/services/warp_engine/image_service.rb +++ /dev/null @@ -1,7 +0,0 @@ -module WarpEngine - class ImageService - def show(input) - WarpEngine::Image.find(input.id) - end - end -end diff --git a/app/services/warp_engine/pipeline_service.rb b/app/services/warp_engine/pipeline_service.rb index 1535b68..c686de1 100644 --- a/app/services/warp_engine/pipeline_service.rb +++ b/app/services/warp_engine/pipeline_service.rb @@ -1,31 +1,31 @@ module WarpEngine class PipelineService - def initialize(client: WoodpeckerClient.new) - @client = client + def initialize(ci: WarpEngine.ci) + @ci = ci end def trigger(pipeline, branch: "main") - @client.trigger_pipeline(pipeline.woodpecker_repo_id, branch: branch) + @ci.trigger(pipeline.remote_repo_id, branch: branch) end - def list_pipelines(pipeline, page: 1) - runs = @client.list_pipelines(pipeline.woodpecker_repo_id, page: page) - refresh_last_pipeline(pipeline, runs.first) if page == 1 && runs.is_a?(Array) + def runs(pipeline, page: 1) + runs = @ci.runs(pipeline.remote_repo_id, page: page) + refresh_last_run(pipeline, runs.first) if page == 1 runs end - def pipeline_detail(pipeline, number) - @client.get_pipeline(pipeline.woodpecker_repo_id, number) + def run(pipeline, number) + @ci.run(pipeline.remote_repo_id, number) end private - def refresh_last_pipeline(pipeline, run) + def refresh_last_run(pipeline, run) return unless run && pipeline.persisted? pipeline.update_columns( - last_pipeline_status: run["status"], - last_pipeline_at: run["created"] ? Time.zone.at(run["created"]) : nil + last_pipeline_status: run.status, + last_pipeline_at: run.created_at ) end end diff --git a/app/services/warp_engine/pipeline_sync_service.rb b/app/services/warp_engine/pipeline_sync_service.rb index 5259d0b..e357aae 100644 --- a/app/services/warp_engine/pipeline_sync_service.rb +++ b/app/services/warp_engine/pipeline_sync_service.rb @@ -1,32 +1,26 @@ module WarpEngine class PipelineSyncService - def initialize(client: WoodpeckerClient.new) - @client = client + def initialize(ci: WarpEngine.ci) + @ci = ci end def sync_all - remote_repos = @client.list_repos + remote_repos = @ci.repos results = { created: [], updated: [], deactivated: [] } - remote_ids = remote_repos.map { |r| r["id"] } + remote_ids = remote_repos.map(&:id) remote_repos.each do |remote| - record = Pipeline.unscoped.find_or_initialize_by( - woodpecker_repo_id: remote["id"] - ) + record = Pipeline.unscoped.find_or_initialize_by(woodpecker_repo_id: remote.id) was_new = record.new_record? record.assign_attributes( - repo_name: remote["name"], - repo_owner: remote["owner"], - active: remote["active"], + repo_name: remote.name, + repo_owner: remote.owner, + active: remote.active?, deleted_at: nil ) - if record.platform.blank? || record.platform == Pipeline::UNKNOWN_PLATFORM - sw = Software.find_by(name: remote["name"]) - record.platform = sw&.platform || Pipeline::UNKNOWN_PLATFORM - record.software = sw if sw - end + assign_platform(record, remote.name) next unless record.save @@ -42,12 +36,12 @@ module WarpEngine end def activate(repo_id) - @client.activate_repo(repo_id) + @ci.activate_repo(repo_id) sync_single(repo_id) end def deactivate(repo_id) - @client.deactivate_repo(repo_id) + @ci.deactivate_repo(repo_id) record = Pipeline.find_by!(woodpecker_repo_id: repo_id) record.update!(active: false) end @@ -55,19 +49,23 @@ module WarpEngine private def sync_single(repo_id) - remote = @client.get_repo(repo_id) + remote = @ci.repo(repo_id) record = Pipeline.unscoped.find_or_initialize_by(woodpecker_repo_id: repo_id) record.assign_attributes( - repo_name: remote["name"], repo_owner: remote["owner"], - active: remote["active"], deleted_at: nil + repo_name: remote.name, repo_owner: remote.owner, + active: remote.active?, deleted_at: nil ) - if record.platform.blank? - sw = Software.find_by(name: remote["name"]) - record.platform = sw&.platform || Pipeline::UNKNOWN_PLATFORM - record.software = sw if sw - end + assign_platform(record, remote.name) record.save! record end + + def assign_platform(record, repo_name) + return unless record.platform.blank? || record.platform == Pipeline::UNKNOWN_PLATFORM + + sw = Software.find_by(name: repo_name) + record.platform = sw&.platform || Pipeline::UNKNOWN_PLATFORM + record.software = sw if sw + end end end diff --git a/app/services/warp_engine/secret_sync_service.rb b/app/services/warp_engine/secret_sync_service.rb index 7eb3f11..1c7a3da 100644 --- a/app/services/warp_engine/secret_sync_service.rb +++ b/app/services/warp_engine/secret_sync_service.rb @@ -2,21 +2,17 @@ module WarpEngine class SecretSyncService SECRET_NAME = "application_token".freeze - def initialize(client: WoodpeckerClient.new) - @client = client + def initialize(ci: WarpEngine.ci) + @ci = ci end def provision(plain_token, pipelines:) results = { synced: [], failed: [] } - pipelines.each do |pipeline| - if secret_exists?(pipeline.woodpecker_repo_id) - @client.update_secret(pipeline.woodpecker_repo_id, SECRET_NAME, value: plain_token) - else - @client.create_secret(pipeline.woodpecker_repo_id, name: SECRET_NAME, value: plain_token) - end + Array(pipelines).each do |pipeline| + @ci.secret_set(pipeline.remote_repo_id, name: SECRET_NAME, value: plain_token) results[:synced] << pipeline - rescue WoodpeckerClient::ApiError, WoodpeckerClient::ConnectionError => e + rescue CI::Error => e Rails.logger.error("[SecretSyncService] failed for #{pipeline.full_name}: #{e.message}") results[:failed] << { pipeline: pipeline, error: e.message } end @@ -24,14 +20,18 @@ module WarpEngine results end - def deprovision(application_token) - pipelines_for_token(application_token).each do |pipeline| - @client.delete_secret(pipeline.woodpecker_repo_id, SECRET_NAME) - rescue WoodpeckerClient::ApiError => e + def remove(pipelines:) + Array(pipelines).each do |pipeline| + @ci.secret_delete(pipeline.remote_repo_id, SECRET_NAME) + rescue CI::Error => e Rails.logger.warn("[SecretSyncService] delete failed for #{pipeline.full_name}: #{e.message}") end end + def deprovision(application_token) + remove(pipelines: pipelines_for_token(application_token)) + end + def rotate(application_token) pipelines = pipelines_for_token(application_token) return { rotated: false, reason: "no pipelines" } if pipelines.empty? @@ -67,14 +67,5 @@ module WarpEngine Pipeline.active.where(software_id: software_ids).to_a end end - - private - - def secret_exists?(repo_id) - secrets = @client.list_secrets(repo_id) - secrets.any? { |s| s["name"] == SECRET_NAME } - rescue WoodpeckerClient::ApiError - false - end end end diff --git a/app/services/warp_engine/woodpecker_client.rb b/app/services/warp_engine/woodpecker_client.rb deleted file mode 100644 index bc9cca9..0000000 --- a/app/services/warp_engine/woodpecker_client.rb +++ /dev/null @@ -1,153 +0,0 @@ -require "net/http" -require "json" -require "uri" - -module WarpEngine - class WoodpeckerClient - class ApiError < StandardError - attr_reader :status, :body - - def initialize(message, status:, body: nil) - super(message) - @status = status - @body = body - end - end - - class ConnectionError < StandardError; end - - def initialize(base_url: nil, token: nil) - @base_url = (base_url || WarpEngine.config.woodpecker_url).to_s.chomp("/") - @token = token || WarpEngine.config.woodpecker_api_token - end - - def list_repos - get("/api/repos") - end - - def get_repo(repo_id) - get("/api/repos/#{repo_id}") - end - - def activate_repo(repo_id) - post("/api/repos", body: { id: repo_id }) - end - - def deactivate_repo(repo_id) - delete("/api/repos/#{repo_id}") - end - - def list_secrets(repo_id) - get("/api/repos/#{repo_id}/secrets") - end - - def create_secret(repo_id, name:, value:, events: %w[push tag deployment]) - post("/api/repos/#{repo_id}/secrets", - body: { name: name, value: value, events: events }) - end - - def update_secret(repo_id, secret_name, value:) - patch("/api/repos/#{repo_id}/secrets/#{secret_name}", - body: { value: value }) - end - - def delete_secret(repo_id, secret_name) - delete("/api/repos/#{repo_id}/secrets/#{secret_name}") - end - - def list_pipelines(repo_id, page: 1, per_page: 25) - get("/api/repos/#{repo_id}/pipelines", - params: { page: page, perPage: per_page }) - end - - def latest_pipeline(repo_id) - get("/api/repos/#{repo_id}/pipelines/latest") - end - - def get_pipeline(repo_id, number) - get("/api/repos/#{repo_id}/pipelines/#{number}") - end - - def trigger_pipeline(repo_id, branch: "main") - post("/api/repos/#{repo_id}/pipelines", - body: { branch: branch }) - end - - private - - def get(path, params: {}) - uri = build_uri(path, params) - request = Net::HTTP::Get.new(uri) - execute(uri, request) - end - - def post(path, body: {}) - uri = build_uri(path) - request = Net::HTTP::Post.new(uri) - request.body = body.to_json - request.content_type = "application/json" - execute(uri, request) - end - - def patch(path, body: {}) - uri = build_uri(path) - request = Net::HTTP::Patch.new(uri) - request.body = body.to_json - request.content_type = "application/json" - execute(uri, request) - end - - def delete(path) - uri = build_uri(path) - request = Net::HTTP::Delete.new(uri) - execute(uri, request) - end - - def build_uri(path, params = {}) - uri = URI.parse("#{@base_url}#{path}") - uri.query = URI.encode_www_form(params) if params.any? - uri - end - - def execute(uri, request) - request["Authorization"] = "Bearer #{@token}" - request["Accept"] = "application/json" - - response = Net::HTTP.start(uri.hostname, uri.port, - use_ssl: uri.scheme == "https", - open_timeout: 10, - read_timeout: 30) do |http| - http.request(request) - end - - handle_response(uri, response) - rescue Errno::ECONNREFUSED, Errno::EHOSTUNREACH, Net::OpenTimeout, - Net::ReadTimeout, SocketError => e - raise ConnectionError, "Cannot reach Woodpecker at #{@base_url}: #{e.message}" - end - - def handle_response(uri, response) - case response - when Net::HTTPSuccess, Net::HTTPNoContent - return nil if response.body.blank? - begin - JSON.parse(response.body) - rescue JSON::ParserError - - raise ApiError.new( - "Expected JSON from #{uri.path} but got: #{response.body.truncate(80)}", - status: response.code.to_i, body: response.body - ) - end - when Net::HTTPNotFound - raise ApiError.new("Not found: #{uri.path}", status: 404, body: response.body) - else - raise ApiError.new( - "Woodpecker API error #{response.code}: #{response.body&.truncate(200)}", - status: response.code.to_i, - body: response.body - ) - end - end - end -end diff --git a/config/routes.rb b/config/routes.rb index 237fa0f..b8990bd 100644 --- a/config/routes.rb +++ b/config/routes.rb @@ -10,7 +10,6 @@ WarpEngine::Engine.routes.draw do get "software", to: "software#index" get "software/highlighted", to: "software_highlighted#index" - get "image/:id", to: "images#show" get "download", to: "downloads#show" get "builds", to: "builds#index" get "softwares/:name/builds", to: "software_builds#show" diff --git a/examples/compose/host_app/app/controllers/api/images_controller.rb b/examples/compose/host_app/app/controllers/api/images_controller.rb new file mode 100644 index 0000000..e61f46a --- /dev/null +++ b/examples/compose/host_app/app/controllers/api/images_controller.rb @@ -0,0 +1,16 @@ +module Api + class ImagesController < ActionController::API + rescue_from ActiveRecord::RecordNotFound do + render json: { error: "Not found" }, status: :not_found + end + + rescue_from Errno::ENOENT do + render json: { error: "Not found" }, status: :not_found + end + + def show + image = Image.find(params[:id]) + send_file image.file_path, type: image.content_type, disposition: "inline" + end + end +end diff --git a/examples/compose/host_app/app/models/image.rb b/examples/compose/host_app/app/models/image.rb new file mode 100644 index 0000000..b2263dd --- /dev/null +++ b/examples/compose/host_app/app/models/image.rb @@ -0,0 +1,32 @@ +class Image < ApplicationRecord + def self.upload_path + ENV.fetch("IMAGE_CONTAINER_PATH", "/images") + end + + has_many :software_images, class_name: "WarpEngine::SoftwareImage", dependent: :restrict_with_error + + default_scope { where(deleted_at: nil) } + + attr_accessor :file_upload + + before_save :process_upload, if: -> { file_upload.present? } + + def self.ransackable_attributes(auth_object = nil) + %w[content_type created_at deleted_at filename id original_filename updated_at] + end + + def file_path + File.join(self.class.upload_path, filename.to_s) + end + + private + + def process_upload + FileUtils.mkdir_p(self.class.upload_path) + self.original_filename = file_upload.original_filename + self.content_type = file_upload.content_type.presence || "application/octet-stream" + ext = File.extname(file_upload.original_filename) + self.filename = "#{SecureRandom.uuid}#{ext}" + IO.copy_stream(file_upload.to_io, file_path) + end +end diff --git a/examples/compose/host_app/config/initializers/warp_engine.rb b/examples/compose/host_app/config/initializers/warp_engine.rb index f32e484..7c4def7 100644 --- a/examples/compose/host_app/config/initializers/warp_engine.rb +++ b/examples/compose/host_app/config/initializers/warp_engine.rb @@ -1,8 +1,15 @@ Rails.application.config.to_prepare do WarpEngine.configure do |c| - c.file_container_path = ENV.fetch("FILE_CONTAINER_PATH", "/softwares") - c.image_container_path = ENV.fetch("IMAGE_CONTAINER_PATH", "/images") + c.file_container_path = ENV.fetch("FILE_CONTAINER_PATH", "/softwares") c.update_secret = ENV["UPDATE_SECRET"] + + c.image_class_name = "Image" + + c.ci_adapter = WarpEngine::CI::Woodpecker::Adapter.new( + url: ENV["WOODPECKER_URL"], + api_token: ENV["WOODPECKER_API_TOKEN"], + repo_owner: ENV["WOODPECKER_REPO_OWNER"] + ) end end diff --git a/examples/compose/host_app/config/routes.rb b/examples/compose/host_app/config/routes.rb index d6ec744..447ee14 100644 --- a/examples/compose/host_app/config/routes.rb +++ b/examples/compose/host_app/config/routes.rb @@ -1,5 +1,9 @@ Rails.application.routes.draw do apipie + namespace :api do + get "image/:id", to: "images#show" + end + mount WarpEngine::Engine => "/" end diff --git a/lib/generators/warp_engine/install/install_generator.rb b/lib/generators/warp_engine/install/install_generator.rb index 7bdd761..cd9c68f 100644 --- a/lib/generators/warp_engine/install/install_generator.rb +++ b/lib/generators/warp_engine/install/install_generator.rb @@ -17,6 +17,12 @@ module WarpEngine template "initializer.rb", "config/initializers/warp_engine.rb" end + def copy_image_library + template "image.rb", "app/models/image.rb" + template "images_controller.rb", "app/controllers/api/images_controller.rb" + migration_template "create_images.rb", "db/migrate/create_images.rb" + end + def copy_migration migration_template "create_warp_engine_tables.rb", "db/migrate/create_warp_engine_tables.rb" end @@ -27,7 +33,11 @@ module WarpEngine WarpEngine telepítve. Következő lépések: 1. rails db:migrate 2. mount WarpEngine::Engine => "/" a config/routes.rb végére - 3. állítsd be a config/initializers/warp_engine.rb-t + 3. a képkönyvtár a hosztod: a routes.rb-be + namespace :api do + get "image/:id", to: "images#show" + end + 4. állítsd be a config/initializers/warp_engine.rb-t MSG end end diff --git a/lib/generators/warp_engine/install/templates/create_images.rb b/lib/generators/warp_engine/install/templates/create_images.rb new file mode 100644 index 0000000..2ecd14d --- /dev/null +++ b/lib/generators/warp_engine/install/templates/create_images.rb @@ -0,0 +1,12 @@ +class CreateImages < ActiveRecord::Migration[8.0] + def change + create_table :images do |t| + t.string :filename, null: false + t.string :original_filename, null: false + t.string :content_type, default: "application/octet-stream", null: false + t.datetime :deleted_at, precision: 3 + t.timestamps + t.index :deleted_at + end + end +end diff --git a/lib/generators/warp_engine/install/templates/create_warp_engine_tables.rb b/lib/generators/warp_engine/install/templates/create_warp_engine_tables.rb index d19679a..b78a73d 100644 --- a/lib/generators/warp_engine/install/templates/create_warp_engine_tables.rb +++ b/lib/generators/warp_engine/install/templates/create_warp_engine_tables.rb @@ -60,21 +60,13 @@ class CreateWarpEngineTables < ActiveRecord::Migration[8.0] t.index :deleted_at end - create_table :images do |t| - t.string :filename, null: false - t.string :original_filename, null: false - t.string :content_type, default: "application/octet-stream", null: false - t.datetime :deleted_at, precision: 3 - t.timestamps - t.index :deleted_at - end - create_table :software_images do |t| t.references :software, null: false, foreign_key: { on_delete: :cascade } - t.references :image, null: false, foreign_key: true, index: false + t.bigint :image_id, null: false t.boolean :is_default, default: false, null: false t.integer :position, default: 0, null: false t.timestamps + t.index :image_id t.index [ :software_id, :image_id ], unique: true t.index [ :software_id, :position ] end diff --git a/lib/generators/warp_engine/install/templates/image.rb b/lib/generators/warp_engine/install/templates/image.rb new file mode 100644 index 0000000..b2263dd --- /dev/null +++ b/lib/generators/warp_engine/install/templates/image.rb @@ -0,0 +1,32 @@ +class Image < ApplicationRecord + def self.upload_path + ENV.fetch("IMAGE_CONTAINER_PATH", "/images") + end + + has_many :software_images, class_name: "WarpEngine::SoftwareImage", dependent: :restrict_with_error + + default_scope { where(deleted_at: nil) } + + attr_accessor :file_upload + + before_save :process_upload, if: -> { file_upload.present? } + + def self.ransackable_attributes(auth_object = nil) + %w[content_type created_at deleted_at filename id original_filename updated_at] + end + + def file_path + File.join(self.class.upload_path, filename.to_s) + end + + private + + def process_upload + FileUtils.mkdir_p(self.class.upload_path) + self.original_filename = file_upload.original_filename + self.content_type = file_upload.content_type.presence || "application/octet-stream" + ext = File.extname(file_upload.original_filename) + self.filename = "#{SecureRandom.uuid}#{ext}" + IO.copy_stream(file_upload.to_io, file_path) + end +end diff --git a/lib/generators/warp_engine/install/templates/images_controller.rb b/lib/generators/warp_engine/install/templates/images_controller.rb new file mode 100644 index 0000000..e61f46a --- /dev/null +++ b/lib/generators/warp_engine/install/templates/images_controller.rb @@ -0,0 +1,16 @@ +module Api + class ImagesController < ActionController::API + rescue_from ActiveRecord::RecordNotFound do + render json: { error: "Not found" }, status: :not_found + end + + rescue_from Errno::ENOENT do + render json: { error: "Not found" }, status: :not_found + end + + def show + image = Image.find(params[:id]) + send_file image.file_path, type: image.content_type, disposition: "inline" + end + end +end diff --git a/lib/generators/warp_engine/install/templates/initializer.rb b/lib/generators/warp_engine/install/templates/initializer.rb index f5dd49d..63dc74a 100644 --- a/lib/generators/warp_engine/install/templates/initializer.rb +++ b/lib/generators/warp_engine/install/templates/initializer.rb @@ -1,4 +1,17 @@ Rails.application.config.to_prepare do WarpEngine.configure do |c| + c.file_container_path = ENV.fetch("FILE_CONTAINER_PATH", "/softwares") + + c.update_secret = ENV["UPDATE_SECRET"] + + c.image_class_name = "Image" + + c.ci_adapter = WarpEngine::CI::Woodpecker::Adapter.new( + url: ENV["WOODPECKER_URL"], + api_token: ENV["WOODPECKER_API_TOKEN"], + repo_owner: ENV["WOODPECKER_REPO_OWNER"], + public_key_url: ENV["WOODPECKER_PUBLIC_KEY_URL"], + platforms: {} + ) end end diff --git a/lib/warp_engine.rb b/lib/warp_engine.rb index c30143f..3442782 100644 --- a/lib/warp_engine.rb +++ b/lib/warp_engine.rb @@ -7,6 +7,8 @@ require "warp_engine/version" require "warp_engine/configuration" require "warp_engine/storage" require "warp_engine/access" +require "warp_engine/images" +require "warp_engine/ci" module WarpEngine @@ -22,10 +24,6 @@ module WarpEngine yield(config) end - def self.woodpecker_configured? - config.woodpecker_url.present? && config.woodpecker_api_token.present? - end - def self.instruments_publish? true end @@ -34,6 +32,14 @@ module WarpEngine Storage.adapter end + def self.images + Images.adapter + end + + def self.ci + CI.adapter + end + def self.access_policy AccessPolicy.current end diff --git a/lib/warp_engine/ci.rb b/lib/warp_engine/ci.rb new file mode 100644 index 0000000..6b34963 --- /dev/null +++ b/lib/warp_engine/ci.rb @@ -0,0 +1,94 @@ +module WarpEngine + + module CI + class Error < StandardError; end + + class ConnectionError < Error; end + + class NotConfigured < Error; end + + class ApiError < Error + attr_reader :status, :body + + def initialize(message, status: nil, body: nil) + super(message) + @status = status + @body = body + end + end + + Repo = Struct.new(:id, :name, :owner, :active, :raw, keyword_init: true) do + def active? = active ? true : false + def full_name = "#{owner}/#{name}" + end + + Run = Struct.new(:number, :status, :branch, :message, :created_at, :url, :raw, keyword_init: true) do + def success? = status.to_s == "success" + + def as_json(*) + { + number: number, + status: status, + branch: branch, + message: message, + createdAt: created_at&.utc&.iso8601, + url: url + } + end + end + + class Null + MESSAGE = "no CI adapter is configured (WarpEngine.config.ci_adapter)".freeze + + def name = "none" + def configured? = false + def platforms = {} + def update_server = nil + + def repos = raise(NotConfigured, MESSAGE) + def repo(_id) = raise(NotConfigured, MESSAGE) + def activate_repo(_id) = raise(NotConfigured, MESSAGE) + def deactivate_repo(_id) = raise(NotConfigured, MESSAGE) + def runs(_repo_id, page: 1) = raise(NotConfigured, MESSAGE) + def run(_repo_id, _number) = raise(NotConfigured, MESSAGE) + def trigger(_repo_id, branch: nil) = raise(NotConfigured, MESSAGE) + def secret_names(_repo_id) = raise(NotConfigured, MESSAGE) + def secret_set(_repo_id, name:, value:) = raise(NotConfigured, MESSAGE) + def secret_delete(_repo_id, _name) = raise(NotConfigured, MESSAGE) + + def verify_config_request(_request) = false + def config_marker(_params) = nil + def pipeline_config(platform:, name:, update_server:) = nil + def config_response(platform:, config:) = {} + end + + class << self + def adapter + configured = WarpEngine.config.ci_adapter + + case configured + when nil, :woodpecker, "woodpecker" then woodpecker_adapter + when :none, "none", :null then null_adapter + else configured + end + end + + def woodpecker? = adapter.is_a?(Woodpecker::Adapter) + + def woodpecker_adapter + @woodpecker_adapter ||= Woodpecker::Adapter.new + end + + def null_adapter + @null_adapter ||= Null.new + end + + def reset! + @woodpecker_adapter = nil + @null_adapter = nil + end + end + end +end + +require "warp_engine/ci/woodpecker" diff --git a/lib/warp_engine/ci/woodpecker.rb b/lib/warp_engine/ci/woodpecker.rb new file mode 100644 index 0000000..493994d --- /dev/null +++ b/lib/warp_engine/ci/woodpecker.rb @@ -0,0 +1,4 @@ +require "warp_engine/ci/woodpecker/client" +require "warp_engine/ci/woodpecker/signature_verifier" +require "warp_engine/ci/woodpecker/pipeline_config" +require "warp_engine/ci/woodpecker/adapter" diff --git a/lib/warp_engine/ci/woodpecker/adapter.rb b/lib/warp_engine/ci/woodpecker/adapter.rb new file mode 100644 index 0000000..28434d5 --- /dev/null +++ b/lib/warp_engine/ci/woodpecker/adapter.rb @@ -0,0 +1,146 @@ +module WarpEngine + module CI + module Woodpecker + class Adapter + SECRET_EVENTS = %w[push tag deployment].freeze + + attr_reader :url, :repo_owner, :update_server + + def initialize(url: ENV["WOODPECKER_URL"], + api_token: ENV["WOODPECKER_API_TOKEN"], + repo_owner: ENV["WOODPECKER_REPO_OWNER"], + public_key: nil, + public_key_url: nil, + platforms: {}, + update_server: nil) + @url = url.presence + @api_token = api_token.presence + @repo_owner = repo_owner.presence + @public_key = public_key.presence + @public_key_url = public_key_url.presence + @update_server = update_server.presence + @config = PipelineConfig.new(platforms: platforms || {}) + end + + def name = "Woodpecker" + + def configured? = @url.present? && @api_token.present? + + def platforms = @config.platforms + + def client + @client ||= Client.new(url: @url, token: @api_token) + end + + def repos + Array(client.list_repos).map { |remote| to_repo(remote) } + end + + def repo(repo_id) + to_repo(client.get_repo(repo_id)) + end + + def activate_repo(repo_id) + client.activate_repo(repo_id) + nil + end + + def deactivate_repo(repo_id) + client.deactivate_repo(repo_id) + nil + end + + def runs(repo_id, page: 1) + Array(client.list_pipelines(repo_id, page: page)).map { |run| to_run(run) } + end + + def run(repo_id, number) + remote = if number.to_s == "latest" + client.latest_pipeline(repo_id) + else + client.get_pipeline(repo_id, number) + end + remote && to_run(remote) + end + + def trigger(repo_id, branch: "main") + to_run(client.trigger_pipeline(repo_id, branch: branch || "main")) + end + + def secret_names(repo_id) + Array(client.list_secrets(repo_id)).map { |secret| secret["name"] } + end + + def secret_set(repo_id, name:, value:) + if secret_names(repo_id).include?(name) + client.update_secret(repo_id, name, value: value) + else + client.create_secret(repo_id, name: name, value: value, events: SECRET_EVENTS) + end + nil + end + + def secret_delete(repo_id, name) + client.delete_secret(repo_id, name) + nil + end + + def verify_config_request(request) + SignatureVerifier.new(request, public_key: @public_key, public_key_url: @public_key_url).valid? + end + + def config_marker(params) + configs = params[:configuration].presence || params[:configs].presence || [] + Array(configs).each do |config| + parsed = parse_marker(config[:data].to_s) + next unless parsed + + return { platform: parsed["platform"].to_s, + name: parsed["name"].presence || params.dig(:repo, :name).to_s } + end + nil + end + + def pipeline_config(platform:, name:, update_server:) + @config.render(platform: platform, name: name, update_server: update_server) + end + + def config_response(platform:, config:) + { configs: [ { name: platform, data: config } ] } + end + + private + + def parse_marker(data) + parsed = begin + YAML.safe_load(data) + rescue Psych::Exception + nil + end + parsed if parsed.is_a?(Hash) && parsed.key?("platform") + end + + def to_repo(remote) + return nil if remote.nil? + + Repo.new(id: remote["id"], name: remote["name"], owner: remote["owner"], + active: remote["active"], raw: remote) + end + + def to_run(remote) + return nil if remote.nil? + + Run.new( + number: remote["number"], + status: remote["status"], + branch: remote["branch"], + message: remote["message"], + created_at: remote["created"] ? Time.zone.at(remote["created"]) : nil, + url: remote["forge_url"] || remote["link_url"], + raw: remote + ) + end + end + end + end +end diff --git a/lib/warp_engine/ci/woodpecker/client.rb b/lib/warp_engine/ci/woodpecker/client.rb new file mode 100644 index 0000000..f2326c1 --- /dev/null +++ b/lib/warp_engine/ci/woodpecker/client.rb @@ -0,0 +1,143 @@ +require "net/http" +require "json" +require "uri" + +module WarpEngine + module CI + module Woodpecker + class Client + def initialize(url:, token:) + @base_url = url.to_s.chomp("/") + @token = token + end + + def list_repos + get("/api/repos") + end + + def get_repo(repo_id) + get("/api/repos/#{repo_id}") + end + + def activate_repo(repo_id) + post("/api/repos", body: { id: repo_id }) + end + + def deactivate_repo(repo_id) + delete("/api/repos/#{repo_id}") + end + + def list_secrets(repo_id) + get("/api/repos/#{repo_id}/secrets") + end + + def create_secret(repo_id, name:, value:, events: %w[push tag deployment]) + post("/api/repos/#{repo_id}/secrets", + body: { name: name, value: value, events: events }) + end + + def update_secret(repo_id, secret_name, value:) + patch("/api/repos/#{repo_id}/secrets/#{secret_name}", + body: { value: value }) + end + + def delete_secret(repo_id, secret_name) + delete("/api/repos/#{repo_id}/secrets/#{secret_name}") + end + + def list_pipelines(repo_id, page: 1, per_page: 25) + get("/api/repos/#{repo_id}/pipelines", + params: { page: page, perPage: per_page }) + end + + def latest_pipeline(repo_id) + get("/api/repos/#{repo_id}/pipelines/latest") + end + + def get_pipeline(repo_id, number) + get("/api/repos/#{repo_id}/pipelines/#{number}") + end + + def trigger_pipeline(repo_id, branch: "main") + post("/api/repos/#{repo_id}/pipelines", + body: { branch: branch }) + end + + private + + def get(path, params: {}) + uri = build_uri(path, params) + execute(uri, Net::HTTP::Get.new(uri)) + end + + def post(path, body: {}) + uri = build_uri(path) + request = Net::HTTP::Post.new(uri) + request.body = body.to_json + request.content_type = "application/json" + execute(uri, request) + end + + def patch(path, body: {}) + uri = build_uri(path) + request = Net::HTTP::Patch.new(uri) + request.body = body.to_json + request.content_type = "application/json" + execute(uri, request) + end + + def delete(path) + uri = build_uri(path) + execute(uri, Net::HTTP::Delete.new(uri)) + end + + def build_uri(path, params = {}) + uri = URI.parse("#{@base_url}#{path}") + uri.query = URI.encode_www_form(params) if params.any? + uri + end + + def execute(uri, request) + request["Authorization"] = "Bearer #{@token}" + request["Accept"] = "application/json" + + response = Net::HTTP.start(uri.hostname, uri.port, + use_ssl: uri.scheme == "https", + open_timeout: 10, + read_timeout: 30) do |http| + http.request(request) + end + + handle_response(uri, response) + rescue Errno::ECONNREFUSED, Errno::EHOSTUNREACH, Net::OpenTimeout, + Net::ReadTimeout, SocketError => e + raise CI::ConnectionError, "Cannot reach Woodpecker at #{@base_url}: #{e.message}" + end + + def handle_response(uri, response) + case response + when Net::HTTPSuccess, Net::HTTPNoContent + return nil if response.body.blank? + + begin + JSON.parse(response.body) + rescue JSON::ParserError + raise CI::ApiError.new( + "Expected JSON from #{uri.path} but got: #{response.body.truncate(80)}", + status: response.code.to_i, body: response.body + ) + end + when Net::HTTPNotFound + raise CI::ApiError.new("Not found: #{uri.path}", status: 404, body: response.body) + else + raise CI::ApiError.new( + "Woodpecker API error #{response.code}: #{response.body&.truncate(200)}", + status: response.code.to_i, + body: response.body + ) + end + end + end + end + end +end diff --git a/lib/warp_engine/ci/woodpecker/pipeline_config.rb b/lib/warp_engine/ci/woodpecker/pipeline_config.rb new file mode 100644 index 0000000..6e6718e --- /dev/null +++ b/lib/warp_engine/ci/woodpecker/pipeline_config.rb @@ -0,0 +1,42 @@ +require "erb" +require "pathname" + +module WarpEngine + module CI + module Woodpecker + class PipelineConfig + PLATFORM_FORMAT = /\A[a-z0-9_-]+\z/ + + def initialize(platforms: {}) + @platforms = platforms.to_h { |key, spec| [ key.to_s, spec.to_h.symbolize_keys ] } + end + + def platforms + @platforms + end + + def render(platform:, name:, update_server:) + platform = platform.to_s + return nil unless platform.match?(PLATFORM_FORMAT) + + spec = @platforms[platform] + return nil if spec.nil? + + path = self.class.templates_dir.join(platform, "pipeline.yaml.erb") + return nil unless path.exist? + + ERB.new(path.read, trim_mode: "-").result_with_hash( + name: name.to_s, + update_server: update_server.to_s, + builder: spec[:builder], + exporter: spec[:exporter] + ) + end + + def self.templates_dir + Pathname.new(__dir__).join("platforms") + end + end + end + end +end diff --git a/app/services/warp_engine/platforms/bevy/pipeline.yaml.erb b/lib/warp_engine/ci/woodpecker/platforms/bevy/pipeline.yaml.erb similarity index 100% rename from app/services/warp_engine/platforms/bevy/pipeline.yaml.erb rename to lib/warp_engine/ci/woodpecker/platforms/bevy/pipeline.yaml.erb diff --git a/app/services/warp_engine/platforms/c64/pipeline.yaml.erb b/lib/warp_engine/ci/woodpecker/platforms/c64/pipeline.yaml.erb similarity index 100% rename from app/services/warp_engine/platforms/c64/pipeline.yaml.erb rename to lib/warp_engine/ci/woodpecker/platforms/c64/pipeline.yaml.erb diff --git a/app/services/warp_engine/platforms/ebitengine/pipeline.yaml.erb b/lib/warp_engine/ci/woodpecker/platforms/ebitengine/pipeline.yaml.erb similarity index 100% rename from app/services/warp_engine/platforms/ebitengine/pipeline.yaml.erb rename to lib/warp_engine/ci/woodpecker/platforms/ebitengine/pipeline.yaml.erb diff --git a/app/services/warp_engine/platforms/godot/pipeline.yaml.erb b/lib/warp_engine/ci/woodpecker/platforms/godot/pipeline.yaml.erb similarity index 100% rename from app/services/warp_engine/platforms/godot/pipeline.yaml.erb rename to lib/warp_engine/ci/woodpecker/platforms/godot/pipeline.yaml.erb diff --git a/app/services/warp_engine/platforms/love/pipeline.yaml.erb b/lib/warp_engine/ci/woodpecker/platforms/love/pipeline.yaml.erb similarity index 100% rename from app/services/warp_engine/platforms/love/pipeline.yaml.erb rename to lib/warp_engine/ci/woodpecker/platforms/love/pipeline.yaml.erb diff --git a/app/services/warp_engine/platforms/phaser/pipeline.yaml.erb b/lib/warp_engine/ci/woodpecker/platforms/phaser/pipeline.yaml.erb similarity index 100% rename from app/services/warp_engine/platforms/phaser/pipeline.yaml.erb rename to lib/warp_engine/ci/woodpecker/platforms/phaser/pipeline.yaml.erb diff --git a/app/services/warp_engine/platforms/tic80/pipeline.yaml.erb b/lib/warp_engine/ci/woodpecker/platforms/tic80/pipeline.yaml.erb similarity index 100% rename from app/services/warp_engine/platforms/tic80/pipeline.yaml.erb rename to lib/warp_engine/ci/woodpecker/platforms/tic80/pipeline.yaml.erb diff --git a/lib/warp_engine/ci/woodpecker/signature_verifier.rb b/lib/warp_engine/ci/woodpecker/signature_verifier.rb new file mode 100644 index 0000000..8b70dae --- /dev/null +++ b/lib/warp_engine/ci/woodpecker/signature_verifier.rb @@ -0,0 +1,145 @@ +require "openssl" +require "base64" +require "net/http" +require "digest" + +module WarpEngine + module CI + module Woodpecker + class SignatureVerifier + CAVAGE_PARAM = /(\w+)="([^"]*)"/ + + @key_cache = {} + @key_mutex = Mutex.new + + class << self + def fetch_public_key(url) + @key_mutex.synchronize do + @key_cache[url] ||= Net::HTTP.get(URI.parse(url)) + end + end + + def reset_key_cache! + @key_mutex.synchronize { @key_cache = {} } + end + end + + def initialize(request, public_key: nil, public_key_url: nil) + @request = request + @public_key = public_key + @public_key_url = public_key_url + end + + def valid? + pem = public_key_pem + if pem.blank? + Rails.logger.error("[CI::Woodpecker] no public key configured — rejecting request") + return false + end + + key = OpenSSL::PKey.read(pem) + if @request.headers["Signature-Input"].present? + rfc9421_valid?(key) + else + cavage_valid?(key) + end + rescue OpenSSL::PKey::PKeyError, ArgumentError => e + Rails.logger.error("[CI::Woodpecker] #{e.class}: #{e.message}") + false + end + + private + + def public_key_pem + return @public_key if @public_key.present? + return nil if @public_key_url.blank? + + self.class.fetch_public_key(@public_key_url) + rescue StandardError => e + Rails.logger.error("[CI::Woodpecker] public key fetch failed: #{e.class}: #{e.message}") + nil + end + + def rfc9421_valid?(key) + input = @request.headers["Signature-Input"].to_s + match = input.match(/\A\s*([\w.-]+)=(\(.*)\z/m) + return false if match.nil? + + label, inner = match[1], match[2] + components = inner[/\((.*?)\)/m, 1].to_s.scan(/"([^"]*)"/).flatten + return false if components.empty? + + signature = @request.headers["Signature"].to_s[/#{Regexp.escape(label)}=:([A-Za-z0-9+\/=]+):/, 1] + return false if signature.blank? + return false unless content_digest_valid?(components) + + lines = components.map do |component| + value = component_value(component) + return false if value.nil? + %("#{component}": #{value}) + end + lines << %("@signature-params": #{inner}) + + key.verify(nil, Base64.decode64(signature), lines.join("\n")) + end + + def component_value(name) + case name + when "@request-target" then @request.fullpath + when "@method" then @request.request_method + when "@target-uri" then @request.original_url + when "@authority" then @request.host_with_port + when "@path" then @request.path + when "@query" then "?#{@request.query_string}" + when /\A@/ then nil + else @request.headers[name] + end + end + + def content_digest_valid?(components) + return true unless components.include?("content-digest") + + digest = @request.headers["Content-Digest"].to_s[/sha-256=:([A-Za-z0-9+\/=]+):/, 1] + return false if digest.blank? + + expected = Digest::SHA256.base64digest(@request.raw_post) + ActiveSupport::SecurityUtils.secure_compare(digest, expected) + end + + def cavage_valid?(key) + params = cavage_params + return false if params.nil? || params["signature"].blank? + + signing_string = cavage_signing_string(params.fetch("headers", "date")) + return false if signing_string.nil? + + key.verify(nil, Base64.decode64(params["signature"]), signing_string) + end + + def cavage_params + header = @request.headers["Signature"].presence + if header.nil? + auth = @request.headers["Authorization"].to_s + header = auth.delete_prefix("Signature ") if auth.start_with?("Signature ") + end + return nil if header.blank? + + header.scan(CAVAGE_PARAM).to_h + end + + def cavage_signing_string(headers_list) + lines = headers_list.split(" ").map do |name| + if name == "(request-target)" + "(request-target): #{@request.request_method.downcase} #{@request.fullpath}" + else + value = @request.headers[name] + return nil if value.nil? + "#{name.downcase}: #{value}" + end + end + lines.join("\n") + end + end + end + end +end diff --git a/lib/warp_engine/configuration.rb b/lib/warp_engine/configuration.rb index 694abac..91d41f6 100644 --- a/lib/warp_engine/configuration.rb +++ b/lib/warp_engine/configuration.rb @@ -2,20 +2,14 @@ module WarpEngine class Configuration attr_accessor :file_container_path, - :image_container_path, :update_secret, :application_token_source, :application_token_owner_class, :max_upload_size, :enforce_software_ownership, - :ci_platforms, - :ci_extension_public_key, - :ci_extension_public_key_url, - :ci_update_server, - :woodpecker_url, - :woodpecker_api_token, - :woodpecker_repo_owner, - :image_owners, + :ci_adapter, + :image_class_name, + :image_adapter, :storage_adapter, :access_policy, :access_token_owner_class, @@ -26,20 +20,14 @@ module WarpEngine def initialize @file_container_path = ENV.fetch("FILE_CONTAINER_PATH", "/softwares") - @image_container_path = ENV.fetch("IMAGE_CONTAINER_PATH", "/images") @update_secret = ENV["UPDATE_SECRET"] @application_token_source = :env @application_token_owner_class = nil @max_upload_size = 500 * 1024 * 1024 @enforce_software_ownership = false - @ci_platforms = {} - @ci_extension_public_key = nil - @ci_extension_public_key_url = nil - @ci_update_server = nil - @woodpecker_url = ENV["WOODPECKER_URL"] - @woodpecker_api_token = ENV["WOODPECKER_API_TOKEN"] - @woodpecker_repo_owner = ENV["WOODPECKER_REPO_OWNER"] - @image_owners = [] + @ci_adapter = :woodpecker + @image_class_name = "Image" + @image_adapter = nil @storage_adapter = :local @access_policy = :open @access_token_owner_class = nil diff --git a/lib/warp_engine/images.rb b/lib/warp_engine/images.rb new file mode 100644 index 0000000..e7d527f --- /dev/null +++ b/lib/warp_engine/images.rb @@ -0,0 +1,68 @@ +module WarpEngine + + module Images + + class HostModel + URL_PREFIX = "/api/image".freeze + + def model_name + WarpEngine.config.image_class_name + end + + def model + model_name.to_s.constantize + end + + def url_for(image_id) + image_id.present? ? "#{URL_PREFIX}/#{image_id}" : nil + end + + def build_from_upload(upload) + model.new(file_upload: upload) + end + + def label_for(record) + record.original_filename + end + + def available?(record) + return true unless record.respond_to?(:file_path) + + File.exist?(record.file_path.to_s) + end + + def select_options + model.order(:original_filename).map { |record| [ label_for(record), record.id ] } + end + end + + class << self + def adapter + configured = WarpEngine.config.image_adapter + + case configured + when nil, :host_model, "host_model" then host_model_adapter + else configured + end + end + + def host_model? = adapter.is_a?(HostModel) + + def host_model_adapter + @host_model_adapter ||= HostModel.new + end + + def reset! + @host_model_adapter = nil + end + + def model_name = adapter.model_name + def model = adapter.model + def url_for(image_id) = adapter.url_for(image_id) + def build_from_upload(upload) = adapter.build_from_upload(upload) + def label_for(record) = adapter.label_for(record) + def available?(record) = adapter.available?(record) + def select_options = adapter.select_options + end + end +end diff --git a/lib/warp_engine/version.rb b/lib/warp_engine/version.rb index 3b9880d..557bd44 100644 --- a/lib/warp_engine/version.rb +++ b/lib/warp_engine/version.rb @@ -1,5 +1,5 @@ module WarpEngine - VERSION = "0.5.2" + VERSION = "0.7.0" VERSION_HEADER = "WarpEngine-Version".freeze end diff --git a/spec/services/woodpecker_client_spec.rb b/spec/ci/woodpecker_client_spec.rb similarity index 87% rename from spec/services/woodpecker_client_spec.rb rename to spec/ci/woodpecker_client_spec.rb index e90d0ed..9210bfc 100644 --- a/spec/services/woodpecker_client_spec.rb +++ b/spec/ci/woodpecker_client_spec.rb @@ -1,10 +1,10 @@ require "rails_helper" require "webmock/rspec" -RSpec.describe WarpEngine::WoodpeckerClient do +RSpec.describe WarpEngine::CI::Woodpecker::Client do let(:base_url) { "https://ci.example.test" } let(:token) { "wp-test-token" } - let(:client) { described_class.new(base_url: base_url, token: token) } + let(:client) { described_class.new(url: base_url, token: token) } def stub_wp(method, path, status: 200, body: nil, request_body: nil) stub = stub_request(method, "#{base_url}#{path}") @@ -91,7 +91,7 @@ RSpec.describe WarpEngine::WoodpeckerClient do it "raises ApiError on 404" do stub_wp(:get, "/api/repos/999", status: 404, body: { "error" => "not found" }) - expect { client.get_repo(999) }.to raise_error(WarpEngine::WoodpeckerClient::ApiError) { |e| + expect { client.get_repo(999) }.to raise_error(WarpEngine::CI::ApiError) { |e| expect(e.status).to eq(404) } end @@ -99,7 +99,7 @@ RSpec.describe WarpEngine::WoodpeckerClient do it "raises ApiError on 500" do stub_wp(:get, "/api/repos", status: 500, body: { "error" => "internal" }) - expect { client.list_repos }.to raise_error(WarpEngine::WoodpeckerClient::ApiError) { |e| + expect { client.list_repos }.to raise_error(WarpEngine::CI::ApiError) { |e| expect(e.status).to eq(500) } end @@ -107,13 +107,13 @@ RSpec.describe WarpEngine::WoodpeckerClient do it "raises ConnectionError on connection refused" do stub_request(:get, "#{base_url}/api/repos").to_raise(Errno::ECONNREFUSED) - expect { client.list_repos }.to raise_error(WarpEngine::WoodpeckerClient::ConnectionError) + expect { client.list_repos }.to raise_error(WarpEngine::CI::ConnectionError) end it "raises ConnectionError on timeout" do stub_request(:get, "#{base_url}/api/repos").to_timeout - expect { client.list_repos }.to raise_error(WarpEngine::WoodpeckerClient::ConnectionError) + expect { client.list_repos }.to raise_error(WarpEngine::CI::ConnectionError) end it "raises ApiError when a 200 response is not JSON" do @@ -121,7 +121,7 @@ RSpec.describe WarpEngine::WoodpeckerClient do .to_return(status: 200, body: "", headers: { "Content-Type" => "text/html" }) - expect { client.list_repos }.to raise_error(WarpEngine::WoodpeckerClient::ApiError, /Expected JSON/) + expect { client.list_repos }.to raise_error(WarpEngine::CI::ApiError, /Expected JSON/) end end end diff --git a/spec/ci_spec.rb b/spec/ci_spec.rb new file mode 100644 index 0000000..e5b7a8c --- /dev/null +++ b/spec/ci_spec.rb @@ -0,0 +1,136 @@ +require "rails_helper" + +RSpec.describe WarpEngine::CI do + after do + WarpEngine.config.ci_adapter = :woodpecker + WarpEngine::CI.reset! + end + + describe ".adapter" do + it "drives Woodpecker unless the host says otherwise" do + expect(WarpEngine.ci).to be_a(WarpEngine::CI::Woodpecker::Adapter) + expect(WarpEngine.ci.name).to eq("Woodpecker") + end + + it "answers with the null adapter for a host that runs no CI" do + WarpEngine.config.ci_adapter = :none + + expect(WarpEngine.ci).to be_a(WarpEngine::CI::Null) + expect(WarpEngine.ci).not_to be_configured + end + + it "hands back whatever object the host named" do + own = Class.new { def name = "Forge Runner" }.new + WarpEngine.config.ci_adapter = own + + expect(WarpEngine.ci).to be(own) + end + end + + describe WarpEngine::CI::Null do + let(:adapter) { described_class.new } + + it "serves no platform and verifies no request" do + expect(adapter.platforms).to be_empty + expect(adapter.verify_config_request(nil)).to be(false) + expect(adapter.config_marker({})).to be_nil + expect(adapter.pipeline_config(platform: "godot", name: "x", update_server: "y")).to be_nil + end + + it "raises rather than pretending to manage repositories" do + expect { adapter.repos }.to raise_error(WarpEngine::CI::NotConfigured) + expect { adapter.trigger(1) }.to raise_error(WarpEngine::CI::NotConfigured) + end + end + + describe WarpEngine::CI::Woodpecker::Adapter do + let(:client) { instance_double(WarpEngine::CI::Woodpecker::Client) } + let(:adapter) do + described_class.new(url: "https://ci.test", api_token: "tok", + platforms: { "godot" => { builder: "registry.test/godot:1" } }, + update_server: "https://games.test") + end + + before { allow(adapter).to receive(:client).and_return(client) } + + it "is inactive without a server and a token" do + expect(described_class.new(url: nil, api_token: nil)).not_to be_configured + expect(adapter).to be_configured + end + + it "normalizes a repository" do + allow(client).to receive(:list_repos).and_return([ + { "id" => 7, "name" => "game", "owner" => "org", "active" => true } + ]) + + repo = adapter.repos.first + + expect(repo.id).to eq(7) + expect(repo.full_name).to eq("org/game") + expect(repo).to be_active + end + + it "normalizes a run, timestamp included" do + allow(client).to receive(:trigger_pipeline).and_return( + { "number" => 4, "status" => "success", "branch" => "main", + "message" => "ship it", "created" => 1_754_500_000 } + ) + + run = adapter.trigger(7, branch: "main") + + expect(run.number).to eq(4) + expect(run).to be_success + expect(run.created_at).to eq(Time.zone.at(1_754_500_000)) + expect(run.as_json[:createdAt]).to be_present + end + + it "creates a secret the repository does not have yet" do + allow(client).to receive(:list_secrets).and_return([]) + allow(client).to receive(:create_secret) + + adapter.secret_set(7, name: "application_token", value: "plain") + + expect(client).to have_received(:create_secret).with( + 7, name: "application_token", value: "plain", events: described_class::SECRET_EVENTS + ) + end + + it "updates a secret the repository already has" do + allow(client).to receive(:list_secrets).and_return([ { "name" => "application_token" } ]) + allow(client).to receive(:update_secret) + + adapter.secret_set(7, name: "application_token", value: "plain") + + expect(client).to have_received(:update_secret).with(7, "application_token", value: "plain") + end + + it "reads the platform marker out of a config request" do + params = ActionController::Parameters.new( + repo: { name: "mygame" }, + configuration: [ { name: ".woodpecker.yaml", data: "platform: godot\n" } ] + ) + + expect(adapter.config_marker(params)).to eq({ platform: "godot", name: "mygame" }) + end + + it "ignores a config request that is not a marker" do + params = ActionController::Parameters.new( + configuration: [ { name: ".woodpecker.yaml", data: "steps:\n - name: build\n" } ] + ) + + expect(adapter.config_marker(params)).to be_nil + end + + it "renders only the platforms it was given" do + expect(adapter.pipeline_config(platform: "godot", name: "mygame", + update_server: "https://games.test")).to include("registry.test/godot:1") + expect(adapter.pipeline_config(platform: "amiga", name: "mygame", + update_server: "https://games.test")).to be_nil + end + + it "shapes the response the way the CI server expects" do + expect(adapter.config_response(platform: "godot", config: "steps: []")) + .to eq({ configs: [ { name: "godot", data: "steps: []" } ] }) + end + end +end diff --git a/spec/dummy/app/models/image.rb b/spec/dummy/app/models/image.rb new file mode 100644 index 0000000..73e4b10 --- /dev/null +++ b/spec/dummy/app/models/image.rb @@ -0,0 +1,27 @@ +class Image < ActiveRecord::Base + def self.upload_path + Rails.root.join("tmp/images").to_s + end + + has_many :software_images, class_name: "WarpEngine::SoftwareImage", dependent: :restrict_with_error + + default_scope { where(deleted_at: nil) } + + attr_accessor :file_upload + + before_save :process_upload, if: -> { file_upload.present? } + + def file_path + File.join(self.class.upload_path, filename.to_s) + end + + private + + def process_upload + FileUtils.mkdir_p(self.class.upload_path) + self.original_filename = file_upload.original_filename + self.content_type = file_upload.content_type.presence || "application/octet-stream" + self.filename = "#{SecureRandom.uuid}#{File.extname(file_upload.original_filename)}" + IO.copy_stream(file_upload.to_io, file_path) + end +end diff --git a/spec/factories/images.rb b/spec/factories/images.rb new file mode 100644 index 0000000..5c5cacb --- /dev/null +++ b/spec/factories/images.rb @@ -0,0 +1,7 @@ +FactoryBot.define do + factory :image, class: "Image" do + sequence(:filename) { |n| "image-#{n}.png" } + original_filename { "image.png" } + content_type { "image/png" } + end +end diff --git a/spec/images_spec.rb b/spec/images_spec.rb new file mode 100644 index 0000000..e2acdb9 --- /dev/null +++ b/spec/images_spec.rb @@ -0,0 +1,51 @@ +require "rails_helper" + +RSpec.describe WarpEngine::Images do + after { WarpEngine.config.image_adapter = nil } + + describe "the default adapter" do + it "is the host model named in the configuration" do + expect(described_class.model_name).to eq("Image") + expect(described_class.model).to eq(Image) + end + + it "serves images from the address the clients have always used" do + expect(described_class.url_for(7)).to eq("/api/image/7") + expect(described_class.url_for(nil)).to be_nil + end + + it "offers the host's images to an admin select" do + image = create(:image, original_filename: "cover.png") + + expect(described_class.select_options).to include([ "cover.png", image.id ]) + end + + it "builds a host image from an upload" do + expect(described_class.build_from_upload(nil)).to be_a(Image) + end + end + + describe "a host adapter" do + it "decides where an image is served from" do + WarpEngine.config.image_adapter = Class.new do + def url_for(image_id) = "https://cdn.example.com/#{image_id}.png" + end.new + + expect(described_class.url_for(9)).to eq("https://cdn.example.com/9.png") + end + end + + it "links the catalog to the host's image model" do + expect(WarpEngine::SoftwareImage.reflect_on_association(:image).klass).to eq(Image) + end + + it "puts the adapter URL in the catalog payload" do + software = create(:software) + image = create(:image) + WarpEngine::SoftwareImage.create!(software: software, image_id: image.id, is_default: true) + + payload = WarpEngine::SoftwareSerializer.render_as_hash(software.reload) + + expect(payload[:imageUrl]).to eq("/api/image/#{image.id}") + end +end diff --git a/spec/migrations_spec.rb b/spec/migrations_spec.rb index 70cd6c3..c73086f 100644 --- a/spec/migrations_spec.rb +++ b/spec/migrations_spec.rb @@ -33,14 +33,14 @@ RSpec.describe "Engine migrations" do end it "does not create a table the install generator also creates" do - template = WarpEngine::Engine.root.join( - "lib/generators/warp_engine/install/templates/create_warp_engine_tables.rb" + templates = Dir.glob( + WarpEngine::Engine.root.join("lib/generators/warp_engine/install/templates/create_*.rb") ) engine_tables = versions.flat_map do |version| file = Dir.glob(WarpEngine::Engine.root.join("db/migrate/#{version}_*.rb")).first File.read(file).scan(/create_table :(\w+)/).flatten end - template_tables = File.read(template).scan(/create_table :(\w+)/).flatten + template_tables = templates.flat_map { |t| File.read(t).scan(/create_table :(\w+)/).flatten } duplicated = (engine_tables & template_tables) - [ "application_tokens" ] expect(duplicated).to be_empty, diff --git a/spec/requests/build_configs_controller_spec.rb b/spec/requests/build_configs_controller_spec.rb index 0c1187c..00c891a 100644 --- a/spec/requests/build_configs_controller_spec.rb +++ b/spec/requests/build_configs_controller_spec.rb @@ -10,11 +10,21 @@ RSpec.describe "Build configs endpoint", type: :request do } end - before do - allow(WarpEngine.config).to receive(:ci_platforms).and_return(ci_platforms) - allow(WarpEngine.config).to receive(:ci_extension_public_key).and_return(signing_key.public_to_pem) + def woodpecker(platforms: nil, public_key: :default) + WarpEngine::CI::Woodpecker::Adapter.new( + url: "https://ci.example.test", + api_token: "wp-token", + platforms: platforms.nil? ? ci_platforms : platforms, + public_key: public_key == :default ? signing_key.public_to_pem : public_key + ) end + def serving(adapter) + allow(WarpEngine).to receive(:ci).and_return(adapter) + end + + before { serving(woodpecker) } + def signed_headers(body, path: "/build/config", digest_body: nil) digest = "sha-256=:#{Digest::SHA256.base64digest(digest_body || body)}:" inner = %{("@request-target" "content-digest");created=#{Time.now.to_i};alg="ed25519"} @@ -69,7 +79,7 @@ RSpec.describe "Build configs endpoint", type: :request do end it "returns 404 when the feature is not configured" do - allow(WarpEngine.config).to receive(:ci_platforms).and_return({}) + serving(woodpecker(platforms: {})) get "/build/config", params: { platform: "godot" } @@ -146,7 +156,7 @@ RSpec.describe "Build configs endpoint", type: :request do payload = extension_payload("platform: godot\n") headers = signed_headers(payload) other_key = OpenSSL::PKey.generate_key("ed25519") - allow(WarpEngine.config).to receive(:ci_extension_public_key).and_return(other_key.public_to_pem) + serving(woodpecker(public_key: other_key.public_to_pem)) post "/build/config", params: payload, headers: headers @@ -170,8 +180,7 @@ RSpec.describe "Build configs endpoint", type: :request do end it "rejects every request when no public key is configured" do - allow(WarpEngine.config).to receive(:ci_extension_public_key).and_return(nil) - allow(WarpEngine.config).to receive(:ci_extension_public_key_url).and_return(nil) + serving(woodpecker(public_key: nil)) payload = extension_payload("platform: godot\n") post "/build/config", params: payload, headers: signed_headers(payload) @@ -182,16 +191,17 @@ RSpec.describe "Build configs endpoint", type: :request do describe "shipped templates" do it "renders every template to valid YAML with non-empty steps" do - templates = Dir[WarpEngine::Engine.root.join("app/services/warp_engine/platforms/*/pipeline.yaml.erb")] + templates = Dir[WarpEngine::CI::Woodpecker::PipelineConfig.templates_dir.join("*/pipeline.yaml.erb")] expect(templates).not_to be_empty templates.each do |path| platform = File.basename(File.dirname(path)) - allow(WarpEngine.config).to receive(:ci_platforms).and_return( - platform => { builder: "registry.example/builder:1", exporter: "registry.example/exporter:1" } + config = WarpEngine::CI::Woodpecker::PipelineConfig.new( + platforms: { platform => { builder: "registry.example/builder:1", + exporter: "registry.example/exporter:1" } } ) - yaml = WarpEngine::CiConfigService.new.render( + yaml = config.render( platform: platform, name: "example", update_server: "https://games.example" ) diff --git a/spec/requests/ci_controller_spec.rb b/spec/requests/ci_controller_spec.rb index 589d011..675e3cf 100644 --- a/spec/requests/ci_controller_spec.rb +++ b/spec/requests/ci_controller_spec.rb @@ -1,15 +1,16 @@ require "rails_helper" RSpec.describe "CI API", type: :request do + let(:ci) { instance_double(WarpEngine::CI::Woodpecker::Adapter, configured?: true, name: "Woodpecker") } + before do - allow(WarpEngine.config).to receive(:woodpecker_url).and_return("https://ci.test") - allow(WarpEngine.config).to receive(:woodpecker_api_token).and_return("wp-token") + allow(WarpEngine).to receive(:ci).and_return(ci) allow(WarpEngine.config).to receive(:update_secret).and_return("s3cret") end describe "GET /api/ci/pipelines" do it "returns active repos" do - repo = create(:pipeline, repo_name: "mygame", platform: "tic80") + create(:pipeline, repo_name: "mygame", platform: "tic80") get "/api/ci/pipelines" @@ -17,10 +18,11 @@ RSpec.describe "CI API", type: :request do json = JSON.parse(response.body) expect(json.size).to eq(1) expect(json.first["repo_name"]).to eq("mygame") + expect(json.first["repo_id"]).to eq(json.first["woodpecker_repo_id"]) end - it "returns 503 when woodpecker not configured" do - allow(WarpEngine.config).to receive(:woodpecker_url).and_return(nil) + it "returns 503 when no CI provider is configured" do + allow(ci).to receive(:configured?).and_return(false) get "/api/ci/pipelines" @@ -29,18 +31,27 @@ RSpec.describe "CI API", type: :request do end describe "GET /api/ci/pipelines/:id/status" do - it "returns repo with pipeline status" do + it "returns the repo with its latest run" do repo = create(:pipeline, repo_owner: "org", repo_name: "game") - client = instance_double(WarpEngine::WoodpeckerClient) - allow(WarpEngine::WoodpeckerClient).to receive(:new).and_return(client) - allow(client).to receive(:get_pipeline) - .and_return({ "number" => 1, "status" => "success" }) + allow(ci).to receive(:run).with(repo.remote_repo_id, "latest") + .and_return(WarpEngine::CI::Run.new(number: 1, status: "success")) get "/api/ci/pipelines/#{repo.id}/status" expect(response).to have_http_status(:ok) json = JSON.parse(response.body) expect(json["pipeline"]["repo_name"]).to eq("game") + expect(json["latest_run"]["status"]).to eq("success") + end + + it "answers without a run when the provider is unreachable" do + repo = create(:pipeline) + allow(ci).to receive(:run).and_raise(WarpEngine::CI::ConnectionError, "down") + + get "/api/ci/pipelines/#{repo.id}/status" + + expect(response).to have_http_status(:ok) + expect(JSON.parse(response.body)["latest_run"]).to be_nil end end @@ -53,12 +64,10 @@ RSpec.describe "CI API", type: :request do expect(response).to have_http_status(:unauthorized) end - it "triggers a pipeline with valid secret" do + it "triggers a pipeline with a valid secret" do repo = create(:pipeline, repo_owner: "org", repo_name: "game") - client = instance_double(WarpEngine::WoodpeckerClient) - allow(WarpEngine::WoodpeckerClient).to receive(:new).and_return(client) - allow(client).to receive(:trigger_pipeline) - .and_return({ "number" => 7, "status" => "pending" }) + allow(ci).to receive(:trigger).with(repo.remote_repo_id, branch: "main") + .and_return(WarpEngine::CI::Run.new(number: 7, status: "pending")) post "/api/ci/pipelines/#{repo.id}/trigger", headers: { "X-Update-Secret" => "s3cret" } @@ -66,6 +75,7 @@ RSpec.describe "CI API", type: :request do expect(response).to have_http_status(:ok) json = JSON.parse(response.body) expect(json["triggered"]).to be true + expect(json["pipeline"]["number"]).to eq(7) end end end diff --git a/spec/requests/version_header_spec.rb b/spec/requests/version_header_spec.rb index 6bd4adb..da57a8c 100644 --- a/spec/requests/version_header_spec.rb +++ b/spec/requests/version_header_spec.rb @@ -15,7 +15,7 @@ RSpec.describe "the WarpEngine-Version header", type: :request do end it "is on an error response" do - get "/api/image/999999" + get "/api/softwares/no-such-software/builds" expect(response).to have_http_status(:not_found) expect(response.headers["WarpEngine-Version"]).to eq(WarpEngine::VERSION) diff --git a/spec/services/pipeline_service_spec.rb b/spec/services/pipeline_service_spec.rb index c88e005..dfd107d 100644 --- a/spec/services/pipeline_service_spec.rb +++ b/spec/services/pipeline_service_spec.rb @@ -1,31 +1,35 @@ require "rails_helper" RSpec.describe WarpEngine::PipelineService do - let(:client) { instance_double(WarpEngine::WoodpeckerClient) } - let(:service) { described_class.new(client: client) } + let(:ci) { instance_double(WarpEngine::CI::Woodpecker::Adapter) } + let(:service) { described_class.new(ci: ci) } + + def ci_run(number:, status: "success", created_at: nil) + WarpEngine::CI::Run.new(number: number, status: status, created_at: created_at) + end describe "#trigger" do it "delegates to client" do repo = build(:pipeline, repo_owner: "org", repo_name: "game") - allow(client).to receive(:trigger_pipeline).and_return({ "number" => 6 }) + allow(ci).to receive(:trigger).and_return(ci_run(number: 6, status: "pending")) result = service.trigger(repo, branch: "main") - expect(result["number"]).to eq(6) - expect(client).to have_received(:trigger_pipeline).with(repo.woodpecker_repo_id, branch: "main") + expect(result.number).to eq(6) + expect(ci).to have_received(:trigger).with(repo.remote_repo_id, branch: "main") end end - describe "#list_pipelines" do - it "returns paginated pipelines and refreshes the repo's cached last pipeline" do + describe "#runs" do + it "returns the runs and refreshes the repo's cached last run" do repo = create(:pipeline, repo_owner: "org", repo_name: "game") - pipelines = [ - { "number" => 2, "status" => "failure", "created" => 1_754_500_000 }, - { "number" => 1, "status" => "success", "created" => 1_754_400_000 } + runs = [ + ci_run(number: 2, status: "failure", created_at: Time.zone.at(1_754_500_000)), + ci_run(number: 1, status: "success", created_at: Time.zone.at(1_754_400_000)) ] - allow(client).to receive(:list_pipelines).with(repo.woodpecker_repo_id, page: 1).and_return(pipelines) + allow(ci).to receive(:runs).with(repo.remote_repo_id, page: 1).and_return(runs) - expect(service.list_pipelines(repo)).to eq(pipelines) + expect(service.runs(repo)).to eq(runs) repo.reload expect(repo.last_pipeline_status).to eq("failure") @@ -35,20 +39,20 @@ RSpec.describe WarpEngine::PipelineService do it "does not touch the cache on later pages" do repo = create(:pipeline, repo_owner: "org", repo_name: "game", last_pipeline_status: "success") - allow(client).to receive(:list_pipelines).with(repo.woodpecker_repo_id, page: 2) - .and_return([{ "number" => 1, "status" => "failure", "created" => 1_754_400_000 }]) + allow(ci).to receive(:runs).with(repo.remote_repo_id, page: 2) + .and_return([ ci_run(number: 1, status: "failure", created_at: Time.zone.at(1_754_400_000)) ]) - service.list_pipelines(repo, page: 2) + service.runs(repo, page: 2) expect(repo.reload.last_pipeline_status).to eq("success") end - it "leaves the cache alone when the repo has no pipelines" do + it "leaves the cache alone when the repo has no runs" do repo = create(:pipeline, repo_owner: "org", repo_name: "game", last_pipeline_status: "success") - allow(client).to receive(:list_pipelines).and_return([]) + allow(ci).to receive(:runs).and_return([]) - expect(service.list_pipelines(repo)).to eq([]) + expect(service.runs(repo)).to eq([]) expect(repo.reload.last_pipeline_status).to eq("success") end end diff --git a/spec/services/pipeline_sync_service_spec.rb b/spec/services/pipeline_sync_service_spec.rb index 282540a..a4a855a 100644 --- a/spec/services/pipeline_sync_service_spec.rb +++ b/spec/services/pipeline_sync_service_spec.rb @@ -1,14 +1,16 @@ require "rails_helper" RSpec.describe WarpEngine::PipelineSyncService do - let(:client) { instance_double(WarpEngine::WoodpeckerClient) } - let(:service) { described_class.new(client: client) } + let(:ci) { instance_double(WarpEngine::CI::Woodpecker::Adapter) } + let(:service) { described_class.new(ci: ci) } + + def ci_repo(id:, name: "mygame", owner: "org", active: true) + WarpEngine::CI::Repo.new(id: id, name: name, owner: owner, active: active) + end describe "#sync_all" do - it "creates new Pipeline records from Woodpecker" do - allow(client).to receive(:list_repos).and_return([ - { "id" => 1, "name" => "mygame", "owner" => "org", "active" => true } - ]) + it "creates new Pipeline records from the CI provider" do + allow(ci).to receive(:repos).and_return([ ci_repo(id: 1) ]) result = service.sync_all @@ -21,9 +23,7 @@ RSpec.describe WarpEngine::PipelineSyncService do it "updates existing records" do existing = create(:pipeline, woodpecker_repo_id: 1, repo_name: "old", platform: "tic80") - allow(client).to receive(:list_repos).and_return([ - { "id" => 1, "name" => "newname", "owner" => "org", "active" => true } - ]) + allow(ci).to receive(:repos).and_return([ ci_repo(id: 1, name: "newname") ]) result = service.sync_all @@ -31,9 +31,9 @@ RSpec.describe WarpEngine::PipelineSyncService do expect(existing.reload.repo_name).to eq("newname") end - it "deactivates repos missing from Woodpecker" do + it "deactivates repos the provider no longer has" do orphan = create(:pipeline, woodpecker_repo_id: 99, active: true) - allow(client).to receive(:list_repos).and_return([]) + allow(ci).to receive(:repos).and_return([]) result = service.sync_all @@ -43,9 +43,7 @@ RSpec.describe WarpEngine::PipelineSyncService do it "auto-detects platform from matching Software" do create(:software, name: "mygame", platform: "godot") - allow(client).to receive(:list_repos).and_return([ - { "id" => 1, "name" => "mygame", "owner" => "org", "active" => true } - ]) + allow(ci).to receive(:repos).and_return([ ci_repo(id: 1) ]) result = service.sync_all @@ -55,11 +53,9 @@ RSpec.describe WarpEngine::PipelineSyncService do end describe "#activate" do - it "calls client and syncs the repo" do - allow(client).to receive(:activate_repo).with(42) - allow(client).to receive(:get_repo).with(42).and_return( - { "id" => 42, "name" => "game", "owner" => "org", "active" => true } - ) + it "calls the provider and syncs the repo" do + allow(ci).to receive(:activate_repo).with(42) + allow(ci).to receive(:repo).with(42).and_return(ci_repo(id: 42, name: "game")) repo = service.activate(42) @@ -69,9 +65,9 @@ RSpec.describe WarpEngine::PipelineSyncService do end describe "#deactivate" do - it "calls client and marks repo inactive" do + it "calls the provider and marks the repo inactive" do repo = create(:pipeline, woodpecker_repo_id: 42, active: true) - allow(client).to receive(:deactivate_repo).with(42) + allow(ci).to receive(:deactivate_repo).with(42) service.deactivate(42) diff --git a/spec/services/secret_sync_service_spec.rb b/spec/services/secret_sync_service_spec.rb index 641df4c..fbcf7e6 100644 --- a/spec/services/secret_sync_service_spec.rb +++ b/spec/services/secret_sync_service_spec.rb @@ -1,8 +1,8 @@ require "rails_helper" RSpec.describe WarpEngine::SecretSyncService do - let(:client) { instance_double(WarpEngine::WoodpeckerClient) } - let(:service) { described_class.new(client: client) } + let(:ci) { instance_double(WarpEngine::CI::Woodpecker::Adapter) } + let(:service) { described_class.new(ci: ci) } before do allow(WarpEngine.config).to receive(:application_token_source).and_return(:database) @@ -10,37 +10,22 @@ RSpec.describe WarpEngine::SecretSyncService do end describe "#provision" do - it "creates secrets on repos that don't have one" do + it "hands the token to the provider for every pipeline" do repo = create(:pipeline) - allow(client).to receive(:list_secrets).with(repo.woodpecker_repo_id).and_return([]) - allow(client).to receive(:create_secret) + allow(ci).to receive(:secret_set) result = service.provision("plaintoken", pipelines: [ repo ]) expect(result[:synced]).to eq([ repo ]) - expect(client).to have_received(:create_secret).with( - repo.woodpecker_repo_id, name: "application_token", value: "plaintoken" - ) - end - - it "updates secrets on repos that already have one" do - repo = create(:pipeline) - allow(client).to receive(:list_secrets).with(repo.woodpecker_repo_id) - .and_return([ { "name" => "application_token" } ]) - allow(client).to receive(:update_secret) - - result = service.provision("newtoken", pipelines: [ repo ]) - - expect(result[:synced]).to eq([ repo ]) - expect(client).to have_received(:update_secret).with( - repo.woodpecker_repo_id, "application_token", value: "newtoken" + expect(ci).to have_received(:secret_set).with( + repo.remote_repo_id, name: "application_token", value: "plaintoken" ) end it "records failed repos without raising" do repo = create(:pipeline) - allow(client).to receive(:list_secrets).and_raise( - WarpEngine::WoodpeckerClient::ConnectionError, "unreachable" + allow(ci).to receive(:secret_set).and_raise( + WarpEngine::CI::ConnectionError, "unreachable" ) result = service.provision("tok", pipelines: [ repo ]) @@ -56,11 +41,11 @@ RSpec.describe WarpEngine::SecretSyncService do sw = create(:software, name: "game1", owner: token.owner) repo = create(:pipeline, :with_software, software: sw) - allow(client).to receive(:delete_secret) + allow(ci).to receive(:secret_delete) service.deprovision(token) - expect(client).to have_received(:delete_secret).with(repo.woodpecker_repo_id, "application_token") + expect(ci).to have_received(:secret_delete).with(repo.remote_repo_id, "application_token") end end @@ -70,8 +55,7 @@ RSpec.describe WarpEngine::SecretSyncService do sw = create(:software, name: "game1", owner: token.owner) repo = create(:pipeline, :with_software, software: sw) - allow(client).to receive(:list_secrets).and_return([]) - allow(client).to receive(:create_secret) + allow(ci).to receive(:secret_set) result = service.rotate(token) @@ -85,8 +69,8 @@ RSpec.describe WarpEngine::SecretSyncService do sw = create(:software, name: "game1", owner: token.owner) create(:pipeline, :with_software, software: sw) - allow(client).to receive(:list_secrets).and_raise( - WarpEngine::WoodpeckerClient::ConnectionError, "down" + allow(ci).to receive(:secret_set).and_raise( + WarpEngine::CI::ConnectionError, "down" ) result = service.rotate(token) diff --git a/warp_engine.gemspec b/warp_engine.gemspec index 60b791c..f9b4011 100644 --- a/warp_engine.gemspec +++ b/warp_engine.gemspec @@ -6,7 +6,7 @@ Gem::Specification.new do |spec| spec.authors = [ "Teletype Games" ] spec.summary = "Retro software catalog engine with a CI-callable release updater" spec.description = "Mountable Rails engine providing a software catalog (softwares, releases, " \ - "release assets, images, platform links), a pipeline-callable update endpoint, " \ + "release assets, image links, platform links), a pipeline-callable update endpoint, " \ "a public read-only JSON API and ActiveAdmin resources for a host-owned admin." spec.license = "MIT" spec.homepage = "https://teletypegames.org"