From 60e196a03ea9a1fcba6d30d02026b6c8e4aa9fd5 Mon Sep 17 00:00:00 2001 From: Zsolt Tasnadi Date: Wed, 5 Aug 2026 06:56:12 +0200 Subject: [PATCH] Add a runnable example compose stack for WarpEngine MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit examples/compose boots everything the engine's workflow assumes: mysql, a minimal Rails host consuming the engine as a path gem, an SSH drop area sharing the softwares volume with the app, and — behind the ci profile — gitea plus woodpecker (agent attached to the stack network so pipeline steps reach droparea/app by service name). host_app doubles as a reference for a brand-new host: Gemfile, the two initializers, apipie + engine mounts in routes.rb; on first boot the entrypoint runs the install generator and db:prepare. The README gains a detailed bring-up walkthrough: quickstart, publishing a release by hand over scp + /update (verified end to end from a clean slate), and the full gitea/woodpecker OAuth wiring. --- README.md | 103 ++++++++++++++ examples/compose/.env.example | 24 ++++ examples/compose/docker-compose.yml | 131 ++++++++++++++++++ examples/compose/host_app/.gitignore | 6 + examples/compose/host_app/Dockerfile | 20 +++ examples/compose/host_app/Gemfile | 10 ++ examples/compose/host_app/Rakefile | 3 + .../compose/host_app/bin/docker-entrypoint | 16 +++ examples/compose/host_app/bin/rails | 4 + examples/compose/host_app/config.ru | 4 + .../compose/host_app/config/application.rb | 23 +++ examples/compose/host_app/config/boot.rb | 3 + examples/compose/host_app/config/database.yml | 8 ++ .../compose/host_app/config/environment.rb | 3 + .../config/environments/development.rb | 10 ++ .../host_app/config/initializers/apipie.rb | 11 ++ .../config/initializers/warp_engine.rb | 9 ++ examples/compose/host_app/config/routes.rb | 6 + 18 files changed, 394 insertions(+) create mode 100644 examples/compose/.env.example create mode 100644 examples/compose/docker-compose.yml create mode 100644 examples/compose/host_app/.gitignore create mode 100644 examples/compose/host_app/Dockerfile create mode 100644 examples/compose/host_app/Gemfile create mode 100644 examples/compose/host_app/Rakefile create mode 100755 examples/compose/host_app/bin/docker-entrypoint create mode 100755 examples/compose/host_app/bin/rails create mode 100644 examples/compose/host_app/config.ru create mode 100644 examples/compose/host_app/config/application.rb create mode 100644 examples/compose/host_app/config/boot.rb create mode 100644 examples/compose/host_app/config/database.yml create mode 100644 examples/compose/host_app/config/environment.rb create mode 100644 examples/compose/host_app/config/environments/development.rb create mode 100644 examples/compose/host_app/config/initializers/apipie.rb create mode 100644 examples/compose/host_app/config/initializers/warp_engine.rb create mode 100644 examples/compose/host_app/config/routes.rb diff --git a/README.md b/README.md index 2bf8968..2cef9ad 100644 --- a/README.md +++ b/README.md @@ -31,6 +31,109 @@ Repository: `https://git.teletypegames.org/tools/warp_engine` - A relational database (developed and tested against MySQL 8) - Optional: ActiveAdmin + Devise in the host app for the admin UI +## Example stack (docker compose) + +`examples/compose` boots everything the engine's workflow assumes, end to +end: the catalog app itself, the SSH drop area the updater contract feeds +from and — behind a compose profile — a Gitea forge with Woodpecker CI, so +you can watch a pipeline publish a release into the catalog. + +| Service | Role | Where | +| --- | --- | --- | +| `app` | Minimal Rails host with the engine mounted as a path gem (headless: API + updater) | `http://localhost:8080` | +| `mysql` | Catalog database | internal | +| `droparea` | SSH server where pipelines drop build artifacts; shares the `softwares` volume with `app` | `ssh drop@localhost -p 2222` | +| `gitea` | Git forge (profile `ci`) | `http://gitea:3000` | +| `woodpecker` + agent | CI wired to gitea (profile `ci`) | `http://woodpecker:8000` | + +### Quickstart — catalog + drop area + +```sh +cd examples/compose +cp .env.example .env # defaults work for a throwaway local demo +docker compose up --build +``` + +The first boot takes a few minutes: the app container bundles, runs +`rails g warp_engine:install` and `rails db:prepare`, then serves on +`http://localhost:8080`: + +- `http://localhost:8080/api/software` — the (empty) catalog +- `http://localhost:8080/api/builds` — the platform build matrix +- `http://localhost:8080/api/docs` — apipie API docs + +### Publish a release by hand + +The updater contract needs nothing but files in the drop area and one HTTP +call, so you can play the role of the CI pipeline yourself: + +```sh +# 1. Fake a build: metadata, a web build and a windows artifact, named by +# convention (the love platform requires the .html.zip web build) +cat > demo-0.1.0.metadata.json <<'JSON' +{ "name": "demo", "title": "Demo Game", "author": "You", "desc": "Hello", "license": "MIT" } +JSON +echo '

demo

' > index.html && zip demo-0.1.0.html.zip index.html +echo hello > game.bin && zip demo-0.1.0-win-x64.zip game.bin + +# 2. Drop them (password: DROP_PASSWORD from .env, default "drop") +scp -P 2222 demo-0.1.0.* drop@localhost:drop/ + +# 3. Trigger the updater +curl -H "X-Update-Secret: example-update-secret" \ + "http://localhost:8080/update?platform=love&name=demo&version=0.1.0" +``` + +`GET /api/software` now lists *Demo Game* with `html` and `win_x64` assets, +`http://localhost:8080/file/demo-0.1.0/index.html` serves the extracted web +build, and `GET /api/download?path=demo-0.1.0-win-x64.zip` serves the +artifact while logging a download record. + +### Full loop — forge + CI (profile `ci`) + +gitea and woodpecker address each other by service name, so let your browser +resolve those names too: + +```sh +echo "127.0.0.1 gitea woodpecker" | sudo tee -a /etc/hosts +``` + +1. `docker compose --profile ci up -d gitea`, open `http://gitea:3000`, + finish the install wizard (SQLite is fine) and create your admin user. +2. In gitea: *Settings → Applications → Manage OAuth2 Applications*, create + an app with redirect URI `http://woodpecker:8000/authorize`; copy the + client id/secret into `WOODPECKER_GITEA_CLIENT` / `WOODPECKER_GITEA_SECRET` + in `.env`. +3. `docker compose --profile ci up -d` — then log in at + `http://woodpecker:8000` (OAuth via gitea) and enable your repository. + +A pipeline publishes a release exactly like the by-hand steps above — build, +`scp` to `droparea`, call `/update`: + +```yaml +# .woodpecker.yaml in a game repo hosted on the example gitea +steps: + publish: + image: alpine + environment: + DROP_PASSWORD: + from_secret: drop_password + UPDATE_SECRET: + from_secret: update_secret + commands: + - apk add --no-cache openssh-client sshpass curl zip + - # ... build your game, produce mygame-1.0.0.metadata.json + artifacts ... + - sshpass -p "$DROP_PASSWORD" scp -P 2222 -o StrictHostKeyChecking=no mygame-1.0.0.* drop@droparea:drop/ + - curl -fs -H "X-Update-Secret: $UPDATE_SECRET" "http://app:3000/update?platform=love&name=mygame&version=1.0.0" +``` + +(The agent attaches pipeline containers to the stack network, so `droparea` +and `app` resolve. For real projects, the per-platform +[`tools/*-tools`](https://git.teletypegames.org) repos ship ready-made +Makefile + pipeline templates implementing this contract.) + +Tear the stack down with `docker compose --profile ci down -v`. + ## Installation From the git repository: diff --git a/examples/compose/.env.example b/examples/compose/.env.example new file mode 100644 index 0000000..6f2a416 --- /dev/null +++ b/examples/compose/.env.example @@ -0,0 +1,24 @@ +# Copy to .env and adjust. Every value except the OAuth pair has a working +# default for a throwaway local demo, so the quickstart runs without edits. + +MYSQL_ROOT_PASSWORD=warpengine + +# Shared secret for the /update endpoint (X-Update-Secret header). +UPDATE_SECRET=example-update-secret + +# Password of the "drop" user on the artifact drop area (SSH, port 2222). +DROP_PASSWORD=drop + +# Published ports. +APP_PORT=8080 +DROPAREA_SSH_PORT=2222 +GITEA_SSH_PORT=2223 + +# --- profile "ci" only ------------------------------------------------------- + +WOODPECKER_AGENT_SECRET=example-agent-secret + +# OAuth2 application created in gitea (Settings -> Applications), redirect URI +# http://woodpecker:8000/authorize — required before the ci profile starts. +WOODPECKER_GITEA_CLIENT= +WOODPECKER_GITEA_SECRET= diff --git a/examples/compose/docker-compose.yml b/examples/compose/docker-compose.yml new file mode 100644 index 0000000..2ce8abb --- /dev/null +++ b/examples/compose/docker-compose.yml @@ -0,0 +1,131 @@ +# Example stack: everything WarpEngine needs to come alive, end to end. +# +# mysql the catalog database +# app a minimal Rails host with the engine mounted from this +# repo checkout (headless: API + updater, no ActiveAdmin) +# droparea SSH server where build pipelines drop artifacts; shares +# the "softwares" volume with the app +# gitea (profile "ci") the git forge +# woodpecker (profile "ci") CI server + agent, wired to gitea +# +# Quickstart (catalog + drop area only): +# cp .env.example .env +# docker compose up --build +# +# Full loop with forge + CI: +# docker compose --profile ci up --build +# +# See ../../README.md ("Example stack") or the WarpEngine wiki page for the +# full walkthrough, including the one-time gitea/woodpecker OAuth wiring. + +services: + mysql: + image: mysql:8 + healthcheck: + test: ["CMD", "mysqladmin", "ping", "-h", "localhost"] + timeout: 20s + retries: 10 + environment: + MYSQL_ROOT_PASSWORD: ${MYSQL_ROOT_PASSWORD:-warpengine} + MYSQL_DATABASE: warp_engine_example + volumes: + - mysql-data:/var/lib/mysql + + app: + build: ./host_app + ports: + - "${APP_PORT:-8080}:3000" + environment: + RAILS_ENV: development + MYSQL_ROOT_PASSWORD: ${MYSQL_ROOT_PASSWORD:-warpengine} + UPDATE_SECRET: ${UPDATE_SECRET:-example-update-secret} + FILE_CONTAINER_PATH: /softwares + IMAGE_CONTAINER_PATH: /images + depends_on: + mysql: + condition: service_healthy + volumes: + - ./host_app:/app + - ../..:/warp_engine # the engine itself, consumed as a path gem + - bundle:/usr/local/bundle + - softwares:/softwares + - images:/images + + # SSH landing zone for build artifacts. Pipelines (or you, with scp) upload + # into ~/drop here; the app sees the same files under /softwares. + droparea: + image: linuxserver/openssh-server + environment: + PUID: 1 + PGID: 1 + SUDO_ACCESS: "false" + PASSWORD_ACCESS: "true" + USER_NAME: drop + USER_PASSWORD: ${DROP_PASSWORD:-drop} + ports: + - "${DROPAREA_SSH_PORT:-2222}:2222" + volumes: + # A subdir of the drop user's home (/config), so sshd's own state files + # never end up in the catalog directory. + - softwares:/config/drop + + # --- profile "ci": the forge + CI producing releases for the catalog ------ + # + # gitea and woodpecker refer to each other by their service names, so your + # browser needs to resolve those names too: + # echo "127.0.0.1 gitea woodpecker" | sudo tee -a /etc/hosts + # gitea: http://gitea:3000 woodpecker: http://woodpecker:8000 + + gitea: + image: gitea/gitea:1.27.0 + profiles: ["ci"] + environment: + DISABLE_REGISTRATION: "true" + ROOT_URL: "http://gitea:3000" + ports: + - "3000:3000" + - "${GITEA_SSH_PORT:-2223}:22" + volumes: + - gitea-data:/data + + woodpecker: + image: woodpeckerci/woodpecker-server:v3.16.0 + profiles: ["ci"] + environment: + WOODPECKER_HOST: "http://woodpecker:8000" + WOODPECKER_OPEN: "true" + WOODPECKER_GITEA: "true" + WOODPECKER_GITEA_URL: "http://gitea:3000" + # Create an OAuth2 app in gitea first — see the README walkthrough. + WOODPECKER_GITEA_CLIENT: ${WOODPECKER_GITEA_CLIENT:-} + WOODPECKER_GITEA_SECRET: ${WOODPECKER_GITEA_SECRET:-} + WOODPECKER_SERVER_ADDR: ":8000" + WOODPECKER_AGENT_SECRET: ${WOODPECKER_AGENT_SECRET:-example-agent-secret} + ports: + - "8000:8000" + volumes: + - woodpecker-data:/var/lib/woodpecker + + woodpecker-agent: + image: woodpeckerci/woodpecker-agent:v3.16.0 + profiles: ["ci"] + environment: + WOODPECKER_SERVER: "woodpecker:9000" + WOODPECKER_AGENT_SECRET: ${WOODPECKER_AGENT_SECRET:-example-agent-secret} + # Attach pipeline containers to the stack network so steps can reach + # gitea, droparea and the app by service name. + WOODPECKER_BACKEND_DOCKER_NETWORK: warp-example + volumes: + - /var/run/docker.sock:/var/run/docker.sock + +networks: + default: + name: warp-example + +volumes: + mysql-data: + bundle: + softwares: + images: + gitea-data: + woodpecker-data: diff --git a/examples/compose/host_app/.gitignore b/examples/compose/host_app/.gitignore new file mode 100644 index 0000000..1077577 --- /dev/null +++ b/examples/compose/host_app/.gitignore @@ -0,0 +1,6 @@ +# Generated on first boot by the entrypoint (install generator + db:prepare). +db/migrate/ +db/schema.rb +log/ +tmp/ +Gemfile.lock diff --git a/examples/compose/host_app/Dockerfile b/examples/compose/host_app/Dockerfile new file mode 100644 index 0000000..c53786b --- /dev/null +++ b/examples/compose/host_app/Dockerfile @@ -0,0 +1,20 @@ +FROM ruby:3.3-slim + +RUN apt-get update && apt-get install -y --no-install-recommends \ + build-essential \ + default-libmysqlclient-dev \ + git \ + tzdata \ + libyaml-dev \ + pkg-config \ + && rm -rf /var/lib/apt/lists/* + +WORKDIR /app + +# The app source, the engine checkout (/warp_engine) and the bundle are all +# mounted at runtime by the compose file; the entrypoint bundles on first boot. +ENTRYPOINT ["./bin/docker-entrypoint"] + +EXPOSE 3000 + +CMD ["bin/rails", "server", "-b", "0.0.0.0", "-p", "3000"] diff --git a/examples/compose/host_app/Gemfile b/examples/compose/host_app/Gemfile new file mode 100644 index 0000000..95ab476 --- /dev/null +++ b/examples/compose/host_app/Gemfile @@ -0,0 +1,10 @@ +source "https://rubygems.org" + +gem "rails", "~> 8.0.0" +gem "mysql2", "~> 0.5" +gem "puma" + +# In this example stack the engine comes straight from the repo checkout the +# compose file mounts at /warp_engine. A real host would use the git source or +# the Forgejo rubygems registry instead — see the engine README. +gem "warp_engine", path: ENV.fetch("WARP_ENGINE_PATH", "/warp_engine") diff --git a/examples/compose/host_app/Rakefile b/examples/compose/host_app/Rakefile new file mode 100644 index 0000000..d1baef0 --- /dev/null +++ b/examples/compose/host_app/Rakefile @@ -0,0 +1,3 @@ +require_relative "config/application" + +Rails.application.load_tasks diff --git a/examples/compose/host_app/bin/docker-entrypoint b/examples/compose/host_app/bin/docker-entrypoint new file mode 100755 index 0000000..877a203 --- /dev/null +++ b/examples/compose/host_app/bin/docker-entrypoint @@ -0,0 +1,16 @@ +#!/bin/sh +set -e + +bundle check || bundle install + +# First boot: run the install generator the way a real host would. It creates +# the create_warp_engine_tables migration; --skip leaves our initializer alone. +if ! ls db/migrate/*create_warp_engine_tables* >/dev/null 2>&1; then + bin/rails generate warp_engine:install --skip +fi + +bin/rails db:prepare + +rm -f tmp/pids/server.pid + +exec "$@" diff --git a/examples/compose/host_app/bin/rails b/examples/compose/host_app/bin/rails new file mode 100755 index 0000000..efc0377 --- /dev/null +++ b/examples/compose/host_app/bin/rails @@ -0,0 +1,4 @@ +#!/usr/bin/env ruby +APP_PATH = File.expand_path("../config/application", __dir__) +require_relative "../config/boot" +require "rails/commands" diff --git a/examples/compose/host_app/config.ru b/examples/compose/host_app/config.ru new file mode 100644 index 0000000..5c59c6f --- /dev/null +++ b/examples/compose/host_app/config.ru @@ -0,0 +1,4 @@ +require_relative "config/environment" + +run Rails.application +Rails.application.load_server diff --git a/examples/compose/host_app/config/application.rb b/examples/compose/host_app/config/application.rb new file mode 100644 index 0000000..88c29cd --- /dev/null +++ b/examples/compose/host_app/config/application.rb @@ -0,0 +1,23 @@ +require_relative "boot" + +require "rails" +require "active_model/railtie" +require "active_record/railtie" +require "action_controller/railtie" +require "action_view/railtie" +require "action_dispatch/railtie" + +Bundler.require(*Rails.groups) +require "warp_engine" + +module HostApp + class Application < Rails::Application + config.load_defaults 8.0 + + config.time_zone = "UTC" + config.active_record.default_timezone = :utc + + # Demo stack: reachable as localhost, gitea-network hostnames, etc. + config.hosts.clear + end +end diff --git a/examples/compose/host_app/config/boot.rb b/examples/compose/host_app/config/boot.rb new file mode 100644 index 0000000..93f96d5 --- /dev/null +++ b/examples/compose/host_app/config/boot.rb @@ -0,0 +1,3 @@ +ENV["BUNDLE_GEMFILE"] ||= File.expand_path("../Gemfile", __dir__) + +require "bundler/setup" diff --git a/examples/compose/host_app/config/database.yml b/examples/compose/host_app/config/database.yml new file mode 100644 index 0000000..d202ccb --- /dev/null +++ b/examples/compose/host_app/config/database.yml @@ -0,0 +1,8 @@ +development: + adapter: mysql2 + encoding: utf8mb4 + username: root + password: <%= ENV.fetch("MYSQL_ROOT_PASSWORD", "warpengine") %> + host: mysql + port: 3306 + database: warp_engine_example diff --git a/examples/compose/host_app/config/environment.rb b/examples/compose/host_app/config/environment.rb new file mode 100644 index 0000000..73a3979 --- /dev/null +++ b/examples/compose/host_app/config/environment.rb @@ -0,0 +1,3 @@ +require_relative "application" + +Rails.application.initialize! diff --git a/examples/compose/host_app/config/environments/development.rb b/examples/compose/host_app/config/environments/development.rb new file mode 100644 index 0000000..42cd3ee --- /dev/null +++ b/examples/compose/host_app/config/environments/development.rb @@ -0,0 +1,10 @@ +Rails.application.configure do + config.enable_reloading = true + config.eager_load = false + config.consider_all_requests_local = true + + config.active_record.migration_error = :page_load + config.active_record.verbose_query_logs = true + + config.logger = ActiveSupport::Logger.new($stdout) +end diff --git a/examples/compose/host_app/config/initializers/apipie.rb b/examples/compose/host_app/config/initializers/apipie.rb new file mode 100644 index 0000000..627be2d --- /dev/null +++ b/examples/compose/host_app/config/initializers/apipie.rb @@ -0,0 +1,11 @@ +Apipie.configure do |config| + config.app_name = "WarpEngine Example Host" + config.api_base_url = "" + config.doc_base_url = "/api/docs" + config.api_controllers_matcher = [ + "#{WarpEngine::Engine.root}/app/controllers/**/*.rb" + ] + config.validate = false + config.translate = false + config.default_version = "1.0" +end diff --git a/examples/compose/host_app/config/initializers/warp_engine.rb b/examples/compose/host_app/config/initializers/warp_engine.rb new file mode 100644 index 0000000..2ad62d8 --- /dev/null +++ b/examples/compose/host_app/config/initializers/warp_engine.rb @@ -0,0 +1,9 @@ +Rails.application.config.to_prepare do + WarpEngine.configure do |c| + c.file_container_path = ENV.fetch("FILE_CONTAINER_PATH", "/softwares") + c.image_container_path = ENV.fetch("IMAGE_CONTAINER_PATH", "/images") + + # Beállítatlan secret esetén a /update endpoint minden kérést elutasít. + c.update_secret = ENV["UPDATE_SECRET"] + end +end diff --git a/examples/compose/host_app/config/routes.rb b/examples/compose/host_app/config/routes.rb new file mode 100644 index 0000000..ff4d071 --- /dev/null +++ b/examples/compose/host_app/config/routes.rb @@ -0,0 +1,6 @@ +Rails.application.routes.draw do + apipie + + # Keep the engine mount the last entry so the host's own routes win. + mount WarpEngine::Engine => "/" +end