Building AI Workflow Studio: from portfolio idea to observable production system

The product question behind the portfolio page

AI Workflow Studio started with a narrow product goal: make workflow automation legible. A user should be able to see what starts a workflow, what each step receives, what it returns, and where a run stopped. The portfolio value would come from demonstrating the system, not from decorating a static diagram.

That distinction split the product into three surfaces. The standalone Next.js application is the editor and operations console. The Go backend owns authentication, validation, persistence, and execution. Supabase Postgres stores definitions, executions, stages, credentials, and audits. The public portfolio is a separate article client, not the workflow runtime.

The evidenced runtime is narrower than the surrounding ecosystem: Manual, Schedule, and Webhook triggers can lead through a linear chain of HTTP Request nodes. Other palette nodes fail as unsupported rather than simulating success.

Borrowing n8n's interaction grammar, not its feature count

The editor uses a familiar n8n-style grammar: a dotted React Flow canvas, compact nodes, a palette, direct rename and delete controls, and a focused inspector. The HTTP Request node expands into Input, Parameters, and Output panes, keeping upstream items distinct from the request body. JSON, table, and schema views support different debugging modes.

The important design choice was restraint. The HTTP node supports a bounded set of methods, query parameters, headers, body text, response format, timeout, redirect policy, status handling, and Header Auth credentials. Controls that have no runtime contract are not presented as if they work. Schedule configuration similarly exposes interval, daily, weekly, and validated five-field cron modes, with an IANA timezone and an explicit skip or run-once misfire policy.

The editor also blocks execution while a workflow is new or dirty. That small guard closes a serious semantic gap: a test should run the definition the user can see, not an older persisted revision.

A graph contract shared across UI and backend

The first workflow representation was a list of labels. That remains as a public-safe summary for compatibility, but it could not express positions, trigger identity, edges, or node configuration. The production contract therefore adds a versioned definition alongside it:

WorkflowDefinitionV1
  version: 1
  nodes: [{ id, type, kind, label, position, config }]
  edges: [{ id, source, target }]

The TypeScript parser rejects duplicate IDs, unknown types, mismatched type/kind pairs, invalid positions, self-edges, duplicate edges, and missing references. Go validates again; the worker never treats browser serialization as executable truth. Credentials are referenced by ID rather than embedded in generic JSON.

React Flow owns the visible node and edge state. Every add, rename, move, configuration change, or delete is converted back into the versioned definition and reported to the editor shell for saving. Edges are derived rather than freely editable: they are non-selectable and non-deletable in the canvas. This keeps a visually flexible editor aligned with a deliberately constrained runtime.

Two roots, one downstream chain

The default definition makes the graph's most important rule visible. Schedule and Manual Trigger are independent roots. Both connect to the first action, then share the same downstream chain. They are alternatives, not sequential steps; a scheduled run must not execute a Manual Trigger first.

The edge builder recomputes this topology whenever nodes move or change. Trigger nodes are laid out at the left, while non-trigger nodes form a shared chain ordered by position. At runtime, the pure Go graph compiler selects exactly one trigger root and follows one unambiguous path. It rejects cycles, edges into triggers, duplicate edges, ambiguous branches, unreachable targets, and unsupported runtime node types.

This enables two execution modes without two graph models. A full run compiles from the selected root to the terminal node. “Execute previous nodes” compiles an inclusive root-to-target prefix, which is how the HTTP inspector can show the real input and output at one node. By contrast, “Execute step” on a trigger returns only that trigger's transient item array; it does not claim the downstream chain ran.

HTTP auth without turning the canvas into a secret store

A useful HTTP node needs authentication, but placing tokens in node headers would leak them through workflow JSON, browser state, execution snapshots, exports, and logs. The Studio instead gives Header Auth its own credential modal. The browser sends a name, header name, and secret value only when creating or rotating a connection. Existing values cannot be revealed; editing requires both fields again. The UI receives only credential metadata and stores a credential ID in the node definition.

The backend encrypts credential data before persistence and binds decryption to a credential-specific scope. The Supabase table has row-level security enabled, and normal list responses select only ID, name, type, status, and timestamps. Delete is a soft revocation, preserving referential history while making future execution fail closed. A connection test checks decryptability and schema without returning the secret.

At execution time, the worker loads an active credential, decrypts it, and injects the header immediately before dispatch. Bodies, status text, and allowed response headers are redacted against known secret values. The HTTP client validates destinations, bounds time, redirects, and response size, and sanitizes transport errors. cURL import remains an untrusted preview path.

Execution is a database-backed state machine

Clicking Run does not ask the browser to walk the canvas. The backend compiles the selected path, then calls a Supabase RPC that atomically inserts one queued execution and its ordered pending stages. Each record carries the trigger, optional target, execution mode, source, source key, workflow revision, retry relationship, and bounded initial input. A unique source key makes repeated schedule or client requests idempotent.

An in-process Go runner polls queued work. Claiming uses FOR UPDATE SKIP LOCKED and a time-bound lease so workers do not claim the same row. Before dispatch, it compares the current workflow timestamp with the queued revision. A change fails as workflow_changed rather than running a different graph under an old record.

For every node, the runner persists sanitized input, marks the stage running, executes, persists sanitized output or error, and advances the lease. Cancellation is cooperative: queued work can become cancelled immediately; running work moves through a cancellation-requested state checked between stages and around dispatch. Retry creates a new execution linked to the original instead of rewriting history.

There is a conservative trade-off around recovery. If a lease expires while an external request was running, the next worker cannot know whether the remote side effect happened. The runtime stops with dispatch_state_unknown instead of issuing a possible duplicate. That sacrifices automatic recovery for safer semantics.

Supabase is the queue; Redis is not

The backend reaches Supabase through PostgREST and RPC endpoints rather than a direct SQL connection. Postgres is both the durable record and the coordination boundary for Studio: definitions, queue rows, stage transitions, leases, idempotency, cancellation, and audits live together. Additive SQL migrations introduced the definition JSON, encrypted credentials, graph queue, stage I/O, ownership checks, and schedule/webhook hardening.

Redis exists in the backend, but the source does not place Studio executions in it. The configured RedisCache serves article caching and shared fixed-window rate limits, with code paths that can fall back when Redis is absent. Keeping the workflow queue in Postgres avoids a split-brain problem between a fast queue and the execution history, at the cost of polling and additional database writes. Naming that boundary matters more than listing Redis in an architecture diagram.

Observability as persisted evidence

Each execution has a durable timeline rather than a client-generated animation. StudioExecutionStage records node ID and type, position, status, timing, bounded input and output, sanitized error information, and detail. Authenticated execution detail can inspect that private data. Public stage and SSE projections explicitly clear input, output, metadata, and private errors before serialization.

The browser subscribes through a same-origin Next.js proxy to Server-Sent Events. The backend polls persisted state, hashes snapshots, emits only changes, sends heartbeats, and bounds stream lifetime. The React hook validates snapshots and reconnects with backoff. SSE improves freshness without becoming the source of truth; reload still reconstructs the run from Supabase.

Audit logs add a second lens. Workflow, execution, node, and credential mutations record actor and state context. Operational logs intentionally report identifiers, hostnames, and error types rather than request secrets. A fail-closed readiness endpoint probes the workflow and execution tables, unlike a general health route that only proves the process is alive.

Shipping the frontend and backend independently

Both applications use multi-stage containers and non-root runtime users. The Studio builds a Next.js standalone artifact with Bun, then runs it on Node. The backend compiles a static Go binary in a cached builder and places it in a small Alpine image with CA certificates and timezone data.

GitHub Actions validates before publishing. Studio runs Bun tests, Biome, a production build, and script checks. The backend runs formatting, go vet, race-enabled shuffled tests, golangci-lint, script checks, and an exact-image smoke test. Images use full commit SHA tags in GHCR; deployment uses pinned SSH host identity and serialized Compose updates.

The backend deployment gates success on the fail-closed readiness probe and restores the previous image if the new one fails. Manual rollback selects an already published full-SHA image rather than rebuilding old source. Frontend and backend remain separate releases, which is useful but creates an ordering constraint: additive database migrations first, compatible backend second, UI capability last.

Failures that shaped the system

Several shortcuts could produce a convincing demo and an unreliable product. A label array could draw nodes but not preserve semantics. Arbitrary edges conflicted with a linear worker. Calling a trigger test a run confused transient output with persisted lifecycle. Public stage JSON risked data exposure; credentials inside definitions spread secrets; an availability-oriented overview was unsafe as a deploy gate.

Some limitations remain intentional. The compiler supports a shared linear chain, not general branches or joins. The in-process schedule loop scans persisted workflows; database idempotency protects occurrences, but it is not a dedicated distributed scheduler. SSE is polling-backed. Credential mutation and audit persistence are adjacent, so an audit failure can be reported after a change. These boundaries are exposed rather than hidden behind optimistic UI.

What this build changed in my engineering approach

The central lesson was to design from evidence backward. Input, output, timing, and failure must be durable fields before styling the timeline. Authentication controls require secret handling across drafts, persistence, dispatch, errors, and public projections. Shared trigger paths must be proved by the compiler, not a screenshot.

The second lesson was that constraints can make a visual system more trustworthy. Derived edges, one selected root, one executable path, version-checked runs, bounded payloads, and fail-closed unsupported nodes are less flexible than a general automation platform. They also make every displayed state easier to explain.

AI Workflow Studio is therefore not presented as a replacement for n8n. It is a focused production case study: a visual contract connected to a durable state machine, with enough security and observability that a failed run remains useful evidence instead of disappearing behind the canvas.