Designing AI workflows that help the product instead of decorating it

Start with the job, not the model

A model is not a product requirement. Start by mapping the work a user is already trying to complete: the input they have, the decision they need, the action that follows, and the consequence of a mistake. A useful AI workflow usually addresses an expensive ambiguity inside that path. It may classify an unstructured request, draft a response from known context, or extract fields that a person would otherwise copy by hand.

Choose the narrowest workflow that changes the outcome. A chat box on every screen may look modern while adding no leverage. By contrast, a draft placed exactly before a review step can shorten the path without pretending to own the decision. Define success in product terms such as fewer manual transitions, clearer handoffs, or more complete inputs. Do not begin with model benchmarks that users never experience.

The workflow runtime in this portfolio is a useful example of that separation. Persisted graphs are compiled into an executable path rooted at a manual, schedule, or webhook trigger. The compiler rejects duplicate identifiers, cycles, invalid edges, ambiguous branches, and unreachable targets before execution. This is ordinary software doing ordinary validation; the uncertain component does not get to redefine the process.

Put deterministic boundaries around uncertain work

Treat a model call as one typed step, not as the operating system for the whole feature. Code should decide authentication, authorization, routing, deadlines, allowed tools, storage, and state transitions. The model can propose content or a structured interpretation inside those boundaries.

Give every step an explicit contract:

• a bounded input schema and maximum size;
• a declared output shape;
• a timeout and cancellation path;
• a policy for side effects;
• a version for prompts, schemas, and workflow definitions.

The repository's execution runner demonstrates why versions matter. When a job is claimed, it checks that the workflow has not changed since the job was queued. Persisted stages must still match the compiled graph. A stale job fails with a specific reason instead of quietly running a different definition. That principle applies equally to prompts: record what contract produced an output, or debugging becomes guesswork.

Keep side effects behind deterministic adapters. If generated text will become an HTTP request, validate the method, URL, headers, body, and destination in code. Credentials should be referenced from protected storage rather than embedded in a prompt or graph. In this backend, request configuration rejects secret-like fields, credentials are resolved separately, response data is redacted before persistence, and oversized execution data is replaced with a bounded record. Those controls are more important than clever prompt wording.

Validate in layers

Validation is not one JSON parse after generation. It is a sequence. First validate the user's input before spending model time. Then validate the model's structure, required fields, enum values, lengths, and relationships. Finally validate the proposed action against current product state. A syntactically valid answer may still reference a deleted record or violate a business rule.

Prefer repair for harmless formatting defects and rejection for semantic uncertainty. A missing optional field can receive a default. An unsupported destination, unknown customer, or contradictory decision should stop the workflow. Never transform a failed validation into a confident-looking success message.

For high-impact changes, make the model produce a proposal rather than execute the change. The review surface should show source context, proposed output, validation warnings, and the exact action approval will trigger. The person reviewing needs meaningful control: edit, approve, reject, or request another draft. A ceremonial confirmation button after the action has effectively happened is not a human checkpoint.

Design failure as a first-class result

Model providers time out, structured outputs drift, dependencies return errors, workers restart, and users cancel work. Each failure needs a stable code, a safe message, and a known state transition. Separate retryable transport failures from invalid inputs and policy failures. Retry only operations that are idempotent, or protect them with idempotency keys.

This distinction is visible in the workflow runner. Executions move through queued, running, completed, failed, cancellation-requested, or cancelled states. Each stage stores its own status. If a worker lease is recovered while an external request may already have been dispatched, the runner stops rather than risking a duplicate side effect. That is the correct bias: uncertainty about a write is not permission to repeat it.

A useful fallback should preserve the core task. Let the user continue manually, save a draft, or retry only the failed step. Do not trap the whole product behind a model outage. When AI is optional acceleration, the non-AI path is part of the feature, not technical debt.

Observe the workflow, not only the endpoint

A successful HTTP response says little about product quality. Capture a trace across workflow, execution, stage, model request, validation, review, and final action. Useful fields include workflow and prompt versions, model identifier, queue time, generation duration, validation outcome, retry count, cancellation, and final disposition. Avoid logging raw prompts or outputs when they may contain private data. Prefer structured, redacted events.

The backend keeps execution stages and audit records, and exposes sanitized public projections separately from authenticated detail. Server-sent events publish changing execution snapshots while removing stage input, output, metadata, and error details from the public view. This is a sound observability boundary: operators need enough evidence to diagnose state, while public progress indicators need far less data.

Review traces as product evidence. Where do people edit drafts heavily? Which validation rule fails repeatedly? Which step dominates latency? A workflow that technically completes but is routinely discarded is not healthy.

Budget latency and cost before launch

Every model step spends time and money, including retries and validation calls. Set a budget per user action, not merely per API request. Count context assembly, model calls, tool calls, retries, storage, and review time. Use the smallest model that satisfies the contract, reduce context to relevant evidence, cache stable derived data, and run independent deterministic work in parallel.

Latency should shape the interface. Fast classification can remain inline. Longer drafting should become an explicit job with progress, cancellation, and resumable state. Streaming can improve perceived responsiveness, but it does not fix a slow or unreliable workflow. Put hard deadlines around network steps; the Studio HTTP node, for example, validates configurable timeouts and returns distinct timeout and transport errors.

Know when not to use AI

Do not use a model when a rule, query, template, or conventional search produces a more reliable answer. Avoid it for exact arithmetic, permission checks, canonical identifiers, irreversible decisions, and tasks with no acceptable review or rollback. Also avoid it when the necessary context cannot be provided safely or when the cost exceeds the value of the ambiguity removed.

The best AI architecture often contains less AI than the first sketch. Keep deterministic work deterministic, isolate the uncertain step, and make its limits visible. The result may look less theatrical, but it will help the product: users finish real work, operators can explain failures, and the team can improve the workflow without rebuilding the system around a model.