AI applications

Define the work

An AI feature is more than a model call. A user action becomes input, passes through policy and context, reaches a model, and returns as something the product must validate and present. Each step can affect cost, response time, quality, and trust. An infrastructure layer can make this path explicit instead of scattering provider-specific decisions through application code.

Name the work before choosing a model. A support reply, document summary, and structured extraction task have different requirements. Define the expected output and what counts as useful. That makes route selection clearer and helps the application recognize when a response needs review or another attempt.

Keep product rules in the application

The application should authenticate the user, choose relevant data, enforce permissions, and shape the experience. Infrastructure can route an eligible request, attach approved context, and return consistent metadata. Validate structured responses and handle refusals or incomplete results. Keep sensitive actions behind the application’s ordinary authorization checks.

Use a small set of stable task names and explicit request fields. Keep provider-specific options at the integration edge where possible. This lets a feature express needs such as structured output without adopting every provider’s vocabulary. Document essential provider-specific capabilities as deliberate constraints.

Observe and change carefully

Capture enough metadata to understand a request without keeping unnecessary user data. A trace may include a request identifier, task name, selected route, latency, available usage counts, and outcome. Keep operational telemetry separate from prompt content, and use approved sampling and retention controls for quality review examples.

Evaluate routes against representative inputs and edge cases. Human review helps with qualities that are hard to score automatically, such as grounding or completeness. Introduce changes gradually, define conditions to pause or roll back, and retain a known-good configuration.