Skip to main content

Configure the AI Gateway

Enable the AI Gateway through the platform chart, then apply its custom resources in the order described below. See AI Gateway for ongoing configuration.

Prerequisites​

  • A corporate identity provider, configured once at global.stacklok.primaryIdp. The gateway ties every request to an identity from that provider. See Configure identity.
  • PostgreSQL, which the platform already requires. Budgets, pricing, and recorded spend live there.
  • Redis or Valkey, only if you intend to enable the detection result cache. It is optional and off by default. See PCI/PII controls.

Enable it​

Set the install toggle in your platform values and upgrade:

values.yaml
global:
  stacklok:
    aiGateway:
      enabled: true

This installs the AI Gateway operator and custom resource definitions. Apply an AIGateway resource to create a gateway instance.

Bring it up in this order​

Complete the following sequence before sending production traffic:

  1. Create budgets for every user or group that will send traffic, before you enable the budget webhook target. A caller with no applicable budget is refused. See Budgets and pricing.

  2. Apply an AIGateway resource with at least one provider and one route. When the operator's budget webhook target is configured, budget enforcement applies across the gateway. See Connect model providers.

  3. Verify. Confirm the gateway reports its providers ready and that budget enforcement probed successfully:

    kubectl get aigw -n <NAMESPACE>
    kubectl get aigw <NAME> -n <NAMESPACE> \
      -o jsonpath='{.status.webhooks}' | jq .

Content screening posture​

Detection failures deny requests by default. An experimental waiver can allow traffic during a rollout or incident, but it is unavailable on the stable release channel.

Next steps​