gateway.yaml is the configuration file for a self-hosted ManyLayers deployment. The gateway, workspace and deployer services all read the same file, passed with -config. Copy configs/gateway.example.yaml to gateway.yaml and edit it. The container image ships configs/gateway.container.yaml at /etc/manylayers/gateway.yaml as its default.
ManyLayers Cloud is configured through the console, not this file.
Interpolation. ${ENV_VAR} and ${ENV_VAR:-default} are expanded when the file loads. An unset variable becomes an empty string, so a key like upstream_api_key: ${OPENAI_API_KEY} with the variable missing is empty, and an empty team API key stops startup. HashiCorp Vault references also work: ${vault:secret/data/path#key} (requires VAULT_ADDR and VAULT_TOKEN); one that cannot be resolved fails startup.
The file is parsed strictly. A key that is not listed on this page makes startup fail with a parse error, rather than being ignored.
GATEWAY_PORT (server.listen), DATABASE_URL (database.url), REDIS_URL (redis.url), SHUTDOWN_TIMEOUT (server.shutdown_timeout), MANYLAYERS_ENCRYPTION_KEY (connectors.encryption_key) and MANYLAYERS_PII_VAULT_KEYS / MANYLAYERS_PII_VAULT_KEY_VERSION (pii.vault). See Self-hosting for the full list.
server
server:
listen: ":8180" # gateway bind address; workspace and deployer use WORKSPACE_PORT and DEPLOYER_PORT
max_body_bytes: 10485760 # 10 MiB request body limit
shutdown_timeout: 20s # time for in-flight requests to finish after SIGTERM
listen applies to the gateway only. A request over max_body_bytes is refused with 413 body_too_large. Changes to this section require a restart.
database
database:
url: ${DATABASE_URL}
url is empty it defaults to postgres://postgres:test@localhost:5532/manylayers?sslmode=disable, the local dev stack.
The long-running services do not migrate the schema. Run dbtool migrate -config <file> (the compose migrate job) before first start and before every upgrade; a service whose schema is behind refuses to start. Changes to this section require a restart.
redis
redis:
url: ${REDIS_URL} # redis://[:password@]host:port/db; takes precedence over addr
# addr: "" # "" with no url = in-memory limits (single replica only)
# password: ${REDIS_PASSWORD}
# db: 0
auth
auth:
oidc:
issuer: "" # "" disables OIDC; set to your IdP URL to enable SSO
# audience: manylayers # required when issuer is set
# jwks_url: "" # "" = auto-discovery via OIDC
# team_claim: team # JWT claim carrying the team name (default "team")
# team_mapping: # map IdP group names to gateway teams
# idp-eng-group: engineering
# role_claim: role # default "role"
# admin_role: admin # claim value that grants admin (default "admin")
# spa_client_id: "" # OIDC client ID for console SSO login; "" hides the SSO button
ml- is verified against the issuer. audience is required once issuer is set, and audience or jwks_url without issuer is an error.
audit
audit:
retention_days: 90 # auto-prune audit entries after N days (default 90)
log_bodies: false # store request/response bodies (post-PII-redaction only); default false
GET /admin/audit/verify, which checks the most recent 10,000 records. See Request Logging & Audit.
pii
pii:
detector: builtin # builtin | http | off
# http_url: "http://localhost:5002/analyze" # required when detector is http
# vault: # optional reversible redaction
# keys:
# v1: ${YOUR_PII_KEY_ENV} # base64 of 32 random bytes, from an env var of your choosing
# current_key: v1
builtin detector uses regex-based entity recognition (email, phone, SSN, credit card and similar). The http detector calls a Presidio-style sidecar at http_url. With vault.keys set, each redacted value becomes a numbered token and is stored encrypted beside the trace, so authorized administrators can reveal it. current_key may be omitted when exactly one key is configured. See PII redaction.
firewall
firewall:
default_mode: audit # off | audit | enforce (default audit)
threshold: 50 # summed rule score that trips the firewall (default 50)
audit mode, detections are recorded but not blocked. In enforce mode, a request over the threshold is refused with 403 prompt_injection. A team can override the default with its own firewall_mode. See Prompt injection.
cache
cache:
enabled: false
max_entries: 1024 # in-process LRU size (default 1024; unused when Redis is configured)
ttl: 5m # default 5m
# semantic:
# enabled: false
# embedding_model: text-embedding-3-small # required when enabled
# threshold: 0.92 # cosine similarity cutoff (default 0.92)
# max_entries: 512 # default 512
# store: memory # memory | qdrant (default memory)
store: qdrant shares the semantic cache across replicas and requires rag.qdrant.url. A request is cached only when the cache is enabled here, caching is enabled for its team (cache_enabled), and the request has temperature: 0 or the model sets cache: true. See Caching.
async
async:
workers: 2 # async job workers per gateway process; 0 disables
# webhook_signing_secret: ${ASYNC_WEBHOOK_SECRET} # HMAC key for webhook payloads
/v1/async/*. See Async requests.
queue
queue:
backend: db # db | kafka (default db)
# kafka:
# brokers:
# - kafka:9092 # required when backend is kafka
# prefix: "" # optional topic prefix, e.g. "prod."
db backend uses Postgres and needs no extra infrastructure. The kafka backend uses consumer groups with at-least-once delivery: a message that fails 5 times is published to <topic>.dlq. Changes to this section require a restart.
router
router:
strategy: least_inflight # least_inflight | round_robin | prefix_affinity | score
# prefix_bytes: 1024 # prompt prefix hashed by prefix_affinity (default 1024)
# session_ttl: 10m # sticky session lifetime (default 10m)
# health_interval: 15s # active probe period (default 15s)
# failure_threshold: 3 # consecutive failures before ejecting an upstream (default 3)
# cooldown: 30s # ejection duration before retrying (default 30s)
# health_threshold: 0 # 0-1; eject below this success rate (0 = off)
# restore_health: 0.5 # 0-1; health an endpoint returns to after ejection
# max_ejection_duration: 0 # cap on the escalating ejection window (0 = uncapped)
# respect_retry_after: true # use a provider's own Retry-After as the ejection window
rag
rag:
enabled: false
vector_store: memory # memory | qdrant
# qdrant:
# url: http://qdrant:6333 # required when vector_store is qdrant
# api_key: ${QDRANT_API_KEY}
# hybrid_search: false # BM25-style keyword scores fused with vector scores (RRF)
# rerank_model: "" # logical model used for a cross-encoder rerank pass
# recency_half_life: 0 # decay for chunks with a timestamp; 0 disables
# embedding_model: "" # model for per-user memory semantic recall
/v1/kb). The memory store is per process and lost on restart; use qdrant for anything persistent.
connectors
connectors:
encryption_key: "" # AES-256-GCM passphrase for stored credentials
MANYLAYERS_ENCRYPTION_KEY overrides this value. When neither is set, each service generates a key in data/credentials.key (or MANYLAYERS_ENCRYPTION_KEY_FILE); with more than one service or replica, set the key explicitly so they can all read each other’s credentials. Changing the key makes existing sealed credentials unreadable.
suite
suite:
enabled: false
# session_ttl: 720h # how long a login lasts before sign-in (default 720h)
# access_token_ttl: 15m # access token lifetime before silent refresh (default 15m)
# local_users_enabled: true # allow email and password logins
# user_approval_required: false
# http_tool_allowlist: # hosts the agent http_request tool may reach
# - api.example.com
# resend_api_key: ${RESEND_API_KEY} # email for sign-in codes, invitations, alerts
# resend_from: "ManyLayers <[email protected]>"
# public_url: https://app.example.com # your console origin, used in invitation links
/auth/*) and the /w/* endpoints for chat, agents, workflows, evals and knowledge bases. Leave it false and the Workspace service serves neither. public_url is your own console address; if empty, invitation links use the origin the inviting administrator was on.
deployer
deployer:
mode: "" # "" (disabled) | mock | kubernetes | dstack
# namespace: manylayers # kubernetes namespace (default manylayers)
# reconcile_interval: 5s
# cold_start_timeout: 2m # wake-from-zero wait before a 504
# dstack: # all three required when mode is dstack
# server_url: http://localhost:3100
# token: ${DSTACK_TOKEN}
# project: main
telemetry
telemetry:
region: global # global | eu | us
environment: "" # label on every span, e.g. production
Other sections
billing (billing webhooks and the on-prem license file), saas (signup_approval_required), notifications.slack and staff (Staff Portal session length) configure operator features of the control plane. The mode: key is accepted and ignored. See Deployment Model.
models
Each entry maps a logical name (what clients send) to one or more upstreams (where requests go):
models:
- logical_name: gpt-4o
provider: openai
upstream_url: https://api.openai.com
upstream_model: gpt-4o
upstream_api_key: ${OPENAI_API_KEY}
encoding: cl100k_base # tokenizer for counting; default cl100k_base
input_price_per_1k: 0.005
output_price_per_1k: 0.015
pii_output_mode: passthrough # passthrough | buffer (default passthrough)
| Field | Notes |
|---|
logical_name | Required and unique. |
provider | One of openai (default), anthropic, azure, gemini, vertex, bedrock, cohere, typesafe, mistral, groq, deepseek, together, perplexity, xai, fireworks, openrouter, cerebras, sambanova, ai21, ollama. |
upstream_url, upstream_model, upstream_api_key | Single-endpoint form. Use either these or upstreams, not both. For the OpenAI-compatible presets (mistral, groq, deepseek, together, perplexity, xai, fireworks, openrouter, cerebras, sambanova, ai21, ollama) upstream_url can be omitted and defaults to the provider’s public endpoint (ollama defaults to http://localhost:11434). |
upstreams | List of endpoints for failover and load balancing: url, model, api_key, weight (default 1), provider, api_version (Azure, default 2024-06-01), region (AWS). |
aws_region | Required for bedrock. The key is ACCESS_KEY:SECRET_KEY[:SESSION_TOKEN], or a Bedrock API key. |
aliases | Extra names that resolve to this model. |
cached_input_price_per_1k, cache_write_price_per_1k | Prices for cached and cache-write prompt tokens. Unset means billed at input_price_per_1k. |
media_prices | image (per image), speech_per_1k_chars, transcription (per request). |
cache | true caches responses even when temperature is not 0. |
pii_output_mode | buffer scans and redacts the completion before it is returned, which delays streaming. |
bedrock_mantle, sagemaker or databricks) cannot be declared in gateway.yaml.
aliases
A model may declare additional names that resolve to it. Aliases let callers keep sending a name they already use while you change what serves it:
models:
- logical_name: house-fast
aliases: ["gpt-4o-mini", "fast"]
provider: groq
upstream_url: https://api.groq.com/openai
upstream_model: llama-3.3-70b-versatile
GET /v1/models, work on GET /v1/models/{model}, and a team policy naming either the alias or the model grants both. A model name always wins over an alias of the same name, so an alias can never reroute traffic away from a real model.
See Providers for per-provider configuration and OpenAI Compatibility for what each provider can honor.
teams
Seeds teams, their limits and API keys:
teams:
- name: engineering
monthly_token_budget: 1000000 # 0 = unlimited
rpm_limit: 60 # requests per minute; 0 = unlimited
tpm_limit: 100000 # tokens per minute; 0 = unlimited
cache_enabled: false
firewall_mode: "" # "" = use gateway default; off | audit | enforce
models: ["gpt-4o-mini"] # logical names or "*"; omit for unrestricted
api_keys:
- name: dev
key: ${MANYLAYERS_DEV_KEY}
role: member # member | admin (default member)
POST /admin/keys or the console’s API Keys page to set them. Every name in models must be a logical_name defined above (or *).
Send SIGHUP to a running service to reload models and teams without downtime. Other sections are read at startup. Sections that the console’s Settings page manages are seeded from this file on first boot and then owned by the database; see Settings & Configuration Precedence.
