Models: GPU or endpoint
Where 3AM's two models run. A GPU on the host, or endpoints you point it at (OpenRouter, vLLM, Ollama, your LLM gateway). Never the CPU. Every option, with copy-paste installs.
3AM uses two models, and neither ever acts (what each does):
| System 1 | The investigator (System 2) | |
|---|---|---|
| Model | togethercomputer/tev1-4b-experimental (4B) | Llama 3.3 70B by default, or Qwen2.5 72B; qwen2.5:7b-instruct on a local GPU |
| API | A decisions endpoint (OpenRouter's /api/alpha/decisions, Ollama's /v1/systemone) | OpenAI-compatible chat completions, ≥ 32k context, JSON output |
| Job | Verifies causes, fixes and every sentence of the note | Investigates through read-only actions; writes the note |
Never on the CPU
The models run on a GPU or behind an endpoint. A host without a GPU must be given both endpoints. There is
no CPU model profile: an earlier install's --profile cpu is moved to auto with a warning.
Why not the CPU
- The investigator's prompts are large: the alerts, the evidence bundle, every method's findings, the read tools and each step's history.
- CPU inference makes everything wait. A 7B model answers in minutes per step on a CPU, with a context too short for those prompts. A 1.5B model is fast enough but can't hold the evidence.
- Incidents share the CPU. Model calls are on the critical path of every investigation; on a CPU they take the cores every other incident's evidence gathering needs.
Choose
Does the 3AM host have a GPU with 8 GB+ VRAM?
├─ yes ─▶ ./install.sh (the gpu profile: models run on it)
└─ no ──▶ do you have a GPU server, an LLM gateway, or may you use a hosted API?
├─ yes ─▶ ./install.sh --model-url … --model … --s1-url … (endpoints)
└─ no ──▶ 3AM won't install. For a lab only: --profile none --allow-no-modelsEndpoints (any host)
./install.sh --model-url https://openrouter.ai/api/v1 --model meta-llama/llama-3.3-70b-instruct \
--model-key-file ~/openrouter.key \
--model-extra '{"provider":{"data_collection":"deny"}}'System 1 uses OpenRouter's decisions endpoint (https://openrouter.ai/api/alpha/decisions) with the same key: no
--s1-url needed. data_collection: deny routes only to providers that neither store nor train on the data. Use
hosted APIs only for data you are allowed to send; secrets and card numbers are masked first.
Refused: an endpoint on the 3AM host itself (localhost, 127.0.0.1, 3am-model) when the host has no GPU, and
a missing System 1 endpoint (unless the investigator is on OpenRouter). The installer says why and stops.
A GPU on the host
| GPU | Needs | Runs |
|---|---|---|
| NVIDIA, 8 GB+ VRAM | NVIDIA Container Toolkit (Docker) or CDI (nvidia.com/gpu=all, Podman) | The investigator (qwen2.5:7b-instruct) in 3am-model, on an internal network with no published port |
| AMD, 8 GB+ VRAM | ROCm (/dev/kfd, /dev/dri) | The ROCm build of the same runtime |
| 24 GB+ (L4, A10G) | As above | The investigator and System 1 side by side, with room for 8–16 concurrent calls |
./install.sh # auto: a GPU is found, the gpu profile is used
./install.sh --profile gpu # the same, and fail if no GPU is foundAn air-gapped GPU install needs a bundle with the model runtime and weights inside; otherwise use endpoints.
Kubernetes
| Value | Meaning |
|---|---|
externalModel.url, .model, .keySecret, .extra | The investigator's endpoint (no model runs in the cluster) |
systemOne.url, .model, .keySecret | System 1's endpoint. Empty: OpenRouter's, when externalModel is OpenRouter |
model.enabled, model.profile: gpu | The model runtime on a GPU node (requests nvidia.com/gpu: 1) |
vllm.*, systemTwo.profile | System 2 on the cluster's GPUs with vLLM (llama-3.3-70b or qwen2.5-72b) |
allowNoModels | Labs only |
The chart fails to render a model pod that doesn't request a GPU, externalModel.url without systemOne.url, or
no models at all without allowNoModels. Kubernetes.
Settings
| Variable | Installer | Meaning |
|---|---|---|
THREEAM_MODEL_URL, THREEAM_MODEL | --model-url, --model | The investigator's endpoint and model |
THREEAM_MODEL_KEY_FILE / THREEAM_MODEL_KEY | --model-key-file | Its key (prefer the file) |
THREEAM_MODEL_EXTRA | --model-extra | JSON merged into each request |
THREEAM_S1_URL, THREEAM_S1_MODEL, THREEAM_S1_KEY_FILE | --s1-url, --s1-model, --s1-key-file | System 1 |
THREEAM_SYSTEM2 | llama-3.3-70b (default) or qwen2.5-72b; THREEAM_MODEL wins | |
THREEAM_MODEL_DAILY_TOKENS, THREEAM_S1_DAILY_TOKENS | A daily token ceiling (0: none). Spent, model steps are skipped until 00:00 UTC (model.budget_exhausted) | |
THREEAM_MODEL_CONCURRENCY, THREEAM_S1_CONCURRENCY | Calls at once: 8 and 16. Raise to what the endpoint is sized for | |
THREEAM_MODEL_TIMEOUT_S, THREEAM_S1_TIMEOUT_S | Request timeouts (System 1: 20 s) | |
THREEAM_MODEL_CASSETTE, THREEAM_MODEL_REPLAY | Record and replay model calls (record, replay, auto): tests and labs re-run for free |
All of them: Environment variables.
Check it
./install.sh --doctor | grep -i model # "model placement" and both models answeringPreflight and 3AM Doctor fail "model placement" on a host without a GPU whose models are missing or local.