Study
Before anything breaks, 3AM rehearses every cause it could be asked to prove, records what normal looks like, reads who talks to whom, and lists exactly what stands in the way.
Study (/study, g y) asks 3AM to get to know your estate before an incident does. It only reads. Owners and
admins start it; everyone can see the report.
What a study does
| Part | What 3AM does |
|---|---|
| Rehearse | For each service, every cause its alerts would ask 3AM to prove is checked now, as an alert about that service would ask. Each cause is can prove, can't yet (with why), or at the alert (only the alert names what to look at) |
| Baseline | Every connection that can describe itself is read: settings, schema, versions. The next incident is compared with these facts |
| Relations | Sessions on each database, code that uses an engine only one database runs, a service and its database by name. Suggested on Estate for a person to confirm |
Safe on a live estate
- One cause at a time, with a pause between them: never a burst.
- Only read-only actions; the check engine refuses anything else.
- It stands aside while an incident is investigated.
- A time limit (5 min to 2 h, 15 by default), and Stop before the next cause.
The report
- A ring: the share of causes 3AM can prove today, with can prove, can't yet and at the alert.
- True right now: causes that hold at this moment with no alert. Worth a look before they become incidents.
- What stands in the way, grouped by kind, most common first; remove one obstacle and every cause behind it becomes provable:
| Kind | Means | Remove it by |
|---|---|---|
| Nothing connected can read it | No connection provides the read the check needs | Connecting the kind of evidence missing (the Guide shows which) |
| Several connections could answer | 3AM never guesses which database or server an incident is on | Naming the service on each connection, or labelling the alerts with instance or host |
| The data isn't there | The metric or query returned nothing | Installing the exporter, or mapping its metric name |
| The read was refused or failed | Usually access | Granting the account 3AM uses |
| 3AM can't place it | It doesn't run anywhere a connected system can see | Connecting where it runs, or placing it on Estate |
- Service by service: a tile per service; open one for what can't be proven yet (with reasons), what's checked at the alert, and what can be proven.
The Guide recommends a first study, a new one after a month, and points at the gaps the last one found.
API
curl -s -X POST -H "Authorization: Bearer $T" -d '{"minutes":15}' $URL/v1/study # 202, or 409 if one is running
curl -s -H "Authorization: Bearer $T" $URL/v1/study # running + last report
curl -s -X POST -H "Authorization: Bearer $T" $URL/v1/study/stopAudit events: study.started, study.finished, study.stop_requested, study.failed. Rehearsals are not written as
an incident's check events.
Estate
Your services, owners, teams, dependencies, alerts, workloads, databases and repositories, where each link came from, what isn't placed yet, and how discovery never drops anything.
Autonomy, drafted fixes and the halt switch
What 3AM may do, service by service (off, shadow, L1, L2), readiness for L1, drafted fixes, and how to stop everything at once.