ServiceNow

ServiceNow is where a bank records what happens to its systems, so 3AM works through it:

• The incident record. 3AM attaches to the open incident on the service if a pager or monitoring integration already opened one. Otherwise it opens one. Every step goes in as a work note: evidence, proposal, approval, action, verification.
• What changed. Every incident shows the changes on the service and the CIs it depends on in the last 24 hours, with the running ones first. The first question in an outage is "what changed?", and the approver sees the answer.
• Change records for 3AM's own actions. Where your policy asks for one, 3AM opens a standard change from a pre-approved template, or an emergency change, before anyone approves. A freeze or blackout window shows up then, not afterwards.
• Your knowledge base. Published articles, such as runbooks and known errors, are searched in every diagnosis, and the incident note cites them.
• Fixes that can only be requested. Some fixes, like group membership, an account unlock, a certificate or a firewall rule, go through the service catalogue. 3AM orders the item, ServiceNow runs its own approval and fulfilment, and 3AM waits for the result.
• ServiceNow itself. 3AM notices when ServiceNow stops answering and proves why. It keeps every ticket update until ServiceNow is back. A MID Server that goes down is restarted, and stuck request fulfilment is traced to its cause.

Before you start

• A ServiceNow admin, once, to create the integration user and an OAuth client.
• Outbound HTTPS from 3AM to your instance. ServiceNow never connects to 3AM.
• For MID Servers that run in Kubernetes, the Kubernetes connector with the MID Server's namespace in its namespaces, so 3AM can restart them.

1. Create the integration userCopy Anchor Link

In ServiceNow, create a user for 3AM (User Administration → Users). Tick Web service access only and give it the roles for the features you'll use:

Sign in with OAuth client credentials (recommended):

1. System OAuth → Application Registry → New → Create an OAuth API endpoint for external clients. Name it 3AM and keep the client secret.
2. Set OAuth Application User to the integration user. Client-credential tokens act as this user.
3. Make sure the system property glide.oauth.inbound.client.credential.grant_type.enabled is true.

A client certificate (mutual TLS, mapped to the user) or a password also work. Passwords expire and get locked, so prefer OAuth.

If 3AM will open change records, create a standard change template for its fixes (Change → Standard Change → Template proposal), named for example 3AM remediation, and have it approved like any other template.

2. Add the connectorCopy Anchor Link

In the console: Settings → Add a connection → ServiceNow. Or in connectors.json:

{ "name": "servicenow", "kind": "servicenow",
  "settings": { "url": "https://bank.service-now.com",
                "client_id": "f1c2…",
                "client_secret": { "file": "/etc/3am/secrets/servicenow-client-secret" },
                "assignment_group": "Payments Ops",
                "standard_change_template": "3AM remediation",
                "knowledge_bases": ["IT Runbooks"],
                "watch_fulfilment": true,
                "catalog_items": ["Add user to AD group", "Unlock service account"] } }

catalog_items is an allow-list: 3AM never orders anything else.

Change recordsCopy Anchor Link

Whether 3AM opens a change record is policy, per service or per action:

"change_records": { "default": "none", "freeze": "warn" },
"rules": [ { "match": { "environment": "prod" }, "change": "standard" } ]

When a change is wanted, 3AM:

1. Opens the change before asking for approval. It describes the proven cause, the evidence, the action, the rehearsal, and how 3AM will verify the fix.
2. Runs ServiceNow's conflict check. A freeze or blackout conflict is shown in the approval request ("freeze": "warn"), or stops 3AM altogether ("freeze": "block").
3. Moves the change to Implement once approved, and only then acts. If the change can't move, nothing runs.
4. Closes the change as successful or unsuccessful, with the verification result. If the fix is rejected or blocked, the change is cancelled.

If policy wants a change and none can be opened, 3AM doesn't act.

3. VerifyCopy Anchor Link

ServiceNow Zurich, signed in with oauth; incidents readable

Test connection also reads every table the features you turned on need. A missing role shows up there, not at 3am during an incident.

What it detectsCopy Anchor Link

What it proves and fixesCopy Anchor Link

When ServiceNow is downCopy Anchor Link

ServiceNow is a hosted service, so 3AM can't restart it. Incidents still get handled:

• Diagnosis and fixes carry on. Approvals go through your other channels.
• Ticket updates are kept in order in 3AM's data directory: the new incident, each work note, the resolution. They're sent once ServiceNow answers again.
• Incidents opened during the outage show pending:<episode> until ServiceNow gives them a number. The incident record is then updated.
• A request ServiceNow rejects as wrong (for example a field that doesn't exist) isn't kept, because it would never succeed. It's in the audit log.

ActionsCopy Anchor Link

CertificationCopy Anchor Link

Live certification against ServiceNow Zurich is in progress. Until it finishes, this connector is tested against a model of ServiceNow's REST APIs, not a live instance.

TroubleshootingCopy Anchor Link