AKS Night Incident Runbook for Care-Facility IT Leads

At 11:40 p.m., the overnight manager calls: the care facility’s booking screen opens, but saving a care record fails. An Azure email exists, yet the handoff omits the target resource, severity, and whether the alert is still firing. Worse, the draft escalation to the IT supplier contains a resident’s name. The operations and IT lead must stop both the investigation delay and the secondary spread of personal data.

This article is for the person accountable for a care facility’s booking and care application on Azure, including its overnight escalation design. It is not a guide for frontline care staff. The goal is one runbook that keeps business continuity decisions separate from AKS investigation while preserving a shared incident timeline.

Azure Kubernetes Service (AKS) is Azure’s managed Kubernetes service for deploying and managing containerized applications. Kubernetes schedules, restarts, and scales groups of containers; Azure takes on the AKS control-plane overhead. Azure does not, however, decide the facility’s operational impact, investigate every workload issue, or own the overnight contact order.

Key takeaways

• Never turn Azure severity directly into the facility’s incident level; assess booking, records, handover, and safety separately.
• Sanitize the Common Alert Schema JSON before Claude Code sees it, then validate required routing fields and direct personal data locally.
• Separate Claude Code tasks, Azure/IT operator decisions, and care-facility incident decisions.
• Without Azure credentials, call this a handoff-contract test, not an AKS or alert-delivery test.
• Measure ROI from comparable incidents using timestamps, rejections, and labor minutes instead of invented improvement rates.

Make the overnight incident one handoff workflow

Phone calls, Azure notifications, affected screens, and contact lists produce conflicting stories when they live in separate notes. Start with business facts: “booking search works,” “care-record save fails,” “paper recording is active,” and “next update at 00:10.” Put Pod, Node, and deployment evidence after those facts.

Azure Monitor is Microsoft’s unified observability service for collecting, analyzing, and acting on metrics, logs, traces, and events from cloud and hybrid environments. Azure Monitor Common Alert Schema is the standardized JSON structure for consuming different Azure Monitor alert notifications. data.essentials carries shared metadata such as severity, while data.alertContext varies by signal and supports investigation.

Microsoft Learn documents severity as Sev0 through Sev4, signalType as Metric, Log, or Activity Log, and monitorCondition as Fired or Resolved. alertTargetIDs is the list of Azure Resource Manager IDs targeted by the alert. These fields can route a handoff; they cannot decide resident safety, paper fallback, or the facility’s recovery declaration.

The primary references checked on July 23, 2026 are What is AKS?, Azure Monitor overview, Common alert schema, Monitor AKS, and Service Health alerts. For nearby internal material, use the Kubernetes deployment guide and the Claude Code security checklist.

flowchart TD
  A["Receive Azure Monitor alert"] --> B["Save sanitized JSON fixture"]
  B --> C{"Local validator passes?"}
  C -- "No" --> D["Remove personal data or restore required fields"]
  D --> B
  C -- "Yes" --> E["Claude Code formats the handoff"]
  E --> F["Azure/IT operator decides technical investigation"]
  E --> G["Care-facility lead decides business continuity"]
  F --> H["Merge evidence at the next update time"]
  G --> H
  H --> I{"Both closure conditions met?"}
  I -- "No" --> H
  I -- "Yes" --> J["Record the post-incident review"]

Service Health is another evidence source for notifications matching configured subscriptions, services, regions, and event types. An empty Service Health view does not prove the application is healthy. Likewise, an AKS alert becoming Resolved does not finish paper-record reconciliation or booking checks.

What Claude Code handles and what people decide

The handoff table should name inputs, outputs, prohibitions, and approvers. Treating Claude Code as the recovery owner mixes unsupported diagnosis and change commands into the same conversation. Here it only organizes approved evidence.

Claude Code tasks

• Extract severity, signalType, monitorCondition, alertTargetIDs, and timestamps from a fixture that already passed validation.
• Separate confirmed facts, unknowns, and pending human decisions; flag a missing next-update time.
• Draft separate facility and supplier notes under one incident ID.
• Suggest observation commands without running them, and state the evidence and permissions each would require.

Azure/IT operator decisions

• Choose the order for inspecting the alert rule, target, monitoring configuration, recent changes, and AKS telemetry.
• Decide whether to collect more evidence in Azure Portal, Container insights, Log Analytics, or an authorized terminal.
• Approve or reject restart, scaling, rollback, and technical escalation.
• Decide whether Resolved is sufficient technical recovery evidence or whether monitoring continues.

Care-facility incident decisions

• Pause booking, switch care records to paper, or prioritize a specific business workflow.
• Direct safety checks, handover, later data entry, and any communication with families or external parties.
• Approve what information can be shared, the next internal update, and the business-recovery declaration.
• Assign reconciliation of missing records or duplicate bookings after technical recovery.

This is not individualized legal advice. Personal-data classification, retention, disclosure, and incident reporting must follow the facility’s policies, contracts, and accountable functions.

Three use cases

Use case 1: Hand off a booking API failure overnight

• Input: sanitized Common Alert Schema fixture, booking-screen check, incident time, and next-update time.
• Output: one contact sheet separating Azure target IDs from facility impact, unknowns, and the supplier request.
• Human review: the Azure/IT operator approves investigation; the facility lead approves the booking fallback and duplicate checks.

Sev1 alone does not justify closing every booking channel. A read failure and a duplicate-write risk require different facility actions. Claude Code separates observed facts from pending approvals rather than making that decision.

Use case 2: Escalate delayed care-record saves

• Input: passing fixture, affected feature, sanitized count, time window, and whether a release was recent.
• Output: technical memo with the routing fields, privacy-safe reproduction conditions, and missing evidence.
• Human review: the Azure/IT operator decides on logs and rollback; the facility lead owns paper records and later reconciliation.

Do not send a screenshot or log containing a resident name to Claude Code. The validator blocks explicit identity keys, emails, and phone numbers, but it cannot recognize every unlabeled name, image, or encoded identifier. Keep visual review in the approval column.

Use case 3: Close business recovery after Resolved

• Input: fixture with monitorCondition: "Resolved", connectivity check, outstanding records, duplicate-booking check, and contact history.
• Output: separate technical and business closure checklists, remaining work, and a morning owner.
• Human review: the Azure/IT operator closes monitoring; the facility lead declares business recovery after records and bookings reconcile.

In Common Alert Schema, Resolved means the condition that fired the alert has cleared. It says nothing about paper records being entered or callbacks being completed. Two closure conditions prevent a technically recovered incident from hiding operational work.

Copy-paste runnable handoff validator

The following validator uses only Node.js standard libraries and reads the local JSON path passed on the command line. It does not sign in to Azure and does not call AKS, Azure Monitor, or kubectl. It tests only the documented Common Alert Schema shape and this runbook’s sanitization contract.

// validate-azure-alert-handoff.mjs
import { readFileSync } from "node:fs";

const REQUIRED = ["severity", "signalType", "monitorCondition", "alertTargetIDs"];
const ALLOWED = {
  severity: new Set(["Sev0", "Sev1", "Sev2", "Sev3", "Sev4"]),
  signalType: new Set(["Metric", "Log", "Activity Log"]),
  monitorCondition: new Set(["Fired", "Resolved"]),
};
const DIRECT_PERSONAL_DATA_KEYS = new Set([
  "residentname", "patientname", "carerecipientname", "serviceusername",
  "staffname", "employeename", "familyname", "guardianname",
  "phone", "phonenumber", "telephone", "mobile",
  "email", "emailaddress", "streetaddress", "postaladdress",
  "dateofbirth", "birthdate", "medicalrecordid", "carerecordid",
  "residentid", "patientid",
]);
const EMAIL = /\b[A-Z0-9._%+-]+@[A-Z0-9.-]+\.[A-Z]{2,}\b/i;
const PHONE = /(?:^|[^A-Za-z0-9])(?:\+\d{1,3}[ .-]?)?(?:\(\d{2,4}\)|\d{2,4})[ .-]\d{2,4}[ .-]\d{3,4}(?:$|[^A-Za-z0-9])/;
const LABELED_NAME = /(?:resident|patient|care recipient|staff|employee|family|guardian|利用者|入居者|患者|職員|家族)(?:\s+name|氏名|名)\s*[:=：]\s*\S+/iu;
const ARM_RESOURCE_ID_PATH = /^\$\.data\.essentials\.(?:alertId|alertRuleId|alertTargetIDs\[\d+\])$/;

function normalizeKey(key) {
  return key.replace(/[^a-z0-9]/gi, "").toLowerCase();
}

function findDirectPersonalData(value, path = "$", findings = []) {
  if (Array.isArray(value)) {
    value.forEach((item, index) => findDirectPersonalData(item, `${path}[${index}]`, findings));
    return findings;
  }
  if (value && typeof value === "object") {
    for (const [key, child] of Object.entries(value)) {
      const childPath = `${path}.${key}`;
      if (DIRECT_PERSONAL_DATA_KEYS.has(normalizeKey(key))) {
        findings.push(`${childPath} uses a prohibited direct-personal-data key`);
      }
      findDirectPersonalData(child, childPath, findings);
    }
    return findings;
  }
  if (typeof value === "string") {
    if (EMAIL.test(value)) findings.push(`${path} contains an email address`);
    if (!ARM_RESOURCE_ID_PATH.test(path) && PHONE.test(value)) {
      findings.push(`${path} contains a phone number`);
    }
    if (LABELED_NAME.test(value)) findings.push(`${path} contains a labeled person name`);
  }
  return findings;
}

export function validate(payload) {
  const errors = [];
  if (!payload || typeof payload !== "object" || Array.isArray(payload)) {
    throw new Error("payload must be a JSON object");
  }
  if (payload.schemaId !== "azureMonitorCommonAlertSchema") {
    errors.push('schemaId must be "azureMonitorCommonAlertSchema"');
  }

  const essentials = payload.data?.essentials;
  if (!essentials || typeof essentials !== "object" || Array.isArray(essentials)) {
    errors.push("data.essentials must be an object");
  } else {
    for (const field of REQUIRED) {
      if (!(field in essentials)) errors.push(`missing data.essentials.${field}`);
    }
    for (const [field, allowed] of Object.entries(ALLOWED)) {
      if (field in essentials && !allowed.has(essentials[field])) {
        errors.push(`data.essentials.${field} has an undocumented value`);
      }
    }
    if ("alertTargetIDs" in essentials) {
      if (!Array.isArray(essentials.alertTargetIDs) ||
          essentials.alertTargetIDs.length === 0 ||
          essentials.alertTargetIDs.some((id) => typeof id !== "string" || id.trim() === "")) {
        errors.push("data.essentials.alertTargetIDs must be a non-empty string array");
      }
    }
  }

  errors.push(...findDirectPersonalData(payload));
  if (errors.length > 0) throw new Error(errors.join("; "));
  return true;
}

function loadJson(filePath) {
  return JSON.parse(readFileSync(filePath, "utf8"));
}

function runSelfTest() {
  const base = loadJson("sanitized-alert.json");
  validate(base);
  const cases = [
    ["direct personal data", (p) => { p.data.customProperties.residentName = "Example Resident"; }],
    ["missing severity", (p) => { delete p.data.essentials.severity; }],
    ["missing signalType", (p) => { delete p.data.essentials.signalType; }],
    ["missing monitorCondition", (p) => { delete p.data.essentials.monitorCondition; }],
    ["missing alertTargetIDs", (p) => { delete p.data.essentials.alertTargetIDs; }],
  ];
  let rejected = 0;
  for (const [label, mutate] of cases) {
    const candidate = structuredClone(base);
    mutate(candidate);
    try {
      validate(candidate);
      console.error(`FAIL self-test: accepted ${label}`);
      process.exitCode = 1;
    } catch {
      console.log(`PASS self-test: rejected ${label}`);
      rejected += 1;
    }
  }
  if (!process.exitCode) console.log(`PASS self-test: ${rejected} rejection cases`);
}

if (process.argv[2] === "--self-test") {
  runSelfTest();
} else {
  const filePath = process.argv[2];
  if (!filePath) {
    console.error("Usage: node validate-azure-alert-handoff.mjs <fixture.json> | --self-test");
    process.exit(2);
  }
  try {
    validate(loadJson(filePath));
    console.log(`PASS fixture: ${filePath}`);
  } catch (error) {
    console.error(`FAIL fixture: ${error.message}`);
    process.exit(1);
  }
}

Save the complete fixture below as sanitized-alert.json beside the validator. IDs, timestamps, and rule names are test data, not observed incident results. customProperties is a documented Common Alert Schema extension point; here it makes the sanitization assertion visible to a reviewer.

{
  "schemaId": "azureMonitorCommonAlertSchema",
  "data": {
    "essentials": {
      "alertId": "/subscriptions/00000000-0000-4000-8000-000000000000/providers/Microsoft.AlertsManagement/alerts/11111111-1111-4111-8111-111111111111",
      "alertRule": "care-api-availability",
      "alertRuleId": "/subscriptions/00000000-0000-4000-8000-000000000000/resourceGroups/rg-care-prod/providers/microsoft.insights/metricAlerts/care-api-availability",
      "severity": "Sev1",
      "signalType": "Metric",
      "monitorCondition": "Fired",
      "monitoringService": "Platform",
      "alertTargetIDs": [
        "/subscriptions/00000000-0000-4000-8000-000000000000/resourceGroups/rg-care-prod/providers/Microsoft.ContainerService/managedClusters/aks-care-prod"
      ],
      "configurationItems": [
        "aks-care-prod"
      ],
      "originAlertId": "sanitized-night-incident-001",
      "firedDateTime": "2026-07-23T14:40:00Z",
      "description": "The production booking and care-record API crossed its approved availability threshold.",
      "essentialsVersion": "1.0",
      "alertContextVersion": "1.0"
    },
    "alertContext": {
      "properties": null
    },
    "customProperties": {
      "environment": "production",
      "service": "booking-care-api",
      "sanitized": "true",
      "runbook": "night-incident-v1"
    }
  }
}

Run these exact commands with Node.js 17 or later. The first reads the valid fixture. The second proves rejection of direct personal data and each of the four required-field failures.

node validate-azure-alert-handoff.mjs sanitized-alert.json
node validate-azure-alert-handoff.mjs --self-test

This is the exact output produced from the published code and fixture on July 23, 2026, not an estimated result.

PASS fixture: sanitized-alert.json
PASS self-test: rejected direct personal data
PASS self-test: rejected missing severity
PASS self-test: rejected missing signalType
PASS self-test: rejected missing monitorCondition
PASS self-test: rejected missing alertTargetIDs
PASS self-test: 5 rejection cases

A pass does not prove Azure configuration, real alert delivery, AKS health, or a recovery procedure. A real connection test needs an environment with Common Alert Schema enabled on the Action Group, a receiver, Azure credentials, network access, permissions, and a test alert. No credentials were used here, so only the documented handoff contract was tested.

Pitfalls: do not call a fixed-object pass an operations test

The first pitfall is validating an object embedded in the script and claiming implementation. The cause is that file reading, malformed JSON, missing fields, and rejection exits never run. Fix it by reading a local fixture and running both success and failure commands during every runbook review.

The second is calling validator success an AKS connectivity test. The cause is using one test name for an input contract and cloud access. Record “handoff contract passed” and keep alert delivery, monitoring, and AKS evidence in separate fields.

The third is mapping Sev0 through Sev4 directly to care-safety levels. Azure severity and facility impact are different scales. Approve Azure severity, booking impact, record impact, resident safety, and paper fallback separately.

The fourth is automatically broadcasting Resolved as incident closure. The alert condition can clear while missing records remain. Keep the incident ID open until both technical and facility closure conditions are complete.

The fifth is treating the validator as a personal-data guarantee. Key and text patterns cannot detect every image, opaque identifier, or unlabeled name. Minimize inputs, review free text and attachments, and follow the facility’s information-governance rules.

Measure ROI across comparable incidents

Build the ROI table from incident IDs, timestamps, and labor minutes. Compare incidents with similar severity bands, overnight windows, and impact scopes. When event volume is low, report individual timelines rather than asserting a rate.

Metric	Recording method	Question answered
IT handoff time	First notification to IT receipt of a passing fixture	How much waiting comes from contract defects?
Impact confirmation time	Notification to approved booking, record, and handover scope	How much back-and-forth separates technology and operations?
Fixture rejection rate	Failed validator submissions divided by all submissions	Are required fields and sanitization becoming routine?
First-pass escalation acceptance	Escalations started without clarification divided by all escalations	Is the handoff complete enough to investigate?
Manual labor time	Minutes by role for transcription, checks, and re-entry	What overnight and morning work remains?

To express value in money, multiply saved minutes by facility-approved loaded labor rates, add approved avoided rework, and subtract runbook design, review, exercise, and maintenance costs. Divide the net benefit by investment cost. If baseline, period, and exclusions are unavailable, report time and rejection trends rather than labeling them ROI.

FAQ

Q. Does this validator connect to Azure Monitor or AKS?

A. No. It only reads local JSON. It cannot support a claim about Azure delivery, credentials, or cluster state.

Q. Why require these four essentials fields?

A. Overnight routing needs severity, signal type, firing or resolved state, and target resources. The validator does not turn every Common Alert Schema field into a private mandatory schema.

Q. Does a pass prove there is no personal data?

A. No. It rejects explicit keys, email addresses, phone numbers, and labeled names. Images and unlabeled free text still require human review.

Q. Should Claude Code run Azure change commands?

A. Not in this runbook. The Azure/IT operator reviews permissions, impact, rollback, and approval, then uses the existing change procedure.

Q. Does an empty Service Health view prove AKS is healthy?

A. No. Service Health alerts reflect configured criteria. Combine them with application, workload, network, and monitoring evidence.

Consultation CTA: incident runbook review

On the ClaudeCodeLab training and consultation page, request an “incident runbook review” for an overnight booking or care-record workflow. The deliverable is a commented current runbook, Common Alert Schema handoff contract, three-role decision table, test procedure, and unresolved-risk list. Bring only a sanitized alert example, contact table, and one recent timeline, with credentials and personal data removed.

What we actually tested

On July 23, 2026, we extracted the published validate-azure-alert-handoff.mjs and sanitized-alert.json into a temporary folder and ran the two shown Node.js commands. The valid fixture exited with code 0. The self-test rejected direct personal data and missing severity, signalType, monitorCondition, and alertTargetIDs, for five rejection cases. Repository checks also cover frontmatter, internal links, official URLs, code fences, and the shared slug across ten locales. We used no Azure credentials, so real AKS access, Action Group configuration, and alert delivery remain untested. Start the runbook review by sanitizing one facility alert example and running these two commands.