Worker health

Use both the RestoreTrust worker detail page and Kubernetes status. The console shows whether a worker session is live and able to accept work; Kubernetes shows why its Pod or restore Jobs are unhealthy.

Console status

An Online worker is not necessarily ready for new Jobs. It can remain connected while temporarily draining, resynchronizing Kubernetes state, or protecting itself from a configuration or local buffering problem.

Capacity fields

A healthy idle worker normally shows:

• Status: Online
• Accepting jobs: Yes
• configured and advertised capacity equal
• effective capacity greater than zero
• a recent Last seen value

Effective capacity limits concurrent work; currently running Jobs consume slots but do not change the configured value.

The Agent image field shows the digest reported at the last session open. Update available means the digest differs from the image offered for new installations or could not be read; Update required means the agent cannot receive checks with the current execution protocol. An absent update status can mean the worker has never connected or no comparison image is configured. See Upgrade a worker before changing the Deployment.

Kubernetes probes

The generated Deployment has:

• a liveness probe that checks whether the agent process is alive;
• a readiness probe that checks Kubernetes observation state and a recently confirmed RestoreTrust session.

A failed readiness probe removes the Pod from Ready but does not restart it. A failed liveness probe can restart the container.

kubectl -n <AGENT_NAMESPACE> get deployment restoretrust-k8s-agentkubectl -n <AGENT_NAMESPACE> get pods -l app.kubernetes.io/name=restoretrust-k8s-agent -o widekubectl -n <AGENT_NAMESPACE> describe pod <AGENT_POD>

Keep replicas: 1 and confirm that no second Pod uses the same identity.

Agent logs

The worker detail page's Logs tab reads the current agent container log on demand. It is available only while the worker has a live session and is not a historical log archive. You can request the last 10, 50, 100, or 500 lines and download the result.

From the cluster:

kubectl -n <AGENT_NAMESPACE> logs deployment/restoretrust-k8s-agent --tail=100kubectl -n <AGENT_NAMESPACE> logs <AGENT_POD> --previous --tail=100

Use --previous after a container restart. Your cluster logging platform is the source for longer retention.

Restore Job health

During a check, inspect the resources in the job namespace:

kubectl -n <JOB_NAMESPACE> get jobs,pods,pvc,networkpolicykubectl -n <JOB_NAMESPACE> describe pod <RESTORE_POD>

Normal behavior includes:

• one Job and Pod for the active check;
• one scratch PVC owned by the Pod;
• the namespace egress floor and one execution-specific egress policy;
• cleanup after RestoreTrust accepts the terminal evidence and approves cleanup for that execution.

If the agent disconnects, running Kubernetes Jobs continue. The worker stops receiving new checks, reconnects, and reconciles existing Jobs when the agent returns. Do not delete an in-flight Job merely because the console temporarily shows Offline.

Reconnection and capacity recovery

A check's terminal status and its worker slot have separate lifecycles. A slot can remain reserved while the agent finishes stopping containers, delivers terminal evidence, and receives cleanup approval. Job and Pod finalizers retain the evidence until that handoff completes; this also keeps their scratch PVCs alive.

If a workload never started, the agent can confirm the absence of both its Job and Pods so RestoreTrust can release the reservation and retry. Repeated unsuccessful starts end as Infrastructure error after the configured start-attempt limit, which is 10 by default. A retry does not create an unlimited sequence of fresh attempts.

For a finalized execution whose previously observed Job and Pods have disappeared, the agent can report that absence so its reservation is released. This does not release a still-running check. Do not manually remove evidence finalizers to recover capacity; inspect the retained resources and agent connection, and escalate a persistent reservation with the check and worker IDs.

What to collect before escalation

Collect:

• worker name, project, status, last seen, and capacity fields;
• Kubernetes version and CNI name;
• agent Deployment and Pod status;
• recent agent logs, including --previous after a restart;
• affected Job and Pod descriptions;
• StorageClass name and PVC status;
• relevant NetworkPolicies and firewall decision;
• check ID and timestamp from RestoreTrust.

Redact Secret values, identity JSON, bearer tokens, backup paths when sensitive, and database credentials.