Workload Management

Track 1: Requirement decomposition

Workload management starts by translating business/use-case goals into technical constraints.

• - Extract functional and non-functional requirements from use case.
• - Identify latency, throughput, and reliability targets.
• - Define success criteria before scheduling design.

Drill: Take one training and one inference use case and derive resource and routing requirements.

Track 2: Resource sizing and placement

Incorrect CPU/GPU/memory sizing leads to poor utilization or frequent job failures.

• - Estimate compute and memory demand by workload stage.
• - Map workload profile to scheduler placement strategy.
• - Validate allocation behavior under scale and contention.

Drill: Create a sizing table for baseline and peak workload windows.

Track 3: Workflow and route design

The exam expects you to determine and validate AI workflow and route behavior.

• - Model workflow stages and dependencies end-to-end.
• - Define routing path for data, model, and inference flow.
• - Validate workflow state transitions and failure handling.

Drill: Draw a workflow route from dataset ingestion to model serving and list failure checkpoints.

Track 4: Security requirements in workload path

Security constraints can alter placement, routing, and runtime decisions.

• - Map security requirements to runtime and network controls.
• - Validate workload execution under least-privilege assumptions.
• - Ensure secret and credential handling aligns with policy.

Drill: Add security controls to an existing workflow and identify new operational checks required.

Track 5: Conversion and workload validation

Model/dataset conversion and runtime validation are explicit objectives.

• - Verify conversion output compatibility with target runtime.
• - Validate workload status and health from scheduler to endpoint.
• - Use smoke and scale tests to qualify production readiness.

Drill: Build a conversion-and-validation checklist for one model serving workflow.

Exam Decision Hierarchy

Prioritize decisions in this order: safety and hardware integrity, baseline consistency, controlled validation, then optimization.

• - If integrity checks fail, stop optimization and remediate first.
• - Compare against known-good baseline before changing multiple variables.
• - Document rationale for each decision to support incident replay.

Operational Evidence Standard

Treat every key action as evidence-producing: command, output, timestamp, and expected vs observed behavior.

• - Evidence should be reproducible by another engineer.
• - Use stable command templates for repeated environments.
• - Keep concise but complete validation artifacts for exam-style reasoning.

Workload design as constraint solving

Workload management is an iterative constraint-solving problem balancing performance, capacity, and security.

• - Start from required outcomes, not platform defaults.
• - Re-evaluate constraints under scale and contention.
• - Make tradeoffs explicit and measurable.

Route-aware operations

Workflow routing decisions shape failure modes and observability requirements.

• - Track state transitions between workflow stages.
• - Validate fallback routes where possible.
• - Instrument route checkpoints for incident response.

Conversion is not deployment completion

Artifact conversion must be followed by runtime and output validation to be operationally meaningful.

• - Validate compatibility against target runtime versions.
• - Run representative payload tests, not only health probes.
• - Capture known-good outputs for regression detection.

Scenario: Inference workload unstable after scale increase

An inference service meets SLO at low QPS but violates latency targets after scale increase.

Client Requests
   |
Gateway -> Inference Route
   |
Scheduler Placement
   |
GPU Worker Pool

kubectl get pods -A -o wide && kubectl describe pod <pod-name>

Pod scheduling, limits, and events align with intended policy.

squeue && scontrol show job <jobid>

Queue and job details explain placement and resource behavior.

Scenario: Model conversion succeeds but runtime output invalid

Converted model deploys successfully, yet output schema or quality checks fail in production tests.

Model Source
    |
Conversion Pipeline
    |
Runtime Deployment
    |
Inference Validation

Workload status and placement runbook

Validate scheduler placement and runtime status against workload intent.

kubectl get pods -A -o wide

Pod placement and state are visible across namespaces.

kubectl describe pod <pod-name>

Event stream clarifies scheduling and runtime transitions.

squeue

Queue reflects active, pending, and priority-ordered jobs.

• - Capture outputs during both normal and peak windows.
• - Correlate placement with observed latency and throughput outcomes.

Conversion and route validation runbook

Validate artifact conversion outcomes and route behavior before production promotion.

cat conversion-report.json | head -n 40

Report includes expected model format, precision, and target runtime details.

curl -sS http://<service-endpoint>/health

Route health endpoint reports ready status.

curl -sS -X POST http://<service-endpoint>/v1/completions -H 'Content-Type: application/json' -d '{"prompt":"validate","max_tokens":8}'

Response schema and fields match expected contract.

• - Do not promote artifacts without runtime output validation.
• - Keep known-good validation payloads for regression checks.

Frequent OOM or eviction events in production workloads

Workflow stalls at intermediate stage

Converted model runs but returns inconsistent outputs

Walkthrough: Requirement-to-workload deployment path

Validate full workload management chain from requirement mapping to stable runtime status.

1. Map use-case requirements to resource spec.

   cat workload-spec.yaml

   Expected: Spec includes explicit CPU/GPU/memory and policy constraints.

2. Deploy workload and inspect placement.

   kubectl apply -f workload-spec.yaml && kubectl get pods -A -o wide

   Expected: Workload is placed according to policy with expected status.

3. Validate runtime behavior under test traffic.

   python3 run_load_test.py --qps 100 --duration 120

   Expected: Latency and error metrics remain within defined target band.

Walkthrough: Conversion and route validation

Validate converted artifact readiness and route correctness before production rollout.

1. Inspect conversion metadata.

   cat conversion-report.json | head -n 30

   Expected: Metadata aligns with runtime and precision requirements.

2. Check endpoint route health.

   curl -sS http://<service-endpoint>/health

   Expected: Endpoint reports ready state.

3. Run output contract validation.

   curl -sS -X POST http://<service-endpoint>/v1/completions -H 'Content-Type: application/json' -d '{"prompt":"contract","max_tokens":8}'

   Expected: Response schema and output quality checks pass.

Lab A: Requirement-to-scheduling translation

Translate use-case requirements into scheduler-ready resource specifications.

• - Capture workload SLO/SLA targets.
• - Map targets to resource requests and limits.
• - Validate placement behavior under normal load.

kubectl get pods -A -o wide

Pod placement and state are visible across namespaces.

Lab B: Workflow and route validation

Validate end-to-end route from data ingestion to inference output.

• - Execute workflow with status tracking per stage.
• - Confirm route policies and service dependencies.
• - Record failure-handling behavior for one injected fault.

kubectl describe pod <pod-name>

Event stream clarifies scheduling and runtime transitions.

Lab C: Security-aware workload execution

Verify that workload can run under required security constraints.

• - Apply role and secret constraints.
• - Run workload and validate allowed operations.
• - Confirm denied actions are blocked.

squeue

Queue reflects active, pending, and priority-ordered jobs.

Lab D: Conversion and runtime compatibility check

Validate converted model/dataset compatibility and runtime health.

• - Run conversion pipeline and capture artifact metadata.
• - Deploy converted artifact in target runtime.
• - Validate status, logs, and output schema.

cat conversion-report.json | head -n 40

Report includes expected model format, precision, and target runtime details.