Data Manipulation and Software Literacy

Track 1: cuDF vs cudf.pandas decision making

Exam scenarios often test whether you can choose the right API path for both speed and compatibility.

• - cudf.pandas is the fastest way to accelerate existing pandas code with minimal refactor.
• - cuDF direct API usually gives tighter control and higher peak performance when your workflow is fully GPU-compatible.
• - cudf.pandas can fall back to CPU pandas for unsupported paths, and each fallback can add host-device transfer overhead.

Drill: Take one pandas notebook, run with cudf.pandas, profile fallback-heavy steps, then rewrite only hot steps in cuDF.

Track 2: Dask-cuDF architecture and scaling model

The domain is heavily NVIDIA-flavored; multi-GPU and distributed patterns are core exam scope.

• - Dask-cuDF extends Dask DataFrame with cuDF partitions; multi-GPU requires a distributed cluster.
• - Use Dask-CUDA and LocalCUDACluster for GPU pinning, diagnostics, and memory controls.
• - Prefer the portable dask.dataframe API with backend set to cudf, and convert only when needed with to_backend.

Drill: Deploy a 2-GPU LocalCUDACluster, run groupby and join workloads, and capture dashboard screenshots for bottleneck analysis.

Track 3: Partitioning, shuffle, and execution semantics

Most performance regressions come from partition shape and expensive all-to-all operations.

• - Typical partition-size target is between 1/32 and 1/8 of single-GPU memory.
• - Shuffle-heavy jobs often do best closer to 1/32-1/16; larger partitions can increase OOM risk.
• - Avoid calling compute on large collections; persist keeps data distributed and avoids client-side concatenation.

Drill: Benchmark one pipeline with three partition sizes and compare runtime, spill behavior, and worker memory headroom.

Track 4: Memory management and spill strategy

Memory pressure is a top exam and production concern in GPU data pipelines.

• - Initialize an RMM pool per worker (for example, rmm_pool_size near 0.8-0.9) to reduce allocation overhead.
• - Enable cuDF spill for ETL-style workflows; evaluate JIT-unspill for mixed DataFrame and array workloads.
• - Minimize repeated CPU-GPU backend switching, because movement overhead can erase acceleration gains.

Drill: Run a memory-stressed join with and without RMM pool and spill enabled, then compare spill counts and total runtime.

Track 5: I/O path optimization (Parquet and JSON)

Ingest is often the bottleneck before transformation or modeling begins.

• - Parquet is usually preferred for column projection and predicate pushdown in distributed workflows.
• - For JSON Lines, use lines=True and use byte-range or multi-source patterns depending on file size profile.
• - String-heavy Parquet workloads can benefit from encoding and compression tuning (for example ZSTD and selective delta encodings).

Drill: Create one Parquet write/read benchmark and one JSON Lines benchmark, then document throughput and CPU utilization.

Track 6: Software literacy for reproducible pipelines

The exam tests practical stack literacy, not only DataFrame syntax.

• - Conda environments should be reproducible from environment.yml and solved with pinned core dependencies.
• - Docker images package runtime, dependencies, and entrypoints for consistent dev-test-prod behavior.
• - Spark RAPIDS and Databricks integrations can accelerate SQL/DataFrame paths with fallback to CPU when operators are unsupported.

Drill: Package one RAPIDS workflow in Docker and run it in a fresh environment to validate reproducibility.

Exam Decision Hierarchy

Prioritize decisions in this order: safety and hardware integrity, baseline consistency, controlled validation, then optimization.

• - If integrity checks fail, stop optimization and remediate first.
• - Compare against known-good baseline before changing multiple variables.
• - Document rationale for each decision to support incident replay.

Operational Evidence Standard

Treat every key action as evidence-producing: command, output, timestamp, and expected vs observed behavior.

• - Evidence should be reproducible by another engineer.
• - Use stable command templates for repeated environments.
• - Keep concise but complete validation artifacts for exam-style reasoning.

Decision model: cudf.pandas first, cuDF for hotspots

For exam scenarios, fastest migration path is usually `cudf.pandas` for compatibility, then selective direct cuDF optimization where fallback or memory pressure remains.

• - Start with least-disruptive acceleration path to establish baseline.
• - Profile unsupported operations and repeated fallback transitions.
• - Refactor only measured hotspots into native cuDF or dask-cuDF patterns.

Distributed execution discipline with Dask-cuDF

Most failures come from partitioning and execution semantics, not DataFrame syntax.

• - Choose partition sizes by workload type; shuffle-heavy jobs often need smaller partitions.
• - Prefer `persist()` for large distributed collections and delay `compute()` until final reduction.
• - Control worker memory with RMM pool and spill settings before scaling up.

I/O and data-type literacy for GPU pipelines

Parquet and JSON ingestion choices directly impact throughput and memory stability.

• - Use Parquet for projection/predicate pushdown in analytics-heavy ETL.
• - Use JSON Lines patterns (`lines=True`, byte-range for large files) to avoid parser bottlenecks.
• - Validate cuDF dtypes and null behavior before joins/groupby to prevent hidden casts and spill.

Scenario A: ETL pipeline is faster on CPU than GPU after migration

A pandas ETL job was switched to `cudf.pandas`, but runtime improved only slightly and sometimes regressed. You need to isolate root cause quickly.

[Raw Files] -> [cudf.pandas pipeline] -> [Dask scheduler/workers] -> [Parquet output]
                     |
          [Fallback + transfer hotspots]

python - <<'PY'
import time, pandas as pd, cudf.pandas
start=time.time()
# run existing ETL notebook entrypoint here
print('elapsed_s=', round(time.time()-start,2))
PY

elapsed_s= 214.37

Scenario B: Multi-GPU groupby job fails with intermittent OOM

A Dask-cuDF job crashes during shuffle-heavy groupby steps on larger batches. You need a stable tuning sequence.

[Scheduler]
   |
[GPU Worker 0] [GPU Worker 1] ... [GPU Worker N]
   |                |
 [RMM Pool + Spill Control + Shuffle]

Cluster bring-up and memory controls

Start a stable local multi-GPU Dask-cuDF session with explicit memory behavior.

python - <<'PY'
from dask_cuda import LocalCUDACluster
from dask.distributed import Client
cluster = LocalCUDACluster(rmm_pool_size='0.85', enable_cudf_spill=True)
client = Client(cluster)
print(client)
PY

<Client: 'tcp://127.0.0.1:8786' processes=4 threads=4>

python - <<'PY'
import dask.dataframe as dd
import dask
with dask.config.set({'dataframe.backend':'cudf'}):
    ddf = dd.read_parquet('data/*.parquet')
    print(ddf.head())
PY

Returns first rows using cuDF backend where supported.

• - Use this baseline before workload tuning so failures are not caused by implicit defaults.
• - Keep startup config in runbook for reproducible exam-style reasoning.

I/O optimization quick checks

Validate Parquet and JSON ingestion behavior before transformation tuning.

python - <<'PY'
import cudf, time
t=time.time()
gdf=cudf.read_parquet('data/*.parquet', columns=['id','ts','value'])
print('rows=', len(gdf), 'elapsed_s=', round(time.time()-t,2))
PY

rows= 18400213 elapsed_s= 9.84

python - <<'PY'
import cudf, time
t=time.time()
gdf=cudf.read_json('logs/*.jsonl', lines=True)
print('rows=', len(gdf), 'elapsed_s=', round(time.time()-t,2))
PY

rows= 9023310 elapsed_s= 14.21

• - Record both throughput and memory profile, not runtime alone.
• - Use consistent input snapshots when comparing formats.

Repeated backend switching erases acceleration gains

Dask-cuDF OOM during shuffle-intensive joins

Walkthrough A: Migrate pandas ETL to GPU with measured optimization

Deliver stable speedup while preserving result parity.

1. Run baseline pandas workflow and capture runtime plus output checksum.

   python - <<'PY'
   # run pandas ETL baseline
   print('baseline_elapsed_s= 420.7')
   print('checksum= 9df7...')
   PY

   Expected: You have baseline performance and parity marker.

2. Enable cudf.pandas and rerun unchanged workflow.

   python - <<'PY'
   import cudf.pandas
   # run ETL entrypoint
   print('gpu_elapsed_s= 260.4')
   PY

   Expected: Initial acceleration is measured and fallback hotspots are identified.

3. Refactor top hotspot into direct cuDF operation and compare again.

   Expected: Runtime improves further with parity preserved against checksum.

Walkthrough B: Stabilize multi-GPU shuffle workflow

Tune partition and memory settings to avoid OOM in join/groupby workload.

1. Start LocalCUDACluster with explicit memory settings.

   python - <<'PY'
   from dask_cuda import LocalCUDACluster
   from dask.distributed import Client
   cluster=LocalCUDACluster(rmm_pool_size='0.8', enable_cudf_spill=True)
   client=Client(cluster)
   print('cluster_ready')
   PY

   Expected: Cluster initializes with predictable memory behavior.

2. Run workload at three partition targets and record runtime/spill.

   Expected: You identify a stable partition range with no worker failures.

3. Publish final runbook defaults for this workload class.

   Expected: Team has a reusable partition and memory baseline.

Lab A: Zero-code-change acceleration baseline

Measure uplift from pandas to cudf.pandas and isolate fallback penalties.

• - Run baseline pandas job with fixed input size.
• - Enable cudf.pandas and rerun unchanged code.
• - List operations that still execute on CPU and estimate their share of total runtime.

python - <<'PY'
from dask_cuda import LocalCUDACluster
from dask.distributed import Client
cluster = LocalCUDACluster(rmm_pool_size='0.85', enable_cudf_spill=True)
client = Client(cluster)
print(client)
PY

<Client: 'tcp://127.0.0.1:8786' processes=4 threads=4>

Lab B: Dask-cuDF scaling and partition tuning

Find stable partition strategy for join and groupby workload.

• - Deploy LocalCUDACluster with at least two workers.
• - Test partition targets at 1/32, 1/16, and 1/8 memory fractions.
• - Record runtime, spill behavior, and OOM incidents.

python - <<'PY'
import dask.dataframe as dd
import dask
with dask.config.set({'dataframe.backend':'cudf'}):
    ddf = dd.read_parquet('data/*.parquet')
    print(ddf.head())
PY

Returns first rows using cuDF backend where supported.

Lab C: Memory stability under pressure

Compare default allocator vs RMM pool and spill controls.

• - Run memory-heavy workflow with default settings.
• - Enable rmm_pool_size and cudf spill, then rerun.
• - Document runtime delta and stability improvements.

python - <<'PY'
import cudf, time
t=time.time()
gdf=cudf.read_parquet('data/*.parquet', columns=['id','ts','value'])
print('rows=', len(gdf), 'elapsed_s=', round(time.time()-t,2))
PY

rows= 18400213 elapsed_s= 9.84

Lab D: I/O throughput optimization

Improve ingest throughput for Parquet and JSON workloads.

• - Benchmark read_parquet with representative file sizes.
• - Benchmark read_json for JSON Lines with mixed file-size distribution.
• - Write recommendations for file format and encoding choices.

python - <<'PY'
import cudf, time
t=time.time()
gdf=cudf.read_json('logs/*.jsonl', lines=True)
print('rows=', len(gdf), 'elapsed_s=', round(time.time()-t,2))
PY

rows= 9023310 elapsed_s= 14.21