Data Analysis

Track 1: Exploratory data analysis workflow

EDA questions test whether you can quickly profile data quality, distribution shape, and data issues before modeling.

• - Start with schema, missingness, cardinality, and duplicate checks before deeper analysis.
• - Use univariate and bivariate analysis to surface skew, outliers, and class imbalance.
• - In GPU workflows, use cuDF and cuxfilter to keep early exploration interactive on larger datasets.

Drill: Take one medium-size tabular dataset and produce a one-page EDA summary with missing-data, distribution, and relationship sections.

Track 2: Summary statistics and robust interpretation

The exam checks whether you can choose statistics that match data shape and noise profile.

• - Mean and standard deviation are sensitive to outliers; median and IQR are often safer for heavy-tailed data.
• - Percentiles communicate spread and tails better than single central tendency values.
• - Segment-level summaries can expose patterns that global averages hide.

Drill: For the same feature, compare mean/std vs median/IQR and explain which pair you trust and why.

Track 3: Probability distributions and sampling intuition

Distribution assumptions drive the validity of tests, confidence statements, and many modeling decisions.

• - Different features can follow normal, skewed, count, or multimodal patterns; verify instead of assuming.
• - Sampling variation decreases with sample size, but bias in collection remains a separate risk.
• - Distinguish population parameters from sample estimates and state uncertainty explicitly.

Drill: Assess two features, state plausible distributions, and justify with visual and summary evidence.

Track 4: Hypothesis testing decisions

You may be asked to pick appropriate tests and correctly interpret p-values and decision thresholds.

• - State null and alternative hypotheses clearly before running any test.
• - Choose tests based on data type, independence assumptions, and variance behavior (for example Welch vs equal-variance t-test).
• - A p-value is not effect size; always pair significance with practical magnitude.

Drill: Run one two-sample test and one contingency-table test, then interpret statistical and practical significance separately.

Track 5: Correlation and covariance interpretation

Correlation questions are common and frequently mixed with causality traps.

• - Covariance scale depends on feature units; correlation normalizes covariance for comparability.
• - Pearson captures linear relationships; low Pearson does not prove no relationship.
• - Correlation does not establish causation and can be confounded by hidden variables.

Drill: Compute covariance and correlation matrices and list one misleading interpretation you intentionally avoid.

Track 6: Evaluation metrics by problem type

Metric selection errors cause wrong model conclusions even when pipelines run correctly.

• - For imbalanced classification, rely on precision/recall/F1 and PR-AUC rather than accuracy alone.
• - ROC-AUC measures ranking quality, while thresholded metrics reflect deployed operating points.
• - For regression, pair absolute and squared-error metrics to capture both average and large-error behavior.

Drill: Given one imbalanced classification output, choose a metric set and justify tradeoffs for a production-like threshold.

Exam Decision Hierarchy

Prioritize decisions in this order: safety and hardware integrity, baseline consistency, controlled validation, then optimization.

• - If integrity checks fail, stop optimization and remediate first.
• - Compare against known-good baseline before changing multiple variables.
• - Document rationale for each decision to support incident replay.

Operational Evidence Standard

Treat every key action as evidence-producing: command, output, timestamp, and expected vs observed behavior.

• - Evidence should be reproducible by another engineer.
• - Use stable command templates for repeated environments.
• - Keep concise but complete validation artifacts for exam-style reasoning.

EDA decision ladder for exam cases

Use a fixed order when reading any dataset: integrity checks, distribution checks, relationship checks, then metric-aligned conclusions.

• - Start with schema, nulls, duplicates, and impossible ranges before charts.
• - Compare global summary with segment-level summary to avoid hidden subgroup effects.
• - Document assumptions before selecting tests or model metrics.

Hypothesis testing selection flow

Select tests using data type and assumptions first, then interpret p-value and effect size separately.

• - Numeric vs numeric two-group comparisons often start with t-test variants or nonparametric alternatives.
• - Categorical association checks commonly use chi-square style tests.
• - Significance decision without effect size or confidence context is incomplete.

Metric selection under class imbalance

In imbalanced classification, threshold-aware metrics and error-cost framing are more reliable than accuracy.

• - Precision/recall/F1 and PR-AUC usually carry more signal than raw accuracy.
• - ROC-AUC is ranking quality; operating threshold still needs business-cost reasoning.
• - Always state threshold, confusion matrix, and expected false-positive/false-negative impact.

Scenario A: High validation accuracy but poor production alert quality

A binary anomaly model shows strong validation accuracy, but production users report many missed anomalies. You need to re-evaluate analysis and metric strategy.

[Raw Events] -> [EDA + Label Audit] -> [Train/Validation Split]
                                  |
                         [Threshold + Metrics]
                                  |
                         [Production Alerts]

python - <<'PY'
from sklearn.metrics import confusion_matrix
import numpy as np
# y_true, y_score loaded earlier
for t in [0.2, 0.4, 0.6, 0.8]:
    y_pred = (y_score >= t).astype(int)
    print(t, confusion_matrix(y_true, y_pred).ravel())
PY

0.2 [tn fp fn tp]
0.4 [tn fp fn tp]
...

Scenario B: Correlation appears strong but deployment decision is risky

A feature shows high correlation with target in one slice. You must decide whether it is stable enough for production decisions.

[Feature Table] -> [Correlation Matrix] -> [Segment Stability Check]
                                       -> [Confounder Review]
                                       -> [Decision Memo]

EDA integrity quick pass

Run a fast quality and distribution baseline before selecting any statistical test.

python - <<'PY'
import pandas as pd
df = pd.read_parquet('dataset.parquet')
print(df.shape)
print(df.isna().mean().sort_values(ascending=False).head(10))
print('duplicates=', df.duplicated().sum())
PY

(1200000, 42)
feature_a    0.182
feature_b    0.104
duplicates= 3912

python - <<'PY'
import pandas as pd
df = pd.read_parquet('dataset.parquet')
print(df['amount'].describe(percentiles=[0.5,0.9,0.99]))
PY

count ...
50% 42.1
90% 381.3
99% 3210.7

• - Run this before hypothesis testing so assumptions are evidence-based.
• - Save the output snapshot for exam-style justification.

Classification metric decision runbook

Compare threshold-dependent and threshold-independent metrics for imbalanced labels.

python - <<'PY'
from sklearn.metrics import average_precision_score, roc_auc_score
print('pr_auc=', average_precision_score(y_true, y_score))
print('roc_auc=', roc_auc_score(y_true, y_score))
PY

pr_auc= 0.41
roc_auc= 0.89

python - <<'PY'
from sklearn.metrics import precision_recall_fscore_support
for t in [0.3,0.5,0.7]:
    y_pred=(y_score>=t).astype(int)
    p,r,f,_=precision_recall_fscore_support(y_true,y_pred,average='binary')
    print(t, round(p,3), round(r,3), round(f,3))
PY

0.3 0.22 0.81 0.346
0.5 0.37 0.58 0.452
0.7 0.61 0.34 0.437

• - Use threshold table when the exam asks for deployment recommendation.
• - Tie final threshold to error cost asymmetry.

Inflated validation score from leakage-prone split logic

Accuracy looks strong while minority-class recall is unacceptable

Walkthrough A: Hypothesis testing with assumption checks

Select and defend the right statistical test for two-group comparison.

1. Profile both groups for shape and variance behavior.

   python - <<'PY'
   import pandas as pd
   from scipy import stats
   df=pd.read_parquet('dataset.parquet')
   a=df[df.grp==0]['metric']
   b=df[df.grp==1]['metric']
   print(stats.describe(a))
   print(stats.describe(b))
   PY

   Expected: You identify skew/outlier behavior and decide if parametric assumptions are acceptable.

2. Run chosen test and record p-value plus effect size.

   python - <<'PY'
   from scipy import stats
   stat,p=stats.ttest_ind(a,b,equal_var=False)
   print('p=',p)
   print('mean_delta=', b.mean()-a.mean())
   PY

   Expected: Decision report includes both significance and practical magnitude.

3. Write final decision statement with assumptions and caveats.

   Expected: Statement clearly separates statistical evidence from operational recommendation.

Walkthrough B: Threshold calibration for imbalanced classification

Choose a deployment threshold using confusion-matrix tradeoffs.

1. Sweep thresholds and capture precision, recall, and F1.

   python - <<'PY'
   from sklearn.metrics import precision_recall_fscore_support
   for t in [x/10 for x in range(1,10)]:
     y_pred=(y_score>=t).astype(int)
     p,r,f,_=precision_recall_fscore_support(y_true,y_pred,average='binary')
     print(t,p,r,f)
   PY

   Expected: You obtain a threshold table that reveals tradeoff knees.

2. Pick candidate threshold and verify confusion matrix impact.

   python - <<'PY'
   from sklearn.metrics import confusion_matrix
   t=0.4
   y_pred=(y_score>=t).astype(int)
   print(confusion_matrix(y_true,y_pred))
   PY

   Expected: Chosen threshold aligns with acceptable miss rate and alert load.

3. Publish threshold recommendation with rationale.

   Expected: Recommendation includes metric evidence plus cost-based reasoning.

Lab A: Fast EDA triage

Produce a repeatable EDA checklist for tabular datasets under time pressure.

• - Profile nulls, duplicates, type consistency, and high-cardinality columns.
• - Create compact visuals for distribution shape and outlier risk.
• - Document top three data-quality risks before any modeling.

python - <<'PY'
import pandas as pd
df = pd.read_parquet('dataset.parquet')
print(df.shape)
print(df.isna().mean().sort_values(ascending=False).head(10))
print('duplicates=', df.duplicated().sum())
PY

(1200000, 42)
feature_a    0.182
feature_b    0.104
duplicates= 3912

Lab B: Test-selection decision lab

Practice selecting and interpreting hypothesis tests correctly.

• - Write hypotheses in plain language before computing p-values.
• - Run at least one two-sample mean comparison and one categorical independence test.
• - Report both significance decision and practical significance.

python - <<'PY'
import pandas as pd
df = pd.read_parquet('dataset.parquet')
print(df['amount'].describe(percentiles=[0.5,0.9,0.99]))
PY

count ...
50% 42.1
90% 381.3
99% 3210.7

Lab C: Correlation discipline lab

Avoid common interpretation errors in covariance/correlation analysis.

• - Compute covariance and correlation matrices for selected features.
• - Identify at least one spurious or confounded relationship candidate.
• - Propose one validation step to test robustness of observed relationships.

python - <<'PY'
from sklearn.metrics import average_precision_score, roc_auc_score
print('pr_auc=', average_precision_score(y_true, y_score))
print('roc_auc=', roc_auc_score(y_true, y_score))
PY

pr_auc= 0.41
roc_auc= 0.89

Lab D: Metric strategy lab

Align evaluation metrics to business and deployment constraints.

• - Build confusion matrix and derive precision, recall, and F1 at multiple thresholds.
• - Compare ROC-AUC with PR-AUC for an imbalanced scenario.
• - Write a threshold recommendation tied to false-positive and false-negative costs.

python - <<'PY'
from sklearn.metrics import precision_recall_fscore_support
for t in [0.3,0.5,0.7]:
    y_pred=(y_score>=t).astype(int)
    p,r,f,_=precision_recall_fscore_support(y_true,y_pred,average='binary')
    print(t, round(p,3), round(r,3), round(f,3))
PY

0.3 0.22 0.81 0.346
0.5 0.37 0.58 0.452
0.7 0.61 0.34 0.437