ecc-mle-workflow

Move hyperparameters and paths into typed configs or dataclasses, pin dependencies, set random seeds, and record dataset version, code SHA, and metrics with each artifact. Save preprocessing logic with the model artifact and make every pipeline step idempotent so retries do not corrupt results.

Check every feature against prediction-time availability and remove anything joined using future information. Define label timestamps, feature freshness SLAs, and point-in-time join rules in an explicit data contract before splitting train, validation, and test sets.

Choose metrics from failure costs: precision when false positives dominate, recall when missed positives dominate, and AUC when ranking quality matters. Add guardrails for latency, calibration, and slice performance, and always compare against the baseline and current production model.

No. The workflow explicitly avoids assuming supervised labels, online serving, feature stores, PyTorch, GPUs, or A/B tests. It scales from a simple data contract and eval script up to full MLOps machinery depending on the system being reviewed.

Common causes include random splits leaking future data, thresholds tuned repeatedly on the test set, and important slices regressing while aggregate metrics rise. Check slice metrics, train-serve transformation equivalence, and feature freshness drift before trusting offline gains.

Do not add complexity until error analysis shows the baseline fails for a reason that more signal or capacity can plausibly fix. Cluster mistakes first and trace each cluster to better labels, features, thresholds, or product fallbacks.