Why Data Science Projects Collapse Under Their Own Weight

Most machine learning projects start as rapid research scripts. A single train.py evolves over six months: someone adds command-line arguments, someone else pastes a custom loss function, an analyst adds manual data filtering logic, and another developer tacks on a quick Flask endpoint at the bottom of the file.

When the project must be onboarded to CI/CD or handed to another engineer, it fails. Testing individual components becomes impossible because importing the model also triggers dataset downloads and GPU allocations.

The Canonical Production ML Directory Structure

ml-project-root/
├── .github/workflows/          # CI/CD test and linting pipelines
│   ├── lint.yml
│   └── test.yml
├── configs/                    # Declarative configuration files (YAML)
│   ├── base.yaml
│   └── train_retina.yaml
├── src/                        # Core Python package
│   └── retina_ai/
│       ├── data/               # Data ingestion, loading & transforms
│       ├── models/             # Pure network architectures (No training logic)
│       ├── training/           # Training loops, loss functions & optimizers
│       └── serving/            # Production inference microservice
├── tests/                      # Automated test suite
└── pyproject.toml              # Modern Python dependency specifications

Core Architectural Rules

  1. Models Have Zero Side Effects: Importing anything from src.retina_ai.models must never trigger disk reads, GPU memory allocations, or network requests. A model definition is pure mathematical architecture.
  2. Declarative Configs over Hardcoded Constants: Never hardcode learning rates, batch sizes, or hidden layer dimensions in python code. Maintain them in configs/*.yaml files loaded via Pydantic or Hydra.
  3. Contract Tests on Input Schemas: Write explicit pytest tests asserting that boundary values behave predictably through your validation layers.