Pipeline & Serving Migration

A ported model still running inside an MXNet-era pipeline hasn't really left. Training jobs on SageMaker MXNet estimators, Gluon data-loading code, MXNet Model Server behind your endpoints, base images built on containers AWS stopped updating — the infrastructure around the models carries the same freeze as the framework itself.

Training pipelines

We replace Gluon training code with equivalents in the target framework — typically PyTorch with your existing experiment tooling — preserving the parts that encode real institutional knowledge: augmentation choices, sampling strategies, loss weighting, early-stopping logic. Where training runs on SageMaker, MXNet estimators and framework containers move to their maintained counterparts, and the job configuration (spot usage, checkpointing, distributed setup) is translated rather than rebuilt from scratch.

Serving infrastructure

Inference is where MXNet's freeze bites operationally: the serving stack pins CUDA and driver versions your platform team can no longer standardize on. We migrate:

• MXNet Model Server / multi-model endpoints to TorchServe, ONNX Runtime serving, or Triton — chosen against your latency, throughput, and hardware profile, not by default;
• batch scoring jobs to the target runtime with the same input/output contracts, so downstream consumers don't change;
• base images and build pipelines to maintained foundations that your security tooling can actually keep patched.

Parallel-run validation

Nothing cuts over on faith. The new pipeline runs alongside the old one on the same live or replayed traffic, and we compare:

• outputs, at the tolerances established during model porting;
• latency and throughput, against the envelope the old system actually delivered — not its spec sheet;
• failure behavior: timeouts, malformed inputs, retry semantics.

Cutover happens when the comparison is clean over a window you're comfortable with, with the old path kept warm for rollback until you decide it isn't needed. Decommissioning the MXNet stack is the last step, not a side effect.

Where this fits

Serving migration usually follows an estate assessment and runs alongside model porting — the parity harness from porting becomes the comparison layer of the parallel run. Like all our work it is hourly time and materials, scoped through an initial conversation. Get in touch.