An assessment that precedes a rewrite is not a design document and not a speedup promise. It is four artefacts: a profiled baseline on representative inputs, bottleneck attribution that separates compute from memory bandwidth from I/O from serialisation, per-target gain estimates bounded by the share of wall-clock the port can actually touch, and an engineering-cost estimate built against the real code rather than a generic rewrite multiplier. Everything else in the report is commentary on those four.
The reason to be strict about the deliverable list is that assessments get compressed. Once a team has privately decided a rewrite is likely, the assessment shrinks into a two-day opinion: a senior engineer reads the codebase, names Python as the bottleneck, and the port gets scoped in the same meeting. That produces a number nobody can re-check three months later when the roadmap shifts or a new VP asks why the quarter went to a migration.
What does a porting and performance assessment actually deliver?
The four deliverables, and what each one has to contain to be usable:
| Deliverable | What it contains | What it is used for | Failure if missing |
|---|---|---|---|
| Profiled baseline | Latency percentiles and throughput on representative production inputs, with the build configuration and hardware recorded | The denominator for every later claim | Gains get measured against a number nobody can reproduce |
| Bottleneck attribution | Wall-clock split across compute, memory bandwidth, host↔device transfer, I/O wait, serialisation, interpreter and framework overhead | Decides whether a port is even the right lever | The port targets a stage that was never the constraint |
| Per-target gain estimate | An estimated range per candidate runtime, explicitly capped by the addressable share of runtime | Sets a calibrated expectation before headcount is committed | Amdahl’s ceiling gets discovered after the rewrite ships |
| Engineering-cost estimate | Person-weeks derived from the actual module boundaries, test coverage, dependency graph and deployment surface | Turns a technical preference into a capital allocation decision | Cost is a multiplier someone read in a blog post |
Each row is traceable to the client’s own code and workload. That traceability is the deliverable — not the recommendation at the end of it.
The baseline has to be representative, not convenient
A synthetic benchmark is easy to run and almost always flatters the port. Batch sizes are uniform, inputs are cached, the request mix has no long tail, and the build was never checked. We treat a baseline as invalid until three things are pinned down: input distribution drawn from production traffic (including the tail cases that dominate p99), a recorded build configuration, and a warm-vs-cold distinction so cache effects are not silently averaged away.
The build configuration matters more than teams expect. A C++ baseline compiled without optimisation flags makes any subsequent port look artificially attractive; the same is true of a Python path where a framework is running in eager mode with no graph compilation applied. Our colleagues covering GCC optimisation flags and what they mean for a port work through that specific trap in more detail.
Profiling itself is boring engineering: py-spy or cProfile for interpreter-level attribution, PyTorch’s profiler or Nsight Systems for the accelerator timeline, perf for the native side, and request-level tracing to catch upstream synchronous calls that never appear in a process-local profile.
Why attribution ordering decides the port
This is the divergence point. Bottleneck attribution is not a list of hot functions — it is a wall-clock budget with named owners, and the ordering of that budget determines whether a language port is a lever or a distraction.
If most of the measured wall-clock sits in data loading, in a network round trip, or inside a library that is already native — a language port moves a small fraction of the total, and the assessment should say so before anyone commits headcount. A pipeline where 70% of the time is a cuBLAS or cuDNN call has almost nothing for a C++ rewrite to reclaim; the interpreter overhead the team wanted to eliminate was never the constraint. That is a genuine finding, not a failed assessment.
The buckets we separate, and why each one routes differently:
- Compute on the accelerator — already native in most stacks. A port rarely helps; kernel fusion, precision changes or a different serving runtime might.
- Memory bandwidth — layout and access-pattern work, not a language problem. Rewriting the same access pattern in C++ reproduces the same stall.
- Host↔device transfer and small-batch dispatch — often the real winner for a port, because batching and pinned-memory handling are structural, not linguistic.
- I/O wait and serialisation — frequently the largest single bucket in inference paths, and usually addressable without changing language at all.
- Interpreter and framework overhead — the bucket a port genuinely removes. Its size sets the honest ceiling on the whole exercise.
Only after that split do target runtimes get ranked. C++, Rust and WebAssembly each answer a different deployment question, and the profile — not preference — narrows the list. The port-target decision framework consumes exactly these four deliverables as its inputs, which is why supplying them properly comes first.
Bounding the gain estimate honestly
No speedup can be promised before profiling, and none should be promised after it either. What an assessment can produce is a range with an explicit cap: the addressable share of wall-clock multiplied by a plausible per-bucket improvement factor, stated as a band rather than a point. If interpreter and dispatch overhead account for 25% of p95 latency, then 25% is the ceiling on what any language port removes — and the realistic band sits below it, because the ported code still has to do the work the old code did.
We state the measurement basis alongside every band, because a figure derived from a profiled bucket and a figure derived from a comparable prior migration are not the same class of evidence and should never sit unlabelled in the same table. In our experience the second class is the one that gets quoted out of context.
Costing against the real code
A generic “rewrites cost 3× the original” multiplier tells a budget holder nothing they can defend. The cost estimate is built bottom-up from what is actually in the repository: how many modules cross the proposed language boundary, how much of the surface is covered by tests that can validate the port’s outputs, which third-party dependencies have no equivalent in the target runtime, what the build and CI change looks like, and who will maintain the result after the consultants leave.
That last item is a cost line, not a footnote. A CUDA or Rust component in a team with no CUDA or Rust maintainers carries an ongoing tax that belongs in the estimate.
The output is person-weeks with named assumptions, which combines with the gain band into a payback window. A budget holder can check the arithmetic. That is the point of the exercise.
When the honest recommendation is “don’t”
An assessment with only one acceptable outcome is not an assessment. The refusal path is explicit in ours: if attribution shows the addressable share is small, or the payback window exceeds the horizon the workload will survive on its current architecture, the report recommends defer or skip and names the cheaper levers it found instead — batching, a serving-runtime change, an I/O fix, a compiler flag.
Avoided rewrites are usually the larger saving, because the cost avoided is the full person-week estimate plus the roadmap quarter it would have consumed. This is the deliverable set the wider R&D consulting engagement model is built around, and the reason the profiling work sits before the commitment rather than after it. Where the workload is accelerator-bound, the same evidence chain feeds our GPU engineering work.
Two access conditions make this possible at all: read access to the code that will be ported, and a representative sample of production workload. Without the second, the baseline is a guess wearing a number.
The question worth carrying into the commissioning conversation is not “how much faster will it be” — it is “which of these four artefacts will I have in hand when I have to defend this decision to someone who was not in the room?”
Frequently Asked Questions
What does a porting and performance assessment actually deliver before any rewrite begins? Looked at closely, Porting Performance Assessment Delivers is this. Porting Performance Assessment Delivers has one honest answer. Four traceable artefacts: a profiled baseline on representative inputs, a wall-clock attribution across compute, memory bandwidth, I/O and serialisation, per-target gain estimates capped by the addressable share of runtime, and an engineering-cost estimate in person-weeks built from the real code. Together they produce a commit-or-defer decision a budget holder can check rather than a speedup promise.
How is the profiling baseline captured so it represents production workloads rather than a synthetic benchmark? Inputs are drawn from production traffic including the tail cases that dominate p99, the build configuration and hardware are recorded, and warm and cold paths are reported separately so caching is not averaged away. A baseline measured on uniform synthetic batches or an unoptimised build systematically flatters the port.
How does bottleneck attribution separate compute, memory-bandwidth, I/O and serialisation cost — and why does that ordering decide the port? Attribution is a wall-clock budget with named owners, produced from interpreter-level profiles, accelerator timelines and request-level tracing. The ordering decides the port because only interpreter, dispatch and transfer overhead are things a language change removes — if native library compute, bandwidth stalls or I/O wait dominate, the port moves a small fraction of the total.
How are per-target gain estimates bounded when no speedup can be promised before profiling? The addressable share of wall-clock sets a hard ceiling, and the estimate is expressed as a band below that ceiling rather than a point figure. Each band carries its measurement basis, so a number derived from a profiled bucket is never confused with one inferred from a comparable prior migration.
How is the engineering-cost estimate built against real code rather than a generic rewrite multiplier? It is assembled bottom-up from the module boundaries the port would cross, the test coverage available to validate ported outputs, dependencies with no target-runtime equivalent, the build and CI change, and the in-house maintenance capability for the target language. The result is person-weeks with named assumptions that can be audited line by line.
What does the assessment look like when the honest recommendation is to defer or skip the rewrite? It states the addressable share and the payback window that failed the threshold, then names the cheaper levers found during profiling — batching, a serving-runtime change, an I/O fix, compiler flags. The saving is the full person-week estimate plus the roadmap quarter the migration would have consumed.
Who needs to be in the room, and what access to code and workloads does the assessment require? The engineers who own the serving path, someone who can authorise the roadmap decision, and whoever will maintain the ported component afterwards. Practically it needs read access to the code proposed for porting and a representative sample of production workload — without the latter, the baseline is a guess wearing a number.
Evidence Over Estimates
Benchmarking the existing codebase on target hardware eliminates assumptions and replaces vendor promises with reproducible proof. The teams that do tend to ship the boring, correct version first.
