Ask a bank’s analytics lead what machine learning they run and you get a list: credit scoring, churn propensity, liquidity and demand forecasting, risk revaluation, some flavour of fraud screening. The list is accurate and almost useless as a starting point, because it tells you nothing about which of those workloads deserves an engineering cycle first. Treating the list as a set of equally weighted candidates is the mistake. Most banking analytics workloads are limited by data quality, feature availability, or governance sign-off — not by compute. A minority are genuinely compute-bound, and those few are usually the ones making everything else on the shared cluster late.

That distinction is the whole point of this piece. Not how any single model is built, but how a portfolio of them should be triaged.

What a bank analytics team builds in-house versus buys

The build/buy split is remarkably consistent across institutions, and it matters here because a bought workload is rarely a candidate for a profiling engagement — you do not own the kernels.

Workload	Typical posture	Why
Credit scoring / PD models	Built in-house	Regulatory ownership, model risk management, bank-specific portfolio segmentation
Churn and propensity	Built in-house, sometimes on a vendor feature store	Cheap to train, tightly coupled to CRM data
Demand and liquidity forecasting	Built in-house	Treasury-specific scenario definitions; no vendor knows the balance sheet
Market/counterparty risk revaluation	Mixed — engine bought, calibration built	Pricing libraries are commoditised; the scenario grid is not
Fraud and AML screening	Mixed — vendor rules plus in-house models	Vendor supplies the rails, the bank supplies the discriminating signal
Document and KYC extraction	Mostly bought	Commodity OCR and extraction; little differentiation in building it

The pattern worth noticing: the workloads a bank builds itself are also the ones whose compute profile the bank controls, and therefore the only ones where profiling can change anything. In our experience, half of an initial scoping conversation is spent establishing which side of that line each workload actually sits on, because the org chart rarely makes it obvious.

How do you tell whether a workload is compute-bound or data-bound?

This is the question that reorders the list. A workload is data-bound when the next unit of improvement comes from better features, longer history, cleaner labels, or a faster upstream feed. It is compute-bound when the model and features are settled and the constraint is wall-clock time against a fixed deadline.

A few reliable tells, in rough order of diagnostic value:

1. Does the run have a hard deadline? Nightly batch that must clear before the ledger opens, or an intraday revaluation cycle. No deadline means no compute constraint — just a slow job nobody is waiting on.
2. Has the model spec been stable for two or more cycles? If the team is still changing the specification, any tuning work will be thrown away.
3. What does the team do when the run is late? Thinning the scenario grid, reducing segmentation, or dropping a sub-portfolio are all admissions that compute is the binding constraint. Waiting an extra hour is not.
4. Where does wall-clock time actually go? Feature computation, data movement, and I/O frequently dominate the matrix maths that everyone assumes is the cost centre. This is measurable in an afternoon with standard tooling — PyTorch profiler traces, nsys timelines, or even coarse stage timings from the orchestration layer.

Item four is the one teams skip. A workload cannot be called compute-bound until someone has looked at where its time goes. In practice, the answer is often that a CUDA-accelerated scoring step finishes in minutes while the Spark job assembling its features takes hours — a data-plumbing problem wearing a GPU problem’s clothes.

The divergence point is the cluster, not the workload

Here is where banking analytics differs from the single-workload optimisation story. A bank rarely runs one model on one machine. It runs scoring, forecasting, risk, and a rotating set of research jobs on one shared GPU cluster, usually under a scheduler that was configured when the estate was half its current size.

Under contention, the workload that appears broken is frequently not the workload causing the problem. A monthly PD re-estimation that grabs the whole cluster for six hours will make every nightly scoring job look marginal, and the scoring team — who raised the ticket — will be the ones asking for optimisation. Optimising their kernels would be a wasted cycle.

So the portfolio-level questions are:

• Which workload holds the most GPU-hours per week, not per run?
• Which one has the longest queue-wait tail behind it?
• Which one’s schedule is inflexible, and which could simply move?
• Is the estate short on capacity, or short on scheduling discipline?

That last one is worth dwelling on. We have seen clusters where reordering the batch calendar recovered more headroom than any kernel change would have. Rescheduling is free; engineering is not. Check it before committing anyone.

A practical triage

The triage below is the deliverable. It is not a maturity model and it produces one output: which workload gets looked at first.

Step 1 — Drop everything you do not own. Bought engines and vendor screening rails come off the list immediately.

Step 2 — Drop everything without a deadline. If nothing downstream waits on the run, it is a cost question, not a performance question, and it belongs in a capacity review instead.

Step 3 — Rank the remainder by GPU-hours consumed per week. Not by how loudly the owning team is complaining. This alone reorders most lists.

Step 4 — For the top two or three, measure the split. Feature assembly and data movement versus actual kernel execution. If more than roughly half the wall-clock is upstream of the GPU, the fix is a data-engineering fix and no amount of kernel work will move it.

Step 5 — Check scheduling before engineering. If moving a job to a different window clears the deadline, do that and re-run the triage in a month.

Step 6 — Whatever survives is the first profiling engagement. One workload, profiled properly, before anything is optimised or purchased.

The workload that survives all six steps is usually not the one that started the conversation. That reordering is the value — the first engineering cycle goes to the job actually starving the cluster rather than the job whose owner asked first.

Where the batch is already overrunning its window and the question is whether to add hardware, that is exactly what a GPU performance audit is for: establishing whether the binding constraint is cluster capacity or kernel-level inefficiency, before capex is committed to either answer. The profile-first, algorithmic-gains-before-micro-tuning discipline is the same one we apply outside finance; banking analytics is simply a case where the shared-cluster contention makes the profiling step non-optional. For the broader picture of where these workloads sit inside a bank’s compute estate, see our coverage of GPU-accelerated finance workloads and where their cost concentrates.

Once a single workload is selected, the depth belongs to that workload’s own analysis — how a probability-of-default model behaves when it is re-estimated across a full scenario grid is a different article, and we cover PD calculation pipelines separately.

What we do, and what we do not

Being plain about this saves everyone a call. On a banking engagement, TechnoLynx profiles and optimises the calculation pipeline: kernel-level work, memory and data-movement patterns, scheduling and cluster utilisation, and the honest capacity answer that follows. We do not build credit models, we do not own model risk management sign-off, and we do not advise on which specification a regulator will accept. Those are the bank’s, and they should stay there.

The boundary matters because it defines what the triage is for. It ranks workloads by how much a compute intervention would help — not by how important they are to the bank. A scoring model that decides who gets a mortgage is far more consequential than a nightly liquidity forecast, and may still be the wrong place to spend the first engineering cycle.

Which leaves an open question most estates cannot currently answer: over a full month, how many GPU-hours did each of your analytics workloads actually consume? If that number is not on hand, the triage has to start there.

Frequently Asked Questions

What does a bank analytics team typically build in-house versus buy? Credit scoring, propensity models, and treasury forecasting are almost always built in-house, because they depend on bank-specific segmentation and carry regulatory ownership. Pricing engines, document extraction, and the rails of fraud and AML screening are usually bought, with the bank supplying calibration or the discriminating signal on top. The practical consequence is that only in-house workloads are candidates for profiling work — you cannot optimise kernels you do not own.

How do you tell whether an analytics workload is compute-bound or data-bound? Check four things: whether the run has a hard deadline, whether the model specification has been stable for at least two cycles, what the team does when the run is late, and where wall-clock time actually goes. Teams that respond to lateness by thinning scenario grids or dropping sub-portfolios are compute-bound; teams that simply wait are not. A workload should not be labelled compute-bound before someone has traced its stage timings.

With several workloads on one GPU cluster, how do you find the one starving the others? Rank by GPU-hours consumed per week rather than per run, then look at queue-wait tails and schedule flexibility. The workload generating the complaint is often downstream of the real consumer — a monthly re-estimation that occupies the cluster for hours makes every nightly job look marginal. Test whether rescheduling clears the deadline before assuming an engineering fix is needed.

What does TechnoLynx do on a banking engagement, and what does it not do? We profile and optimise the calculation pipeline — kernels, memory and data movement, cluster scheduling and utilisation — and establish whether a capacity shortfall is real before hardware is added. We do not build credit or risk models, do not take model risk management ownership, and do not advise on regulatory acceptability of a specification. Those stay with the bank.

Why most bank ML projects never reach production

Regulatory approval cycles and model governance committees kill more experimental models than poor accuracy ever will. If Bank Machine Learning Analytics is on your roadmap, the next step is to map it onto your own constraints rather than copy a reference architecture.