Ask an Azure team whether Databricks or Data Factory should run a stage of their AI pipeline, and the honest answer is usually “whichever we already know.” That is the wrong basis for the decision, and it is where GPU money quietly leaks. The two services are not interchangeable orchestration options, even though they are often framed that way in procurement conversations. Azure Data Factory moves and orchestrates data. Azure Databricks runs compute — including the GPU clusters that train models and serve inference. Treating them as substitutable options and defaulting to team familiarity is exactly how you end up with expensive accelerators sitting idle while a serial data-copy step finishes. The decision that actually matters is not “which tool.” It is “where does the GPU spend live, and how long do GPUs sit idle waiting on data that another tool should have moved?” You cannot answer the tool question until you have answered that one. What Azure Databricks and Data Factory each actually do Start with the division of labour, because the naming obscures it. Azure Data Factory (ADF) is an orchestration and data-integration service. It schedules pipelines, copies data between stores, triggers external activities, and coordinates the sequence of a workflow. Its Copy activity and mapping data flows move bytes from A to B and apply lightweight transformations. ADF does not run your PyTorch training loop and it does not host a GPU. When people say ADF “processes” data, they usually mean it orchestrates a step that runs somewhere else. Azure Databricks is a compute platform built on Apache Spark, with cluster types that include GPU-backed instances (NC, ND, and NV series VMs carrying NVIDIA accelerators). This is where the CUDA kernels run — where a distributed training job, a batch inference sweep, or a feature-engineering job that genuinely needs parallel compute lives. Databricks can also orchestrate through Workflows, which is part of why the boundary blurs in practice. The blur is the problem. Because both can trigger steps, teams reason about them as if they occupy the same slot. They do not. One is a conductor; the other is the orchestra section that happens to be expensive to keep warm. Which pipeline stages belong in Data Factory versus Databricks when GPU compute is involved? Here is the split we reach for when architecting an Azure AI pipeline. The evidence class throughout this table is observed-pattern — drawn from infrastructure engagements, not a published benchmark suite. Pipeline stage Tool Why Ingesting raw data from source systems (databases, APIs, blob) Data Factory Pure I/O and scheduling; no GPU needed Landing and staging into a lakehouse / Delta table Data Factory Movement and format conversion, not compute Triggering the overall workflow on a schedule or event Data Factory (or Databricks Workflows) Orchestration is ADF’s core competency Heavy feature engineering over large tables Databricks (CPU cluster) Distributed Spark compute, rarely GPU-bound Model training / fine-tuning Databricks (GPU cluster) This is where the accelerators earn their cost Batch or streaming inference at scale Databricks (GPU cluster) Sustained GPU compute, autoscaled to demand Writing results back to serving stores Data Factory Movement again, not compute The rule underneath the table: if a stage is not saturating a GPU, it should not be holding one open. Data movement is I/O-bound and often serial; GPU compute is throughput-bound and parallel. Mixing them in the same billed cluster is how idle capacity accumulates. How the split affects GPU utilisation and cost per useful FLOP The metric that should drive this decision is cost per useful FLOP, not cost per purchased GPU-hour. Those two numbers diverge exactly in proportion to how long your GPUs sit idle. This is the same lens we apply when comparing cloud providers on cost per useful FLOP — the provider and the tool split are two facets of one question. Consider a common failure shape. A Databricks GPU cluster is spun up at the start of a nightly pipeline. Data Factory — or worse, a Databricks notebook doing the copy inline — then spends a large fraction of the wall-clock time pulling data from source systems and staging it. The GPUs are billed the entire time. They compute during a fraction of that window and idle through the rest. In configurations we have profiled, the ingestion-and-staging portion of a batch AI pipeline can consume a substantial share of end-to-end wall-clock time before the first FLOP of useful compute runs (observed-pattern; the exact fraction depends heavily on source-system latency and data volume). Every minute of that share, if it happens on a warm GPU cluster, is GPU-hours billed for waiting. Separating the concerns fixes the shape. Data Factory does the slow serial movement on cheap orchestration infrastructure. The GPU cluster in Databricks comes up only when compute-ready data is staged, runs at high utilisation, and scales down. The purchased GPU-hours converge toward useful GPU-hours, and cost per useful FLOP drops without buying a single additional accelerator. How do I stop GPU clusters sitting idle while Data Factory moves data? Three levers, in the order we usually apply them: Decouple the trigger from the cluster lifetime. Let Data Factory complete data movement first, then have it trigger the Databricks job. The GPU cluster boots after the data is staged, not before. This alone removes the largest idle window. Right-size Databricks autoscaling instead of over-provisioning. Autoscaling matches cluster size to the compute actually queued. A fixed cluster sized for peak sits partly idle the rest of the time. Profile before you tune. You cannot right-size a workload you have not measured. Utilisation-per-workload data tells you which stages are GPU-bound and which are masquerading as compute while really waiting on I/O. Our note on ML observability tools for GPU cost covers how that telemetry turns into cloud decisions. When is Databricks autoscaling worth it versus fixed GPU clusters? Autoscaling is not free lint-free money — it has warm-up latency and can thrash under bursty load. The decision is conditional. Autoscaling tends to win when the compute demand is variable: batch jobs with uneven sizes, inference traffic that follows a daily curve, or exploratory training where you cannot predict cluster size in advance. The idle savings outweigh the boot-latency cost. A fixed cluster tends to win when the workload is steady and latency-sensitive — a continuously running streaming inference service where the boot delay of scaling up would breach a latency budget, or a training job that runs at a constant, known size where autoscaling adds churn without saving anything. Some teams pin a small fixed baseline and autoscale on top of it, which is a reasonable hybrid when the floor of demand is well understood. The honest answer is that this depends on the demand curve, and the demand curve is something you measure, not assume. If nobody on the team can produce a utilisation timeline for the workload, the autoscaling-versus-fixed debate is premature. How do I profile an Azure AI pipeline to find where GPU capacity is wasted? The diagnostic is mechanical once you decide to look. Instrument the pipeline so you can answer, for every stage: is a GPU cluster warm during this stage, and is that GPU actually doing floating-point work? Pull GPU utilisation and memory-throughput metrics per cluster (Databricks exposes Ganglia / cluster metrics; NVIDIA’s DCGM gives finer-grained SM occupancy). Overlay them on the pipeline stage timeline from Data Factory’s run history. Flag any interval where a GPU cluster is warm but utilisation is low — that is the wasted capacity. Cross-check against data-movement durations: long ADF Copy activities running while a GPU cluster is warm are the smoking gun. The overlap of “GPU warm” and “GPU idle” is the number you are hunting. It is the operational counterpart to the utilisation reasoning we develop in the three pillars of observability applied to GPU utilisation — metrics, logs, and traces stop being abstract once they are pinned to a billed accelerator. Where the waste hides is not always the obvious stage. Sometimes it is the query layer feeding the GPU job; our piece on where GPU compute costs hide in the query layer walks through that adjacent leak. Total cost of ownership: orchestrating versus running compute The TCO difference between the two tools is not primarily a per-hour rate comparison — it is a difference in what you pay while waiting. Data Factory’s cost model is dominated by activity runs and data-movement units; it is cheap to keep an ADF pipeline waiting on a slow source. Databricks GPU compute is billed by the cluster-second, and idle GPU-seconds cost the same as busy ones. That asymmetry is the whole argument. Put the waiting on the cheap-to-wait tool and the computing on the expensive-to-idle tool. When teams invert that — waiting on Databricks GPUs, computing nowhere useful — the bill reflects capacity that was never converted into work. The fix is architectural, not a discount negotiation. For teams running Databricks on Kubernetes-backed infrastructure, the same idle-capacity logic extends to node pools; our note on private-cloud Kubernetes for GPU AI workloads covers how scheduling interacts with GPU cost there. And the broader framing — utilisation as the real cost driver — lives on our [GPU engineering practice page](GPU engineering). FAQ How should you think about azure databricks vs data factory in practice? Azure Data Factory orchestrates and moves data between stores; Azure Databricks runs compute, including GPU clusters for training and inference. In practice, the choice is not which tool to standardise on but which pipeline stage belongs where — data movement to Data Factory, GPU-bound compute to Databricks — so that expensive accelerators are not billed while a slow serial copy runs. Which pipeline stages belong in Data Factory versus Databricks when GPU compute is involved? Ingestion, staging, format conversion, and writing results back are movement stages that belong in Data Factory. Model training, fine-tuning, and scaled inference are compute stages that belong on Databricks GPU clusters. The underlying rule: a stage that is not saturating a GPU should not be holding one open. How does the Databricks-vs-Data-Factory split affect GPU utilisation and cost per useful FLOP? The relevant metric is cost per useful FLOP, not cost per purchased GPU-hour, and the two diverge in proportion to GPU idle time. Routing data movement through Data Factory and letting a Databricks GPU cluster boot only after data is staged converges purchased GPU-hours toward useful ones, cutting cost without buying more hardware. How do I stop GPU clusters sitting idle while Data Factory moves data? Decouple the trigger from the cluster lifetime so the GPU cluster boots after Data Factory finishes staging, right-size Databricks autoscaling instead of provisioning for peak, and profile utilisation per workload before tuning. The largest idle window usually disappears once the GPU cluster stops waiting for data that a cheaper tool should be moving. When is Databricks autoscaling worth it versus fixed GPU clusters for AI workloads? Autoscaling wins for variable demand — uneven batch sizes, daily-curve inference, exploratory training — where idle savings outweigh boot latency. Fixed clusters win for steady, latency-sensitive workloads where scale-up delay would breach a budget. The decision depends on a measured demand curve, not an assumption. How do I profile an Azure AI pipeline to find where GPU capacity is wasted? Pull GPU utilisation and memory-throughput metrics per Databricks cluster, overlay them on Data Factory’s stage timeline, and flag intervals where a GPU cluster is warm but utilisation is low. Long data-movement activities running while a GPU cluster is warm are the clearest sign of wasted capacity. What are the total-cost-of-ownership differences between orchestrating with Data Factory and running compute in Databricks? Data Factory is billed by activity runs and data-movement units, so it is cheap to keep waiting on a slow source; Databricks GPU compute is billed by the cluster-second, where idle seconds cost the same as busy ones. Putting the waiting on the cheap-to-wait tool and the computing on the expensive-to-idle tool is the structural way to lower TCO — an architecture change, not a discount. Before committing to a tool split, one question is worth answering with data rather than intuition: across a full pipeline run, how many of your billed GPU-seconds were spent computing versus waiting? A GPU Performance Audit measures utilisation per workload and quantifies exactly which stages are converting purchased capacity into useful FLOPs — and which are burning it waiting on data movement that never needed a GPU in the first place.
