Batch effects in a Cell Painting assay are technical variation from reagent lots, processing times, equipment, laboratories and well position, and they can bury the biology you ran the screen to see. The way to handle them has three stages. First, detect them with correlation heatmaps, PCA and UMAP coloured by plate, batch, row and column. Second, pick a correction method that fits how complicated your data is. Data from one lab with many replicates may need little more than baseline preprocessing, while Harmony and Seurat RPCA were the two methods that ranked in the top three in every scenario of the JUMP benchmark. Third, check the result with metrics that measure batch removal and metrics that measure preserved biology, side by side. One limit is worth knowing before you start: with more than a few hundred compounds and more than one microscope type, none of the benchmarked methods removed the batch effects adequately.
What the Cell Painting assay measures, and why technical noise hits it so hard
The Cell Painting protocol uses six fluorescent dyes, imaged in five channels, to reveal eight cellular components or organelles. Automated image analysis then measures roughly 1,500 morphological features per cell, covering size, shape, texture and intensity. In the JUMP dataset, each well is represented by a 4,762-dimensional CellProfiler feature vector.
A profiling assay aims to measure hundreds to thousands of readouts rather than one positive-versus-negative separation, which creates a challenge for optimisation. Within a single laboratory, the Arevalo benchmark notes that profiles can still drift because of unintended changes in lamp intensity, staining concentration or cell seeding and growth rate. The data is also hierarchical: each readout comes from a region in a well, on a plate, in an experimental batch, at a particular laboratory.
Batch, row and column: the three effects in Cell Painting data
The cpDistiller authors (Nature Communications, 2025) group the technical variation in Cell Painting data into “triple effects”: batch effects plus row effects and column effects, which together make up well-position effects. A standard 384-well plate has 16 rows and 24 columns, and every well is affected by both its row and its column.
Well-position effects behave differently from the batch effects familiar from single-cell RNA-seq. They follow a gradient: the further apart two rows or columns are, the larger the difference between them, and row and column effects interact, so they need to be corrected together. In the ORF profiles, generated at a single site, colouring a UMAP by batch showed no significant batch effect, but colouring it by row or column name produced a clear striping pattern that particular samples sitting in those positions could not explain.
This is particularly important when the plate layout confounds position with treatment. In the ORF experiments, every replicate of a given ORF sat in the same well position across plates, so position effects and perturbation effects cannot be cleanly separated. Our reading is that this is a design problem first and a correction problem second: no algorithm can reliably separate two signals that the layout has fused.
How do I tell whether my plates cluster by batch instead of biology?
Look for structure that follows metadata rather than treatment. A heatmap of sample-level correlations that shows blocks of high similarity lining up with specific plates or experimental dates, rather than with treatments, is the classic sign. In PCA, separation along a component that tracks plate or batch membership points the same way. UMAPs coloured by plate ID or well position can reveal “islands” of cells separated by technical variables. Batch effects show up as distinct clusters per batch, while row and column effects appear as colour gradients.
A quick diagnostic checklist, run before any correction:
- Correlation heatmap of well profiles, ordered by plate and date: are the blocks technical?
- PCA coloured by batch and plate: does any leading component split by batch?
- UMAP coloured separately by batch, row and column: clusters for batch, gradients or stripes for position.
- Positive control replicates (for example Target2 compounds) across batches: do identical compounds from different batches find each other?
- Plate layout: are replicates of a treatment always in the same well? If so, position and treatment are confounded.
Treat these plots as screening tools rather than proof. The Arevalo authors warn that dimensionality reduction can distort what you see, so clustering by biology suggests success and clustering by batch suggests failure, but neither settles it on its own. Running these checks early pays off because what you find decides the normalisation and correction strategy.
Normalisation comes before correction, and the order matters
The JUMP pipeline aggregates single-cell measurements into a per-sample mean profile, normalises each feature per plate by subtracting the median and dividing by the mean absolute deviation (MAD), then removes redundant and invariant features. The Arevalo benchmark used four preprocessing steps: low-variance feature removal, plate-wise MAD normalisation, a rank-based inverse normal transformation and correlation-based feature selection. In the Pycytominer implementation of that selection step, any feature pair correlated above 0.9 loses the feature with the highest total correlation to the others.
Typical analysis pipelines use Pycytominer for these steps. It is a set of functions for processing high-dimensional image-based profiling data, with no explicit link between them, which you can chain in Python or through pipeline frameworks such as the Profiling-recipe or CytoSnake. It normalises each plate using either all wells or only the negative control wells, with the RobustMAD method. All wells are appropriate when they are not expected to be enriched for a phenotype, and negative controls when there are enough of them. DMSO wells play that negative control role in JUMP, supporting the detection and correction of plate-to-plate variation.
In the cpDistiller study, batch effects across 12 ORF batches became less prominent after MAD normalisation, though the authors still corrected them further for reliable cross-batch comparison.
Sphering, also called whitening, is another transformation in the toolkit. Pycytominer implements it as a multivariate standardisation that uses negative controls. Sphering Cell Painting profiles was found to raise percent replicating from 24–30% to 83–84% for compounds at 10 µM, but these results are likely confounded by plate layout and have not been consistently high across studies. For self-supervised deep learning features, a review of image-based profiling reports that sphering followed by robust z-scoring against plate statistics is the most effective post-processing, and that z-scoring before sphering substantially degraded performance in most cases. That ordering finding comes from deep learning features, so we would test it rather than assume it carries over to CellProfiler features.
Which batch correction method fits my Cell Painting scenario?
A benchmark built for exactly this question is Arevalo et al. (Nature Communications, 2024), which benchmarked ten scRNA-seq batch correction methods on JUMP data across five scenarios of rising difficulty: several batches in one lab, several labs with the same microscope (with few and with many compounds), and several labs with different microscopes (with few and with many compounds). Nearest-neighbour methods such as Seurat, fastMNN and Scanorama scored best on batch-removal metrics, Harmony and Seurat did consistently well on biology metrics, and Harmony and Seurat RPCA reached a top-three mean rank in every scenario while remaining computationally efficient.
Difficulty changes the answer. For data from one laboratory with many replicates per compound, the uncorrected baseline already fixed most of the batch effect, with fastMNN and the Seurat methods only slightly better. As technical and biological variance grew, linear models such as ComBat and sphering fell short of aligning the data. ComBat assumes additive and multiplicative linear batch effects, which may oversimplify nonlinear technical variation, and sphering needs reliable negative controls in every batch. Methods that rely on overlapping biological subpopulations (MNN, fastMNN, Scanorama, Seurat) struggle when profiles lack shared states or when the whole phenotype shifts.
| Scenario | Reasonable starting point | What the evidence says |
|---|---|---|
| One lab, several batches, many replicates | Baseline preprocessing only; optionally fastMNN or Seurat | Baseline corrected most batch effects; fastMNN and Seurat slightly better |
| Several labs, same microscope | Harmony or Seurat RPCA | Performance dropped versus one lab, but rankings held; Harmony and nearest-neighbour methods (except MNN) led |
| Several labs, different microscopes | Harmony or Seurat RPCA; linear methods not recommended | ComBat and sphering fall short as variance grows |
| Many compounds and more than one microscope type | Harmony or Seurat RPCA, with modest expectations | No tested method adequately removed batch effects |
| Strong row and column gradients | Row/column-aware correction such as cpDistiller | Single-label methods left row and column stripes; cpDistiller outperformed five baselines on triple-effect correction |
| Very large, highly heterogeneous collections | Consider scalable methods such as BALANS | BALANS showed a clear advantage on the large, high-diversity JUMP-2 scenario and scaled to 5 million points in under an hour on synthetic data |
The hardest row of that table deserves plain wording. In the most difficult scenarios, with more than a few hundred compounds and more than one microscope type, none of the methods could adequately remove batch effects, and the best methods gave their largest improvements in the easiest scenarios. If your plan relies on merging a large compound screen across imaging platforms, budget for residual batch structure and design the validation to measure it.
Correcting well-position effects alongside batch
Harmony, Seurat v5 (run with its CCA method) and Scanorama work on low-dimensional representations, so they can remove batch, row and column effects one after another by passing each label in turn. To remove triple effects with these methods, the cpDistiller authors passed batch labels first, then row, then column. Order made a measurable difference: for Harmony and Scanorama, correcting rows before columns was significantly better than the reverse.
There is a trade-off in that iterative route. In the same study, Harmony was the most effective baseline at removing row and column effects but did poorly at preserving biological heterogeneity, and Scanorama and Seurat v5 tended to overcorrect. cpDistiller, which combines a pre-trained segmentation model with a semi-supervised Gaussian mixture variational autoencoder trained with contrastive and domain-adversarial learning, outperformed five baselines on both single-batch well-position correction and simultaneous triple-effect correction. Arevalo et al. also note that their scenarios used a single batch label each, and that multi-level correction (plates, then batches, then sources) might help but may not suit methods that transform the feature space.
Operational constraints that shape the choice
Metadata and maintenance differ between methods. Every method in the Arevalo benchmark except sphering needs batch labels, and sphering needs negative control samples instead. FastMNN, MNN, Scanorama and Harmony have to be recomputed over the whole dataset when new profiles arrive, while sphering, ComBat, scVI and DESC do not, though they do not guarantee good corrections for profiles from an unseen source. For a screen that adds plates every month, that recomputation cost is a pipeline design decision, not a footnote.
Correction also changes what your features mean. The JUMP hub notes that batch correction moves features into a new space where they no longer map to the original size, shape and intensity measurements, which is why JUMP publishes “interpretable” profiles taken just before that step. We would keep both versions: corrected profiles for retrieval and clustering, interpretable ones for explaining a hit to a biologist.
Validating a correction: batch metrics alone will mislead you
A correction that mixes batches well can still fail to preserve the biological signal. That is why the Arevalo benchmark scored four batch-removal metrics and six biology-preservation metrics, all from the scib package and normalised to a 0–1 range. Its overall score weighted biology metrics at 0.6 and batch metrics at 0.4. The BALANS paper used the same two-family logic, with graph connectivity, kBET, LISI and silhouette scores alongside clustering measures such as Leiden ARI and NMI.
The cpDistiller results show why both families matter: Harmony was the most effective baseline at removing row and column effects yet did poorly at preserving biological heterogeneity. On the biology side, task-based measures include replicate retrieval (finding a compound’s replicate across batches or labs), phenotypic activity (whether replicates are consistent and distinguishable from negative controls) and phenotypic consistency (whether perturbations with shared function give similar profiles). Percent replicating and percent matching, used to optimise the JUMP protocol, ask whether pairs of wells that should look alike correlate above the 95th percentile of a null distribution built from random pairs.
One caveat sits over all of this. The 2025 review states that the field still lacks consensus on which metrics best capture each aspect of performance. In our view, the most robust approach is to report at least one metric from each family together with a retrieval task on controls you know well, rather than optimising a single composite score.
Plate design that makes batch correction possible
The optimised Cell Painting protocol recommends that every batch include an extra negative control plate (DMSO only) and an extra positive control plate such as JUMP-Target-1 or JUMP-Target-2. In JUMP, most partners ran at least one untreated or DMSO plate and one Target2 plate per batch so batches could be aligned within and across sources. Target2 plates are needed because unrelated plates from the same batch correlate strongly with each other compared with related plates from other batches.
Layout comes next. JUMP’s 384-well layouts placed controls in the outer columns and treatments in the inner columns to limit well-position effects on the treatment samples. Placing a compound in the same well position across plates can inflate phenotypic activity metrics, and scrambling compound locations avoids that. For replicate numbers, the 2025 review recommends as many as feasible, typically at least four per treatment.
Acquisition choices count too. In the protocol optimisation work, profile quality did not measurably degrade between plates imaged on the day of staining and plates imaged 14 days later, but declined between weeks two and four, so imaging within 28 days of staining is recommended. Imaging a plate twice cost roughly 10% in percent replicating. Widefield and confocal imaging showed no consistent difference, even on two different microscopes. Plate and well metadata need to be complete, because CellProfiler processes images per plate and downstream profiling depends on linking layout to images.
Combining JUMP Cell Painting data with your own screens
The JUMP principal dataset (cpg0016) has features that make it ideal as a reference dataset. The JUMP hub describes it as 116k chemical and about 22k gene perturbations across 12 data-generating centres, all in U2OS cells. Images were acquired on five microscope types across 12 sites (Opera Phenix, ImageXpress Micro confocal, Yokogawa CV8000 and CV7000, and Operetta). Source counts vary between documents: the dataset paper says the compound set was generated at 11 sources, while the BALANS paper counts 13 data-producing sites.
The Arevalo paper describes JUMP-Target-2-Compound as 302 diverse compounds run with every batch, which allows alignment within JUMP and with future datasets generated outside the consortium. The dataset paper gives 306 compounds for the same plate, so check the plate map you download rather than relying on either count. The protocol therefore recommends a Target2 or similar positive control plate in each of your own batches.
The data lives in the Cell Painting Gallery on the AWS Registry of Open Data, which holds images (TIFF), extracted features and metadata, released under CC0 with a request to cite each dataset’s publication. The JUMP hub flags that cross-modality matching (ORF, CRISPR and compounds) is still being improved, while within-modality matching is generally reliable. A merge spanning more than one microscope type and more than a few hundred compounds is the scenario where no benchmarked method adequately removed batch effects, so validation has to measure what remains.
The engineering side of moving this pipeline from images to corrected profiles at scale (segmentation, feature extraction and storage for millions of wells) is covered in our piece on high-throughput image analysis in biotechnology. The question that stays open, though, is a scientific one. Before committing to a large multi-site merge, decide which biological comparison has to survive correction, and measure that comparison directly on controls. A batch-mixing score only tells you the batches overlap, not that the comparison you care about still holds.
FAQ
How do I use Harmony for batch correction on Cell Painting profiles?
In Python, one documented route is to reduce dimensionality with scanpy’s PCA (default 50 components) and pass batch labels into scanpy.external.pp.harmony_integrate with default parameters. The BALANS authors followed the default hyperparameters that Arevalo and colleagues recommend. For well-position effects, pass batch, then row, then column labels sequentially.
What kinds of cell morphology does Cell Painting capture?
Cell Painting measures size, shape, texture and intensity features across eight stained components, including the nucleus, nucleolus, endoplasmic reticulum, Golgi, mitochondria, plasma membrane, cytoplasm and cytoskeleton.
