Biology Models

44% of the Way There: Illumina Won the Virtual Cell Challenge With Statistical Transfer, Not a Big Model

November 22, 20261 views阅读中文原文
44% of the Way There: Illumina Won the Virtual Cell Challenge With Statistical Transfer, Not a Big Model

44% of the Way There: Illumina Won the Virtual Cell Challenge With Statistical Transfer, Not a Big Model

The short version

  • Who won: On November 19, 2026, the Arc Institute announced the results of the second Virtual Cell Challenge. Illumina's BioInsight AI Lab took first place with an algorithm called CellularAI, scoring 0.4399 on the final test set. FlowState finished second at 0.3348, and Xaira Therapeutics' X-team third at 0.3263. The top prize is $100,000, paid half in cash and half in NVIDIA Brev credits.
  • What the task was: Teams received only the unperturbed control cells of each new cell line — 18,400 cells profiled with 46 non-targeting guides — and no perturbation data at all. They had to predict how gene expression shifts after 300 CRISPRi knockdowns, generating 400 virtual cells per knockdown. The three test lines were AC16 (cardiomyocytes), MOLM13 (acute myeloid leukemia) and U-87MG (glioblastoma).
  • What 0.4399 means: On Arc's scale, 0 is the cell-context mean (the average response across all knockdowns) and 1 is an independent replicate of the experiment. The winning entry therefore covered 44% of the distance between the mean and a replicate. Arc states plainly on the same page that the number is not accuracy, and the six component metrics diverge sharply: Illumina's PDS reached 0.8313 while its JACCARD was 0.0642.
  • How it won: Not with an end-to-end foundation model, but with statistical transfer. The pipeline first pools what happened when the same gene was knocked down in other cell lines, drawing on public and proprietary screens; it then weights each dataset by how closely its control cells resemble the new line, how noisy its estimates are, and how efficient its knockdown was; finally it rewrites the new line's real control cells by layering the predicted fold changes on top. Genes with no supporting evidence are left untouched.
  • What shipped alongside it: Illumina open-sourced a first version of the algorithm the same day at Illumina/CellularAI under Apache-2.0. GitHub records the repository as created on November 12 and last pushed on November 19.

> This article is compiled from multiple media reports: the organizer's wrap-up, Illumina's press release and technical blog, TIME's reporting, an independent technical analysis, and the GitHub repository.

Why this round was harder — and more informative — than the first

The 2025 challenge asked models to predict held-out perturbations within a single context: participants got H1 human embryonic stem cell perturbation data and were tested on perturbations they had not seen in the same cells. The 2026 edition removed that scaffolding entirely. There was no training set, only the baseline state of each new line, and models had to extrapolate across cell types.

Arc's wrap-up explains why that is the more useful question. The cell types biologists most want to perturb are often the ones they cannot: rare cell types, primary cells that stop dividing outside the body, diseased cells that cannot be cultured faithfully, and tissue states that depend on their native microenvironment. A virtual cell becomes valuable when it can take the experiments that are feasible and make useful predictions about the ones that are not.

The dataset was built to that standard. Each of six cell lines was profiled across more than 10,000 CRISPRi knockdowns; the experimental teams handled more than a billion cells and deeply sequenced more than 30 million cells (median 20,000 UMIs per cell), producing more than 800 billion UG100 reads per cell line. Only perturbations with at least 80% median on-target reduction were kept, and each was downsampled to 400 cells so that coverage, depth and knockdown efficiency stayed comparable across all six lines.

Inside the winning method: three steps, none of them generative

The flow described by Arc and by Illumina's technical blog condenses into three stages. The third is what makes the approach distinctive.

Step one: pool the evidence into a consensus

Each training dataset is analyzed with covariate-adjusted regression to obtain a log fold change, a standard error and a significance value for every gene and perturbation, and these estimates are centered across perturbations to correct for inflation. The datasets are then combined into a prediction for the unseen line through a random-effects meta-analysis. Each dataset's weight reflects three things: how closely its control cells resemble the new line, how variable and noisy its estimates are, and how efficient its knockdown was.

The critical design choice is that the meta-analysis also estimates how much the datasets disagree beyond their own noise. When sources conflict, no single dataset dominates simply because it is larger. Illumina's blog uses a weather-station analogy: a forecaster does not rely on the nearest station alone, but pools readings weighted by instrument reliability while acknowledging that terrain and elevation produce real disagreement.

Step two: correct with the new line's own control cells

A consensus effect cannot be applied as-is, because expression backgrounds differ between cell lines. The team used singular value decomposition to identify the dominant response patterns across datasets, then fit a ridge regression that learns how a dataset's control expression shifts its log fold changes along those patterns. Feeding in the new line's control profile yields a correction to the predicted effects.

Step three: rewrite real cells instead of generating new ones

This is where CellularAI departs most sharply from typical cell-generation models: it does not build cells from scratch. It starts from the new line's real unperturbed cells and pushes each gene's expression up or down by the predicted log fold change, leaving genes with no supporting evidence exactly as they were.

The benefit is that virtual cells inherit the natural variation of real single-cell data — most genes are detected in only some cells, and the long tail of expression is preserved. The cost is that outputs are doubly constrained, by the reference experiments and by the predicted effects. The method answers "can knockdown responses measured in similar cells transfer to a new background," not "what happens under any intervention in any cellular state."

Illumina's blog lays out several principles for gene-level calls: predict strongly only when multiple cell types agree; default to "no change" when evidence is weak, sparse or inconsistent; weight similar cell lines more heavily but cap any single close match; and shrink predictions conservatively for lowly expressed genes, since a gene that is barely expressed has little room to fall.

Two training details are worth recording. Beyond the Billion Cell Atlas, the team used three public genome-scale CRISPRi Perturb-seq datasets (in iPSCs, human pluripotent stem cells, and primary CD4+ T cells). Method iteration was partly automated by LLM-based research agents running on eight NVIDIA H200 GPUs, but prediction and cell synthesis ran on CPU nodes.

The leaderboard and the component scores

| Team | Overall | PDS | MSE | NMAE | FIDELITY | REACH | JACCARD |

| --- | --- | --- | --- | --- | --- | --- | --- |

| BioInsight AI Lab (Illumina) | 0.4399 | 0.8313 | 0.6842 | 0.4399 | 0.1908 | 0.4288 | 0.0642 |

| FlowState | 0.3348 | 0.8363 | 0.3632 | 0.3286 | 0.1695 | 0.3128 | −0.0018 |

| X-team (Xaira) | 0.3263 | 0.7346 | 0.4681 | 0.3038 | 0.0659 | 0.3186 | 0.0667 |

The six metrics divide the work as follows: PDS asks whether a predicted effect is closer to its own measured effect than to those of other knockdowns; MSE measures squared expression error relative to the size of the real effect after removing sampling noise; NMAE measures error in predicted log fold changes for genes that truly change; FIDELITY checks whether genes called differentially expressed move in the right direction, with credit scaled by whether the prediction flags as many genes as the data do; REACH measures how far down a prediction's own significance ranking the directions stay correct; and JACCARD measures overlap between predicted and measured sets of significant genes.

Three observations stand out:

  1. Discriminating between perturbations works; reconstructing detail does not. All three teams scored above 0.73 on PDS, and FlowState's PDS (0.8363) actually edged out the winner's. Yet JACCARD was 0.0642 for the champion, 0.0667 for third place, and negative for second. Arc notes that even the best entry managed only 0.07 on JACCARD.
  2. Difficulty is uneven across metrics. Among final entries, more than nine in ten beat the context mean on PDS and REACH, and eight in ten on NMAE — but only about six in ten did so on FIDELITY, four in ten on MSE, and a third on JACCARD.
  3. The two leaderboards are not directly comparable. Validation used Cal-33, HeLa and Jurkat; testing used three different lines with a different target panel (129 of the 300 test knockdowns overlapped the validation panel, 171 were new). Across the 638 teams appearing on both boards, validation and test scores correlated at a Spearman coefficient of 0.78 — but among the top 20, that correlation fell to 0.49.

The caveat that matters most: how much of 0.4399 is transfer?

Arc put this in its own write-up, and it deserves to be quoted in substance: of the 300 test targets, almost all had already been knocked down in other cell lines. 277 appeared in the genome-wide K562 screen from Replogle et al. (2022), 298 in the X-Atlas/Orion screens of HCT116 and HEK293T, and 21 had been targeted in U-87MG by a kinome screen. Only one of the 300 had no public perturbation data anywhere.

As a result, simply carrying measured responses over to the control cells of a new line — with no learning at all — already recovers a large part of the perturbation-specific signal, because much of a knockdown's downstream signature is shared across cell types. Arc states directly that all three winning teams built on this kind of reference transfer; they differed in how they weighted the evidence, adapted it to a new line, and turned predicted effects into cells.

The word "zero-shot" needs the same care. It means the challenge supplied no perturbation data for those six lines — not that the models had never seen the genes. Teams could train on any public or private data. Illumina used its own Billion Cell Atlas; Xaira used more than 37 million cells of in-house Perturb-seq data; the second-place FlowState was built by two independent researchers on public data alone — 222 datasets from 128 studies spanning 86 cellular contexts.

TIME's coverage makes the point more directly: data volume showed up in the results, and Illumina owns its own atlas. TIME also spoke with challenge head Hani Goodarzi, Arc's AI research lead and a professor of biochemistry at UCSF, and with Shana Kelley, a professor of chemistry and biomedical engineering at Northwestern and president of bioengineering at the Chan Zuckerberg Initiative's Biohub, who was not involved in the challenge. Goodarzi called the turnout "a staggering number of participants" and said "this year, everyone jumped in." Kelley's estimate of what matching AlphaFold would require: "probably going to be billions, maybe trillions, of datasets."

TIME adds an observation worth keeping: the models did well on the general picture but less well on the individual quirks of different cell types — precisely the part a virtual cell most needs.

What was open-sourced, and what was not

Illumina released a first version of the algorithm the same day at Illumina/CellularAI under Apache-2.0. The repository contains prediction, validation, merging and evaluation code, a method guide, configuration files, and a synthetic example. GitHub's API records creation on November 12, 2026 and a final push on November 19, 2026.

"Open source" here needs to be unpacked into three layers:

  • Code: readable and modifiable. Pooling, context-similarity weighting, residual correction, cell synthesis and quality gates are all present.
  • Data: separately gated. The README states that a full run requires a precomputed, access-controlled summary bundle entered through catalog.json; the Billion Cell Atlas is available through a consortium partnership model and requires contacting the company. The repository does not include the preprocessing chain that turns raw training data into those summary tables.
  • Results: not directly reproducible. The bundled synthetic example uses fictional data — 30 genes, three perturbations, 1,200 cells — and the README says explicitly that it validates data flow and output structure, not predictive accuracy or biological realism.

There is also a subtle tension between two official statements. Illumina's blog says this release "does not include the Billion Cell Atlas nor was it trained on the Billion Cell Atlas," describing CellularAI as an analytical framework that can operate on a reference library such as the Atlas. The GitHub README, meanwhile, says the method is driven by precomputed summary tables derived from the Billion Cell Atlas together with public Perturb-seq datasets. The two statements most likely refer to different things — the open-source package versus the reference data behind the winning run — but neither document maps configurations to specific summary bundles item by item. What can be confirmed from both: a full run requires a separately supplied data bundle under its own license terms.

One more number needs its provenance labeled. Illumina ran held-out evaluations on three contexts with known ground truth — H1 (2025 challenge data), K562 (the Replogle screen) and a HEK293 screen — each time removing that cell line's own data from the reference library. It reports roughly 2× the performance of the cross-cell response mean. That is the company's own validation narrative; a detailed manuscript is promised for early 2027, no third party has replicated it, and it cannot be converted into Arc's 0.4399.

How to use it: what three kinds of teams can do now

Teams with their own Perturb-seq data. Follow the README: clone the repository, build a Python 3.12 environment (Linux for the parallel stages), run examples/synthetic/run_example.py first to confirm the pipeline and output format, then prepare your own reference summaries, task gene list, perturbation list, and at least one control .h5ad for the target context. Treat step one as a pipeline check only — "it ran" is not "it is accurate."

Teams that want to evaluate rather than deploy. The most valuable move right now is to reproduce the evaluation protocol, not the score: hold out entire cellular contexts rather than randomly splitting cells within one context, and report three baselines side by side — the predicted mean, uncalibrated reference transfer, and the full CellularAI pipeline — stratified by context and by metric.

Readers who just want the state of the field. Three coordinates suffice: discriminating between perturbations (PDS 0.83) is largely working; recovering the actual set of differentially expressed genes (JACCARD 0.06) has barely begun; and none of the three winning entries was an end-to-end model.

Good fit

  • You already hold perturbation data from similar cell types and want testable hypotheses about a new context you cannot easily perturb, before spending wet-lab budget.
  • You need to prioritize candidate genes or targets ahead of bench work, especially where public Perturb-seq corpora (Replogle, X-Atlas/Orion, ENCODE, LINCS and others) can be exploited.
  • You want to audit an interpretable transfer method: which reference data entered, with what weights, how the new background was corrected, and under what conditions output is refused. In CellularAI these are inspectable steps, not a black box.

Not a good fit

  • Do not use it to predict drugs. The task is defined as CRISPRi knockdown to transcriptome. There is no dose, no compound, no combination perturbation in the input, and no spatial or phenotypic readout in the output. Using it to talk about drug efficacy changes the question.
  • Do not treat it as a cell simulator. It emits expression count matrices, not cell function, morphology or tissue behavior; the underlying mechanism is statistical transfer, not physical or causal simulation.
  • Do not use it to replace wet-lab decisions. A score of 0.4399 means roughly 56% of the distance to a replicate experiment remains, and Arc itself says the field has "a long way to go."
  • Do not expect to reproduce the leaderboard out of the box. Without the official data bundle and the competition configuration, a change in reference coverage or preprocessing version can move results more than the algorithm does.

Pitfalls: four easy misreadings

  1. Reading 0.4399 as accuracy. It is a position between two anchors — the context mean and a replicate — not a percentage correct. The JACCARD component of 0.0642 shows that significant-gene sets barely line up.
  2. Reading zero-shot as "never saw these genes." 299 of the 300 targets had public perturbation data in other cell lines; only one was genuinely new. The zero-shot property applies to the cell line dimension only.
  3. Crediting the model for the transfer baseline. All three winners build on reference transfer, and Arc flags that simple carry-over already recovers much of the signal. Any evaluation of a similar model must run the direct-transfer and mean baselines first.
  4. Equating open source with reproducibility. The code is genuinely Apache-2.0; the data must be requested, the preprocessing chain is absent, and the bundled example is fictional. A Chinese-language analysis published November 22 by Biocloudservice separates these layers and concludes that reading the code and inspecting the workflow is not the same as reproducing 0.4399.

Cost and time

  • Compute: Illumina says prediction and cell synthesis run on CPU nodes and has not published per-task runtime or cost. The H200s were used for automated method iteration, which users do not need to replicate.
  • Data: this is the real barrier. The Billion Cell Atlas is available through consortium partnership, and the summary bundle requires separate approval under its own license. Public Perturb-seq corpora alone are enough to run something — that is what FlowState did — but coverage and results will differ.
  • Manuscript: a full methods paper is planned for early 2027, at which point self-reported figures such as the 2× gain will have checkable detail.

What to watch next

If you only track one thing, make it the holdout design in the 2027 methods paper; if you want a next step you can take this month, run the mean baseline and the direct-transfer baseline on your own data and see how much of your model's edge survives.

  1. The early-2027 methods paper. Look at the holdout design (are whole cellular contexts held out?), the baselines (are direct transfer and the mean included?), and exactly how the 2× figure is defined.
  2. An independent reproduction on public data alone. If a team using only public Perturb-seq summaries, run strictly per the README, consistently beats the cross-context direct-transfer baseline across pre-registered held-out contexts, the claim that open source has produced an independently verifiable transfer method holds up. If only the synthetic example runs, the value of the release stays mostly at the transparency level.
  3. The third challenge. Goodarzi says Arc will run it "every year, until we have a virtual cell"; the next edition starts in summer 2027. Whether the task adds drugs, doses or combination perturbations will be the clearest signal of where the field's boundary sits.
  4. How open the Billion Cell Atlas becomes. Its 200-plus cell lines and more than 500 million single cells are consortium assets today. Whether outside groups can audit that data determines whether reproducibility in this field is held by one company or shared by the industry.

Sources

  • Arc Institute, "Virtual Cell Challenge 2026: Winners and Wrap-Up," November 19, 2026: https://arcinstitute.org/news/virtual-cell-challenge-2026-wrap-up
  • Illumina press release, "Illumina Wins Virtual Cell Challenge, Powered by Billion Cell Atlas," November 19, 2026: https://www.illumina.com/company/news-center/press-releases/2026/29411.html
  • Illumina technical blog, "Introducing the CellularAI algorithm," November 19, 2026: https://www.illumina.com/science/genomics-research/articles/introducing-the-cellularai-algorithm--predicting-single-cell-res.html
  • TIME, "Scientists Are Pursuing the Dream of a 'Virtual Cell'," November 20, 2026: https://time.com/article/2026/11/20/scientists-are-pursuing-the-dream-of-a-virtual-cell/
  • Snape Report, "Illumina's CellularAI wins Arc Institute's 2026 Virtual Cell Challenge": https://snapereport.com/s/illumina-cellularai-wins-arc-institute-virtual-cell-challenge-2026
  • Biotech Today, "Illumina Opens CellularAI After Virtual Cell Challenge Win," November 20, 2026: https://biotech-today.com/illumina-opens-cellularai-virtual-cell-challenge-win/
  • Biocloudservice, "CellularAI Is Open Source: What Still Stands Between You and the Winning Score," November 22, 2026: https://www.biocloudservice.com/wordpress/cellularai-open-source-reproduction-20261122/
  • GitHub, Illumina/CellularAI (Apache-2.0): https://github.com/Illumina/CellularAI

Comments

0 published
Log inorsign up to leave a comment.

Loading comments...