Gemma4Science.

Trials take years because screening takes hours per patient.

80% of clinical trials miss their enrollment timeline^[2], and screening is the bottleneck. A coordinator spends ~50 minutes per chart and 88% of screened patients don't qualify, meaning it takes ~8 hours to enroll a single patient.^[1]

To bring this to life, consider PARADIGM-HF, the Phase 3 trial that brought us Entresto, a game-changing drug for reducing heart failure.^[3]

Recruitment	Patients	Sites	Countries
2009-2012 (~3 years)	8,442	1,043	47

The ideal solution for decades has been AI-assisted chart review, starting with expert systems in the 80s. After multiple generations of clinical NLP (RNNs, BERT, medBERT, etc.), LLMs emerged in ~2022 as a real step change in capability. In 2026, Gemma 4 represents a step change on top of that as a strong open weights model capable enough to be generally reliable, and small enough to stay on the hospital's hardware to protect patient data.

To prove this out, I built Gemma4Science to simulate AI-assisted trial recruitment for PARADIGM-HF. Built on Gemma 4, the system reads every patient's chart locally and hands the coordinator a candidate queue with citations. For each LLM-assisted criterion, the system outputs one of three verdicts along with a verbatim chart quote:

Verdict	On an inclusion criterion	On an exclusion criterion
PASS	Has the required condition	Does not have the disqualifying condition
FAIL	Does not have the required condition	Has the disqualifying condition
UNKNOWN	Evidence not present in the chart	Evidence not present in the chart

Note that PASS always means the patient passes the criteria

Not every trial criterion is LLM-assisted. Some can be answered by structured queries when the data is well-documented; others must be assessed during the in-person screening visit.

Criterion Type	Examples
LLM-assisted	NYHA class, medication tolerance, angioedema history in notes
Structured SQL	Age, ICD-coded diagnoses, medication
Screening visit	Patient consent, latest labs, current decompensation, regimen reconciliation

When there is sufficient evidence available for a patient, Gemma 4 turns a ~50min chart review process to a ~2min verification process, a 25x speedup. This is the premise for clinical trial acceleration.

We adhere to the strict data use agreement of the MIMIC-IV dataset, which prohibits non-credentialed data sharing and only permits local LLM inference.

This project is made possible by MIMIC-IV + MIMIC-IV-Note, which are de-identified datasets containing ~364K patients and ~331K physician notes, sourced from emergency department and ICU visits at the Beth Israel hospital system. The data use agreement (DUA) allows analysis on credentialed user machines and Google BigQuery, but prohibits exposing any amount of data publicly or sending raw data to LLM APIs (OpenAI / Anthropic endpoints are off the table).

This creates a number of challenges that we solved.

Inference is local-only

Stack	Setup	Notes
Cloud	vLLM on ephemeral spot H100 VMs	Credentialed user's GCP project, torn down after each run
Local	Ollama on M3 MacBook	Most flexible option for broad OS and hardware-acceleration support

Data access is gated

• All BigQuery calls routed through a gated query interface: a pre-tool-call check blocks forbidden statements that would return raw data, and a post-tool-call static analyzer (regex) verifies returned rows don't carry raw text
• Scaled text analysis is performed with Gemma 4 local models, as permitted by the DUA

Evals with a silver dataset instead of gold

• Golden-dataset annotation is not feasible within the hackathon's reasonable-resources clause: would require credentialing clinical experts for review
• Silver dataset created through synthetic data generation, producing notes that contain specified clinical details by construction
• The generation spec combines Gemma 4-derived note observations (structure + quirks) so synthetic notes resemble the originals. I had Gemma 4 read thousands of notes and write a structural report on note layout (as emitted by the Beth Israel EHR) and recurring quirks (deidentification artifacts, physician shorthand, etc.)
• This approximates the real-data distribution but leaves a notable gap. A true solution in future work would be an expert-annotated golden dataset to evaluate Gemma 4

Hackathon submission without MIMIC-IV data

Artifact	What ships
Public Kaggle notebook	Gemma4Science pipelines with locked prompts and the synthetic silver dataset
Public live demo site	Gemma4Science UI on a fully-redacted static data source (verdict structure visible, no MIMIC-IV text)

Gemma4Science needed to implement PARADIGM-HF's 14 eligibility criteria for enrollment.

How each criterion is handled

Inclusion:

#	Criterion	Handled by
Incl 1	Informed consent	Screening visit
Incl 2	Age ≥ 18	Structured SQL
Incl 3	NYHA II-IV (3a), LVEF ≤ 35% (3b), NT-proBNP ≥ 400 (3c)	LLM for 3a, 3b, and 3c that comes from note; Structured SQL for 3c that comes from labs
Incl 4	ACEi or ARB at stable dose ≥ 4 weeks	LLM for tolerance + tenure + dose from notes; Structured SQL for active rx
Incl 5	β-blocker at stable dose ≥ 4 weeks	LLM for tolerance + tenure from notes; Structured SQL for active rx

Exclusion:

#	Criterion	Handled by
Excl 1	Other investigational drugs ≤ 30 d / 5 half-lives	Screening visit
Excl 2	ACEi/ARB hypersensitivity	Screening visit (allergy interview)
Excl 3	Previous intolerance at target dose	Screening visit (med-history interview)
Excl 4	History of angioedema	Structured SQL (ICD diagnosis); LLM (note mention)
Excl 5	Concurrent ACEi + ARB	Screening visit (regimen reconciliation)
Excl 6	Current acute decompensated HF	Screening visit (current state)
Excl 7	Symptomatic hypotension or SBP < 100	Screening visit (current state)
Excl 8	eGFR < 30 mL/min/1.73 m²	Screening visit (fresh lab)
Excl 9	Serum K+ > 5.2 mmol/L	Screening visit (current lab)

The LLM does the chart-reading work for 6 observations: NYHA class, LVEF, NT-proBNP cited in note, ACEi/ARB tolerance, β-blocker tolerance, angioedema mentioned in notes. Structured SQL covers age, recorded lab and rx values, and the angioedema ICD hard gate. The 9 remaining criteria will continue to be owned by the trial team as they can be completed at the screening visit.

Limitations

It's worth noting that PARADIGM-HF enrollment was focused on outpatient while the dataset we have is ICU and emergency department only. This is fine from a data quality perspective - ICU and ED visits are often better documented than outpatient visits - but it does pose a data completeness challenge since outpatient visits are far more common. I would expect real-world performance to be better than what is visible in MIMIC-IV.

The evaluation (silver) dataset is 400 synthetic discharge summaries with verdicts known by construction, generated to look like the real corpus of notes from MIMIC-IV.

In order to evaluate Gemma4Science and optimize the prompts, I needed notes with ground-truth verdicts per criterion.

The problem

Expert annotation on real MIMIC-IV notes was out of the question for the hackathon scope: note labeling is slow (~50 min per chart review), required clinical expertise bills at ~$1,000/hr, and the MIMIC-IV data use agreement would force an unrealistic credentialing timeline.

The approach

I can generate synthetic notes instead. For example, If I need a ground-truth PASS for NYHA II-IV, I should generate a note that explicitly contains the NYHA II-IV characterization, and the ground-truth label is PASS by construction. This, however, creates two new problems:

• Will the synthetic notes look like real MIMIC-IV notes?
• If Gemma 4 generates the silver notes, could it be biased toward accurately predicting the label on its own outputs in a subtle, invisible way?

The solution

1. I analyzed the real MIMIC-IV note corpus with Gemma 4 for two things: note structure (the standard sections that appear in every note) and note quirks (artifacts from de-identification, physician shorthand, and other unexpected content).
2. I analyzed the same corpus for ways that criterion-specific keywords show up in text, one analysis per criterion, along with the prevalence of each.
3. The output is 8 spec files that drive synthetic-note generation downstream:

Spec	Purpose
Note structure	Standard sections and ordering of a discharge summary
Note quirks	De-identification artifacts, shorthand, recurring oddities
6× criterion specs	Per-criterion keyword and idiom guide for NYHA, LVEF, NT-proBNP, ACEi/ARB tolerance, β-blocker tolerance, angioedema history

4. The specs feed a generator/verifier loop: a weaker/cheaper model handles at-scale generation, a stronger/more expensive model verifies. I used Claude Sonnet for generation and Claude Opus for verification. This was acceptable because both models operate only on the spec files (aggregate findings, no raw MIMIC-IV content). A second benefit: the generator is not Gemma 4, which partially addresses the self-bias problem. Hard to say how much, but enough for me to rely on the evals at silver-dataset quality. It would be interesting to revisit this approach in the future in a more rigorous way.

5. The loop ran on 38 reasonable case permutations (across the 6 criteria), designed to populate the 6×3 criterion-by-verdict cells without creating clinically-absurd scenarios.

Final state

• 38 cases, 400 notes (50/50 dev/test split)
• Label distribution matches real-cohort prevalence: NT-proBNP citations and angioedema mentions dominated by UNKNOWN, other criteria spread across PASS / FAIL / UNKNOWN

Achieved 97% verdict accuracy and 100% citation faithfulness with Gemma 4 E2B

I needed a model + prompt combination that's accurate enough to be clinically useful and small enough to run on a typical laptop made available to a clinical trial coordinator.

The requirements

Requirement	Target
Memory footprint	≤ 8 GB to support local inference
Hardware	Supporting GPU, Apple Silicon, GPU
Cloud run	GPU-accelerated VM
Coordinator deployment	Local machine without cloud round-trip

The starting point

Gemma 4 31B (BF16) on vLLM was the first thing I tried, just to check the task is possible on the largest/most capable model in the family. We hit 100% accuracy on 5 of 6 criteria, ~90% accuracy on the 6th, with span faithfulness at 100%. Now we can get smaller.

The target

Gemma 4 E2B at 4-bit quantization (~8 GB at Q4_K_M) generally the local memory footprint. Naive prompts on E2B dropped per-label accuracy to 94.75%, with Excl4 collapsing to 41%. Two prompt optimizations recovered the gap:

Prompt Optimization	What it did	Impact
Detailed per-criterion rewrite	Mechanical decision rules, "either path alone is sufficient" framing, step-by-step decision flows, loud anti-patterns based on dev mistakes	Excl4 41% → 98%; per-label avg 94.75% → 98.34%
Removed few-shot examples	Small model was reciting examples as citations	Improved both accuracy and citation faithfulness

Citation faithfulness

Gemma 4's emitted quotes drift from the source in two ways, one recoverable and one not:

Quirk	Share of unfaithful citations	Recovery
Style drift	~70%	Rewrite based on static rules: lowercase substring match, length-anchored slice, longest verbatim sub-match. Only rewrite if similarity > 0.85.
Structural fabrication	~30%	Drop-to-UNKNOWN: any row with an unrewriteable quote demotes the verdict to UNKNOWN and drops the quotes.

Raw citation faithfulness goes from 92.25% to 98.92% after the rewrite pass. The drop-to-UNKNOWN policy ensures all citations are valid when rendered to the user: every PASS/FAIL row surfaces a verbatim citation, and a fabricated PASS becomes UNKNOWN.

Speed

The pipeline achieved x notes/min on cloud inference and y notes/min on local inference. Two notable optimizations helped it get there:

Optimization	What it did	Impact
All-six mode	One LLM call emits all 6 verdicts per note, instead of 6 per-criterion calls	~2× wallclock throughput (6× fewer calls, each doing more work)
MTP speculative decoding	Small drafter model proposes tokens; target model verifies in parallel	~2× throughput on H100, no quality loss

Some notes:

• All-six mode required some prompt optimization work as well to clarify that all 6 verdicts will be generated from a single response.
• MTP is vLLM-only for now. Ollama doesn't seem to have MTP available yet for Gemma4 E2B - once it is available, it should hopefully be a free 2x speedup pending.

Final eval results

Cloud (vLLM bf16 MTP, silver test, 200 notes):

criterion     n   v_acc  s_faith  macroF1
Incl3a      200  96.5%  100.0%   0.87
Incl3b      200  98.5%  100.0%   0.97
Incl3c      200  96.0%  100.0%   0.92
Incl4       200  94.5%  100.0%   0.87
Incl5       200  99.0%  100.0%   0.98
Excl4       200  97.5%  100.0%   0.95
avg              97.0%  100.0%   0.93

Local (Ollama Q4_K_M on M3 MacBook, silver test, 200 notes):

criterion     n   v_acc  s_faith  macroF1
Incl3a      198  93.4%  100.0%   0.81
Incl3b      198  98.5%  100.0%   0.97
Incl3c      197  98.0%  100.0%   0.95
Incl4       198  96.0%  100.0%   0.90
Incl5       198  98.5%  100.0%   0.96
Excl4       198  97.5%  100.0%   0.94
avg              97.0%  100.0%   0.92

• Local Q4_K_M matches cloud bf16 on the held-out set (97.0% v_acc both, 0.92 vs 0.93 macroF1)
• 2/200 parse failures (1%) on local, within tolerance for Ollama serving

Kaggle T4 x2 reproduction (same Ollama path, clean environment): 91% v_acc, 0.85 macroF1.

• Regression vs M3 Ollama test: −6 pp v_acc, −0.07 macroF1
• Possible root causes that represent future enhancements opportunities:
  • Regressions were concentrated to 3 of 6 criteria. These are also the ones tuned hardest against M3 Ollama dev mistakes, which suggests overfitting
  • Serving environment variance - CUDA T4 vs MPS, Ollama version 0.24 (Kaggle) vs 0.23.1 (Local M3 Macbook)

Every adult heart-failure admission in MIMIC-IV's trial window goes through Gemma4Science in a single 6h 35m run on one H100, for $23

I pulled 14,517 potential patients with heart failure and analyzed 41,208 physician notes using Gemma 4 E2B, identifying 37 patients ready to be enrolled pending a screening visit / consent, and soft-passing 1,027 patients who are highly likely candidates.

The run

• Model and prompt: run with the configuration with the strongest evals, model details were Gemma 4 E2B-it, vLLM with MTP (speculative decoding), all-six verdicts per call, citation rewrites, drop-to-UNKNOWN at write time
• Throughput: 1.6 calls/sec on one H100 with 16-way concurrency. I observed the run was compute constrained so 8-way concurrency is probably sufficient here.
• Wallclock: 6h 35m
• Cost: ~$23 in compute using spot VMs
• Parse failures: 21 / 41,208 (0.05%), not retried

Note-grain verdicts after post-process:

Criterion	n	PASS	FAIL	UNKNOWN	Rewrite	Drop
Incl3a	38,027	21,805	515	15,707	7,071	1,314
Incl3b	38,027	7,520	17,674	12,833	3,118	1,582
Incl3c	38,027	7,621	1,061	29,345	189	688
Incl4	38,027	14,066	5,983	17,978	3,494	586
Incl5	38,027	22,851	3,754	11,422	7,663	476
Excl4	38,025	202	178	37,645	78	6

• Incl3b (HFrEF, EF ≤ 40%): largest source of explicit FAIL
• Incl3c (NT-proBNP elevated): 77% UNK — discharge summaries rarely print BNP values in narrative
• Excl4 (history of angioedema): 99% UNK — discharge notes don't typically state the absence of a rare reaction

Patient funnel

Patient-grain verdicts are rolled up from note-grain: any-PASS over the patient's notes wins, then any-FAIL, then UNKNOWN.

Stage	Count	Filter
MIMIC-IV database	~364K patients	None; source dataset
Pre-note cohort	14,517 patients	Adult, HF diagnosis, admission in 2009-2012, no ICD-coded angioedema
Screened by LLM	14,429 patients	At least one note in the materialized cohort parsed successfully
Ineligible	9,102 patients	At least one criterion FAIL
Candidate - Indeterminate	4,300 patients	At least one UNKNOWN
Candidate - Soft-pass	990 patients	All 5 inclusions PASS; Excl4 UNKNOWN (expected as angioedema is rare, so most discharge notes don't state its absence)
Candidate - Eligible	37 patients	All 6 criteria PASS; recommending outreach for screening visit

What MIMIC-IV doesn't cover

A real-world deployment can close three limitations in the MIMIC-IV dataset:

• Setting: MIMIC-IV is inpatient only (admissions, ICU, ED); PARADIGM-HF enrolled through outpatient cardiology, where the chronic baseline lives
• Other note types: admission histories, daily progress notes, cardiology consult notes, echo reports, allergy intake; the worst-performing criteria here (Excl4 99% UNK, Incl3c 77% UNK) are exactly what those notes are written to document
• Claims data: pharmacy + prescription claims = continuous medication exposure history; Incl4 (ACEi/ARB tolerance, 47% UNK) and Excl4 (prior angioedema, 99% UNK) both have stronger signal in claims than any single hospital note

Each site only needs to enroll ~8 patients for PARADIGM-HF. Gemma4Science found 37 likely candidates using only 6 hours of GPU time.

The benefit

Using Gemma4Science, we can shift the trial coordinator's job from ~50min chart review to ~2min verification of LLM recommendations, while prioritizing patients who are highly likely to be trial candidates. Of course, trial recruitment still requires screening visits, patient consent, confirmation of labs, etc. - manual tasks that are not straightforwardly automated. Even so, academic research has consistently flagged chart review to be the most time consuming activity for clinical trial recruitment^[1] - accomplishing this 25x speedup could reasonably reduce PARADIGM-HF's 3-year enrollment window to less than 1 year.

The impact

PARADIGM-HF showed that Entresto provides a 20% reduction in cardiovascular death or HF hospitalization versus the prior standard of care.^[3] Entresto now prescribed to millions of patients globally and is the recommended first-line therapy for HFrEF. The trial finished enrolling in 2012, and results were published in 2014.

Back in 2014, PARADIGM-HF was actually halted 6 months early because Entresto worked so well that it was unethical to withhold from a control group as it prevents hospitalization and saves lives. If Gemma 4 were available back then, the trial team could have brought Entresto to patients 2 years earlier by compressing the 3-year recruitment timeline. Making this possible for the next Entresto - that's Gemma 4 Good.

Sources