ScienceBuddy: The Science Agent Learning With Researchers Through Nested Self-Improvement

A team led by PhAI Labs opened a workspace for the daily work of scientists with a September 15, 2026 study: ScienceBuddy. ScienceBuddy records on GitHub split a 224-tool science kit into 22 functional groups, stretching from gene science to molecular and cancer biology, from drug knowledge to image work, from literature scans to database queries. The thesis in one line: collaboration built with researchers should turn into better work next round. Fixing one answer in chat is not enough; working procedures and the model running them must improve for good.

The idea in plain words is this: in a one-floor house you live and work in one room; in a two-floor house the ground floor is the workshop and the top floor is the master. The inner loop tidies the workshop: the model stands still while procedures get revised. The outer loop trains the master: the model grows stronger through reinforcement learning in the chosen workshop. As the workshop changes, the master work changes; as the master grows, the workshop order gets revisited. The technical report on arXiv calls this two-way bond the heart of the study.

Procedures inside, the model outside: the nested loop

In the inner loop the task model stays fixed; a separate fixed helper model inspects faults and suggests bounded fixes to instructions, skill packs or context settings. A candidate is accepted only if it meets edit limits and advances on paired development measures. Later runs carry proof to the next fix. Development jobs stay apart from policy-training jobs and from a final held-out test set; the final set informs neither fixes nor choices. Well-formed job packs are stored runnable with their instructions and inputs.

In the outer loop, once the rig improves, once-hard jobs turn routine and lose training value. So environments get tuned to the current model and chosen rig through pilot runs, and researcher-derived jobs get varied with scientific inputs. A fixed rubric composer derives job criteria from the source collaboration trace for every job: correctness checks plus proof and output needs pile up weighted. The reward lands at trace end and spreads to produced tokens as shared advantage. While training runs, procedures and criteria stay fixed; past interactions give task definitions and diagnostic proof, not online training samples.

The workspace gathers chat, trace, compute and results panels in one view. Researchers give the question with data, inspect the resulting review, refine the work through later messages. Execution follows a ReAct-style reason-act-observe loop; the ReAct record on arXiv counts as the founding idea of this reason-action loop. Python, R and the command line are served, online databases and a local data lake complete each other. Uploaded papers, tables, sequences and images rest in durable space. The rig stays apart from infrastructure: instructions, skill packs and context procedures are editable parts. HarborFramework verifier docs tell how a job pack runs and gets graded with its instructions and inputs.

Three rounds ran back to back: 10 rig-search steps per round, then 20 reinforcement updates under the chosen rig; the starting model is Qwen-3.8-4B. Qwen docs on HuggingFace tell how small models run agent jobs. Rig measures climbed from 38.9 to 44.4, 34.4 to 46.7 and 61.1 to 70.0 percent per round, with mean training reward rising across all three rounds. Literature reading, database judgments, protocol fault fixing plus gene and variant rating formed four science families. The Stanford-rooted Biomni study laid the ground for such biomedical agent jobs. LAB-Bench records on GitHub show how literature and protocol jobs get graded.

Three rounds, four families: what the numbers say

At the end, single-try accuracy climbed from 42.2 to 73.3 percent; 33.3 percent of jobs flipped from wrong to right, with only 2.2 percent sliding from right to wrong. Gains appear in all four families; improvement reached jobs never solved before, widening system coverage. This ground is solid: same test sets, paired start-finish comparison. The weak side is written honestly: the reflecting helper model stands still, so task success rose but the improvement rig itself cannot count as strengthened.

Two separate cases test the parts one by one. Rig adaptation saw a best 75.0 percent across 24 batches; the chosen rig climbed from 31.1 to 51.1 percent on a separate validation set, a 20-point gain with model weights untouched. Four instruction entries plus nine scoped skills of the chosen rig cover Python runs, source and schema checks, bounded record search and open answer delivery. The model-learning side carried four-try coverage from 48.3 to 67.8 percent with nearly two hours of reinforcement; rig and try budgets stayed the same.

Real interaction cases exist too: a gene-focused plan was built for a JAK1 review in small-cell lung fate, assigning Seurat and Scanpy to single-cell reviews, UCell and AUCell to signature scoring, CellChat and NicheNet to cell talk. In an ARL4C deck request, candidate screening got tied to cell and molecule proof through Python and PyPDF2, each claim pinned to its backing panel. The GEPA line argues reflective prompt evolution can pass reinforcement; GEPA records on GitHub explain this line.

The honest limits are written in the report: the reflector is fixed, no multi-node work, thin generalization to more goals. Still the lesson is clear: when collaboration traces turn into task definitions, tasks into criteria and criteria into rewards, the system carries both its procedures and its model forward every round. Any team with a growing training bill learns the same lesson: first write with researchers what gets taught, then test procedures and models apart.