An auto-research system repeatedly chooses what to investigate next. Each experiment consumes time, and its outcome changes what the system knows. This makes the search strategy and the use of past results central design choices.
The projects in Awesome-Auto-Research-Tools approach those choices through code edits, trees, graphs, optimization rules, and persistent memory. This article compares the mechanisms behind those approaches. For the broader end-to-end workflow—from experiments to claims and human oversight—see How End-to-End Auto-Research Systems Work.
Search requires a testable candidate
In autoresearch, a candidate is an edit to a training script. The agent runs each edited version under the same time budget and retains improvements in the validation metric. This is greedy hill-climbing: the next candidate starts from the best script found so far. Git records experiments, and a human can change the instructions in program.md.
The AI Scientist packages the executable problem in a template: code, data, and an evaluation protocol. The agent generates an idea within that setting, implements it, evaluates it, and writes up the result. The template narrows the search space and makes candidate ideas testable. Its successor, AI-Scientist-v2, starts from a topic instead. The agent must design the experiments as part of its search.
A search method is only useful when the system can express a candidate as an action and evaluate its outcome. The definition of that action determines what kind of research the system can perform.
Branches preserve alternatives
Greedy search follows the best current version. Tree search retains alternatives. AI-Scientist-v2 represents experimental states as nodes and uses best-first tree search to expand a promising node. Other branches remain available when an experiment fails or a different line of work becomes attractive.
EvoScientist builds separate searches for ideas and experiments. Its Researcher Agent proposes, reviews, and refines ideas in a tree. Pairwise Elo comparisons rank candidates on qualities such as novelty and feasibility. Its Engineer Agent then searches implementation and experimental choices. The separation matters: selecting a promising research idea and finding a workable implementation are different problems.
InternAgent uses a graph for another kind of search. FlowSearch treats literature research as dependent subquestions: an answer may require earlier concepts or results, and the graph changes as those dependencies become clear. The structure supports parallel exploration of several questions and deeper investigation of prerequisites.
Research budgets shape selection
DeepScientist treats the value of a proposed method as an unknown function that is expensive to evaluate. An LLM reviewer estimates usefulness, quality, and exploration value; an upper-confidence-bound-style rule chooses what to test next. The exploration term gives uncertain candidates a chance, while the other terms favor candidates expected to produce valuable results. Findings from earlier experiments inform later estimates.
RD-Agent has a more specific allocation problem in quantitative research: should the next effort optimize a factor or a model? A contextual two-armed bandit uses the current performance state to allocate work between them. Its Knowledge Forest stores factor and model hypotheses, so successful directions can be extended and failed ones can be revised.
These selection mechanisms address different scales. A tree chooses a branch; a ranking chooses an idea; an optimization rule spends compute on a candidate; a bandit decides which type of work receives the next budget. They can coexist within one research process.
Memory changes future decisions
Persistence alone is not learning. A file of past experiments becomes useful when later searches retrieve and act on it.
DeepScientist’s Findings Memory distinguishes three stages: an untested idea, an implemented and tested finding, and a result promoted after exceeding a baseline. This records what has been proposed, what has been measured, and what counts as progress. Failed paths stay in memory to reduce repeated exploration.
EvoScientist keeps separate memories for promising or failed idea directions and for reusable experimental strategies. An Evolution Manager updates them from earlier runs and retrieves relevant entries for the agents’ next prompts. InternAgent combines outcome records with an IdeaGraph that connects similar ideas. Its PromptEvolver also revises the idea-generation prompt from accumulated experience. The search policy itself changes over time.
AutoResearchClaw encodes lessons from failed runs as reusable skills. ARIS records workflow events and uses /meta-optimize to propose improvements to its skills. These systems treat experience as guidance for how to conduct the next run, not just a list of previous results.
Sources of new hypotheses
Search also depends on where its candidates come from. autoresearch generates hypotheses by editing code. AI-Scientist generates ideas from a task and available context. RD-Agent draws on historical hypothesis trees. InternAgent’s FlowSearch organizes questions arising from the literature.
Idea2Paper constructs a methodological knowledge graph before a research run. It extracts reusable Method Units from papers, records relations among them, and uses peer-review feedback as a signal when retrieving and refining patterns. Online research then retrieves and adapts these structured patterns. The expensive work of organizing sources is done offline.
Biomni also builds infrastructure from literature, but its output is an action space: biomedical tools, software, databases, and workflows. Its agent retrieves know-how and executes tasks within that environment. Idea2Paper structures possible methods; Biomni structures possible operations. Both expand what an agent can do with external knowledge.
Evaluation determines what improves
A system can optimize only what its feedback distinguishes. autoresearch uses a validation metric, RD-Agent evaluates factors and models through backtesting, and DeepScientist promotes findings that exceed a baseline. These signals guide repeated experiments.
Other work tests broader research outcomes. AI-Researcher introduces Scientist-Bench tasks ranging from guided instructions to open-ended topics. AutoResearchClaw’s ARC-Bench covers open research topics in several disciplines. InternAgent’s ResearchClawBench asks agents to reproduce key findings from a paper and its data. These benchmarks evaluate different abilities: following a research plan, developing one, and reproducing existing evidence.
Scientific claims also need checks beyond a performance score. ARIS separates an executor from a zero-context reviewer and audits code, results-to-claims, reported figures, and citations. claude-scholar requires evidence review before source annotations become accepted knowledge or writing. These controls specify when a measured outcome can support a research claim.
The central design question is therefore concrete: what counts as progress in this research task? The answer determines which hypotheses the system tests, what it remembers, and which results it can present as findings.
modifications:
- Oct 1, 2026: rewrite in English.