LLM as Scientist
Systems intended to connect hypothesis generation, experiments, analysis, and paper drafting. Their autonomy and evidence vary by project and research domain.
WHAT TO CHECK BEFORE RUNNING AN AI SCIENTIST
- Match the published experiments to your domain. A result in one task does not establish general scientific autonomy.
- Check model access, compute costs, code-execution requirements, and the project's licence before running it.
- Inspect experiments, citations, and generated conclusions yourself. A generated paper is not a validated research finding.
For narrower planning or analysis workflows, compare LLM as Analyst. Benchmarks in this list evaluate systems; they are not deployable research assistants.
Listed alphabetically by intended research role. A level is not a quality score or a verified capability rating; benchmark entries describe evaluation resources for that role.
TOOLS IN THIS LEVEL
14 indexedCollaborative AI agent research lab
Autonomous ML research agent
Automated research workflow engine
Research workspace for multiple coding agents
Research agents with evolving memory and skills
Autonomous web research agent
Agent framework for long-running tasks
Personal AI research assistant
Open-source co-scientist agent
Autonomous PhD-style research agent
R&D agent for data-driven research
Researcher skill pack for Claude
Template-based automated ML research
Automated ML research via agentic tree search
WHAT DO THE LEVELS MEAN?
- L0 · Reference — Reference Resources: Curated lists, awesome-repos, and initiatives that support AI-in-research workflows — starting points, not end-products.
- L1 · Tool — LLM as Tool: AI tools that augment a single researcher task — literature search, writing, reviewing, figure generation, content transformation.
- L2 · Analyst — LLM as Analyst: Research agents with stronger analysis, modeling, and experiment-execution abilities — they plan and reason across steps.
- L3 · Scientist — LLM as Scientist: Systems intended to connect hypothesis generation, experiments, analysis, and paper drafting. Their autonomy and evidence vary by project and research domain.