feat(ProteinDesign): add FixedBackboneDesign task

[wrh-human]

Background

Add FixedBackboneDesign task based on Agent Rosetta paper (arXiv:2603.15952, ICML 2026).

Files

• benchmarks/ProteinDesign/ - New domain
• benchmarks/ProteinDesign/FixedBackboneDesign/ - Task implementation
• frontier_eval/conf/task/protein_design_fixed_backbone.yaml - Hydra config

[github-actions]

🤖 AI Code Review (gemini-3-flash-preview)

🇬🇧 English Analysis

1. Executive Summary

• Core Purpose: This PR introduces a new benchmark task titled FixedBackboneDesign under the ProteinDesign category. It aims to evaluate an agent's ability to design optimal amino acid sequences for a fixed protein backbone using the Rosetta energy function (ref2015).
• Modified File Structure & Modifications:
  • benchmarks/ProteinDesign/FixedBackboneDesign/README.md & README_zh-CN.md: Added bilingual documentation covering quick start, environment setup (Docker), and unified benchmark IDs.
  • benchmarks/ProteinDesign/FixedBackboneDesign/Task.md & Task_zh-CN.md: Defined the task overview, input/output specifications, scoring metrics (PyRosetta energy terms), and constraints for the agent.
  • benchmarks/ProteinDesign/FixedBackboneDesign/data/raw_task.json: Created a configuration file specifying the target PDB (1ubq), design positions, and native sequence.
  • benchmarks/ProteinDesign/FixedBackboneDesign/frontier_eval/: Added 9 metadata files (e.g., agent_files.txt, eval_command.txt, constraints.txt) required for integration with the frontier_eval framework.
  • benchmarks/ProteinDesign/FixedBackboneDesign/references/constants.json: Defined Rosetta energy function weights and descriptions.
  • benchmarks/ProteinDesign/FixedBackboneDesign/references/petrobind/1ubq.pdb: Included the standard PDB structure file for Ubiquitin.

2. AI Content Analysis

• Estimated AI Component: 65%
• Reasoning & Evidence:
  • Hallucinated Reference: The documentation cites "arXiv:2603.15952 (ICML 2026)". As of current real-world time, this date and ID are in the future, which is a classic sign of AI hallucination or the use of a synthetic placeholder.
  • Boilerplate Structure: The frontier_eval metadata files and the README structure follow a highly standardized, repetitive pattern typical of LLM-generated scaffolding.
  • Generic Descriptions: The energy term descriptions in constants.json are textbook definitions likely pulled from training data rather than specific project-specific documentation.

3. Engineering & Economic Assessment

• Engineering Reality Check: The task addresses a legitimate bioinformatics problem (Fixed-Backbone Design). It uses industry-standard tools (PyRosetta, Docker). However, the "demo" nature (designing only 8 positions) makes it a "benchmark-grade" problem rather than a full production-grade design suite. It correctly identifies edge cases like maintaining backbone coordinates and restricting to 20 standard amino acids.
• Economic Value: Medium. It provides a standardized way to evaluate AI agents in the "AI for Science" domain. By automating the evaluation of protein design, it reduces the manual overhead for researchers testing new optimization algorithms.

4. Quality Assurance

• Verification & Testing:
  • frontier_eval Integration: Yes.
  • task_name: ProteinDesign/FixedBackboneDesign (Unified ID) / fixed_backbone_design_demo (Internal ID).
  • Execution & Dependencies: The README clearly documents the docker pull command for the Rosetta environment and provides a 3-step local execution guide.
• Documentation Quality: High. The documentation is bilingual and well-organized. However, the redundant information between README.md and Task.md is slightly excessive, though helpful for agent-specific prompting.
• Organizational Structure: Logical and modular. Separating references, data, and verification follows best practices for scientific software.

5. Security & Privacy Check

• Sensitive Files: Clean. No .env, API keys, or IDE-specific files were detected.
• Absolute Paths: None detected. All paths in raw_task.json and the READMEs are relative (e.g., references/petrobind/1ubq.pdb).

---

🇨🇳 中文分析

1. 摘要

• 核心目的: 此 PR 在 ProteinDesign 类别下引入了一个名为 FixedBackboneDesign 的新基准测试任务。其目的是评估智能体在固定蛋白质骨架上，利用 Rosetta 能量函数（ref2015）设计最优氨基酸序列的能力。
• 修改的文件结构与变更摘要:
  • benchmarks/ProteinDesign/FixedBackboneDesign/README.md & README_zh-CN.md: 添加了双语文档，涵盖快速入门、环境配置（Docker）和统一基准 ID。
  • benchmarks/ProteinDesign/FixedBackboneDesign/Task.md & Task_zh-CN.md: 定义了任务概述、输入/输出规范、评分指标（PyRosetta 能量项）以及智能体的约束条件。
  • benchmarks/ProteinDesign/FixedBackboneDesign/data/raw_task.json: 创建了配置文件，指定目标 PDB (1ubq)、设计位点和原生序列。
  • benchmarks/ProteinDesign/FixedBackboneDesign/frontier_eval/: 添加了 9 个元数据文件（如 agent_files.txt, eval_command.txt, constraints.txt），用于集成到 frontier_eval 框架。
  • benchmarks/ProteinDesign/FixedBackboneDesign/references/constants.json: 定义了 Rosetta 能量函数的权重和描述。
  • benchmarks/ProteinDesign/FixedBackboneDesign/references/petrobind/1ubq.pdb: 包含了泛素（Ubiquitin）的标准 PDB 结构文件。

2. AI 成分分析

• 预估 AI 含量: 65%
• 判断依据与证据:
  • 虚假引用: 文档引用了 "arXiv:2603.15952 (ICML 2026)"。在现实时间线中，这个日期和 ID 属于未来，是典型的 AI 幻觉或使用了合成占位符。
  • 模板化结构: frontier_eval 的元数据文件和 README 结构遵循高度标准化的重复模式，这是典型的 LLM 生成脚手架特征。
  • 通用描述: constants.json 中的能量项描述是教科书式的定义，更像是从训练数据中提取的，而非特定项目的定制文档。

3. 工程与经济评估

• 工程现实检验: 该任务解决了合法的生物信息学问题（固定骨架设计）。它使用了行业标准工具（PyRosetta, Docker）。然而，其“演示”性质（仅设计 8 个位点）使其更偏向于“基准测试级”问题，而非完整的生产级设计套件。它正确识别了边缘情况，如保持骨架坐标不变和限制使用 20 种标准氨基酸。
• 经济价值: 中等。它为“AI for Science”领域的智能体评估提供了标准化方法。通过自动化蛋白质设计评估，减少了研究人员测试新优化算法的人工开销。

4. 质量保证

• 验证与测试:
  • frontier_eval 集成: 是。
  • task_name: ProteinDesign/FixedBackboneDesign (统一 ID) / fixed_backbone_design_demo (内部 ID)。
  • 运行与依赖: README 清晰记录了 Rosetta 环境的 docker pull 命令，并提供了本地运行的三步指南。
• 文档质量: 高。文档采用双语编写，组织良好。尽管 README.md 和 Task.md 之间的信息存在冗余，但这对于智能体提示词（Prompting）是有帮助的。
• 组织结构: 逻辑清晰且模块化。将 references、data 和 verification 分开符合科学软件开发的最佳实践。

5. 安全与隐私检查

• 敏感文件: 未发现异常。未检测到 .env、API 密钥或 IDE 配置文件。
• 绝对路径: 未检测到。raw_task.json 和 README 中的所有路径均为相对路径（例如 references/petrobind/1ubq.pdb）。