# TraderHarness full documentation Canonical source: https://github.com/HephaestLab/TraderHarness This file is generated from the public README and documentation pages. --- # TraderHarness

A contamination-resistant backtesting environment for LLM trading agents — and a training-data synthesizer built on every run.
Real A-share data · strict point-in-time masking · fingerprinted replay → trajectory export

CI PyPI License Python Status · 中文 · Docs

LLM agents are moving into real trading. Research, decision-making, and order placement are increasingly done end-to-end by a model, not by a hand-coded strategy. But "backtest the agent" still has no standardized rig. A general-purpose model may recognize the dates, the companies, or the very history it is being tested on — leakage that quietly invalidates the evaluation. Fill conventions are ad hoc, runs cannot be reproduced, and the conclusions are hard to trust. TraderHarness is that rig: a contamination-resistant execution environment for LLM trading agents — and, because every model call is recorded at full fidelity, a data synthesizer that turns those runs into trajectory training data. **Start with evidence:** [explore the zero-credential Masked vs Unmasked experiment](https://hephaestlab.github.io/TraderHarness/masking-ab-showcase/) · [run the no-key replay](#quick-start) · [compare its scope with TradingAgents, StockBench, and Qlib](docs/comparison.md) · [read the LLM-agent backtesting guide](https://hephaestlab.github.io/TraderHarness/guides/agent-stock-backtesting/) · [中文文档](https://hephaestlab.github.io/TraderHarness/zh/)

TraderHarness streaming historical replay console

Captured from the local research console (webui/scripts/capture-demo.mjs). Regenerate with npm run capture:demo after any UI change; this GIF will be refreshed alongside the v1.0 acceptance run.

> If leakage control, execution fairness, or reproducible LLM evaluation matters to your work, consider starring the > repository. New public benchmarks, replay cassettes, and dataset updates will land here. ## Field test: a 21-trading-day mini benchmark in a brutal drawdown From 2026-06-20 to 2026-07-20 the A-share market sold off hard — the CSI 300 returned **-9.12%** over the window. We ran the same "trend swing" agent card (dual date + entity masking, seed 42, ¥1,000,000 initial cash) against four frontier models; each completed all 21 trading days independently, seeing only masked prices, announcements, and fundamentals: | Model | Return | vs. CSI 300 | Sharpe | Max drawdown | Trades | Win rate | |---|---|---|---|---|---|---| | Claude Opus 4.8 | **+1.45%** | +10.57 | 2.61 | 0.76% | 6 | 66.7% | | Kimi K2.6 | +0.34% | +9.46 | 0.69 | 0.37% | 2 | 100% | | Qwen3.7-Max | -1.64% | +7.48 | -3.77 | 1.64% | 12 | 0% | | DeepSeek V4-Pro | -2.69% | +6.43 | -6.57 | 2.69% | 10 | 20.0% | All four beat the benchmark by a wide margin. The interesting part is the behavioral spread: Kimi traded only twice all month, while Qwen fired 12 orders without a single winner — same market, same tools, same prompt contract, visibly different model temperaments. That is exactly what a standardized harness is for. > This is a small-sample, single-agent-style smoke benchmark that demonstrates the evaluation loop; it is not a model ranking or investment advice. Every call is fingerprint-replayable — see [training data](docs/training-data.md) for the full contract. ## A standardized harness for LLM trading agents TraderHarness hands the model a historically valid market, research tools, and a portfolio. It can investigate, write analysis code, revise its thesis, and place orders through one controlled path — nothing about the environment is a live market or a broker. The first production dataset covers China A-shares: five years of full-market daily and 5-minute bars, announcements, policy news, fundamentals, valuation, dividends, and the CSI 300 benchmark. - **Contamination-resistant by construction.** Strict point-in-time masking at every data boundary, plus deterministic calendar and company anonymization, so the agent cannot recognize the period or the stocks it is being tested on. - **Fair execution.** Progressive 5-minute visibility with minute-level matching; every fill goes through one path, `TradingBus.place_order()`, at unadjusted historical prices. - **Deterministic.** Zero data I/O after preload — the same visible data and action sequence always produce the same environment result. ## An LLM trading data synthesizer Every run doubles as a data-generation pass, with a three-step chain: 1. **Full-fidelity trajectory.** Each LLM call is persisted with the complete message list, the full tool schema, assistant content and reasoning, every tool call and its arguments, and phase/sub-window metadata. 2. **Fingerprinted replay.** The cassette replays deterministically with no API key, and `traderharness audit` scans serialized artifacts for leaked entities or dates before anything is shared. 3. **Training trajectories.** Full-fidelity trajectories can feed reinforcement learning, behavior analysis, and evaluation; the built-in `traderharness export sft` converter also emits OpenAI-style JSONL, gated by masking and leakage checks. ```bash traderharness audit result.json replay.jsonl traderharness export sft result.json --output training.jsonl ``` The research console renders the same evidence as a per-trade dossier: the executing 5-minute K-line, the agent's stated reasoning, and the exact tool call that produced the fill.

TraderHarness result research workbench with benchmark, drawdown, K-lines, trades, and decision evidence

The full contract is in [Training data](docs/training-data.md); curated exports are published as the [traderharness-a-share dataset](https://huggingface.co/datasets/ANTICH/traderharness-ashare-5y) on Hugging Face. ## The research console The bundled local console (FastAPI + React) turns a run into something you can watch: a pixel-art operations floor streams the replay live, while performance, per-trade dossiers, and cross-run comparisons update in the same window. It is a local research tool — keep it bound to localhost and never expose it to the public internet.

TraderHarness live control room streaming a historical replay

Pixel-office operations floor, close-up Live performance panel during a run
Pixel office. Each agent is a desk on the floor; the replay clock, phases, and tool activity stream in real time. Live performance. Equity curve, benchmark, drawdown, and positions update as the day unfolds.
Per-trade review with execution K-line and agent reasoning Cross-run comparison view
Trade review. Every fill links back to the 5-minute K-line, the agent's reasoning, and the tool call behind it. Run compare. Rank agents or checkpoints across runs on identical masked data.
## Why TraderHarness | Capability | Typical agent demo | TraderHarness | |---|---:|---:| | Point-in-time fundamentals and news | Partial | Enforced at every tool boundary | | Date and company anonymization | No | Deterministic entity + calendar masks | | Fair intraday execution | Often close-price fills | 5-minute visibility and minute-level matching | | Reproducible LLM runs | Prompt logs | Fingerprinted replay cassettes | | Full training trajectory | Usually truncated | Full request, response, reasoning, tools, and results | | Multi-agent support | Shared-chat simulation | Isolated comparison + single-executor committees | | Backtest-time data I/O | Common | Zero I/O after preload | | Historical playback | Ad hoc scripts | Streaming, phase-bounded historical replay — never a live feed | TraderHarness deliberately owns the historical environment, information boundary, execution fairness, and evidence artifact; it does not prescribe one trading methodology. For a fuller comparison against TradingAgents, StockBench, Qlib, and traditional backtesting engines, see [Project comparison](docs/comparison.md). ## Four-agent showcase TraderHarness ships four reference agent cards under `traderharness/agents/builtin/`, each with a distinct, inspectable persona rather than a single generic "trader": | Agent id | Style | Risk profile | Holding period | |---|---|---|---| | `trend-breakout` | Momentum/breakout continuation, relative strength, volume-confirmed, mechanical stops | Aggressive | 3–10 trading days | | `quality-compounder` | Profitability quality, balance-sheet, and valuation discipline; low turnover | Conservative | 20–60 trading days | | `event-hawk` | Announcement/policy/news catalysts with timestamp and source discipline | Aggressive | 1–5 trading days | | `quant-researcher` | Reproducible cross-sectional factor checks via the Python sandbox | Balanced | 2–20 trading days | The reference acceptance run compares all four, head-to-head, over **2024-03-04 → 2024-03-29** with entity masking enabled and `deepseek-v4-pro` in thinking mode as the executor model: ```bash export DEEPSEEK_API_KEY="..." traderharness compare \ --agent trend-breakout \ --agent quality-compounder \ --agent event-hawk \ --agent quant-researcher \ --start 2024-03-04 \ --end 2024-03-29 \ --mask-entities \ --entity-mask-seed 42 \ --record-replay showcase_mar2024 \ --output showcase_mar2024/comparison.html ``` `--record-replay` captures a fingerprinted cassette so the exact run can be replayed deterministically without an API key, then audited and shared. See [Quick start](#quick-start) and [Training data](docs/training-data.md). **Acceptance result** (2024-03-04 → 2024-03-29, entity mask seed `42`, `deepseek-v4-pro` thinking high). Ranked by Sharpe after `traderharness audit` passed with zero findings; replay the recorded bundle with `--replay showcase_mar2024` to reproduce without an API key: | Agent | Total return | Annualized return | Sharpe | Max drawdown | Win rate | Trades | |---|---:|---:|---:|---:|---:|---:| | `quality-compounder` | +0.52% | +7.92% | 1.09 | 0.76% | 33% | 9 | | `trend-breakout` | +0.09% | +1.32% | −0.20 | 1.07% | 38% | 16 | | `event-hawk` | −0.70% | −9.74% | −3.22 | 0.93% | 0% | 5 | | `quant-researcher` | −1.33% | −17.75% | −4.19 | 1.73% | 40% | 19 | | CSI 300 (benchmark) | −0.10% | — | — | — | — | — | ## Dual masking Evaluation leakage is a systems problem, not a prompt instruction. - **Temporal mask:** daily bars satisfy `date < current_date`; intraday bars are truncated to the active sub-window; fundamentals satisfy `pub_date <= current_date`. - **Calendar mask:** agent-facing dates become offsets such as `D-1` and `D+0`. - **Entity mask:** real codes and company aliases map deterministically to neutral pseudo-identities while preserving board rules such as 20% price limits. - **Output sanitation:** responses, reasoning, tool arguments, committee memos, trajectories, and replay cassettes pass through the same masks. - **Leakage audit:** JSON, JSONL, and Parquet artifacts can be scanned before publication or trajectory export.

Original historical announcement transformed by date and entity masks

The v1.0 acceptance run scanned the complete one-month DeepSeek trajectory after serialization and reported zero detected real entity aliases or absolute calendar dates. That is an automated egress audit, not a claim that semantic re-identification is impossible — see [Preventing data leakage](docs/contamination.md). ## Quick start Python 3.10–3.12 is supported. Three commands, no API key (the first run downloads and verifies the full dataset): ```bash pip install "traderharness[llm,data,ui]" traderharness data download --full traderharness demo ``` `traderharness demo` replays a real masked one-day run without an API key. It is a **streaming historical replay** of real local market data, progressively revealed phase by phase — not a live feed and not a synthetic substitute. Launch the local research console: ```bash traderharness ui # open http://127.0.0.1:8000 ``` Run your own agent: ```bash export DEEPSEEK_API_KEY="..." traderharness run \ --agent trend-breakout \ --start 2024-03-04 \ --end 2024-03-29 \ --mask-entities \ --record-replay run.jsonl ``` On PowerShell, use `` $env:DEEPSEEK_API_KEY="..." `` and backtick line continuations instead of `\`. Prefer a container: ```bash docker compose up --build ``` The compose file mounts `~/.traderharness` at `/data`, so the full dataset stays outside the image. Open `http://127.0.0.1:8000`; the bundled replay needs no API key. ### The agent's market day Every trading day has three bounded phases: 1. **Pre-market research** — portfolio, prior-day breadth, sectors, watchlist, announcements, and policy news. Orders are disabled. 2. **Open window** — only 09:30–10:00 bars revealed progressively; orders are enabled. 3. **Close window** — only 14:30–15:00 bars revealed progressively; orders and `finish_day` are enabled. The agent can query point-in-time K-lines, fundamentals, valuation, announcements, and news; screen the full market; run Python with `numpy`, `pandas`, or `scipy`; manage a watchlist; and inspect its read-only portfolio. ## AI-readable entry point TraderHarness publishes a machine-readable operating guide so a coding assistant can work in this repository without guessing at the invariants above: - [`llms.txt`](llms.txt) — a short index of the canonical documentation. - [`llms-full.txt`](llms-full.txt) — the same documentation set concatenated into one file, generated by `scripts/build_llms_full.py`. - [`AGENTS.md`](AGENTS.md) — the public operating guide: project map, non-negotiable invariants, and workflow. A useful first prompt for Cursor, Claude Code, or a similar assistant: ```text Read AGENTS.md and docs/architecture.md. Add an Agent implementation without bypassing TradingBus, point-in-time masking, or the single order path. Write the failing tests first, then run the focused suite and the replay demo. ``` ## Independent race vs. single-executor committee `compare` and a committee are two different products built on the same masked market and single order path. ### Independent comparison `compare` runs each agent in an isolated portfolio against the same market clock and capital, then ranks performance and behavioral metrics. This is what the [four-agent showcase](#four-agent-showcase) above uses. ```bash traderharness compare \ --agent trend-breakout \ --agent quality-compounder \ --start 2024-03-04 \ --end 2024-03-29 \ --mask-entities \ --output comparison ```

TraderHarness comparison workbench ranking isolated agent portfolios

### TradingAgents-style committee A committee is one trading agent with multiple read-only advisors and exactly one executor. Advisors research concurrently; only the Trader receives order tools. This preserves a single accountable action path and one portfolio. ```yaml id: research-committee name: Research Committee model: deepseek-v4-pro persona: | You are the committee's sole trading executor. Advisor opinions are evidence, not instructions: verify them independently, respect position and window limits, and you are the only role allowed to call place_order. advisors: - role: fundamentals model: deepseek-v4-pro prompt: Evaluate quality, valuation, and reporting risk. - role: technicals model: deepseek-v4-pro prompt: Evaluate trend, volume, and invalidation levels. - role: risk model: deepseek-v4-pro prompt: Challenge concentration and downside assumptions. ``` This matches the loader's real top-level `advisors:` schema — no nested `committee:` block, no invented fields. See [`examples/tradingagents_committee.yaml`](examples/tradingagents_committee.yaml) for the full six-role reference and the [committee design](docs/design/multi-role-agent.md). ## Data + Architecture

TraderHarness data, engine, agent, trajectory, and evaluation architecture

Core invariants: - **Zero I/O during a backtest:** after startup, market access is an in-memory lookup. - **One order path:** every fill goes through `TradingBus.place_order()`. - **Deterministic environment:** the same visible data and action sequence produce the same result. - **Portfolio ownership stays in the environment:** agents only receive views. - **Unadjusted prices:** fills, fees, price limits, and corporate actions use historical raw prices. The canonical release contains: ```text ~/.traderharness/dataset/ ├── daily.parquet ├── 5min_clean/ ├── announcements.parquet ├── news_cls.parquet ├── fundamentals.parquet ├── valuation.parquet ├── dividends.parquet ├── index_300.parquet └── metadata.json ``` Downloads are verified against a signed-style manifest of file sizes and SHA-256 checksums, then installed atomically. Incremental updates are idempotent: ```bash traderharness data update ``` Public news data retains templated headlines and omits licensed full text. Review upstream licenses and market-data terms before redistribution or commercial use. See [Data and licensing](docs/data.md) and [Core architecture](docs/architecture.md). ## Build with us TraderHarness is easiest to extend at specific, load-bearing seams: data source adapters, tools, sandbox backends, evaluation metrics, a live broker adapter, and the research console frontend. Each has a short contract in [Extending TraderHarness](docs/extensions.md); the process and required checks are in [`CONTRIBUTING.md`](CONTRIBUTING.md). ```bash git clone https://github.com/HephaestLab/TraderHarness cd TraderHarness python -m venv .venv .venv/Scripts/python -m pip install -e ".[all]" .venv/Scripts/python -m pytest tests/ --no-header -q .venv/Scripts/python -m ruff check traderharness cd webui npm ci npm test npm run build npm run test:e2e ``` Linux and macOS users should replace `.venv/Scripts/python` with `.venv/bin/python`. ## Roadmap A short version — see [`docs/roadmap.md`](docs/roadmap.md) for the full picture, including explicit non-goals: | Status | Item | |---|---| | ✅ Delivered in v1.0 | Backtest engine, masking, replay/trajectory export, comparison, committees, research console | | 🚧 Next | Paper trading (simulated live-forward mode on the same engine and masking contract) | | 📋 Planned | Live broker adapter | | 📋 Planned | A hardened sandbox for untrusted or third-party agent cards | | ❌ Non-goal | A public leaderboard, hosted multi-tenant service, or market-impact modeling | ## Scope and limitations - The maintained production dataset currently covers China A-shares; adapters for other markets are welcome. - There is no paper-trading or live-broker mode yet; see [Roadmap](docs/roadmap.md). Historical simulation cannot model market impact or guarantee live performance. - Provider terms may restrict redistribution of some source fields. - TraderHarness is research infrastructure, not investment advice or a brokerage system. ## Star History GitHub stars — click for Star History ## Citation If TraderHarness supports your research, please cite it. A machine-readable record is in [`CITATION.cff`](CITATION.cff) — GitHub's "Cite this repository" button reads it directly: ```bibtex @software{traderharness2026, title = {TraderHarness: A Contamination-Resistant Environment for Autonomous Trading Agents}, author = {{HephaestLab}}, year = {2026}, url = {https://github.com/HephaestLab/TraderHarness}, version = {1.0.0} } ``` ## License Apache-2.0 © HephaestLab. See [`LICENSE`](LICENSE). --- # TraderHarness

面向 LLM 交易 Agent 的抗污染回测环境——同时也是一个构建在每次运行之上的训练数据合成器。
真实 A 股数据 · 严格时点掩码 · 指纹回放 → 轨迹导出

English · CI · PyPI · Research Beta · 文档站 · Apache-2.0

LLM Agent 正在走进真实交易：研究、决策、下单，越来越多地由模型端到端完成，而不是人工编写的策略。 但"给 Agent 做回测"至今没有规范化的框架。通用模型可能认得被测的日期、公司，甚至那段历史走势本身——这种数据泄漏会悄悄让评测失效。成交口径五花八门，跑出来的结果无法复现，结论自然难以采信。 TraderHarness 就是这套规范：一个抗污染的 LLM 交易 Agent 回测执行环境——同时，由于每次模型调用都被全保真记录，它也是一个把这些运行一键变成轨迹训练数据的数据合成器。 **先看证据：** [Agent 炒股如何回测？完整中文指南](https://hephaestlab.github.io/TraderHarness/zh/guides/agent-stock-backtesting/) · [零凭据体验 Masked vs Unmasked 实验](https://hephaestlab.github.io/TraderHarness/zh/masking-ab-showcase/) · [三步运行免 Key 回放](#快速开始) · [看它和 TradingAgents / StockBench / Qlib 的边界](docs/comparison.md) · [English documentation](https://hephaestlab.github.io/TraderHarness/)

TraderHarness 流式历史回放控制台

该 GIF 由本地研究控制台截图生成（webui/scripts/capture-demo.mjs）。UI 有变化时执行 npm run capture:demo 重新生成；将随 v1.0 验收跑一并更新。

> 如果你也在解决 LLM 交易评测里的数据泄漏、成交公平或结果复现问题，欢迎在仓库右上角点 **Star**。后续公开 benchmark、回放盒带和数据集更新都会在这里发布。 ## Agent 炒股如何回测 可信的 AI / LLM 股票 Agent 回测，需要用历史时钟限制日线、分钟线、公告、新闻和基本面的每一个出口；Agent 只能通过受控工具研究和下单，决策后才逐步揭示可成交价格，环境负责账户、撮合和风控，并完整记录模型请求、工具调用、订单与成交。仅把一段 K 线交给大模型、再询问“该买还是该卖”，无法排除未来数据、模型历史记忆和成交偏差。 完整方法、A 股实例和可运行命令见[《Agent 炒股如何回测？AI / LLM 股票交易 Agent 回测指南》](docs/guides/agent-stock-backtesting.md)。 ## 魔鬼行情实测：21 个交易日小基准 2026 年 6 月 20 日 – 7 月 20 日，A 股经历一轮急速回撤，沪深 300 区间收益 **-9.12%**。我们在这个"魔鬼窗口"里用同一张「趋势波段」Agent 卡（日期与实体双重遮罩、种子 42、初始资金 100 万）实测了 4 个主流模型——每个模型独立跑完 21 个交易日，看不到真实日期与公司名，只能靠行情结构、公告和基本面做判断： | 模型 | 区间收益 | 相对沪深 300 | 夏普比率 | 最大回撤 | 交易笔数 | 胜率 | |---|---|---|---|---|---|---| | Claude Opus 4.8 | **+1.45%** | +10.57 | 2.61 | 0.76% | 6 | 66.7% | | Kimi K2.6 | +0.34% | +9.46 | 0.69 | 0.37% | 2 | 100% | | Qwen3.7-Max | -1.64% | +7.48 | -3.77 | 1.64% | 12 | 0% | | DeepSeek V4-Pro | -2.69% | +6.43 | -6.57 | 2.69% | 10 | 20.0% | 四个模型全部大幅跑赢基准。更有意思的是风格差异：Kimi 全月只出手 2 次、Qwen 出手 12 次却无一盈利——同样的市场、同样的工具、同样的提示词，模型性格一目了然。这正是规范化回测框架要回答的问题。 > 这是一组小样本、单 Agent 风格的冒烟基准，用于演示评测链路，不构成模型排名或投资建议。每一次调用都可指纹回放，完整契约见[训练数据](docs/training-data.md)。 ## 规范化回测框架 TraderHarness 交给模型一个严格对齐历史时点的市场、一套研究工具和一个账户。模型可以自主检索、写分析代码、修正判断，并通过唯一受控入口下单——环境里没有实盘市场，也没有券商接口。首个生产数据集覆盖中国 A 股：五年全市场日线与 5 分钟线、公告、政策快讯、基本面、估值、分红以及沪深 300 基准。 - **构造上抗污染。** 每个数据出口强制执行时点掩码，外加确定性的日期与公司匿名化——Agent 无法认出被测的时间段和股票。 - **公平成交。** 5 分钟行情渐进可见、分钟级撮合；每一笔成交都经过唯一路径 `TradingBus.place_order()`，按不复权的历史真实价格结算。 - **确定性。** 预载完成后零数据 I/O——相同的可见数据与动作序列，必然得到相同的环境结果。 ## LLM 交易数据合成器 每一次回测同时就是一次数据生产，链路分三步： 1. **全保真轨迹。** 每一次 LLM 调用都会持久化完整消息列表、完整工具 schema、模型回答与推理内容、每次工具调用及其参数，以及阶段/子窗口元数据。 2. **指纹回放。** 生成的 cassette 可以在没有 API Key 的情况下确定性重放；分享之前，`traderharness audit` 会扫描序列化产物中泄漏的实体与日期。 3. **训练轨迹。** 全保真轨迹可用于强化学习、行为分析与评测；内置的 `traderharness export sft` 还可转换为 OpenAI 风格 JSONL，并由掩码与泄漏检查把关。 ```bash traderharness audit result.json replay.jsonl traderharness export sft result.json --output training.jsonl ``` 研究控制台会把同一份证据渲染成逐笔复盘档案：成交时刻的 5 分钟 K 线、Agent 当时的推理陈述，以及产生这笔成交的具体工具调用。

TraderHarness 回测结果研究工作台：基准、回撤、K 线、成交与决策证据

完整契约见[训练数据](docs/training-data.md)；经过筛选的导出数据发布在 Hugging Face 的 [traderharness-ashare-5y 数据集](https://huggingface.co/datasets/ANTICH/traderharness-ashare-5y)。 ## 研究控制台 内置的本地控制台（FastAPI + React）让回测变成可以围观的现场：像素风办公区实时直播回放过程，绩效、逐笔档案和跨 run 对比在同一个窗口里联动更新。它是本地研究工具——请保持只监听 localhost，不要暴露到公网。

TraderHarness 实时控制室正在直播历史回放

像素办公区特写 运行中的实时绩效面板
像素办公区。每个 Agent 是工位上的一个角色；回放时钟、阶段和工具活动实时流动。 实时绩效。净值曲线、基准、回撤和持仓随交易日推进同步更新。
逐笔复盘：成交 K 线与 Agent 推理 跨 run 对比视图
逐笔复盘。每笔成交都能回溯到对应的 5 分钟 K 线、Agent 当时的推理和触发它的工具调用。 跨 run 对比。在同一份掩码数据上给不同 Agent 或不同检查点排名。
## 为什么不是普通 Agent Demo | 能力 | 常见 Demo | TraderHarness | |---|---:|---:| | 基本面与新闻时点对齐 | 局部 | 每个数据出口强制执行 | | 日期与公司匿名化 | 无 | 确定性实体/日期遮罩 | | 盘中公平成交 | 常用收盘价 | 5 分钟可见性 + 分钟级撮合 | | LLM 结果复现 | 仅日志 | 请求指纹校验的 replay | | 训练轨迹 | 经常截断 | 请求、推理、工具、结果全保真 | | 多 Agent | 共享聊天模拟 | 隔离组合对比 + 单执行者委员会 | | 回测期间数据 I/O | 常见 | 启动预载后零 I/O | | 历史回放 | 临时脚本拼接 | 流式、按阶段渐进披露的历史回放——绝非实时行情 | TraderHarness 负责历史环境、信息边界、公平撮合和证据产物，不限定交易方法论。与 TradingAgents、StockBench、Qlib 等项目的完整边界对比见[项目对比](docs/comparison.md)。 ## 四智能体展示阵容 TraderHarness 在 `traderharness/agents/builtin/` 下内置四个人设清晰、互不重复的参考 Agent 卡片： | Agent id | 风格 | 风险画像 | 持仓周期 | |---|---|---|---| | `trend-breakout` | 量价突破、相对强度、成交量确认、机械止损 | 进取 | 3–10 个交易日 | | `quality-compounder` | 盈利质量、资产负债表与估值纪律，低换手 | 保守 | 20–60 个交易日 | | `event-hawk` | 公告/政策/新闻催化，重视时间戳与来源核验 | 进取 | 1–5 个交易日 | | `quant-researcher` | 沙箱内可复现的横截面因子检验 | 均衡 | 2–20 个交易日 | 验收参考跑是这四个 Agent 在 **2024-03-04 → 2024-03-29** 区间的正面对比，执行模型为 `deepseek-v4-pro` 的 thinking（深度推理）模式，全程开启实体遮罩： ```powershell $env:DEEPSEEK_API_KEY="..." traderharness compare ` --agent trend-breakout ` --agent quality-compounder ` --agent event-hawk ` --agent quant-researcher ` --start 2024-03-04 ` --end 2024-03-29 ` --mask-entities ` --entity-mask-seed 42 ` --record-replay showcase_mar2024 ` --output showcase_mar2024/comparison.html ``` `--record-replay` 会保存一份带指纹校验的 cassette，可以在没有 API Key 的情况下确定性重放、审计和分享。参考[快速开始](#快速开始)和[训练数据](docs/training-data.md)。 **验收结果**（2024-03-04 → 2024-03-29，实体遮罩 seed `42`，`deepseek-v4-pro` thinking high）。 按夏普排序；该跑已通过 `traderharness audit`（零发现），可用 `--replay showcase_mar2024` 无 Key 复现： | Agent | 累计收益 | 年化收益 | 夏普比率 | 最大回撤 | 胜率 | 成交次数 | |---|---:|---:|---:|---:|---:|---:| | `quality-compounder` | +0.52% | +7.92% | 1.09 | 0.76% | 33% | 9 | | `trend-breakout` | +0.09% | +1.32% | −0.20 | 1.07% | 38% | 16 | | `event-hawk` | −0.70% | −9.74% | −3.22 | 0.93% | 0% | 5 | | `quant-researcher` | −1.33% | −17.75% | −4.19 | 1.73% | 40% | 19 | | 沪深 300（基准） | −0.10% | — | — | — | — | — | ## 双重遮罩 数据泄漏是系统问题，不能只靠 prompt 约束。 - **时序遮罩**：日线严格满足 `date < current_date`；分钟线只暴露已发生子窗口；基本面严格满足 `pub_date <= current_date`。 - **日历遮罩**：Agent 看到的日历日期变为 `D-1`、`D+0` 等相对偏移。 - **实体遮罩**：股票代码和公司别名映射为确定性中性实体，同时保留创业板/科创板等交易规则（如 20% 涨跌停）。 - **输出清洗**：模型回答、推理、工具参数、委员会备忘、轨迹和 replay 全部经过同一套遮罩。 - **泄漏审计**：发布或导出轨迹前，可对 JSON、JSONL、Parquet 进行扫描。

历史公告经过日期与实体双重遮罩后的 Agent 视图

v1.0 发布验收已对完整的一月 DeepSeek 轨迹做序列化后出口审计，未检出真实公司别名或绝对日历日期。这里指自动化出口审计结果，不代表语义再识别风险为零——参见[抗污染机制](docs/contamination.md)。 ## 快速开始 支持 Python 3.10–3.12。三步上手，无需 API Key（首次运行需要下载并校验完整数据集）： ```bash pip install "traderharness[llm,data,ui]" traderharness data download --full traderharness demo ``` `traderharness demo` 使用真实市场数据回放一段已经遮罩的单日轨迹，不需要 API Key。这是对真实本机行情的**流式历史回放**，按阶段渐进披露——不是实时行情，也不会用模拟价格替代真实数据。 启动本地研究控制台： ```bash traderharness ui # 打开 http://127.0.0.1:8000 ``` 运行真实模型回测： ```powershell $env:DEEPSEEK_API_KEY="..." traderharness run ` --agent trend-breakout ` --start 2024-03-04 ` --end 2024-03-29 ` --mask-entities ` --record-replay run.jsonl ``` 也可以用容器启动： ```bash docker compose up --build ``` Compose 会把本机 `~/.traderharness` 挂载到 `/data`，不会把完整数据集烘焙进镜像。打开 `http://127.0.0.1:8000`，内置回放不需要 API Key。 ### 每个交易日 每个交易日分三个有边界的阶段： 1. **盘前研究**：账户、昨日市场宽度、板块、自选股、公告和政策；禁止下单。 2. **开盘窗口**：09:30–10:00 的 5 分钟线按子窗口逐步可见；允许下单。 3. **尾盘窗口**：14:30–15:00 的数据逐步可见；允许下单并结束当天。 Agent 可查询历史 K 线、基本面、估值、公告和新闻，进行全市场筛选，在沙箱里自由使用 `numpy/pandas/scipy`，并查看只读账户。所有成交都必须经过 `TradingBus.place_order()`。 ## 面向 AI 的入口 TraderHarness 提供机器可读的操作指南，让编程助手不用猜测上面这些不变量： - [`llms.txt`](llms.txt) —— 官方文档的精简索引。 - [`llms-full.txt`](llms-full.txt) —— 同一份文档集合合并成一个文件，由 `scripts/build_llms_full.py` 生成。 - [`AGENTS.md`](AGENTS.md) —— 公开操作指南：项目地图、不可协商的不变量与工作流程。 推荐先给 Cursor、Claude Code 等工具以下指令： ```text 先阅读 AGENTS.md 和 docs/architecture.md。新增 Agent 时不得绕过 TradingBus、时点遮罩和单一下单路径；先写失败测试，再运行相关测试和 replay demo。 ``` ## 独立赛马 vs 单执行者委员会 `compare` 和委员会是建立在同一个遮罩市场和同一条下单路径上的两种不同产品。 ### 独立组合对比 `compare` 让每个 Agent 拥有独立账户，在相同市场时钟和初始资金下并行运行，再按收益、风险和行为指标排名。[四智能体展示阵容](#四智能体展示阵容)就是用这个命令跑的。 ```bash traderharness compare \ --agent trend-breakout \ --agent quality-compounder \ --start 2024-03-04 \ --end 2024-03-29 \ --mask-entities \ --output comparison ```

TraderHarness 对比工作台：独立账户 Agent 排名

### TradingAgents 风格委员会 委员会是"一个交易 Agent + 多个只读顾问 + 一个 Trader 执行者"。顾问并发研究，但只有 Trader 能获得下单工具，保证一个账户只有一条可追责的订单路径。 ```yaml id: research-committee name: Research Committee model: deepseek-v4-pro persona: | 你是委员会的唯一交易执行者。顾问意见只是证据输入；你必须独立核验， 遵守仓位与交易窗口约束，并且只有你可以调用 place_order。 advisors: - role: fundamentals model: deepseek-v4-pro prompt: 关注盈利质量、估值、资产负债表和公告中的基本面变化。 - role: technicals model: deepseek-v4-pro prompt: 关注趋势、量价、波动、支撑阻力和信号失效条件。 - role: risk model: deepseek-v4-pro prompt: 从组合暴露、仓位集中、成交约束和尾部风险提出否决意见。 ``` 这份示例严格对应加载器真实的顶层 `advisors:` schema，没有嵌套的 `committee:` 结构，也没有虚构字段。完整六角色参考见 [`examples/tradingagents_committee.yaml`](examples/tradingagents_committee.yaml) 和[设计文档](docs/design/multi-role-agent.md)。 ## 数据与架构

TraderHarness 数据、引擎、Agent、轨迹和评估架构

核心不变量：回测期间零 I/O、单一下单路径（`TradingBus.place_order()`）、环境确定性、Agent 不持有 Portfolio、使用不复权真实价格。 全量数据下载会校验文件大小和 SHA-256，再原子安装到： ```text ~/.traderharness/dataset/ ├── daily.parquet ├── 5min_clean/ ├── announcements.parquet ├── news_cls.parquet ├── fundamentals.parquet ├── valuation.parquet ├── dividends.parquet ├── index_300.parquet └── metadata.json ``` 增量更新幂等： ```bash traderharness data update ``` 公开新闻数据保留模板化标题，去除有版权的正文。数据再分发和商业使用前请确认上游许可。详见[数据与许可](docs/data.md)和[核心架构](docs/architecture.md)。 ## 和我们一起建设 TraderHarness 最值得扩展的几个接缝是：数据源适配器、工具、沙箱后端、评测指标、实盘 Broker adapter，以及研究控制台前端。每一个都在[扩展指南](docs/extensions.md)里有简短契约；流程和必跑检查见 [`CONTRIBUTING.md`](CONTRIBUTING.md)。 ```bash .venv\Scripts\python.exe -m pip install -e ".[all]" .venv\Scripts\python.exe -m pytest tests/ --no-header -q .venv\Scripts\python.exe -m ruff check traderharness cd webui npm ci npm test npm run build npm run test:e2e ``` ## 路线图 完整版本（含明确的非目标）见 [`docs/roadmap.md`](docs/roadmap.md)： | 状态 | 事项 | |---|---| | ✅ v1.0 已交付 | 回测引擎、遮罩、replay/轨迹导出、多 Agent 对比、委员会、研究控制台 | | 🚧 下一阶段 | 模拟盘（Paper Trading，复用同一引擎和遮罩契约的前向仿真模式） | | 📋 规划中 | 实盘 Broker adapter | | 📋 规划中 | 面向不可信/第三方 Agent 卡片的强化沙盒 | | ❌ 非目标 | 公开排行榜、多租户托管服务、市场冲击建模 | ## 边界与限制 - 目前维护的生产数据集覆盖中国 A 股；欢迎为其他市场提供适配器。 - 目前没有模拟盘或实盘接口，参见[路线图](docs/roadmap.md)。历史仿真不模拟市场冲击，也不保证实盘表现。 - 部分上游字段的再分发可能受供应商条款限制。 - TraderHarness 是研究基础设施，不是投资建议，也不是券商系统。 ## Star 历史 GitHub stars — 点击查看 Star History ## 引用 如果 TraderHarness 对你的研究有帮助，请引用。机器可读记录见 [`CITATION.cff`](CITATION.cff)——GitHub 的 "Cite this repository" 按钮会直接读取它： ```bibtex @software{traderharness2026, title = {TraderHarness: A Contamination-Resistant Environment for Autonomous Trading Agents}, author = {{HephaestLab}}, year = {2026}, url = {https://github.com/HephaestLab/TraderHarness}, version = {1.0.0} } ``` ## 许可证 Apache-2.0 © HephaestLab。详见 [`LICENSE`](LICENSE)。 --- --- seo_title: TraderHarness Quickstart — Replay an LLM Trading Agent in Three Steps description: Install TraderHarness, download five years of China A-share data, run a no-key replay, open the local console, and compare isolated LLM trading agents. lang: en --- # Quickstart ## Install === "pip" ```bash pip install "traderharness[llm,data,ui]" ``` === "Source / Windows" ```powershell git clone https://github.com/HephaestLab/TraderHarness cd TraderHarness python -m venv .venv .venv\Scripts\python.exe -m pip install -e ".[all]" ``` === "Docker" ```bash docker compose up --build ``` ## Install market data ```bash traderharness data download --full ``` The installer verifies file size and SHA-256 against the release manifest before atomically replacing `~/.traderharness/dataset`. ## Run the no-key replay ```bash traderharness demo ``` The cassette contains a recorded masked LLM trajectory. No API key is required; the engine still re-evaluates it against local canonical market data. ## Open the research console ```bash traderharness ui ``` Open [http://127.0.0.1:8000](http://127.0.0.1:8000). The service binds to loopback by default and rejects accidental public exposure unless explicitly enabled. ![TraderHarness live control room](https://hephaestlab.github.io/TraderHarness/assets/live-control-room.png) ## Run a fresh model agent ```powershell $env:DEEPSEEK_API_KEY="..." traderharness run ` --agent trend-breakout ` --start 2024-03-04 ` --end 2024-03-29 ` --mask-entities ``` Compare the four built-in reference cards under one market clock and isolated portfolios: ```powershell traderharness compare ` --agent trend-breakout ` --agent quality-compounder ` --agent event-hawk ` --agent quant-researcher ` --start 2024-03-04 ` --end 2024-03-29 ` --mask-entities ` --output showcase ``` Add `--record-replay cassette.jsonl` to save a deterministic, leakage-auditable replay cassette. --- --- title: How to Backtest LLM Trading Agents seo_title: How to Backtest LLM Trading Agents Without Look-Ahead Bias | TraderHarness description: A practical guide to point-in-time data, model-memory contamination, fair execution, deterministic replay, and auditable LLM trading-agent evaluation. author: HephaestLab lang: en schema_type: TechArticle datePublished: 2026-07-29 dateModified: 2026-07-29 faq: - question: Can an LLM trading backtest be deterministic? answer: >- Fresh hosted-model generation is not guaranteed to be deterministic, but complete exchanges can be recorded and replayed against canonical request fingerprints while matching, accounting, and metrics remain deterministic. - question: Does point-in-time data eliminate all backtest bias? answer: >- No. It controls information timing, but survivorship, costs, market impact, missing data, multiple testing, model selection, and historical memorization still require explicit treatment. - question: Why mask company names as well as dates? answer: >- A ticker, company, product, or announcement can reveal a historical event even when its date is hidden. Entity masking reduces that memory channel while preserving market rules needed for simulation. - question: Is TraderHarness a live trading bot? answer: >- No. TraderHarness is local research infrastructure for historical evaluation and trajectory generation. It does not route live orders or claim that historical performance predicts future returns. --- # How to backtest LLM trading agents without look-ahead bias **Short answer:** place the agent inside a market environment that advances on a historical clock. The agent may read only information that was public at that moment and may research or trade only through controlled tools. The environment owns matching, accounting, and risk controls and records the complete path from model request to final fill. [TraderHarness](https://github.com/HephaestLab/TraderHarness) implements that contract for China A-shares. [Run the no-key replay](../quickstart.md){ .md-button .md-button--primary } [Open the masked experiment](../masking-ab-showcase.md){ .md-button } !!! warning "A backtest is not investment advice" Historical simulation cannot establish future profitability. TraderHarness does not connect to a live brokerage account, provide copy trading, or promise returns. ## The three leakage problems ### 1. Ordinary look-ahead bias A decision at time *t* must not receive a closing price, filing, revised fundamental, or news item that became available after *t*. Calendar-date filtering alone is insufficient when sources have different publication timestamps and revision histories. Every data outlet must enforce the simulated clock: daily bars end before the current day, intraday bars stop at the current sub-window, announcements and news satisfy publication-time constraints, and fundamentals are selected by publication date rather than reporting period alone. ### 2. Historical memorization by the model A general-purpose LLM may recognize a famous company, date, crash, rally, or announcement from training data. It can then answer from memory even when the data pipeline is point-in-time correct. TraderHarness can replace absolute dates with relative offsets such as `D+0` and map companies to deterministic neutral identities. This is a contamination control, not proof that semantic re-identification is impossible; public artifacts still need a leakage audit. ### 3. Non-reproducible model calls Hosted models can change even at temperature zero. Saving only the final trade is therefore inadequate. A useful replay artifact retains messages, tool schemas, model response fields, tool calls, tool results, provider configuration, phase, and sub-window. TraderHarness fingerprints each canonical request. Replay fails closed when a prompt, tool schema, visible input, or call sequence differs. ## A minimum credible evaluation contract | Boundary | Minimum requirement | Evidence to retain | |---|---|---| | Market data | Filter every outlet by the simulated clock | Source time, simulated time, visible rows | | Identity | Mask dates and entities when contamination matters | Policy, seed, leakage audit | | Execution | Reveal fillable prices only after the decision | Order time, visible window, fill time and price | | Portfolio | Give the agent a read-only view | Orders, fills, cash, positions, corporate actions | | Model | Record the complete request/tool loop | Model ID, provider, request fingerprint | | Comparison | Use one clock and isolated accounts | Configuration, seed, benchmark series | | Publication | Audit every shared artifact | Code revision, data manifest, audit report | The core rule is: **the same visible data and action sequence must produce the same environment result.** Fresh model generation can be recorded and fingerprint-replayed; matching, accounting, and metrics must remain deterministic. ## A practical workflow 1. Define an agent card with its model, persona, allowed tools, risk limits, and three trading phases. 2. Prepare unadjusted prices and publication-time-aware daily, intraday, announcement, news, and fundamental data. 3. Run masked and unmasked versions under the same clock, capital, tools, and execution rules. 4. Route every order through the environment’s single matching path. 5. Record the complete model/tool trajectory and replay it by request fingerprint. 6. Report behavior and risk, not just return. ```bash pip install "traderharness[llm,data,ui]" traderharness data download --full traderharness demo ``` Record and audit a new run: ```bash traderharness run \ --agent trend-breakout \ --start 2024-03-04 \ --end 2024-03-29 \ --mask-entities \ --record-replay run.jsonl traderharness audit result.json run.jsonl ``` ## What results should report - period return, CSI 300 relative return, maximum drawdown, and risk-adjusted metrics; - orders, fills, rejections, inactive days, turnover, exposure, and concentration; - date range, universe, capital, fees, execution rules, and masking configuration; - model and provider, agent card, data version, code revision, and artifact audit; - omitted effects such as market impact, live latency, and provider drift. ## Frequently asked questions ### Can an LLM trading backtest be deterministic? Fresh generation usually cannot be guaranteed bit-for-bit, but its complete interaction can be recorded and replayed against request fingerprints. The market environment’s matching and accounting remain deterministic. ### Does point-in-time data eliminate all backtest bias? No. It controls information timing, but not survivorship, costs, market impact, missing data, multiple testing, model selection, or historical memorization. ### Why mask company names as well as dates? A ticker, product, executive, or distinctive announcement can identify a historical event. Entity masking reduces those cues while retaining the board and market metadata required for correct trading rules. ### Is TraderHarness a live trading bot? No. It is local historical research infrastructure and does not route live orders. ## Next steps - Follow the [quickstart](../quickstart.md). - Read the [contamination controls](../contamination.md). - Compare TraderHarness with [TradingAgents, StockBench, and Qlib](../comparison.md). - Inspect the implementation and tests on [GitHub](https://github.com/HephaestLab/TraderHarness). --- --- seo_title: Preventing Data Leakage and Historical Memorization in LLM Trading Backtests description: How TraderHarness enforces point-in-time masking, relative dates, company pseudonyms, progressive intraday visibility, and artifact leakage audits. lang: en --- # Backtest LLM trading agents without data leakage TraderHarness treats historical contamination as an environment boundary, not a promise in a prompt. ## Core controls **Point-in-time masking:** every value exposed to an agent is filtered against the simulated clock. Daily bars end before the trading day, 5-minute bars stop at the current sub-window, and fundamentals must have been published by the current date. **Date anonymization:** absolute calendar dates become offsets relative to the simulated present. Today is `D+0`; the previous calendar day is `D-1`. Intraday times remain visible. **Entity masking:** real A-share codes and known aliases are mapped through a deterministic run-wide bijection to neutral identities. Codes are shuffled inside compatible board groups so historical price-limit rules still apply. **Three-phase trading:** pre-market research cannot trade; 09:30–10:00 and 14:30–15:00 windows reveal 5-minute data progressively. ## Egress coverage {#egress-coverage} Masking applies to: - daily and intraday bars; - screening, fundamentals, valuation, announcements, and policy news; - portfolio and watchlist views; - DataFrames returned inside the Python sandbox; - model responses, reasoning fields, tool arguments, and committee memos; - cross-day memory, saved trajectories, replay cassettes, comparisons, and exports. Agent-visible pseudocodes are resolved internally before matching. The portfolio is rendered through the same forward mapping, so an agent never needs the real code. ![Date and entity masking](https://hephaestlab.github.io/TraderHarness/assets/dual-mask.svg) ## Artifact audit ```bash traderharness audit result.json replay.jsonl export.parquet ``` The auditor checks known company aliases, six-digit A-share codes, absolute ISO and Chinese dates, and month-day forms. Passing this lexical contract does not prove that a model cannot infer a company from distinctive products, executives, financial patterns, or events. Public results should disclose masking configuration and seed and compare masked with unmasked runs when semantic re-identification matters. ## Execution leakage Information masking is meaningless if a model can inspect a complete intraday window and then choose an earlier favorable fill. TraderHarness reveals one sub-window, requests a decision, and only then considers eligible prices. Every order passes through `TradingBus.place_order()`. --- --- seo_title: TraderHarness Architecture — LLM Agent Backtesting, Matching, and Replay description: TraderHarness architecture for point-in-time data, three-phase agent loops, one matching path, read-only portfolios, complete trajectories, and fingerprinted replay. lang: en --- # Architecture ![TraderHarness architecture](https://hephaestlab.github.io/TraderHarness/assets/architecture.svg) ```mermaid flowchart TD D[(Canonical market data)] -->|preload| E[BacktestEngine] E --> B[TradingBus per agent] B --> T[Masked tools] B --> V[PortfolioView] B --> X[Python sandbox] T --> A[Agent loop] V --> A X --> A A -->|place_order| B E --> C[Trajectory collector] C --> R[Replay / RL export / reports] ``` ## Non-negotiable invariants ### No backtest-time I/O The engine preloads every required market slice before the first trading day. Agent tools query memory only and cannot fetch provider data or read the canonical dataset directly. ### Strict historical visibility Daily bars use `date < current_date`; fundamentals use `pub_date <= current_date`; intraday bars stop at the current phase and sub-window; agent-facing absolute dates become relative offsets. ### One order path `TradingBus.place_order()` enforces board lots, suspension, cash, holdings, price limits, fees, and visible-price checks. Agents, committees, and sandbox code have no second path. ### Environment-owned portfolios Agents receive read-only views. Only validated orders change state; dividends, corporate actions, and daily equity are deterministic engine operations. ## Replay contract Each recorded LLM request has a canonical SHA-256 fingerprint. Replay rejects modified requests and exhausted cassettes and never falls back to a network model. Regression failures remain explicit instead of becoming silently nondeterministic. --- --- seo_title: TraderHarness vs TradingAgents, StockBench, Qlib, and Traditional Backtesters description: Compare agent orchestration, historical market simulation, contamination controls, execution, replay, and trajectory evidence across adjacent projects. lang: en --- # Comparison with adjacent projects TraderHarness is environment and evidence infrastructure. It can host different agent architectures without prescribing an investment methodology. | Project | Primary responsibility | Native decision unit | What integration still needs | |---|---|---|---| | **TraderHarness** | Historically valid market, execution, portfolio, evaluation, replay, trajectories | Autonomous agent, isolated comparison, or single-executor committee | An agent card or external framework | | **TradingAgents** | Multi-role analysts, debate, risk, and trader workflow | Prescribed role graph | A strict benchmark market and order contract | | **StockBench** | Standardized stock-reasoning benchmark tasks | Benchmark task or prediction | Persistent portfolio and autonomous tool loop | | **Qlib** | Quant data, models, experiments, and strategies | ML model or code strategy | LLM-native tools and language-output contamination controls | | **Backtrader / vn.py** | Strategy execution and trading infrastructure | Code strategy | Autonomous LLM research loop, masking, and trajectory contract | ## Independent agents versus a committee `traderharness compare` is a race: every agent has its own cash, positions, memory, and portfolio. All agents share a market clock and are scored independently. A committee is one competitor: read-only advisors may research, but exactly one trader owns the order tool and one portfolio. See [multi-role committees](design/multi-role-agent.md). ## Integrate your framework An external LangGraph, TradingAgents, or custom orchestrator should implement the public agent protocol and return a final decision. Market reads and orders still pass through TraderHarness, inheriting the same clock, masks, progressive intraday visibility, and matching rules. --- --- seo_title: Backtesting Multi-role LLM Trading Agents with Read-only Advisors description: TraderHarness committee design keeps research and risk advisors read-only, gives one trader the order tool, and preserves deterministic replay and portfolio accountability. lang: en --- # Multi-role agent adapter ## Goal Support TradingAgents-style committees without changing TraderHarness market, portfolio, matching, masking, or replay semantics. A committee is one evaluated agent with one portfolio. Specialist roles advise; only the executor may call `place_order`. This differs from `traderharness compare`, where independent agents own separate portfolios and compete with each other. ## Model ```text BacktestEngine (shared immutable market snapshot) └─ CommitteeAgent (one Portfolio + one TradingBus) ├─ fundamentals advisor ───┐ ├─ technical advisor ──────┼─> phase memo ├─ news advisor ───────────┤ ├─ bull researcher ────────┤ ├─ bear researcher ────────┘ └─ trader/executor -> AgentLoop -> TradingBus.place_order() ``` ## Invariants 1. Advisors receive only masked, agent-visible messages. 2. Advisors never receive an order tool. 3. The executor uses the existing `ToolRegistry`; `TradingBus.place_order()` remains the only matching path. 4. Roles, phases, prompts, responses, models, and order are recorded for deterministic replay. 5. Every role shares one run-level `EntityMasker`. 6. Advisor failures remain visible in the memo and trajectory. ## Extension surface ```python class Advisor(Protocol): role: str async def advise(self, messages: list[dict], phase: str) -> str: ... class CommitteeCoordinator: async def build_memo( self, messages: list[dict], phase: str, sub_window: str | None, ) -> CommitteeMemo: ... ``` Advisors run concurrently with `asyncio.gather`; their labeled memo is injected before the first executor call for each `(day, phase, sub_window)`. The executor may accept or reject each recommendation. ## TradingAgents mapping - market, news, and fundamental analysts become specialist advisors; - bull and bear researchers become adversarial advisors; - research and risk managers become synthesis advisors; - Trader becomes the TraderHarness executor. External portfolio and tool objects are not imported. The adapter keeps the reasoning topology while replacing execution with masked observations and the controlled order path. ## Configuration ```yaml id: tradingagents-reference name: TradingAgents Reference Committee model: deepseek-chat persona: ... advisors: - role: fundamentals model: deepseek-chat prompt: ... - role: technical model: deepseek-chat prompt: ... - role: bull model: deepseek-chat prompt: ... - role: bear model: deepseek-chat prompt: ... ``` See the loadable [reference committee](https://github.com/HephaestLab/TraderHarness/blob/main/examples/tradingagents_committee.yaml). ## Acceptance criteria - advisors never receive `place_order`; - advisors are scheduled concurrently; - exactly one executor can trade; - replay reproduces the same memo and action sequence; - real-data runs pass leakage audits; - `compare` can rank committees and ordinary agents under identical data, capital, seed, and benchmark settings. --- --- seo_title: LLM Trading Agent Trajectories for Replay, RL, and Evaluation description: Capture complete messages, tool schemas, reasoning fields, actions, fills, and phase metadata for auditable replay, reinforcement learning, behavior analysis, and optional SFT export. lang: en --- # Full-fidelity trajectories for RL and evaluation TraderHarness persists every executor LLM exchange as a complete masked request/response pair. These trajectories support reproducible research, reinforcement learning, behavior analysis, and evaluation. An optional converter emits OpenAI-style SFT JSONL, but recorded decisions are not automatically high-quality training targets. ![Auditable trade review](https://hephaestlab.github.io/TraderHarness/assets/trade-review.png) ## Capture a trajectory ```bash traderharness run \ --agent trend-breakout \ --start 2024-03-04 \ --end 2024-03-29 \ --mask-entities ``` Every `llm_exchange` step contains: - the complete message list sent to the executor; - the complete tool schema available for that call; - assistant content and provider-exposed reasoning fields; - every tool call and argument; - phase and sub-window metadata. Assistant text is not truncated in newly generated results. Compatibility `assistant` and `tool_call` steps remain available. ## Optional SFT conversion ```bash traderharness export sft \ ~/.traderharness/results/_result.json \ --output ./training.jsonl ``` Each call becomes one JSONL row containing `messages`, `tools`, and metadata such as agent, phase, sub-window, day index, and call index. Absolute trading dates do not enter export metadata; agent-visible dates remain relative. ## Safety gates By default, export: 1. rejects runs without entity masking; 2. rejects legacy trajectories without full-fidelity exchanges; 3. scans output for entity and date leakage; 4. exits non-zero while any finding remains. `--allow-unmasked` is an explicit escape hatch for private research. Such output must not be published as contamination-resistant training data. ## Selection is still required Full fidelity preserves bad decisions as well as good ones. Before training, filter by outcomes, drawdown, rule compliance, tool failures, and human review. Confirm that provider terms allow generated reasoning and responses to be used for training. --- --- seo_title: TraderHarness China A-share Dataset — Daily, 5-minute, and Point-in-time Data description: Five years of full-market A-share daily and 5-minute bars, announcements, policy news, fundamentals, valuation, dividends, CSI 300, integrity checks, and licensing boundaries. lang: en --- # Data and licensing The canonical A-share release contains five years of full-market daily and 5-minute bars plus announcements, policy news, fundamentals, valuation, dividends, and a CSI 300 benchmark. ## Integrity `traderharness data download --full` verifies every object against the release manifest before atomically replacing the local dataset. `traderharness data update` uses watermarks, deterministic deduplication, and atomic writes. The data doctor checks required schemas and date ranges, natural-key duplicates, annual intraday coverage, stale symbols, non-A-share announcement codes, and metadata consistency. The v1.0 canonical build contains 284,219,844 deduplicated 5-minute records. Release audits reported complete annual symbol coverage for the active daily universe, no stale symbol at the final 5-minute watermark, and zero natural-key duplicates in the verification sample. ## Public release policy Public news tables retain templated titles and omit licensed full text. Company templates resolve to neutral identities only at runtime. This preserves a usable point-in-time dataset without redistributing restricted content. ## Storage layout ```text ~/.traderharness/dataset/ ├── daily.parquet ├── 5min_clean/ ├── announcements.parquet ├── news_cls.parquet ├── fundamentals.parquet ├── valuation.parquet ├── dividends.parquet ├── index_300.parquet └── metadata.json ``` Market-data licensing varies by provider and jurisdiction. Review upstream terms before redistribution or commercial deployment. --- --- seo_title: TraderHarness CLI and Local API Reference description: Commands and localhost APIs for data installation, agent backtests, comparison, replay, auditing, trajectory export, REST endpoints, and WebSocket events. lang: en --- # CLI and local API ## Core commands ```text traderharness run Run one agent traderharness compare Run isolated agents under one market clock traderharness demo Replay the bundled masked run without an API key traderharness ui Start the local FastAPI + React console traderharness audit Scan artifacts for entity and calendar leakage traderharness export Convert trajectories to optional SFT JSONL traderharness data Download, update, and inspect the dataset ``` Use `traderharness --help` for the authoritative options. ## Agent protocol Custom agents implement the public protocol in `traderharness.agents.protocol`. The environment supplies controlled contexts for pre-market research, the progressive open window, and the progressive close window. Read-only advisors can be composed behind one executor; see [multi-role committees](design/multi-role-agent.md). ## Local HTTP API `traderharness ui` exposes: - `GET /api/status` — dataset, provider, and local security status; - `GET/POST /api/agents` — agent card collection; - `GET/PUT/DELETE /api/agents/{id}` — one agent card; - `POST /api/runs` — start a backtest; - `GET/DELETE /api/runs/{id}` — inspect or cancel a run; - `WS /api/runs/{id}/events` — reconnectable ordered event stream; - `GET /api/results` — persisted result summaries; - `GET /api/results/{file}` — complete artifact; - `GET /api/results/{file}/analysis` — normalized research dossier; - `POST /api/demo` — start the bundled replay; - `GET /api/health` — process health. The HTTP API is a local research tool, not an authenticated public service. Keep its default localhost binding. --- --- seo_title: Extending TraderHarness — Data Providers, Agent Tools, and Metrics description: Contracts for adding providers, tools, sandbox capabilities, metrics, and frontend views without weakening point-in-time safety, the single order path, or deterministic execution. lang: en --- # Extending TraderHarness Extensions must preserve zero backtest-time I/O, one order path, strict point-in-time visibility, deterministic execution, and environment-owned portfolios. Review the [architecture](architecture.md), [`AGENTS.md`](https://github.com/HephaestLab/TraderHarness/blob/main/AGENTS.md), and [`CONTRIBUTING.md`](https://github.com/HephaestLab/TraderHarness/blob/main/CONTRIBUTING.md) before a large change. ## Data providers - Implement providers under `traderharness/data/providers/`; do not bypass `traderharness/data/datasets.py` or mutate tables used by a running backtest. - Give every record a stable natural key and every non-price record a publication timestamp that masking can enforce. - Extend data-doctor checks for required columns, date coverage, and duplicates. - Add a small real-data fixture and loader tests. Do not substitute synthetic prices in acceptance runs. ## Agent tools - Tool handlers receive the run’s masking context and may not read the canonical dataset or another agent’s state. - Return structured, actionable errors for invalid symbols, unavailable dates, suspensions, and ignored parameters. - Add JSON-schema validation and tests for each failure mode. - Never create a second order path. Trading always uses `TradingBus.place_order()`. ## Sandbox backends - Apply the existing path guard and wall-clock timeout. - Resolve new `traderharness_api` capabilities through masked accessors. - Never return unmasked frames or real entity codes. - Never start nested backtests or call the engine’s order path. ## Metrics Metrics under `traderharness/metrics/` are pure functions over completed daily equity, fills, and decisions. Document formulas and edge cases and add report/JSON tests. Cross-agent rankings belong in `metrics/comparison.py`. ## Broker adapters The current release has no live broker adapter. Historical matching and live authorization are separate trust boundaries and must not share `TradingBus`. ## Frontend The React console reads existing REST/WebSocket APIs and must not reconstruct unmasked fields client-side. New components need Vitest coverage; new flows need browser E2E coverage. ## Before a pull request 1. Add the smallest failing contract test. 2. Implement without weakening repository invariants. 3. Run focused tests, then the full suite and lint checks. 4. For engine, masking, tools, data, or sandbox changes, run a real replay/backtest and inspect the trajectory. 5. State exactly which real-data checks were and were not performed. --- --- seo_title: TraderHarness Roadmap — Agent Backtesting, Paper Trading, and Security Boundaries description: Delivered capabilities, planned forward paper trading, possible broker adapters, sandbox hardening, and explicit non-goals for TraderHarness. lang: en --- # Roadmap This page distinguishes delivered capability from planned work. It is not a date commitment. ## ✅ Delivered in v1.0 - Five years of full-market A-share data with atomic updates and integrity checks. - Zero market-data I/O after preload and one `TradingBus.place_order()` path. - Point-in-time masking across daily, intraday, news, announcements, fundamentals, and sandbox outlets. - Relative dates and deterministic company pseudonyms that preserve board semantics. - Pre-market, progressive open-window, and progressive close-window agent phases. - Isolated multi-agent comparison and a reference read-only-advisor committee with one executor. - Full-fidelity exchanges, fail-closed fingerprint replay, and trajectory export. - Serialized-artifact leakage auditing. - Local FastAPI/React console, non-root container, PyPI packaging, and CI. See the [changelog](https://github.com/HephaestLab/TraderHarness/blob/main/CHANGELOG.md) for release-level details. ## 🚧 Next: forward paper trading A simulated forward mode would reuse the current engine, masks, and tools while replacing full preload with a streaming feed. This feature is in design and **is not currently available**. It must retain the same order path and risk checks, apply masking to every agent-facing outlet, and prevent tools or sandbox code from reading beyond the simulated clock. ## 📋 Planned: broker adapter {#live-broker-adapter} A future adapter boundary may connect a research agent to a real brokerage API, after paper trading exists and a credential, authorization, and order-risk threat model matches the project’s [security policy](https://github.com/HephaestLab/TraderHarness/blob/main/SECURITY.md). There is no broker integration today. ## 📋 Planned: hardened sandbox {#hardened-sandbox} The current sandbox protects one trusted researcher from accidentally reading canonical data or starting nested backtests. Future hardening may add resource isolation for third-party agent cards, narrower capability domains, and structured sandbox audit logs. ## ❌ Non-goals - A public multi-tenant hosted service in the current security model. - Market-impact modeling. - A prescribed trading methodology. - Agents interacting with each other or sharing fills in real time. See [extensions](extensions.md) for contribution contracts. --- --- seo_title: TraderHarness FAQ — LLM Agent Backtesting, Leakage, and Replay description: Answers about historical memorization, point-in-time data, no-key replay, independent agents versus committees, A-share data, paper trading, and local security. lang: en --- # Frequently asked questions ## Why test a new LLM on history it may have seen? A knowledge cutoff is not an information barrier. TraderHarness removes absolute dates and company identities at agent-facing boundaries, reveals intraday bars progressively, and audits serialized artifacts. Famous events may still be inferred semantically, so serious work should disclose masks and include masked/unmasked controls when appropriate. ## Does entity masking change market rules? No. Real codes are shuffled within compatible A-share board groups. A pseudocode preserves the historical board’s price-limit behavior, while matching uses the internally resolved real symbol. ## Is the full dataset synthetic? No. Acceptance runs and public artifacts use historical full-market data. The no-key demo replays a real masked run rather than substituting generated prices. ## Does `compare` use one shared portfolio? No. Every compared agent has an isolated portfolio. A TradingAgents-style committee instead has read-only advisors, one executor, and one portfolio. ## Can replay silently call a model provider? No. Requests are fingerprinted. A mismatch or exhausted cassette fails closed. ## Can I expose the console as a hosted service? Not safely in its current form. Agent-authored Python can execute behind the local HTTP server. The sandbox protects backtest integrity, not a hostile multi-tenant deployment. Keep it on localhost. ## Does a profitable backtest mean the strategy is deployable? No. TraderHarness does not model market impact, and historical performance does not guarantee future returns. It is research infrastructure, not a broker or investment advisor. ## Is `traderharness demo` paper trading? No. `demo` replays a recorded masked historical run without network calls. Forward paper trading over streaming data is planned but does not exist today. --- --- seo_title: TraderHarness 快速开始：三步回放 A 股 LLM 交易 Agent description: TraderHarness 快速上手：pip/Docker 安装、五年 A 股数据下载、免密回放演示、本地研究控制台与多 Agent 对比。 lang: zh-CN --- # 快速上手 ## 安装 === "pip" ```bash pip install "traderharness[llm,data,ui]" ``` === "源码 / Windows" ```powershell git clone https://github.com/HephaestLab/TraderHarness cd TraderHarness python -m venv .venv .venv\Scripts\python.exe -m pip install -e ".[all]" ``` === "Docker" ```bash docker compose up --build ``` ## 安装行情数据 ```bash traderharness data download --full ``` 下载完成后会按发布清单逐项校验文件大小与 SHA-256，再原子安装到 `~/.traderharness/dataset`。 ## 跑免密回放演示 ```bash traderharness demo ``` 盒带里是一段真实、经过掩码的模型轨迹，不需要 API key；引擎仍然会用本地真实行情对它进行评测。 ## 打开 Web 控制台 ```bash traderharness ui ``` 浏览器打开 [http://127.0.0.1:8000](http://127.0.0.1:8000)。服务默认只绑定回环地址，除非显式开启，否则拒绝意外的公网暴露。 ![回测控制室：像素办公室、实时净值与决策事件流](assets/live-control-room.png) *回测控制室：左侧像素办公室里每个 Agent 各司其职，右侧实时净值曲线与决策事件流同步滚动。* ## 跑一个真实的模型 Agent ```powershell $env:DEEPSEEK_API_KEY="..." traderharness run ` --agent trend-breakout ` --start 2024-03-04 ` --end 2024-03-29 ` --mask-entities ``` `trend-breakout` 是内置的参考 Agent 卡片之一——与 `quality-compounder`、`event-hawk`、`quant-researcher` 并列，各自拥有独立人设，默认执行模型为 `deepseek-v4-pro`（thinking 深度推理模式）。把四位选手放进同一市场时钟正面对决： ```powershell traderharness compare ` --agent trend-breakout ` --agent quality-compounder ` --agent event-hawk ` --agent quant-researcher ` --start 2024-03-04 ` --end 2024-03-29 ` --mask-entities ` --output showcase ``` 加 `--record-replay cassette.jsonl` 可以把整段运行录成确定性的、可过泄漏审计的回放盒带。 ![多 Agent 对比工作台](assets/compare-workbench.png) *对比工作台：同一市场时钟下多个 Agent 的权益曲线、风险与行为指标横向排名。* --- --- title: Agent 炒股如何回测？ seo_title: Agent 炒股如何回测？AI / LLM 股票交易 Agent 回测指南 | TraderHarness description: Agent 炒股如何回测？本指南用 A 股实例讲清时点数据、未来函数、模型记忆污染、分钟级撮合、轨迹回放和结果审计，并给出可运行的 TraderHarness 流程。 author: HephaestLab lang: zh-CN schema_type: TechArticle datePublished: 2026-07-29 dateModified: 2026-07-29 faq: - question: Agent 炒股如何回测？ answer: >- 把 Agent 放进一个严格按历史时钟推进的市场环境，让它只能读取当时已经公开的数据，通过受控工具研究和下单；环境负责撮合、账户与风控，并完整记录模型请求、工具调用、订单、成交和审计证据。 - question: AI 炒股回测和普通量化回测有什么不同？ answer: >- 普通量化策略通常是确定性代码；LLM Agent 还会调用工具、生成自然语言判断，并可能记得历史事件。因此除了未来数据和成交偏差，还必须处理模型记忆污染、调用漂移、工具权限与轨迹复现。 - question: 为什么要隐藏日期和股票名称？ answer: >- 通用模型可能从训练语料中认出著名公司、日期或历史行情。日期与实体掩码减少这些记忆线索，用于检验 Agent 是否依赖当前可见数据，而不是背诵历史。 - question: LLM 调用不确定，回测还能复现吗？ answer: >- 新鲜模型调用通常不能保证逐字确定，但可以完整记录请求、响应和工具序列，并用请求指纹进行失败即报错的回放；撮合、账户和指标计算则保持确定性。 - question: TraderHarness 支持 A 股吗？ answer: >- 支持。公开数据集覆盖五年全市场 A 股日线、5 分钟线、公告、政策新闻、基本面、估值、分红和沪深 300 基准。 - question: 回测收益好是否说明 Agent 可以实盘盈利？ answer: >- 不能。历史回测仍受样本选择、市场冲击、交易成本、模型漂移和数据质量影响；它只能说明 Agent 在明确的历史评测合同下产生了某种行为和结果。 --- # Agent 炒股如何回测？ **简短答案：**回测炒股 Agent，不是把一段 K 线塞给大模型后询问“该买还是该卖”。可信的方法是把 Agent 放进一个严格按历史时钟推进的市场环境：它只能看到当时已经公开的数据，只能通过受控工具研究和下单；环境负责撮合、账户与风控，并保留从模型请求到最终成交的完整审计证据。 [TraderHarness](https://github.com/HephaestLab/TraderHarness) 是面向这套评测合同的开源 Python 环境，目前聚焦中国 A 股。它同时处理时点数据、日期与公司实体掩码、5 分钟级渐进撮合、模型调用记录、指纹回放和轨迹导出。 [三步运行免 Key 回放](../quickstart.md){ .md-button .md-button--primary } [查看 Masked vs Unmasked 实验](../masking-ab-showcase.md){ .md-button } [English guide](https://hephaestlab.github.io/TraderHarness/guides/agent-stock-backtesting/){ .md-button } !!! warning "回测不是投资建议" 历史模拟不能证明未来盈利，也不能替代持牌投资咨询。TraderHarness 不连接真实券商账户，不提供跟单或收益承诺。 ## 为什么普通股票回测方法不够 传统量化策略通常是一段确定性代码：输入行情，输出仓位。LLM 交易 Agent 则会研究、调用工具、生成判断、修改计划并下单，因此多出三类特殊问题。 ### 1. Agent 可能看到未来信息 最常见的问题是未来函数或前视偏差。例如，Agent 在开盘阶段看到了当天收盘价，看到了一份收盘后才发布的公告，或者下单后以决策前不可见的低价成交。仅按“日期”过滤数据不够，日线、分钟线、公告、新闻和财务数据都有不同的实际发布时间。 ### 2. 大模型可能记得历史 即使数据接口完全遵守时点，大模型仍可能认出股票代码、公司产品、著名公告或某段历史行情。它输出的答案看似正确，实际上可能来自训练语料记忆，而不是回测时允许看到的证据。 这也是为什么 LLM Agent 回测需要把普通的 look-ahead bias 与模型记忆污染分开处理。TraderHarness 可以把绝对日期改写为 `D+0`、`D-1`，并把公司名称和代码替换为确定性伪名，同时保留交易规则需要的市场和板块语义。 ### 3. 同一个模型也可能返回不同答案 云端模型会升级，推理服务会变化，即使温度为零也不保证逐字一致。只保存最终买卖记录，无法解释结果为什么变化。可信的回测必须记录完整消息、模型与供应商配置、工具 schema、模型响应、工具调用和工具结果。 ## 一套可信的 Agent 炒股回测合同 | 边界 | 最低要求 | 应保留的证据 | |---|---|---| | 市场数据 | 每个出口都按模拟时钟过滤 | 来源时间、模拟时间、可见行 | | 公司与日期 | 必要时做日期和实体掩码 | 掩码策略、种子、泄漏审计 | | 成交 | 决策后才逐步揭示可成交价格 | 下单时间、可见窗口、成交时间与价格 | | 账户 | Agent 只能读取账户视图 | 订单、成交、现金、持仓、公司行动 | | 模型 | 记录完整请求、响应和工具循环 | 模型 ID、请求指纹、调用模式 | | 对比 | 相同市场时钟，账户相互隔离 | 配置、随机种子、基准序列 | | 发布 | 公开前审计全部工件 | 代码版本、数据清单、审计报告 | 最核心的原则是：**相同的可见数据和动作序列，必须产生相同的环境结果。**模型生成可以先完整记录、再指纹回放；撮合、账户和指标计算必须由环境确定性地执行。 ## 六步完成一次 LLM 股票 Agent 回测 ### 第一步：定义 Agent，而不是只写一句提示词 明确 Agent 的研究风格、可使用工具、模型、风险限制和交易阶段。TraderHarness 的 Agent 卡片会参与三个阶段： 1. 盘前研究，不能下单； 2. 开盘窗口，5 分钟数据逐步揭示； 3. 尾盘窗口，继续渐进揭示并允许交易。 多角色系统还要区分只读顾问和唯一执行者。不能让每个角色都持有交易工具，否则账户责任和决策归因都会失真。 ### 第二步：准备严格时点有效的数据 至少需要未复权日线、盘中分钟线、交易日历、股票状态和基准。若 Agent 使用公告、新闻、基本面或估值，还必须按真实发布时间过滤，而不是只按报告期或自然日过滤。 历史成交和公司行动应使用未复权价格；前复权序列适合画图，却不能直接替代真实成交价格。 ### 第三步：同时控制未来数据和历史记忆 时点过滤解决“Agent 看到了未来”，日期与实体掩码解决“Agent 可能记得过去”。二者不是同一问题，也不能互相替代。 建议对同一张 Agent 卡进行 masked/unmasked 配对实验：市场、模型、工具、资金和执行规则保持一致，只改变日期与公司身份是否可见，再比较行为与结果。TraderHarness 的[一键实验页面](../masking-ab-showcase.md)展示了这种可审计结构。 ### 第四步：让所有订单经过同一撮合路径 Agent 不应直接修改持仓，也不应自行声明成交。所有订单必须通过环境的唯一入口完成校验和撮合。TraderHarness 使用 `TradingBus.place_order()` 处理下单路径，由环境拥有现金和持仓状态。 盘中数据必须先揭示一个子窗口，再允许 Agent 决策，然后才能使用之后可见的价格撮合。这样可以阻止 Agent 看完整天走势后回头选择早盘最优价格。 ### 第五步：记录并回放完整模型轨迹 每次调用都保存完整请求和响应，并为规范化请求计算 SHA-256 指纹。回放时，提示词、工具定义、可见输入或调用顺序只要发生变化，就应立即失败，而不是悄悄返回不匹配的缓存答案。 这使测试能够区分两件事：模型本次生成了什么，以及市场环境如何执行这些动作。 ### 第六步：报告行为和风险，而不只报告收益 至少应报告： - 区间收益、沪深 300 相对收益、最大回撤和风险调整指标； - 订单数、成交数、拒单数、空仓天数和换手率； - 暴露度、集中度、持有周期与工具使用行为； - 日期范围、股票池、初始资金、费用、撮合规则和掩码配置； - 模型与供应商、Agent 卡片、数据版本、代码提交和工件审计结果； - 未建模因素，例如市场冲击、实盘延迟和供应商模型漂移。 ## 用 TraderHarness 运行 A 股 Agent 回测 安装包和完整数据集，然后运行内置的 masked replay： ```bash pip install "traderharness[llm,data,ui]" traderharness data download --full traderharness demo ``` 这个演示不需要模型 API Key。它会回放已经记录的 LLM 轨迹，但仍使用本地规范行情重新执行市场环境和指标计算。 记录一次新的模型运行： ```bash traderharness run \ --agent trend-breakout \ --start 2024-03-04 \ --end 2024-03-29 \ --mask-entities \ --record-replay run.jsonl ``` 公开或导出训练数据之前执行审计： ```bash traderharness audit result.json run.jsonl traderharness export sft result.json --output training.jsonl ``` 如果要比较多个独立 Agent，使用 `traderharness compare` 为每个 Agent 分配隔离账户；如果要复现 TradingAgents 风格的多角色委员会，应让顾问保持只读，只允许一个执行者下单。两者是不同的实验合同，不能混在同一排行榜中解释。 ## 常见的无效回测方式 - 把整段历史行情一次性发给模型，再要求它逐日“模拟”； - 在开盘决策时提供当天最高、最低或收盘价； - 用前复权价格直接模拟真实成交和公司行动； - 只告诉模型“不要使用未来信息”，却不在工具层强制过滤； - 只保存最终交易，不保存模型请求和工具调用； - 在同一账户里并发运行多个执行 Agent； - 只挑选盈利轨迹公开，不报告失败、拒单和空仓行为； - 把一次历史收益直接宣传为模型能力或未来收益。 ## Agent 炒股回测常见问题 ### Agent 炒股如何回测？ 使用按历史时钟推进的市场环境，让 Agent 只能读取当时已公开的数据，通过受控工具研究和下单；环境负责撮合、账户、风控和完整轨迹记录。不要让模型直接读取整段历史，也不要让它自行修改持仓。 ### AI 炒股回测和普通量化回测有什么不同？ LLM Agent 除了未来数据和成交偏差，还会遇到历史记忆污染、模型调用漂移、工具越权和自然语言决策不可复现等问题，因此需要额外的掩码、轨迹和指纹回放机制。 ### 为什么要隐藏日期和股票名称？ 通用模型可能从训练语料中认出著名公司、日期或行情。掩码用于减少记忆线索，检验 Agent 是否依赖当前允许看到的数据。它是污染控制手段，不是模型绝对无法反向识别的证明。 ### LLM 调用不确定，回测还能复现吗？ 新鲜调用通常不能保证完全确定，但可以保存完整交互并用请求指纹严格回放。这样，模型响应可以原样重放，市场环境的撮合和账户结果仍保持确定性。 ### TraderHarness 支持 A 股吗？ 支持。公开数据集覆盖五年全市场 A 股日线、284,219,844 条去重 5 分钟线，以及公告、政策新闻、基本面、估值、分红和沪深 300 基准，使用 SHA-256 发布清单校验。 ### 回测收益好是否说明 Agent 可以实盘盈利？ 不能。回测只能描述明确数据、模型和执行规则下的历史结果。市场冲击、真实延迟、交易成本、样本选择和模型升级都可能改变实盘表现。 ## 下一步 - 按[快速上手](../quickstart.md)运行免密回放和本地研究控制台； - 阅读[无数据泄漏的 LLM 交易回测](../contamination.md)了解时点与实体掩码； - 查看[TraderHarness 与 TradingAgents、StockBench、Qlib 的区别](../comparison.md)； - 在 [GitHub](https://github.com/HephaestLab/TraderHarness) 检查源码、测试和公开工件。 --- --- seo_title: 如何防止 LLM 交易 Agent 回测的数据泄漏与历史记忆污染？ description: TraderHarness 如何在每个数据出口执行时点掩码、日期相对化与公司实体匿名化，防止 LLM 回测中的记忆污染与执行层泄漏。 lang: zh-CN --- # 无数据泄漏的 LLM 交易回测 TraderHarness 把历史污染当作环境边界来处理，而不是提示词层面的口头约定。 ## 基本概念 **时点掩码（Point-in-time masking）**：任何暴露给 Agent 的数值，在离开环境之前都要先过模拟时钟的过滤。日线严格早于当前交易日，5 分钟线截断到当前子窗口，基本面记录的发布日期不得晚于当前日期。 **日期匿名化**：把 Agent 看到的绝对日历日期替换为相对模拟当前的偏移量。今天是 `D+0`，前一个自然日是 `D-1`。一天内的时刻保持可见。 **实体掩码**：真实 A 股代码与已知公司别名到中性假名之间的确定性、全运行范围双射。代码在兼容的板块分组内打散，使涨跌停规则在匿名化后依然成立。 **三阶段交易循环**：一个有边界的交易日——禁止下单的盘前研究、渐进揭示的 09:30–10:00 开盘窗口、渐进揭示的 14:30–15:00 尾盘窗口。 ## 掩码覆盖范围 {#egress-coverage} 掩码作用于： - 日线与盘中 K 线； - 选股筛查、基本面、估值、公告与政策新闻； - 账户与自选股视图； - Python 沙箱中 `traderharness_api` 返回的 DataFrame； - 模型回复、推理字段、工具参数与委员会备忘录； - 跨日记忆、落盘轨迹、回放盒带、对比结果与轨迹导出。 Agent 通过工具回传的伪代码会在内部解析还原后再撮合；账户视图再经同一正向映射渲染，Agent 全程无需接触真实代码。 ![日期与实体双重掩码的变换过程](assets/dual-mask.svg) *同一条历史公告，两种身份：左侧是仅限环境的原始记录（真实日期 + 真实公司），右侧是 Agent 可见的视图（相对日期 `D-9` + 伪身份 `公司-600731`）。* ## 工件审计 ```bash traderharness audit result.json replay.jsonl export.parquet ``` 审计器检查已知真实公司别名、六位 A 股代码泄漏、绝对 ISO/中文日期以及月-日形式。v1.0 发布验收中对一段序列化的一个月期 DeepSeek 轨迹做了审计，在最终出口修复之后未检出任何真实实体别名或绝对日期。 这个结论的含义是窄的：它验证的是已知的词法与日历出口契约。它无法证明模型不能从独特的财务模式、产品、高管或事件推断出某家知名公司。要发布可公开的评测结果，请报告掩码配置与种子、保留审计输出，并在语义重识别影响重大时对比遮罩与未遮罩运行。 ## 执行层泄漏 如果撮合引擎允许模型先看完整个盘中窗口、再回头选择更早的优惠价格，信息掩码就失去了意义。TraderHarness 先揭示每个开盘/尾盘子窗口、再给出该窗口的合格成交，并让每一笔订单都经过 `TradingBus.place_order()`。因此动作序列不可能选中决策时刻不可见的价格。 --- --- seo_title: TraderHarness 架构：LLM Agent 回测引擎、撮合与指纹回放 description: TraderHarness 核心架构说明：时点安全数据出口、三阶段 Agent 循环、唯一撮合路径、只读账户、完整轨迹和确定性指纹回放。 lang: zh-CN --- # 核心架构 ![TraderHarness 数据、引擎、Agent、轨迹与评估架构](assets/architecture.svg) ```mermaid flowchart TD D[(规范全市场数据)] -->|启动时预加载| E[BacktestEngine] E --> B[TradingBus（每 Agent）] B --> T[掩码工具] B --> V[PortfolioView] B --> X[Python 沙箱] T --> A[Agent 循环] V --> A X --> A A -->|place_order| B E --> C[轨迹采集器] C --> R[回放 / 轨迹导出 / 报告] ``` ## 不可妥协的不变量 ### 运行期零 I/O 引擎在第一个交易日之前预加载所需的全部行情切片。Agent 的工具调用只做内存查询，绝不回源拉取供应商数据，也不直接读取规范数据集。 ### 严格的历史可见性 日线使用 `date < current_date`；基本面使用 `pub_date <= current_date`；5 分钟线截断到当前阶段与子窗口；面向 Agent 的绝对日期一律变为相对偏移。 ### 唯一下单路径 `TradingBus.place_order()` 统一施加整手、停牌、现金、持仓、涨跌停、费用与可见价格校验。不存在供 Agent、委员会或沙箱绕行的第二条快速通道。 ### 环境托管账户 账户由环境所有。Agent 只获得只读视图，只能通过校验过的订单改变状态。分红、送转与每日净值是确定性的引擎操作。 ## 回放契约 每条记录的 LLM 请求都有规范的 SHA-256 指纹。回放会拒绝被改动的请求与耗尽的盒带，也绝不回退到联网模型——这让回归失败显式暴露，而不是悄无声息地变得不确定。 --- --- seo_title: TraderHarness 与 TradingAgents、StockBench、Qlib 有什么区别？ description: 对比 TraderHarness、TradingAgents、StockBench、Qlib 和传统回测引擎，说明 Agent 编排、历史市场环境、抗污染、撮合、回放与训练轨迹的边界。 lang: zh-CN --- # 与相邻项目的对比 TraderHarness 是一套环境与证据基础设施。它可以承载不同的 Agent 架构，但不规定它们的投资方法论。 | 项目 | 主要职责 | 原生决策单元 | 集成后还需要什么 | |---|---|---|---| | **TraderHarness** | 历史有效的市场、执行、账户、评估、回放与训练轨迹 | 自主 Agent、隔离对比或单执行者委员会 | 一个 Agent 人设或外部框架 | | **TradingAgents** | 多角色分析师、辩论、风控与交易员工作流 | 规定的角色图 | 可用于基准测试的严格市场模拟器与下单契约 | | **StockBench** | 标准化的股票推理基准任务 | 基准任务/预测 | 支持自主工具调用的持久账户环境 | | **Qlib** | 量化数据、模型、实验与策略研究 | ML 模型或代码策略 | LLM 原生的工具循环与抗污染语言出口 | | **Backtrader / vn.py** | 策略执行与交易基础设施 | 代码策略 | 自主 LLM 研究循环、掩码与轨迹契约 | ## 独立 Agent 与委员会的区别 `traderharness compare` 是一场赛跑：每个 Agent 拥有自己的现金、持仓、记忆与账户。所有 Agent 共享同一市场时钟，各自独立计分。 委员会则是一个选手：只读顾问可以并发研究，但只有 Trader 这一个角色持有下单工具。这保证了一条可问责的动作路径和一个账户。详见[多角色委员会](design/multi-role-agent.md)。 ![跨回测权益曲线叠加与关键指标对比](assets/run-compare.png) *跨回测对比：把多次已完成回测的权益曲线叠加在一起，横向比较累计收益、夏普、最大回撤、胜率与成交次数。* ## 接入你自己的框架 外部 LangGraph、TradingAgents 或自定义编排器应通过公开的 Agent 协议返回最终决策。行情读取与下单仍然经过 `TradingBus`，因此框架天然继承同样的时点掩码、实体/日期匿名化、盘中渐进可见性与撮合规则。 --- --- seo_title: 多角色 LLM 交易 Agent 如何回测？只读顾问与唯一执行者 description: TraderHarness 多角色交易 Agent 设计：研究员和风险顾问保持只读，唯一 Trader 获得下单工具，并保证账户隔离、顺序确定和轨迹可回放。 lang: zh-CN --- # 多角色 Agent 适配器 ## 目标 在不改变 TraderHarness 的市场、账户、撮合、掩码与回放语义的前提下，支持 TradingAgents 风格的委员会。委员会是被评估的一个 Agent、一个账户。专家角色只提供建议；唯一执行者才能调用 `place_order`。 这与 `traderharness compare` 不同：后者中独立 Agent 各自持有账户并相互排名。 ## 模型 ```text BacktestEngine（共享不可变市场快照） └─ CommitteeAgent（一个 Portfolio + 一个 TradingBus） ├─ 基本面顾问 ───────┐ ├─ 技术面顾问 ───────┼─> 阶段备忘录 ├─ 新闻顾问 ─────────┤ ├─ 多头研究员 ───────┤ ├─ 空头研究员 ───────┘ └─ trader/执行者 -> 现有 AgentLoop -> TradingBus.place_order() ``` 多个独立委员会之间仍可由引擎并行： ```text 委员会 A + 账户 A ────┐ 委员会 B + 账户 B ────┼─ 每个交易日 asyncio.gather 单 Agent + 账户 C ────┘ ``` ## 不变量 1. 顾问只收到已经掩码的 Agent 可见消息。 2. 顾问没有下单工具。只读工具访问是后续扩展。 3. 执行者使用现有 `ToolRegistry`；`TradingBus.place_order()` 仍是唯一撮合路径。 4. 委员会调用对回放是确定的：角色、阶段、提示词、响应、模型与顺序都是轨迹记录。 5. 一个运行级的 `EntityMasker` 由所有顾问与执行者共享。 6. 顾问失败在备忘录与轨迹中显式可见；不做静默兜底。 ## 扩展面 ```python class Advisor(Protocol): role: str async def advise(self, messages: list[dict], phase: str) -> str: ... class CommitteeCoordinator: async def build_memo( self, messages: list[dict], phase: str, sub_window: str | None, ) -> CommitteeMemo: ... ``` `AgentLoop._run_phase()` 在每个 `(day, phase, sub_window)` 的首次执行者调用前请求一份备忘录。顾问通过 `asyncio.gather` 并发执行；产出作为带标签的系统消息注入。执行者可以接受或否决每一条建议。 ## TradingAgents 适配器 适配器把外部图节点映射为 `Advisor` 实现： - 市场/新闻/基本面分析师 -> 专家顾问 - 多/空研究员 -> 对抗顾问 - 研究经理/风险经理 -> 综合顾问 - Trader -> TraderHarness 执行者 外部工具与账户对象不会被引入，取而代之的是 TraderHarness 的掩码观测与单一账户/下单路径。这样既保留外部推理拓扑，又保证回测公平性。 ## 配置 加载器（`PromptAgent`）通过顶层 `advisors:` 列表识别委员会——不存在嵌套的 `committee:` 或 `executor:` 块。`id`、`name` 与执行者自己的 `model`/`persona` 都在顶层，与单 Agent 卡片完全一致： ```yaml id: tradingagents-reference name: TradingAgents Reference Committee model: deepseek-chat persona: ... advisors: - role: fundamentals model: deepseek-chat prompt: ... - role: technical model: deepseek-chat prompt: ... - role: bull model: deepseek-chat prompt: ... - role: bear model: deepseek-chat prompt: ... ``` 完整可加载的参考委员会见 [`examples/tradingagents_committee.yaml`](https://github.com/HephaestLab/TraderHarness/blob/main/examples/tradingagents_committee.yaml)。 ## 验收标准 - 单元测试证明顾问永远不会拿到 `place_order`。 - 单元测试证明所有顾问被并发调度。 - 集成测试证明恰好只有一个执行者能下单。 - 回放能复现同样的备忘录与执行者动作序列。 - 真实一日、三日与一个月运行通过泄漏审计。 - `compare` 能在相同数据、现金、掩码种子与基准下，把委员会与普通 Agent 一起排名。 ## 暂缓项 - 按顾问划分的只读工具预算。 - 任意环形图与 Agent 间消息总线。 - 多执行者共享账户（有意排除——它会让订单归属与回放变得含糊）。 --- --- seo_title: LLM 交易 Agent 轨迹数据：回放、审计与强化学习导出 description: TraderHarness 全保真轨迹采集与轨迹导出：完整消息、工具 schema、推理内容逐次落盘，导出 OpenAI 风格 SFT JSONL。 lang: zh-CN --- # 全保真轨迹与轨迹导出 TraderHarness 可以持久化每一次执行者 LLM 调用的完整掩码请求/响应对。它的用途是可复现研究与下游监督微调（SFT），并不代表每一条生成的决策都是高质量训练目标。 ![逐笔复盘：K 线上下文、下单理由与执行证据](assets/trade-review.png) *每一次决策都可回放审计：成交时 K 线、已记录的下单理由、工具调用参数与执行结果完整留档。* ## 采集轨迹 开启实体掩码运行回测： ```bash traderharness run \ --agent trend-breakout \ --start 2024-03-04 \ --end 2024-03-29 \ --mask-entities ``` 每个 `llm_exchange` 轨迹步骤包含： - 发给执行者的完整消息列表； - 该次调用可用的完整工具 schema； - assistant 内容与可选的推理内容； - 完整的工具调用与参数； - 阶段与子窗口元数据。 兼容性的 `assistant` 与 `tool_call` 步骤也会保留。新生成的结果中 assistant 文本不再截断。 ## 导出 OpenAI 风格 JSONL ```bash traderharness export sft \ ~/.traderharness/results/_result.json \ --output ./training.jsonl ``` 每次 LLM 调用输出一行： ```json { "messages": [ {"role": "system", "content": "..."}, {"role": "user", "content": "..."}, {"role": "assistant", "content": "...", "tool_calls": []} ], "tools": [], "metadata": { "agent_id": "trend-breakout", "phase": "pre_market", "sub_window": null, "day_index": 1, "call_index": 1 } } ``` 绝对交易日期不会进入导出元数据；Agent 可见日期保持相对形式（`D+0`、`D-1` 等）。 ## 安全闸口 默认情况下，导出会： 1. 拒绝未开启实体掩码的运行； 2. 拒绝缺少全保真 `llm_exchange` 记录的旧轨迹； 3. 对输出运行实体/日期泄漏检测； 4. 只要仍有检出就以非零码退出。 `--allow-unmasked` 是面向私有研究的显式逃生门，这类输出不得作为抗污染训练数据发布。 ## 筛选仍然必要 全保真意味着错误决策与好决策都会被保留。训练前请按结果、回撤、规则合规、工具错误与人工复核过滤轨迹，并确认所选模型供应商的条款允许将生成的推理与回复用于训练。 --- --- seo_title: TraderHarness A 股回测数据集：日线、5 分钟线与时点数据 description: TraderHarness 五年全市场 A 股数据说明，覆盖日线、5 分钟线、公告、政策新闻、基本面、估值、分红、沪深 300、完整性校验与许可边界。 lang: zh-CN --- # 数据与许可 规范 A 股发布版包含五年全市场日线与 5 分钟线，以及公告、政策新闻、基本面、估值、分红和沪深 300 基准。 ## 完整性 `traderharness data download --full` 会按发布清单逐项校验后才原子替换本地数据集；`traderharness data update` 使用水位线、确定性去重与原子写入。 仓库自带的数据医生（data doctor）检查： - 必需 schema 与日期范围 - 自然键重复 - 5 分钟线年度覆盖率 - 过期标的与数据集对齐 - 非 A 股公告代码非法值 - 元数据一致性 v1.0 规范构建包含 284,219,844 条去重后的 5 分钟记录。发布审计中，活跃日线股票池的年度标的覆盖率达到 100%，最终 5 分钟水位线处无滞后标的，验证样本中自然键零重复。 ## 公开发布策略 公开新闻表只保留模板化标题，移除有授权限制的正文。公司模板只在运行时解析为中性身份。这在保护评测完整性的同时，让源数据集依然可用于时点过滤。 ## 存储结构 ```text ~/.traderharness/dataset/ ├── daily.parquet ├── 5min_clean/ ├── announcements.parquet ├── news_cls.parquet ├── fundamentals.parquet ├── valuation.parquet ├── dividends.parquet ├── index_300.parquet └── metadata.json ``` 行情数据许可因供应商与司法辖区而异。再分发或商用部署前请核实上游条款。 --- --- seo_title: TraderHarness CLI 与本地 API：运行、回放、审计和 Agent 管理 description: TraderHarness 命令行与 localhost API 参考，包含数据下载、Agent 回测、compare、replay、audit、轨迹导出、REST 和 WebSocket 接口。 lang: zh-CN --- # CLI 与本地 API 参考 ## 核心 CLI ```text traderharness run 运行单个 Agent traderharness compare 在同一市场时钟下隔离运行多个 Agent traderharness demo 免密回放内置的掩码运行 traderharness ui 启动本地 FastAPI + React 控制台 traderharness audit 扫描工件中的实体与日历泄漏 traderharness export 把轨迹转换为 SFT JSONL traderharness data 下载、更新与检查数据集 ``` 各命令的具体参数以 `traderharness --help` 为准。 ## Agent 协议 自定义 Agent 实现 `traderharness.agents.protocol` 中的公开协议，分别在盘前、开盘窗口与尾盘窗口收到环境控制的上下文。只读顾问可以组合在单一执行者之后，详见[多角色委员会](design/multi-role-agent.md)。 ## 本地 HTTP API `traderharness ui` 提供： - `GET /api/status` — 数据集、供应商与本地安全状态； - `GET/POST /api/agents` — Agent 卡片集合； - `GET/PUT/DELETE /api/agents/{id}` — 单张 Agent 卡片； - `POST /api/runs` — 启动回测； - `GET/DELETE /api/runs/{id}` — 查看或取消运行； - `WS /api/runs/{id}/events` — 可重连的序号化事件日志； - `GET /api/results` — 已落盘结果摘要； - `GET /api/results/{file}` — 完整工件； - `GET /api/results/{file}/analysis` — 归一化的 UI 研究档案； - `POST /api/demo` — 启动内置回放； - `GET /api/health` — 进程健康检查。 HTTP API 是本地工具，不是带鉴权的公共服务。请保持默认的 localhost 绑定。 --- --- seo_title: 如何扩展 TraderHarness：数据 Provider、Agent 工具与评估指标 description: 为 TraderHarness 添加数据 Provider、Agent 工具、沙箱后端、评估指标和前端视图时必须遵守的时点安全、唯一下单路径与确定性边界。 lang: zh-CN --- # 扩展 TraderHarness TraderHarness 的设计允许在不削弱核心不变量的前提下扩展：回测期零 I/O、唯一下单路径、严格时点可见性、确定性执行、环境托管账户（见[核心架构](architecture.md)与 [`AGENTS.md`](https://github.com/HephaestLab/TraderHarness/blob/main/AGENTS.md)）。本页是几个常见扩展点的简要契约。大型改动前请先开 issue；流程见 [`CONTRIBUTING.md`](https://github.com/HephaestLab/TraderHarness/blob/main/CONTRIBUTING.md)。 ## 数据源适配器 新市场，或现有 A 股数据集的新供应商，都应产出符合[数据与许可](data.md)中规范 schema 的数据。 契约： - 在 `traderharness/data/providers/` 下实现 provider；不得绕过 `traderharness/data/datasets.py`，也不得直接写入正在运行的回测的内存表。 - 保持时点完整性：每条记录需要稳定的自然键；非价格类记录还需要可供掩码层过滤的发布时间戳（`pub_date` 式列）。 - 为新表添加或扩展数据医生检查（`scripts/data_doctor.py`）：必需列、日期覆盖与重复键不变量。 - 在 `tests/fixtures/` 提供小体量真实数据夹具与加载测试；验收验证不得用合成价格顶替（见真实数据工作区规则）。 ## 工具 面向 Agent 的工具位于 `traderharness/tools/`，通过 `traderharness/tools/registry.py` 注册。 契约： - 工具处理器接收本次运行的掩码上下文；它绝不能越过上下文读取规范数据集或其他 Agent 的状态。 - 每条失败路径都返回结构化、可行动的错误，区分"代码不存在""该日期前无数据""停牌""参数被忽略"——笼统异常不可接受。 - 新工具需要 JSON-schema 参数校验、每种失败模式一个单元测试；若内置 Agent 应使用它，还要加入相应 Agent 卡片的 `allowed_tools`。 - 工具不得新增第二条下单路径。交易始终走 `TradingBus.place_order()`。 ## 沙箱后端 `execute_code` 工具与 `traderharness_api` 模块是 Agent 对掩码数据运行任意分析代码的受认可方式（见[防数据泄漏](contamination.md#egress-coverage)）。 契约： - 沙箱后端必须执行同样的路径防护，阻断直接读取数据集（`sandbox/guard.py`），并使用同样的 wall-clock 超时。 - `traderharness_api` 的新增能力必须经由现有掩码访问器解析；不得添加返回未掩码 DataFrame 或真实实体代码的代码路径。 - 任何沙箱后端都不得启动嵌套回测，也不得回调引擎的下单路径。 - 沙箱隔离的演进方向见[路线图](roadmap.md#hardened-sandbox)；缩小可信面的贡献尤其受欢迎。 ## 评估指标 绩效与行为指标位于 `traderharness/metrics/`。 契约： - 新指标是对已完成运行的每日净值、成交与决策的纯函数——不得要求重跑回测或调用供应商 API。 - 在 docstring 中写清公式与边界情况（空成交历史、单交易日、缺基准数据），并添加报告/JSON 导出测试。 - Agent 间比较类指标（如排名）属于 `traderharness/metrics/comparison.py`，不属于单 Agent 报告。 ## 券商适配器 v1.0 没有实盘券商适配器，见[路线图](roadmap.md#live-broker-adapter)。欢迎以 issue 形式讨论设计与原型，但券商集成不应接进回测用 `TradingBus`——历史模拟与实盘下单是不同的信任边界，必须保持分离。 ## 前端（webui） 研究控制台是 `webui/` 下的纯本地 React 应用。 契约： - 新视图从现有 REST/WebSocket API（`traderharness/server/app.py`）取数；不得在客户端用原始字段重新推导掩码数据。 - 中文文案统一走 `webui/src/locale.ts`，不要在组件里硬编码字面量。 - 新组件配 Vitest 单元测试；新页面或新流程配 `webui/tests/e2e` 下的 Playwright 场景（若应出现在 README GIF 中，同步扩展 `webui/scripts/capture-demo.mjs`）。 ## 提交 PR 之前 1. 先写一个能证明新契约的失败测试。 2. 实现时不得削弱 [`AGENTS.md`](https://github.com/HephaestLab/TraderHarness/blob/main/AGENTS.md#non-negotiable-invariants) 中任何不变量。 3. 跑聚焦测试套件，再跑全量（`pytest tests/ --no-header -q`、`ruff check`；涉及引擎/掩码/工具/数据/沙箱的改动还需一次真实回放或回测并检查轨迹）。 4. 如实声明做过哪些真实数据运行、哪些没做。 --- --- seo_title: TraderHarness 路线图：A 股 Agent 回测、模拟盘与安全边界 description: TraderHarness 已交付能力、模拟实盘 paper trading 设计、未来券商适配器、沙箱加固和公开多租户服务等明确非目标。 lang: zh-CN --- # 路线图 本页记录 TraderHarness 已经交付了什么、计划做什么，方便集成方做"自建还是等待"的决策，而不必从 issue 列表里猜。以下内容均不构成日期承诺。 ## ✅ v1.0 已交付 - 五年全市场 A 股数据集（日线、5 分钟、公告、政策新闻、基本面、估值、分红、沪深 300），带原子增量更新与完整性检查。 - 预加载后回测期零行情 I/O；`TradingBus.place_order()` 唯一下单路径。 - 日线、盘中、新闻、公告、基本面与沙箱出口的严格时点掩码。 - 确定性日历（`D+0`、`D-1`……）与保留板块语义的公司实体匿名化。 - 盘前 / 开盘窗口 / 尾盘窗口三阶段 Agent 循环，盘中渐进可见。 - 独立多 Agent 对比（`traderharness compare`），以及单执行者多角色委员会参考实现（顾问只读；唯一 Trader 持有下单工具）。 - 全保真 LLM 交互轨迹、失败即报错的指纹回放、OpenAI 风格轨迹导出。 - 序列化工件泄漏审计（`traderharness audit`）。 - 本地 FastAPI + React 研究控制台、非特权 Docker 镜像、PyPI 打包与 CI。 逐项发布说明见 [`CHANGELOG.md`](https://github.com/HephaestLab/TraderHarness/blob/main/CHANGELOG.md)。 ## 🚧 下一步：模拟实盘（paper trading） 一种模拟的实时前推模式：复用现有引擎、掩码与工具契约，但数据源从全量预加载改为流式推送，从而无需改动代码就能对一张 Agent 卡片做面向未来的评估。该功能在设计中，**目前不可用**，仓库中任何内容都不应被解读为相反 claim。 从回测引擎继承的约束： - 同样的 `TradingBus.place_order()` 路径与风控检查； - 任何面向 Agent 的出口都遵守同样的掩码契约； - 不存在让沙箱或工具看到模拟时钟之后数据的捷径。 ## 📋 规划：实盘券商适配器 {#live-broker-adapter} 一个适配器边界，让模拟实盘或研究 Agent 可以对接真实券商 API。它依赖模拟实盘先落地，并需要与项目整体安全姿态相匹配的凭据与下单授权威胁模型（见 [`SECURITY.md`](https://github.com/HephaestLab/TraderHarness/blob/main/SECURITY.md)）。目前没有任何券商集成。 ## 📋 规划：沙箱加固 {#hardened-sandbox} 当前 Python 沙箱（`execute_code` + `traderharness_api`）的定位是防止单一可信研究者意外读取规范数据集或启动嵌套回测——见 [本地服务器安全](https://github.com/HephaestLab/TraderHarness/blob/main/AGENTS.md#local-server-security)。未来版本计划的加固： - 适合运行不可信或第三方 Agent 卡片的资源与 wall-clock 隔离； - 更窄的默认 `traderharness_api` 面，按工具做能力域划分； - 与现有轨迹记录并列的结构化沙箱审计日志。 ## ❌ 非目标 - **公开排行榜或托管多租户服务。** TraderHarness 是本地研究基础设施；见[本地服务器安全](https://github.com/HephaestLab/TraderHarness/blob/main/AGENTS.md#local-server-security)。 - **市场冲击建模。** 历史成交使用不复权价格，不模拟 Agent 自己的订单对市场的推动。 - **规定交易方法论。** 环境保持 Agent 架构中立；见[项目对比](comparison.md)。 - **Agent 之间实时互动、互相影响成交。** 每个 Agent（或委员会）都在自己的隔离账户中对同一历史时钟交易。 ## 如何参与 上述路线图条目是最可能被快速接受的贡献方向。贡献契约见[扩展开发](extensions.md)，流程见 [`CONTRIBUTING.md`](https://github.com/HephaestLab/TraderHarness/blob/main/CONTRIBUTING.md)。 --- --- seo_title: TraderHarness 常见问题：LLM Agent 回测、数据泄漏与确定性回放 description: 回答 TraderHarness 的历史记忆污染、时点数据、免 Key 回放、多 Agent 与委员会区别、A 股数据、模拟盘和本地安全等常见问题。 lang: zh-CN --- # 常见问题 ## 为什么要在模型可能见过的历史上评测新 LLM？ 知识截止日不是信息屏障。TraderHarness 在每一个面向 Agent 的边界移除绝对日期与公司身份，盘中分钟线渐进揭示，并对序列化工件做词法泄漏审计。著名事件仍可能被语义推断出来，因此严谨的工作应报告掩码设置，并在合适时加入遮罩/未遮罩对照。 ## 实体掩码会改变交易规则吗？ 不会。真实代码在兼容的 A 股板块分组内打散。创业板或科创板的伪代码保留其历史板块的涨跌停行为。撮合始终使用内部解析还原后的真实标的。 ## 完整数据集是合成的吗？ 不是。验收测试与公开运行都使用历史全市场真实数据。免密演示是一段真实掩码运行的确定性回放，不用生成的价格顶替。 ## `compare` 是共享账户的多 Agent 吗？ 不是。`compare` 给每个 Agent 一个隔离账户。想要 TradingAgents 式的结构，请用委员会：顾问只读，唯一 Trader 管理一个账户。 ## 回放会偷偷调用模型供应商吗？ 不会。请求带指纹。指纹不匹配或盒带耗尽都会直接失败（fail-closed）。 ## 控制台能暴露成托管服务吗？ 以当前形态不安全。Agent 编写的 Python 在本地 HTTP 服务后可执行。沙箱防护保护的是回测数据边界，而不是敌意多租户部署。请保持 localhost 绑定。 ## 回测盈利等于可上线策略吗？ 不等于。TraderHarness 不建模市场冲击，历史表现不保证未来收益。它是研究基础设施，不是投资建议，也不是券商。 ## README 展示用哪些 Agent 和模型？ 四 Agent 展示对比内置的 `trend-breakout`、`quality-compounder`、`event-hawk`、`quant-researcher` 四张卡片，区间为 2024-03-04 至 2024-03-29，开启实体掩码，执行模型为 thinking 模式的 `deepseek-v4-pro`。绩效数字只在该次运行真实完成并通过 `traderharness audit` 后发布——README 不会把估算或占位数字当作真实结果发布。 ## `traderharness demo` 等于模拟实盘吗？ 不等于。`demo` 回放一段已录制的掩码历史运行，全程无网络调用。模拟实盘——面向流式行情的前推模拟模式——尚不存在，见[路线图](roadmap.md)。