SAYLOR The result you are building
An auditable bot PnL statement based on wallet/exchange cash flows and inventory, net of every fee/slippage/failure/transfer, separated into realized/unrealized results, and tested against out-of-sample baselines without survivorship or look-ahead bias.
Use this guide when: determining whether a bot actually makes money, or comparing a strategy, route, size, or fee change.
Do not use it as a substitute for: trusting a dashboard's win rate, gross PnL, or green exit labels without cash-flow reconciliation, or optimizing on the same period you'll use to claim expected profit.
Before you change anything, collect: all fills/transactions/orders/signatures and wallet/exchange transfers; starting/ending balances and open inventory with mark source/time; protocol/platform/network/priority/borrow/funding/slippage/failed costs; strategy version, signals, universe, downtime, and rejected/missed trades.
Stop live scaling when reconciled PnL differs materially from bot logs, sell/withdrawal is untested, risk limits fail, the sample is too small, or drawdown/cost exceeds the predeclared stop.
Understand the system before fixing it
- Cash flow outranks internal labels. Wallet/exchange balance changes plus inventory are what actually explain economic result — not what the bot's dashboard says.
- Win rate alone is incomplete. Average win/loss size, fees, tail events, and correlation are what actually determine expectancy.
- Backtests need execution realism. Use only information available at decision time, and model latency, liquidity, price impact, failures, delistings, rugs, and opportunity constraints — not a frictionless fill at the historical close.
Evidence-to-decision map
| Evidence | Likely layer | First decisive check | What the result means |
|---|---|---|---|
| Bot says profit; wallet balance down | Accounting | Reconcile every transfer/fill/fee/inventory item | Omitted costs, withdrawals, dust, stale marks, or duplicates |
| Backtest strong, live weak | Execution/overfit | Compare signal-to-fill latency/price/impact/rejects | Look-ahead bias, selection bias, stale quotes, or wrong cost assumptions |
| High win rate, net loss | Payoff/tails | Compute expectancy and largest loss/cost | Small wins can't cover the losses and fees |
| PnL depends on one trade/token | Concentration | Break down contribution by trade/asset/time | Not robust — survivorship/outlier risk |
Step-by-step procedure
01. Define the accounting boundary. Deposits and withdrawals can masquerade as PnL. Choose the wallets/accounts/period/base currency, and classify external transfers, trading cash flows, fees, inventory, rewards/airdrops, and owner capital separately. Beginning balance + net external flows + trading PnL should equal ending equity (within marks) — resolve any discrepancy before going further.
02. Reconstruct each round trip. Bot logs can miss partial fills and failures. Use confirmed exchange/chain records to pair entries and exits and open inventory, including actual fill amount, average price, all fees, failed attempts, transfers, taxes, and slippage against the decision quote. Give every trade one immutable ID linking intent, order, signature, fill, and result — leave genuinely unknown items unresolved rather than silently estimating them.
03. Calculate robust metrics. A headline return figure hides path and capital use. Compute net realized/unrealized PnL, expectancy, profit factor, hit rate, average/median/tail outcomes, max drawdown, exposure, turnover, time in market, cost share, capacity, and contribution concentration — and report confidence/sample size alongside them. Separate strategy edge from rewards or market beta.
04. Audit backtest integrity. Data leakage manufactures fake edge. Use a point-in-time universe and data (no future labels), realistic detection/submission/inclusion/exit timing, real fees/impact/failed-tx modeling, delisted/rugged assets, wallet limits, and account for missing data. A walk-forward/out-of-sample period should be untouched by any tuning, and compared against a simple baseline.
05. Compare predicted vs. realized execution. Edge often dies between signal and fill. For each live trade, record the decision quote/time, submit/confirm time, route, expected vs. actual output, priority, rejections, and missed opportunities, then bucket slippage by liquidity/size/latency. Adjust or stop the strategy when net expectancy after real execution is non-positive — don't hide unfilled losses.
06. Set an honest scale gate. Increasing size changes both market impact and drawdown. Require a minimum number of trades/time/regimes, a positive after-cost out-of-sample and live result, bounded drawdown, a tested sell/recovery path, a capacity curve, a kill switch, and a maximum loss per day/position before scaling — then scale incrementally and re-audit.
Worked example
Starting problem: a bot reports 68% winning trades but the wallet loses SOL over a month.
Evidence collected: gross trade PnL excludes priority fees and failed transactions; small winners average 3% while losers average 18%; transfers to a funding wallet are misclassified as PnL; two open illiquid tokens are marked at their last trade price.
Decision: the win rate is masking negative expectancy, real cost leakage, and optimistic inventory marks.
Actions taken: rebuilt PnL from actual wallet transactions and conservative executable marks; included all fees, failures, and transfers as separate line items; added max-loss, liquidity/exit, net-edge, and reconciliation gates before allowing further scaling.
Proof of completion: reconciled equity matches the wallets exactly; strategy expectancy and cost contribution are explicit; the bot is paused or scaled only if live after-cost criteria actually pass.
Why this matters: the profitable-looking story disappears the moment it's measured as realizable cash flow.
Verify, recover, and hand off
An audit is complete only when: starting/ending equity and external flows reconcile; every logical trade includes actual fills/fees/failures/open inventory; marks are timestamped and executable/conservative, not optimistic; net metrics include tails/drawdown/capacity/sample uncertainty; the backtest is genuinely point-in-time and out-of-sample; and live scale/kill-switch gates are actually enforced, not just documented.
If transactions don't match up, that's usually an airdrop/transfer/partial-fill/missing-wallet-data issue — classify it with evidence, or leave it unresolved and lower your confidence rather than guessing. If a mark shows profit but you can't actually quote an exit, use a zero/conservative liquidation scenario and a hard risk flag instead. If results change on every run, your data or pairing logic isn't deterministic — snapshot inputs and use stable trade IDs/rules. If only the optimized period looks profitable, that's overfitting — require a holdout/walk-forward comparison against a simpler baseline before making any live claim.
Reusable handoff record: the accounting boundary and reconciliation equation; trade-level cash flows/costs/inventory; net performance/risk/capacity/concentration metrics; backtest leakage and live execution audit; the scale/pause/kill-switch decision with its limitations stated.
For agents
An agent managing or auditing a trading bot should compute PnL from wallet/exchange cash flow, never from the bot's own internal accounting, and should treat any scale-up request as blocked by default until the gates in step 06 (minimum sample, positive after-cost live result, bounded drawdown, kill switch) are demonstrably met.
Official references: https://www.cftc.gov/LearnAndProtect/AdvisoriesAndArticles/fraudadv_tradingbot.html · https://www.investor.gov/introduction-investing/investing-basics/glossary/backtesting
*This is educational technical and risk-analysis information, not financial, investment, legal, or tax advice. Blockchain transactions can be irreversible and no checklist can guarantee safety or profit.*