System Overview
Our platform follows a closed research loop: data ingestion → feature engineering → agent training → evaluation → promotion → (eventually) live trading. Each stage is gated — an agent cannot advance without clearing measurable criteria.
At a high level, the data flows through three planes:
- Data Plane: Historical OHLCV from broker APIs → storage → feature pipeline
- Model Plane: Feature vectors → custom trading environment → RL agent → checkpoint registry
- Simulation Plane: Trained policy → backtest engine → simulated broker with realistic costs → performance metrics
The same agent code runs against both the simulated broker (for validation) and the live broker (for eventual deployment) through a common interface. This ensures that what we validate is what we trade.
Reinforcement Learning
We use actor-critic methods with continuous action spaces and twin value networks. The algorithm is sample-efficient, entropy-regularized, and off-policy — all desirable properties for financial data where experience is limited.
We also maintain implementations of PPO, TD3, A2C, and DQN for comparative benchmarking. The primary algorithm has shown promising results in our experiments.
Walk-Forward Validation
Standard train/test splits are insufficient for financial time series because they assume i.i.d. data. Markets are non-stationary — a model trained on a bull market will fail in a bear market.
Our walk-forward protocol uses rolling windows:
- Train on multi-year historical window
- Validate on subsequent months
- Test on held-out period after validation
- Step forward and repeat
This produces ~20-30 independent folds. An agent must generalize across all folds — not just one lucky period.
Dual Training Backends
We maintain two independent training implementations:
- JAX (primary research): XLA-compiled, pure functional, vectorized. Fastest training speed. Runs on Linux with CUDA.
- PyTorch (rapid iteration): Desktop native, easier debugging, rapid prototyping. Runs on desktop OS with CUDA.
Both implementations are validated against each other — identical hyperparameters should produce statistically equivalent results. This cross-validation catches implementation bugs and ensures that our findings are backend-agnostic.
NSE-Realistic Cost Modeling
Ignoring transaction costs is the most common failure mode in retail algo trading. Our simulation includes:
- STT (Securities Transaction Tax): Material percentage on both sides for delivery
- Exchange charges: Small percentage per turnover
- SEBI turnover fee: Fixed per-crore charge
- Stamp duty: Small percentage on buy side
- GST: Levied on brokerage + exchange + SEBI
- DP charges: Fixed per-sell-company charge
- Slippage: Configurable percentage based on liquidity
Effective round-trip costs for delivery trades are material. A strategy must generate sufficient alpha per trade just to break even.
Risk-Aware Reward Design
Standard RL maximizes cumulative return. In trading, this produces reckless agents that take excessive risk. Our reward function incorporates risk discipline directly:
- Portfolio value change: The primary signal — log returns scaled appropriately
- Transaction cost penalty: Penalizes excessive trading proportional to costs incurred
- Drawdown aversion: Penalizes declines from peak portfolio value
- Cash underutilization: Mild penalty for holding too much cash
- Concentration limits: Penalizes over-concentration in single positions
- Holding incentives: Rewards letting winners run, penalizes stale positions
The weights are tuned to produce risk-adjusted returns rather than raw return maximization.
Promotion Gates
An agent advances through a strict promotion ladder:
- SIM → PAPER: Checkpoint exists + backtest metrics pass basic thresholds
- PAPER → CANARY: Sustained paper trading period + all health checks green
- CANARY → FULL: Extended stable period + no manual interventions + risk within limits
Each gate has explicit, measurable criteria. No exceptions. An agent that cannot pass paper trading does not see real capital.