1. Core Problem
Given a universe of Indian equities from NIFTY indices with historical OHLCV data at multiple granularities, learn a trading policy that:
- Selects which stocks to hold in a multi-slot portfolio
- Decides when to enter and exit positions
- Maximizes risk-adjusted returns after realistic costs
- Respects transaction costs, slippage, and broker constraints
- Generalizes across market regimes (bull, bear, sideways, crisis)
2. Formal MDP Formulation
The problem is formulated as a finite-horizon Markov Decision Process (MDP):
- State space S: Continuous, high-dimensional. Includes per-stock technical features, portfolio state, and global market context.
- Action space A: Discrete with HOLD, CLOSE position, and OPEN position actions.
- Transition dynamics T(s'|s,a): Determined by market price evolution (exogenous) plus position tracking (endogenous). The agent cannot influence market prices.
- Reward function R(s,a,s'): Composite, risk-conditioned signal combining portfolio value change, transaction cost penalty, drawdown aversion, cash underutilization penalty, and profit-taking bonuses.
- Discount factor γ: 0.99 (standard)
- Horizon H: 252 bars per episode (1 trading year of daily data)
3. Key Challenges
Financial markets present unique difficulties for reinforcement learning:
Non-stationarity
Markets exhibit regime changes. A policy trained on 2019-2022 bull market data may fail catastrophically in 2022-2025 consolidation.
Partial observability
The agent observes price/volume features but not order flow, institutional positioning, corporate events, or macroeconomic surprises.
High transaction costs
Indian equity markets charge meaningful fees per trade (STT, stamp duty, exchange charges, GST). Excessive trading destroys alpha.
Sparse signal
Daily returns have very low signal-to-noise ratio. Most price movements are noise; genuine alpha signals are rare and weak.
Multi-objective nature
Maximizing returns conflicts with minimizing drawdowns. No single reward function captures all preferences.
Sample efficiency
With ~252 trading days per year and limited historical data, the effective training dataset is small relative to the observation dimensionality.
Survivorship bias
Historical data only includes stocks that still exist. Stocks that were delisted, merged, or delisted due to poor performance are missing.
Execution risk
The gap between simulated and live execution (slippage, partial fills, market impact, latency) can erode theoretical alpha.
4. Market Context: Indian Equities
Market Structure
- Exchange: National Stock Exchange (NSE) — primary venue
- Trading hours: 09:15 - 15:30 IST (375 minutes per trading session)
- Trading days: ~252 per year (excludes weekends + 15-18 NSE holidays)
- Settlement: T+1 for equities (since January 2023)
- Circuit breakers: ±5%, ±10%, ±20% for individual stocks; market-wide at 10%, 15%, 20%
NSE-Specific Market Microstructure
- Tick size: ₹0.05 for most stocks (≥₹1 price)
- Lot sizes: 1 share for equity (no minimum lot in cash segment)
- Order types: Market, Limit, SL, and advanced variants
- Pre-open session: 09:00-09:08 (price discovery), 09:08-09:12 (order matching)
- Closing price: Weighted average of last 30 minutes (15:00-15:30)
- FII flows: Major driver of short-term volatility
- Retail participation: ~50% of NSE turnover since 2020 (post-COVID spike)
Market Regimes in Recent History
| Period | Regime | NIFTY50 Annualized Return | Max Drawdown |
|---|---|---|---|
| 2015-2017 | Moderate bull | +8-12% | -12% |
| 2018-2019 | Consolidation/correction | +2-5% | -17% |
| 2020 Q1 | COVID crash | N/A (crash) | -38% |
| 2020 Q2 - 2021 | Strong bull (liquidity) | +60-80% | -8% |
| 2022 | Rate hike consolidation | -2% | -18% |
| 2023-2024 | Gradual recovery | +15-20% | -10% |
| 2025 | Mixed (election, global) | +5-8% | -15% |
These regime shifts are the primary challenge. A model trained on 2020-2021 bull market data will not generalize to 2022 consolidation without explicit regime awareness.
Universe Composition
| Index | Stocks | Liquidity | Spread | Use Case |
|---|---|---|---|---|
| NIFTY50 | 50 | Very high | 3-5 bps | Primary training universe |
| NIFTY NEXT 50 | 50 | High | 5-10 bps | Extension universe |
| NIFTY 100 | 100 | High-Medium | 5-15 bps | Balanced universe |
| NIFTY 500 | 500 | Mixed | 10-50 bps | Research (slippage concerns) |
| Full NSE | 700+ | Many illiquid | >50 bps for small-cap | Data coverage only |
5. Transaction Cost Structure
Indian equity markets have a complex fee structure that significantly impacts strategy profitability:
| Component | Delivery (CNC) | Intraday (MIS) |
|---|---|---|
| Brokerage | ₹0 (free) | Small percentage or flat fee |
| STT | Material % on both sides | Material % on sell side only |
| Exchange charges | Small % per turnover | Small % per turnover |
| SEBI turnover fee | Fixed per-crore charge | Fixed per-crore charge |
| Stamp duty (buy only) | Small % on buy | Tiny % on buy |
| GST | Levied on (brokerage + exchange + SEBI) | Levied on (brokerage + exchange + SEBI) |
| DP charges (sell) | Fixed per company | N/A |
Effective cost per leg is material for delivery trades. A strategy must generate sufficient alpha per trade just to break even after costs. With frequent rebalancing, turnover costs compound significantly.
6. Walk-Forward Validation
We use a rigorous walk-forward protocol to prevent curve fitting:
For each fold: 1. Train on multi-year historical window 2. Validate on subsequent months 3. Test on held-out period after validation 4. Step forward and repeat 5. Repeat Aggregate: mean(Sharpe), std(Sharpe), max(drawdown), consistency ratio
Observed walk-forward degradation: Validation Sharpe → Test Sharpe degradation is significant on NSE. This gap is inherent to regime shifts and is one of our primary research targets.
7. Evaluation Protocol
Statistical Rigor Requirements
- Multiple seeds: Minimum 5 random seeds per configuration
- Confidence intervals: Report mean ± std, and 95% CI for Sharpe ratio
- Paired comparisons: Wilcoxon signed-rank test for algorithm A vs B
- Multiple testing correction: Bonferroni or FDR when comparing >2 algorithms
- Effect size: Cohen's d for Sharpe ratio differences
Validation Metrics
We evaluate using standard risk-adjusted metrics: Sharpe ratio, Sortino ratio, maximum drawdown, Calmar ratio, win rate, and profit factor. All reported on held-out test data the agent never saw during training.
8. Baseline Benchmarks
Every RL strategy must be compared against:
- Buy-and-hold NIFTY50: Equal-weight buy and hold
- 60/40 equity/bond: NIFTY50 60% + India 10Y bond 40%
- Momentum factor: Long top-20% momentum, short bottom-20%
- Mean-reversion factor: Buy oversold, sell overbought (RSI-based)
- Random policy: Uniform random action selection
- Always hold: Never trade (zero cost baseline)
NIFTY50 Buy-and-Hold Reference
Buy-and-hold NIFTY50 is our primary benchmark. Historical performance varies significantly by period, from strong bull markets to severe drawdowns during crises. Any active strategy must convincingly outperform this benchmark on a risk-adjusted basis after costs to justify the complexity.