RL for Trading — Key Papers

Paper	Year	Contribution	Relevance
Moody & Saffell, "Learning to trade via direct RL"	2001	First RL trading agent using policy gradient methods	Historical foundation. Showed RL could learn trading policies before deep learning existed.
Deng et al., "Deep direct RL for financial signal representation"	2017	Deep RL for financial feature learning with autoencoder pre-training	Feature engineering insight: pre-training on price structure helps RL convergence.
Jiang et al., "Deep RL for portfolio management"	2017	Continuous portfolio weights with CNN encoder for price images	Action space design: continuous weights are more expressive than discrete slot selection.
Liang et al., "Adversarial deep RL in portfolio management"	2018	Adversarial training for robustness to market shocks	Generalization: adversarial perturbations during training improve walk-forward performance.
Yang et al., "Deep RL for automated stock trading (FinRL)"	2020	Open-source RL trading framework with benchmark datasets	Closest comparable system. Our architecture differs in NSE-specific cost modeling and validation rigor.
Liu et al., "FinRL: DRL framework for quantitative finance"	2021	Comprehensive FinRL library with multiple algorithms and markets	Baseline comparison: FinRL's default cost models are US-centric and underestimate Indian fees.
Fang et al., "Universal trading for order execution"	2021	Execution optimization with RL using universal value function	Market impact: our slippage model is simpler, but this is the direction for future work.
Sun et al., "DeepScalper: RL-based high-frequency trading"	2022	Intraday RL strategies with minute-bar data	Intraday extension: higher SNR but higher infrastructure requirements.

Meta-Learning & Regime Adaptation

Paper	Year	Contribution	Relevance
Finn et al., "Model-Agnostic Meta-Learning (MAML)"	2017	Gradient-based meta-learning for rapid adaptation	Core algorithm for our Priority 1 research direction: regime-aware meta-learning.
Li et al., "Meta-learning for financial trading"	2020	Meta-RL for market regime adaptation	Directly relevant: shows MAML can adapt trading policies across regimes.
Xu et al., "Meta-learning for trading with switching regimes"	2022	HMM + meta-policy for regime detection and adaptation	Hybrid approach: explicit regime detection + meta-learning. Promising for NSE.

Indian Market Specific

Paper	Year	Contribution	Relevance
Jain & Jain, "ML for Indian stock market prediction"	2019	NSE-specific ML approaches and feature selection	Feature relevance: confirms technical indicators have predictive power on NSE, but lower than US markets.
Gupta et al., "DRL for Indian equity portfolio"	2021	SAC/PPO on NIFTY50 with simplified cost models	Closest Indian RL work. Our work extends this with realistic costs, walk-forward validation, and multi-granularity data.
Mondal et al., "Algorithmic trading in Indian markets"	2023	Market microstructure analysis of NSE	Microstructure: bid-ask spreads, liquidity patterns, and impact costs on NSE. Informs our slippage model.

Risk Management in RL

Paper	Year	Contribution	Relevance
Tamar et al., "Risk-sensitive RL"	2015	CVaR optimization in MDPs	Theoretical foundation for risk-aware reward design. We use CVaR-inspired penalties.
Chow et al., "Risk-constrained RL with CVaR"	2017	Constrained optimization for tail-risk control	Constrained RL: hard constraints on drawdown rather than soft penalties. Future direction.
Dabney et al., "Distributional RL with quantile regression"	2018	QR-DQN for modeling full return distribution	Tail risk: modeling the full distribution rather than just mean Q-values. Relevant for Q20.

Our Commentary

The RL-for-trading literature has grown rapidly, but several gaps remain:

• Cost realism: Most papers use simplified cost models (flat 0.1% or less). Indian markets have complex, non-linear fee structures that significantly impact strategy design.
• Validation rigor: Many papers report results on a single train/test split. Walk-forward validation with multiple folds and statistical testing is rare but essential.
• Market specificity: Papers trained on US equities rarely generalize to Indian markets due to different microstructure, liquidity patterns, and regulatory environments.
• Reproducibility: Few papers release code or exact hyperparameters. We aim to be transparent about methodology even if source code remains proprietary.