Reinforcement Learning Math

Welcome to the documentation for Reinforcement Learning Math. This project focuses on providing detailed mathematical foundations and derivations in the field of reinforcement learning (RL), with an emphasis on deep reinforcement learning (DRL) methods. The aim is to build a comprehensive and intuitive guide to understanding the theoretical underpinnings of RL algorithms.

Overview

Reinforcement learning is a powerful paradigm for decision-making and control tasks. It involves agents learning to make decisions by interacting with an environment to maximize cumulative rewards. To facilitate a deeper understanding, this documentation includes:

• A formal definition of commonly used mathematical symbols in reinforcement learning.

• Detailed derivations of key algorithms, starting with policy gradient methods in deep reinforcement learning.

Contents

• Notations
  • Bellman Expectation Equation
  • References
• Markov Decision Processes
  • Introduction
  • Markov Processes
  • Markov Reward Processes
  • Markov Decision Processes
  • References
• Monte Carlo Methods
  • Introduction
  • Monte Carlo Prediction
  • Monte Carlo Control
  • Importance Sampling
  • Incremental Implementation
  • Conclusion
  • References
• Temporal Difference Learning
  • Introduction
  • TD Prediction
  • TD Control
  • Eligibility Traces
  • Conclusion
  • References
• Policy Gradient
  • Policy Gradient Theorem
  • Proof of Policy Gradient Theorem
  • References
• Soft Actor-Critic (SAC) Algorithm
  • Introduction
  • Theoretical Derivation
  • Algorithmic flow
  • References
• Bayes Theorem
  • Introduction
  • Mathematical Formulation
  • Proof
  • Intuitive Explanation
  • Key Insights
  • Conclusion
• Attention Is All You Need
  • Introduction
  • Architecture Overview
  • Self-Attention Mechanism
  • Multi-Head Attention
  • Code Implementation
  • Conclusion
  • References
• Understanding GPT as an Attention-Driven Decoder
  • Introduction
  • Decoder-Only Architecture
  • Principles
  • Mathematical Formulation
  • Code Implementation
  • Conclusion
  • References