MohnishM21/RL-Missile-Guidance

# 🚀 Autonomous Missile Guidance using Reinforcement Learning      # 📌 Overview This project implements a custom aerospace reinforcement learning environment where an autonomous missile learns to: * Navigate toward a target * Avoid hostile interceptors * Maintain stable flight dynamics * Optimize trajectories under nonlinear aerodynamic constraints The framework combines: * **Reinforcement Learning (PPO)** * **Missile-interceptor pursuit-evasion dynamics** * **Aerodynamic flight modeling** * **Trajectory optimization** * **Hierarchical guidance concepts** * **Physics-based simulation** The environment is built entirely from scratch using **Gymnasium**, **PyTorch**, and **Stable-Baselines3**. # 🎯 Project Objectives The goal of this research-oriented project is to investigate how Reinforcement Learning can autonomously learn: ✅ Missile guidance laws ✅ Evasive maneuvering ✅ Stable nonlinear flight control ✅ Energy-efficient trajectories ✅ Pursuit-evasion strategies ✅ Autonomous decision-making in aerospace systems # 🧠 Core Features ## ✈️ Custom Missile Dynamics The missile model includes: * Nonlinear translational dynamics * Pitch dynamics * Aerodynamic lift and drag * Gravity effects * Torque-based control * Supersonic aerodynamic modeling ## 🎯 Interceptor Pursuit Logic The interceptor implements: * Pursuit guidance behavior * PD-based tracking controller * Autonomous missile chasing * Dynamic engagement geometry ## 🤖 Reinforcement Learning Pipeline Implemented using **Proximal Policy Optimization (PPO)**: * Stable-Baselines3 * Continuous action-space control * Dense reward shaping * Trajectory optimization * Long-horizon learning ## 📊 Visualization System The project automatically generates: * Missile trajectory plots * Interceptor pursuit paths * Training evolution animations * PPO learning curves * TensorBoard logs # 🏗️ Environment Architecture Missile RL Environment │ ├── Missile Dynamics ├── Interceptor Dynamics ├── Aerodynamic Solver ├── Reward System ├── PPO Agent ├── Physics Integration (RK45) ├── Visualization Callback └── TensorBoard Logging # 🧮 Mathematical Modeling The simulation incorporates: ## Missile Equations of Motion * Velocity dynamics * Flight path angle dynamics * Angular rate dynamics * Pitch control torque ## Aerodynamic Forces * Lift coefficient approximation * Drag modeling * Compressibility corrections * Supersonic flow approximations ## Numerical Integration * SciPy RK45 solver * Continuous-time state propagation # 🛠️ Tech Stack | Component | Technology | | ---------------- | ----------------- | | RL Algorithm | PPO | | RL Library | Stable-Baselines3 | | Deep Learning | PyTorch | | Environment | Gymnasium | | Numerical Solver | SciPy RK45 | | Visualization | Matplotlib | | Logging | TensorBoard | | Language | Python | # 📂 Project Structure RL-Missile-Guidance/ │ ├── RL4.py ├── README.md ├── requirements.txt │ ├── models/ │ └── PPO trained models │ ├── plots/ │ └── trajectory visualizations │ ├── gifs/ │ └── training evolution GIFs │ ├── tensorboard/ │ └── PPO logs │ └── reports/ └── project reports and documentation # ⚙️ Installation ## Clone Repository git clone https://github.com/MohnishM21/RL-Missile-Guidance.git cd RL-Missile-Guidance ## Create Virtual Environment python -m venv venv Activate environment: ### Windows .\venv\Scripts\activate ### Linux/Mac source venv/bin/activate ## Install Dependencies pip install stable-baselines3 torch gymnasium scipy matplotlib numpy pillow tensorboard ipython # ▶️ Running the Project Run the training environment: python RL4.py # 📈 TensorBoard Visualization Launch TensorBoard: tensorboard --logdir=ppo_missile_tensorboard Open: http://localhost:6006 You can monitor: * PPO rewards * Policy loss * Value loss * Entropy * Learning progress # 📷 Sample Outputs ## Missile Trajectory Evolution * Early unstable trajectories * Mid-training stabilization * Learned glide trajectories * Interceptor avoidance behavior ## Training Animation The framework generates animated GIFs showing policy evolution during training. # 🔬 Research Relevance This project relates to: * Autonomous missile guidance * UAV autonomy * Aerospace reinforcement learning * Pursuit-evasion differential games * Flight control systems * AI for aerospace applications # 🚀 Future Improvements Planned extensions include: * 3D six-DOF missile dynamics * Multi-agent adversarial RL * Advanced guidance laws * Attention/Transformer policies * Unreal Engine / AirSim integration * Domain randomization * Sim-to-real transfer # 📚 Potential Applications * Autonomous UAV guidance * Missile interception research * Defense AI systems * Aerospace trajectory optimization * Intelligent flight control * Guidance and navigation systems # 👨‍💻 Author ## Mohnish M B.Tech Aerospace Engineering Indian Institute of Technology Madras (IIT Madras) ### Interests * Reinforcement Learning * Autonomous Systems * Guidance Navigation & Control * Flight Dynamics * UAV Systems * Aerospace AI GitHub: [MohnishM21](https://github.com/MohnishM21?utm_source=chatgpt.com) # ⭐ Acknowledgements * Stable-Baselines3 * OpenAI Gymnasium * PyTorch * SciPy * Aerospace Guidance & Control Research Community # 📜 License This project is intended for research and educational purposes.