offline-rl

What problem does it solve?

This skill enables practitioners to perform offline reinforcement learning from fixed datasets without interacting with an environment, addressing issues like distribution shift and extrapolation errors by adopting conservative value estimation and robust evaluation strategies.

Core Features & Use Cases

• Conservative offline RL methods: CQL, IQL, and BCQ to mitigate overestimation when data is limited or distribution shifts occur.
• Offline training & evaluation: Train purely from logged data and evaluate using offline techniques to estimate policy performance without live rollout.
• Real-world scenarios: Robotics from logged demonstrations, medical treatment policy learning from historical data, and recommendations improved from historical user interactions.

Quick Start

Prepare a fixed dataset, choose a method (CQL, IQL, or BCQ), train the model, and perform offline evaluation using RIS or model-based estimates.