This blog post concerns a famous toy problem in Reinforcement Learning, the [FrozenLake environment](https://gym.openai.com/envs/FrozenLake-v0/). We compare solving an environment with RL by reaching **maximum performance** versus obtaining the **true state-action values** $Q_{s,a}$.