RL Gridworld

Select a grid and Q-learning or SARSA. Train runs a bounded number of episodes; Pause and Continue preserve the current episode and random sequence. One step performs one environment transition and shows the old Q value, update target and new value. Changing settings clears learning.

Select any accessible cell to inspect its four Q values and actual ε-greedy action probabilities. Arrows show the best actions; equally best actions share the greedy probability. Exploration chooses among all four actions. S marks the start, G the goal; the inset outline shows the agent, blue borders the last/current trace.

The goal gives +1 and ends the episode. Every other move gives −0.01, including hitting a wall or boundary; these moves leave the agent in place. Episodes stop after 200 steps. Q-learning uses reward + γ max Q(next); SARSA uses reward + γ Q(next, sampled action). Terminal targets contain only the reward. The update is Q ← Q + α(target − Q).

Seed reproduces a run with identical settings. Limits: 1–1000 episodes, ε/α/γ between 0 and 1. The reward plot shows each completed episode, including capped episodes. This is a small deterministic grid demonstration with stochastic actions, not a guarantee of convergence or an optimal learned policy. Neither method decays exploration automatically.