Gradient Descent Explorer

Explore how the learning rate changes gradient descent. In 1D, choose a function and start x, then Step or Run. For (x−2)², x=0 and η=0.1, f′(x)=−4 and the next x is 0.4, with f(x)=2.56. Each step uses xₙ₊₁=xₙ−ηf′(xₙ). The table keeps the latest 20 steps; the curve includes the full path and adjusts its range.

In 3D, Render renders the chosen preset. A click on the surface selects the common start; dragging rotates and Ctrl/Meta+wheel zooms. Compare GD, Momentum (β=0.9), Adam (β₁=0.9, β₂=0.999), and simplified fixed-step CG/BFGS. Pause/Run preserves optimizer memory. Changing the rate, selection, landscape or smoothing restarts the paths. Plain wheel scrolls the page.

These are normalized sampled landscapes, not a trained model: peaks, a saddle, or log(1+Rosenbrock)/5. Images use smoothed grayscale normalized to 0–1, at most 72 samples per side. Brighter pixels are higher loss, coordinates are pixel indices. Gradients use numerical differences of bilinear interpolation; the rendered triangles approximate that surface. CG/BFGS have no line search and can fail. No stochastic minibatches.

Runs stop at 500 steps, a small gradient, or an invalid step. 3D paths stop at the image boundary rather than falsely declaring convergence; 1D stops if |x| exceeds 10⁶. A small gradient can also be a saddle or maximum. Image limit: 5 MiB / 16 megapixels. All calculations stay in the browser; WebGL is only needed for the 3D view. Hidden tabs pause the run.

CSV exports the entire recorded path at full numerical precision, including steps no longer shown in the table.

Only input values are remembered in this browser. Calculated results and experiment histories are not saved.