CartPole Solutions - RL Algorithm Explorer

Episode

Current Score

Avg (Last 20)

Best Score

Ready - Select an algorithm and click Start

CartPole Environment

Training Progress

Speed: 1x

Hand-Coded Rules

The simplest approach: if the pole tilts right, push right; if it tilts left, push left. This combines the pole angle with its angular velocity for smoother control.

Formula: action = (angle + 0.1 * angular_velocity) > 0 ? RIGHT : LEFT

No Learning Deterministic Instant

Configuration (Click to Expand)

▼

Presets:

Learning Rate 0.01

Higher = faster but unstable

Discount (γ) 0.99

How much to value future rewards

Batch Size

Memory Size

Hidden Layer Size

Target Update Freq

Exploration Strategy

Initial ε 1.0

Final ε 0.01

ε Decay Rate 0.995

ε = ε * decay after each episode

Temperature 1.0

Higher = more random, Lower = more greedy

UCB Constant (c) 2.0

Exploration bonus coefficient

Noise Scale 0.5

Standard deviation of parameter noise

Angle Penalty (penalize large angles)

Velocity Penalty (penalize fast movement)

Center Bonus (reward staying centered)

Survival Bonus (increasing reward over time)

Reward Scale 1.0

reward = 1.0

Random Seed (for reproducibility)

Same seed = same results (for comparing settings)

State Normalization (zero mean, unit variance)

Curiosity Bonus (intrinsic motivation for novel states)

Curiosity Weight 0.1

Imitation Learning (pre-fill memory with expert demos)

Demo Episodes

Expert demonstrations before learning starts

Environment Noise 0.0

Add randomness to physics (harder task)