Generalist vs.Specialist
Pretrained Decision Models Against Reinforcement Learning Agents in a Real-Time Arena Combat Game
A game-trained PPO agent and a frozen pretrained decision model will compete under shared observations and controls to evaluate combat performance and robustness to unfamiliar mechanics.
A compact game for 1v1 arena combat
Combat inspired by Brawl Stars, with original characters and assets, provides observable and repeatable agent matches.
- 01Arena
A top-down 1v1 map with walls and cover.
- 02Combat
Projectiles with travel time and spread, health and attack cooldowns.
- 03Characters
Different stats and a special ability for each character.
Godot 4 + C++
C++ gameplay through GDExtension / godot-cpp.
Two agents with different learning histories
PPO
- Learning
- Train from experience in this arena.
- Opponents
- Scripted opponents, followed by self-play.
- Input
- Structured observation vector.
- Evaluation
- Freeze the trained policy before testing.
Pretrained
- Knowledge
- Pretrained capabilities + supplied game rules.
- Execution
- Local GPU inference, batched across matches.
- Input
- Equivalent state information rendered as text.
- Evaluation
- Keep model weights frozen throughout.
A fair comparison on shared game-time ticks
Both agents observe the same simulation instant, and the game waits for both decisions before advancing, so slower inference gives neither side extra combat time; wall-clock decision latency is measured separately.
One information set
Encode the same observable facts as a vector and as text.
Both choose actions
No access to the opponent’s pending decision. Inference pauses game time.
Shared controller
Execute both choices over the same fixed interval, then repeat.
Who wins—and what happens when rules change?
Keep agent weights fixed during evaluation, train PPO in independent runs, repeat matches with swapped sides, and report confidence intervals rather than a single win percentage.
Combat performance
Compare the agents across the implemented character matchup matrix and against scripted baselines.
Generalization without retraining
Test character configurations and rule changes not used in PPO training, with the changed mechanics included in both agents’ observations.
Win rate
Primary outcome, with draws and confidence intervals.
- Damage dealt / taken & time to defeatCombat effectiveness
- Ability usage & positioningBehavioural diagnostics and replay analysis
- Performance under unfamiliar conditionsRobustness to new characters and rule changes
- Training time, latency, throughput & memorySupporting resource measurements
Build the game to compare PPO and a pretrained model
The project combines an original arena game, shared observations and controls, and a reproducible experimental comparison.
Proposal & foundations
Review relevant literature, define the scope and experiments, and validate the engine bridge and local model.
Working prototype
Build a stable arena, test the combat rules, integrate both agents, and collect initial training and evaluation evidence.
Results & final report
Compare character matchups and unfamiliar game conditions, then analyse performance, failure cases and study limitations.
Deliverables: game + agents + replays + results + report.