MARIOAI

A Python-focused base for the Mario AI tool, evolved with Genetic Programming to create autonomous Mario controllers.

Report: IAIN_J_VAZ_L_SILVA_REPORT.pdf

---

Overview

This project applies Tree-based Genetic Programming (GP) to evolve autonomous Super Mario Bros. controllers using the DEAP framework. Unlike neural networks, evolved GP controllers remain directly interpretable as symbolic Python programs.

The work is detailed in the report "Evolving Autonomous Mario Controllers using Genetic Programming" by João Vaz and Leonardo Silva (Department of Informatics Engineering, University of Coimbra).

---

Key Results

Move Forward Task

• Best generalization: 64.4% level completion rate across unseen seeds and difficulties
• Distance improvement: +1089.7% over random baseline (1929.2 vs. 162.1 units)
• Best config: pop=300, generations=100, crossover=0.8, mutation=0.2

Hunter Task

• Best generalization: 44.4% completion, 1.33 average kills per run
• Kills improvement: +1109.1% over random (1.33 vs. 0.11)
• Distance improvement: +376.4% over random (1692.2 vs. 355.2 units)
• Best config: pop=300, generations=200, crossover=0.9, mutation=0.2

Critical finding: In the Move Forward task, the highest-training-fitness configuration overfits and generalizes poorly (44.4% completion), while higher mutation rates (pm=0.2) produce more robust controllers (64.4% completion). In the Hunter task, the highest-fitness configuration also achieves the best generalization.

---

Project Structure

Mario_Controller_Genetic_Programming/
├── code/                    # Source code
│   ├── marioai/             # Core package
│   │   ├── __init__.py      # Package init
│   │   ├── agent.py         # Base Agent class
│   │   ├── environment.py   # TCP connection to MarioAI server
│   │   ├── task.py          # Task base class
│   │   ├── experiment.py    # Experiment runner
│   │   ├── utils.py         # Observation parsing
│   │   ├── move_forward.py # MoveForwardTask fitness function
│   │   └── hunter.py        # HunterTask fitness function
│   ├── tasks/               # Task definitions
│   │   ├── move_forward.py
│   │   └── hunter.py
│   ├── data/                # Evaluation data and best agents
│   │   ├── hp_results/      # Hyperparameter search results
│   │   └── gp_best_agents/  # Best evolved agents per config
│   ├── evaluate_best_agent.py  # Evaluate a GP champion
│   ├── mario_random_search_gp.py  # Random search baseline
│   ├── mario_gp_evolution.py     # GP evolution script
│   ├── hp_search.py         # Hyperparameter search
│   ├── plot_convergence.py      # Convergence plotting
│   ├── random_agent.py      # Random agent baseline
│   └── install_requirements.py  # Install dependencies
├── docs/                    # Documentation
│   ├── IAIN_J_VAZ_L_SILVA_REPORT.pdf  # Full research report
│   ├── NIAI_MEI_Project_2026.pdf
│   └── Screencast%20from%202026-06-09%2000-46-05.webm  # Demo video
├── code/SuperMarioServer/   # MarioAI server
├── papers/                  # Related papers
└── README.md                # This file

---

Genetic Programming Details

Primitive Set

The GP evolves executable Python scripts as Abstract Syntax Trees (ASTs) using:

Internal nodes (functions):

• if-then: Condition → Expr
• if-then-else: Condition, Expr, Expr → Expr
• sequence: Expr, Expr → Expr
• set-action: Key, Bool → Expr
• check-enemy: Pos, Pos, Comp, EnemyType → Condition
• check-landscape: Pos, Pos, Comp, LandscapeType → Condition
• and, or, not: Condition → Condition

Leaf nodes (terminals):

• Actions: RIGHT, JUMP, LEFT, SPEED, DOWN (Key type)
• TRUE, FALSE (Bool type)
• on_ground, can_jump (Condition type)
• Positional offsets: {−1, 0, 1, 2, 3} (Pos type)
• Enemy types: Goomba, Koopa, Piranha, etc. (11 types) (EnemyType type)
• Obstacle types: −11, −10, 0, 16, 20, 21 (LandscapeType type)
• Comparisons: ==, != (Comp type)
• pass (Expr type)

Evolutionary Process

• Elitist (µ + λ) generational loop
• Parent selection: Tournament selection (t=3)
• Crossover: One-point subtree crossover (pc)
• Mutation: Node replacement mutation (pm)
• Bloat control: Static limit on tree height (max 17 nodes)
• Elitism: Best 3 individuals copied unchanged

Hyperparameter Search Results

Move Forward Task (Table 2):

Pop	Gen	pc	pm	Mean Fitness	Std	Min	Max
300	200	0.9	0.1	20246.05	4483.06	12130.23	23804.11
300	200	0.9	0.2	17454.95	4926.88	11664.58	24063.06
300	100	0.8	0.2	16675.77	3936.99	11862.88	23293.37

Hunter Task (Table 3):

Pop	Gen	pc	pm	Mean Fitness	Std	Min	Max
300	200	0.9	0.2	20367.57	2924.62	17442.95	23292.19
300	200	0.8	0.2	19100.83	4314.04	14786.80	23414.87
300	200	0.9	0.1	16606.24	1819.44	14786.80	18425.68

---

Tasks

Move Forward Task

• Objective: Maximize distance traveled through procedurally generated levels
• Fitness: (Δx)^1.2 × c per step (c=1.0 on ground, c=2.5 in air), minus penalties for enemies, pipes, and pits
• Key insight: The reward shaping was developed iteratively (V1-V12) to address failure modes like idling, hazard ignorance, and slow airborne movement

Hunter Task

• Objective: Evolve controllers that actively hunt and eliminate enemies
• Fitness: Move Forward reward + kill bonus (+250) + stomp bonus (+700 for airborne kills)
• Kill detection: Uses compute_killed() to filter false positives (enemies falling into pits or leaving view)
• Positioning bonus: +5 for being airborne above an enemy
• Key insight: Precise kill detection + strong signals (+250/+700) produce the richest, most interpretable programs with multi-directional enemy checks

---

Experimental Setup

Environment

• Connects to MarioAI simulator via TCP on port 4242
• Levels: procedurally generated, increasing difficulty
• Observation: 22×22 grid centered on Mario + enemy position data
• Actions: [backward, forward, crouch, jump, speed/bombs]

Hyperparameter Configuration

• Population size: 200-300 (300 chosen for balance)
• Generations: 100-200
• Crossover probability: 0.8-0.9 (0.9 dominates)
• Mutation probability: 0.1-0.2 (0.2 better for Hunter task)
• Elite size: 3 individuals
• Tournament size: 3
• Tree max depth: 17 nodes

Convergence Analysis

• Move Forward: Two-phase pattern — rapid improvement (0-20 gens), plateau (20-80), modest recovery near gen 80
• Hunter: Slow growth until gen 40, breakthrough gen 40-80 (jump-based combat), then flatline

---

Getting Started

Prerequisites

• Python 3.10
• Docker (for evaluation server)

Installation

# Create clean environment
conda create -n NIAI python=3.10
conda activate NIAI

# Navigate to project and install
cd Mario_Controller_Genetic_Programming
python install_requirements.py

Launch Evaluation Server

docker compose up

Wait about 1 minute, then verify at http://localhost:3000.

Test the Setup

Run the random search files to verify everything works:

python code/mario_random_search_gp.py

Or evaluate the best GP agents:

python code/evaluate_best_agent.py

---

Using the Project

Making a Python Bot

Inherit the marioai.Agent class:

from marioai import Agent

class MyAgent(Agent):
    def sense(self, obs):
        super().sense(obs)
        # Access: self.can_jump, self.on_ground, self.mario_floats, 
        #         self.enemies_floats, self.level_scene

    def act(self):
        # Return [backward, forward, crouch, jump, speed/bombs]
        return [0, 1, 0, 0, 0]  # Move right

    def give_rewards(self, reward, cum_reward):
        pass

Sensorial Information (default)

• can_jump: boolean — if Mario can jump
• on_ground: boolean — if touching ground
• mario_floats: 2-tuple — Mario position (x, y)
• enemies_floats: list of 3-tuples — enemy positions (type, x, y)
• level_scene: 22×22 numpy array — level elements around Mario (Mario at [11, 11])

Control Signals

Each position in the action array:

• [backward, forward, crouch, jump, speed/bombs]

Example: return [0, 1, 0, 0, 0] → move right Example: return [1, 0, 0, 1, 0] → jump backward

Tasks Folder

Edit the tasks/ folder — contains the reward function acting as fitness for your algorithms.

Agents Module

Edit/create agents in the marioai module.

---

References

de Freitas, J.M., de Souza, F.R., Bernardino, H.S.: Evolving controllers for mario ai using grammar-based genetic programming. In: 2018 IEEE Conference on Computational Intelligence and Games (2018)