Autonomous Agents AI Simulation

A versatile simulation framework featuring reactive and deliberative agents navigating complex environments.

Explore the Documentation »

Report Bug · Request Feature

Table of Contents

1. About The Project
   • Built With
   • Features & Key Highlights
   • Part 4: Deliberative Agents & Markov Decision Processes (MDP)
   • Part 3: Deliberative Agents & State-Space Search (PEE)
   • Part 2: Reactive Agents in Python
   • Part 1: Reactive Agents in Java
2. Lessons Learned
3. Getting Started
   • Prerequisites
   • Installation & Running Locally
4. Contact
5. Acknowledgments

---

About The Project

vidDemoSimulador4agente.mp4

Demonstration of a deliberative agent navigating a complex maze. The agent evaluates the environment, formulates a sequence of optimal actions using State-Space Search, and executes its plan step-by-step to reach the final goal.

If the embedded video above is not displaying correctly, click here to watch/download the video demo »

This repository contains Autonomous Agents AI Simulation, an academic project developed for the Inteligência Artificial para Sistemas Autónomos course at ISEL (Instituto Superior de Engenharia de Lisboa).

🇵🇹 Documentação em Português: A full, comprehensive academic report detailing the entire theoretical background, architecture, and implementation of this project is available in Portuguese here: academic_documentation/Relatório_IASA_51827.pdf.

Note on Authorship: The vast majority of the codebase was developed from scratch by me, including the core logic, state-space search algorithms, and Markov Decision Processes within the lib package. The development process was guided by the professor (Luís Morgado) using sequential, structural, and class UML diagrams. The only pieces of code provided by the professor are explicitly marked with @author: Luís Morgado in the source code (primarily the sae graphical simulation environment interface).

The primary goal of this project is to explore and implement different artificial intelligence architectures. It covers basic stimulus-response reactive behaviors and advances to deliberative planning using state-space search and Markov Decision Processes.

(back to top)

---

Built With

• 
• 

Built from Scratch: The logic of this project was developed using pure Python and pure Java without any external frameworks. Apart from a strictly defined graphical simulation interface (sae module) provided by the professor—which is clearly marked with @author: Luís Morgado—all the algorithm implementations, object-oriented abstractions, and math operations were fully coded by me using only the languages' built-in features.

(back to top)

---

Features & Key Highlights

• Reactive Agents: Implements a stimulus-response architecture allowing agents to react instantly to environment events without internal planning.
• State-Space Search: Features various search algorithms including Breadth-First, Depth-First, Uniform Cost, Greedy, and A* to find optimal paths.
• Deliberative Planning: Agents evaluate internal models of the world to generate multi-step action plans before execution.
• Markov Decision Processes: Advanced deliberative agents use policies and utilities to navigate environments with uncertainty.

(back to top)

---

Part 4: Deliberative Agents & Markov Decision Processes (MDP)

In the final phase of the project, a fully deliberative architecture was developed that factors in environmental uncertainty. The agent combines classic search logic with Markov Decision Processes (MDP) to calculate optimal policies based on the Bellman Equation, maximizing expected long-term rewards rather than just finding the shortest path.

(The deterministic pathfinding behavior of a deliberative agent can be seen in the featured video at the top of this page).

vidDemoSimulador4agenteDelibPDM.mp4

This simulation displays the agent utilizing Markov Decision Processes. A utility map and a policy vector field are generated across the grid, allowing the agent to continuously adapt its next move to maximize the expected reward, safely guiding it to its target even if unexpected transitions occur.

Conceptual block diagram of the Deliberative Agent Architecture developed in this phase, mapping Perception, Internal Representation, Action Plans, and Execution.

(back to top)

---

Part 3: Deliberative Agents & State-Space Search (PEE)

The third part shifted focus towards deliberative architectures, providing the agent with the ability to reason and plan its actions using State-Space Search (PEE). Multiple classic search algorithms were implemented—including Breadth-First, Depth-First, Uniform Cost, Greedy, and A*.

To evaluate the efficiency of these algorithms, they were tested against a Counting Problem. In this problem, the agent's "state" is a number (e.g., 0), and it must reach a target number (e.g., 9) by applying mathematical increments (e.g., +1, +2, -1). The search algorithms navigate the possible mathematical combinations (the state-space) until they find a sequence of increments that hits the target.

demoOutputProblemaContagemMecanismosProcura.mp4

The video presents a terminal output comparing the execution of the various search algorithms resolving the Counting Problem. It highlights the differences in time complexity, memory usage (space complexity), number of expanded nodes, and path optimality, proving why informed searches like A* often yield the most balanced and efficient results.

Visualization of reasoning through search, demonstrating the transition from an initial state to the objective within a state-space graph.

(back to top)

---

Part 2: Reactive Agents in Python

The second iteration migrated to Python and deepened the complexity of reactive architectures. It introduced the concept of compound behaviors using mechanisms like subsumption and dynamic priority hierarchies, allowing the agent to deal with conflicting goals in a dynamic world.

vidDemoSimulador1agenteReact.mp4

This video demonstrates the Python-based reactive agent operating in a 2D grid. The agent reacts to multiple simultaneous stimuli: it actively seeks out targets (green blocks) while concurrently avoiding obstacles (gray walls). Its behavior is dictated by a priority-based action selection mechanism, preventing it from getting stuck and keeping it moving fluidly.

(back to top)

---

Part 1: Reactive Agents in Java

The first part of the project focuses on the introduction to intelligent agents and their environment, developed in Java. Utilizing core Software Engineering principles—such as abstraction, modularization, and object-oriented design—a state machine was built to handle a basic perception-action cycle.

jogoJavaVidDemo.mp4

In this video, a text-based Java agent interacts with a simulated environment via terminal inputs. Based on specific environmental stimuli (like hearing a noise or spotting an animal), the agent immediately triggers an associated reaction, showcasing a purely reactive architecture without any internal memory or planning capabilities.

(back to top)

---

Lessons Learned

Beyond the technical artificial intelligence concepts, this project was fundamental in heavily improving good programming practices and managing software complexity.

• Software Engineering & Best Practices: Emphasized the strict application of Object-Oriented Programming (OOP) concepts, high cohesion, low coupling, and rigorous modularization. Abstracting systems into interfaces and clearly separating responsibilities made the codebase scalable and robust.
• Comparing Language Paradigms: Developing the project across Java and Python highlighted the pros and cons of strong vs. dynamic typing. Java forced rigorous specifications and interface compliance, catching visibility and typing errors early. Python sped up development but required extra caution regarding subtle bugs (like absolute vs. relative import issues and variable naming typos).
• AI Architectures: Consolidated the theoretical differences between reactive, deliberative, and hybrid agent architectures, and how internal memory and planning drastically change an agent's capabilities.
• Algorithm Complexity: Learned firsthand how time and space complexities diverge between uninformed and informed search algorithms when dealing with large state spaces.
• Handling Uncertainty: Gained practical experience applying the Bellman equation to solve Markov Decision Processes, moving from deterministic environments to probabilistic realities.

(back to top)

---

Getting Started

Follow these instructions to set up a local copy of the project on your machine.

Prerequisites

• Git
• Python 3
• Java JDK

Installation & Running Locally

1. Clone the repository:

   git clone https://github.com/GuilhermeGraca/python-ai-autonomous-systems.git
   cd python-ai-autonomous-systems

2. Run the Java Interactive Simulator (Part 1): This part runs a text-based reactive agent state machine.

   cd iasa_jogo/src
   javac jogo/Jogo.java
   java jogo.Jogo

   Type events like s (silence), r (noise), a (animal), or f (flee) to see the agent react.

3. Run the Python Visual Simulator (Parts 2, 3, and 4): Make sure to set the PYTHONPATH so the modules resolve correctly.

   For Windows (PowerShell):

   cd iasa_agente
   $env:PYTHONPATH="src;src/lib"
   python src/teste.py

   For Linux/Mac:

   cd iasa_agente
   export PYTHONPATH="src:src/lib"
   python src/teste.py

   Note: Edit teste.py to switch between AgenteReact, AgenteDelib, or AgenteDelibPDM.

4. Run the Search Algorithms Comparison: This executes multiple algorithms (A*, Breadth-First, Depth-First, etc.) to solve a math problem and compares their metrics.

   cd iasa_agente
   $env:PYTHONPATH="src;src/lib"
   python src/contagem/contagem.py

(back to top)

---

Contact

Guilherme Graça - LinkedIn - GitHub

(back to top)

---

Acknowledgments

• ISEL (Instituto Superior de Engenharia de Lisboa) - For providing the foundational knowledge and the Inteligência Artificial para Sistemas Autónomos course framework.
• Professor Luís Morgado - For the guidance and theoretical materials provided throughout the semester.

(back to top)