SimpleGrad

SimpleGrad is a lightweight automatic differentiation library written in C++ with Python bindings.

Prerequisites

• Python >= 3.9

Prebuilt wheels are published to PyPI for all major platforms (Linux, macOS, and Windows), so no C++ compiler or build tools are required to install SimpleGrad.

Installation

1. If you are using uv (preferred), you can add SimpleGrad to your project by running the following command:

uv add simplegrad

2. Install via pip:

pip install simplegrad

Features

• Multi-layer perceptron (MLP) which can be used for regression and classification tasks
• Supports basic arithmetic operations
• Lightweight and easy to use
• Gradient computation
• Backpropagation
• Numpy compatibility

Usage

Here's a quick example of how to use MLP in SimpleGrad:

from simplegrad import MLP, Node
from sklearn import datasets

# Define the model
X, y = datasets.make_classification(
        n_samples=1000,
        n_features=10,
        n_classes=2,
        random_state=42,  # for reproducibility
    )


lr = 0.01
batch_size = 16
epochs = 10

# Define the model
model = MLP(
    10, [12, 1]
)  # 2 input nodes, 2 hidden layers with arbitrary sizes, 1 output node

# Training data
n_batches = (len(X) + batch_size - 1) // batch_size  # Ceiling division

for epoch in range(epochs):
    epoch_loss = 0.0
    for i in range(0, len(X), batch_size):
        batch_X = X[i : i + batch_size]
        batch_y = y[i : i + batch_size]
        current_batch_size = len(batch_X)  # Handle last batch

        batch_loss = 0.0
        #model.zero_grad()  # gradients are automatically reset after step function

        # Accumulate gradients over batch
        for x, y_true in zip(batch_X, batch_y):
            y_hat = model(x)[0]
            y_true = Node(y_true)
            loss = (y_hat - y_true) ** 2
            loss = loss * (1.0 / current_batch_size)  # Normalize loss
            batch_loss += loss.data()
            loss.backward()

        model.step(lr)  # Update weights using accumulated gradients
        epoch_loss += batch_loss

    # Average loss over all batches
    print(f"Epoch {epoch+1}, Average Loss: {epoch_loss/n_batches:.3f}")

You can execute the above code by running the following command:

make run

Testing

Tests are written to ensure the correctness of the Node class. Thus making sure MLP works as expected. You can run tests with following command:

make test

Development

If you want to build SimpleGrad from source or contribute to the project, you'll need the C++ toolchain in addition to Python:

• Python >= 3.9
• g++/gcc
• CMake

Build the extension locally and run the example or the tests with:

make build   # configure with CMake, compile, and install into the local venv
make run     # run the example script (py-simplegrad/main.py)
make test    # run the test suite with pytest

Run make help to see all available commands.

License

This project is licensed under the MIT License.

Acknowledgements

This project was inspired by the micrograd project by Andrej Karpathy.

TODO

• Compile and build for other platforms (DONE)
• Publish on pypi (DONE)
• Cyclic reference optimization (DONE)