Fix Gini Impurity

[andrewdalpino]

src/Classifiers/ClassificationTree.php:260-266 accumulates 1 - (count/n)² per class, so for k classes present it returns k - Σpᵢ². Correct Gini impurity is 1 - Σpᵢ². The extra k-1 makes impurity() return up to k-1 too high, and because k varies per candidate subset, it distorts:

Split selection — DecisionTree::splitImpurity() (DecisionTree.php:396) picks the argmin of weighted per-node impurities.
Pruning — Split::purityIncrease() (Split.php:135) and the gate in DecisionTree::grow() (DecisionTree.php:219).
Feature importances — DecisionTree::featureImportances() (DecisionTree.php:296).
This is real, not a constant offset: different candidate splits partition into different k, so the bias changes the ordering. Git history (commit 77b9f93 "Fixed gini impurity computation") shows the canonical form was Σpᵢ² then 1 - Σpᵢ² — later regression reintroduced the per-class 1-pᵢ².

[Copilot]

Pull request overview

This PR corrects the Gini impurity computation used by ClassificationTree, removing a class-count-dependent bias that can distort split selection, pruning decisions, and feature importance calculations.

Changes:

• Fix leaf-node impurity calculation in terminate() to use 1 - Σp² (sum of squared class probabilities).
• Fix impurity() to use the canonical Gini form 1 - Σp² instead of summing 1 - p² per class.
• Document the fix in the changelog for the next release.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
src/Classifiers/ClassificationTree.php	Corrects Gini impurity computation in both terminate() and impurity() to the canonical 1 - Σp² form.
CHANGELOG.md	Adds an entry noting the Classification Tree Gini impurity fix in 2.5.6.

Suppressed comments (2)

src/Classifiers/ClassificationTree.php:270

• $ss is the only occurrence of this abbreviation in the codebase and doesn’t communicate what is being summed here. Renaming it (and using an explicit $p variable) would make the Gini computation easier to follow and keep it consistent with similar impurity implementations.

        $ss = 0.0;

        foreach ($counts as $count) {
            $ss += ($count / $n) ** 2;
        }

src/Classifiers/ClassificationTree.php:272

• There’s no regression test asserting the Gini impurity formula (1 - Σp²). Since this bug previously slipped through the existing end-to-end accuracy tests, consider adding a unit test that exercises impurity/terminate on a known label distribution (e.g., [A,A,B] => 1 - (4/9 + 1/9) = 4/9) to prevent future regressions.

        return 1.0 - $ss;

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.