GLIDE is a Python library for rigorous evaluation of GenAI systems using hybrid human/proxy annotations.
GLIDE implements methods from the field of prediction-powered inference β the science of system evaluation that combines a small set of labeled data with a large set of proxy-labeled data to produce valid, debiased estimates. See the implemented algorithms below.
- π€ GenAI applications are everywhere β and imperfect. Deployed systems make mistakes, and measuring how often matters.
- βοΈ LLM-as-judge is biased. Proxy evaluators (models, heuristics) are cheap but systematically over- or under-estimate true performance.
- π§ Rigorous evaluation requires a human in the loop. Ground-truth labels from humans are expensive, so only a small subset is feasible.
- π GLIDE bridges the gap. It combines a small set of human annotations with a large set of proxy predictions to produce statistically valid metrics β correcting proxy bias without requiring full human labeling.
See prediction-powered inference in action: tune the sample size, human annotation budget, and LLM-as-judge quality, and watch GLIDE combine both signals into a rigorous confidence interval.
π Play with the simulator π
Install the package with your favorite package manager :
uv add glide-pyor
pip install glide-pyAnd look at our practical quickstart.
Explore the full documentation β from practical tutorials and user guides to scientific deep dives into the methods behind GLIDE.
Contributions are welcome! Please read the contributing guide for setup instructions, an architectural overview, and the checklist to follow before opening a pull request. Feel free to open an issue to report a bug or suggest a feature.
This project follows Semantic Versioning (SemVer): MAJOR.MINOR.PATCH.
This project follows SPEC 0 for dependency support windows.
This project is licensed under the Apache 2.0 License.
If you use GLIDE in your work, please cite us using the "Cite this repository" button on the GitHub repository page.
| Name | Class | Reference Paper(s) | Original Implementation |
|---|---|---|---|
| Stratified Sampling | samplers.StratifiedSampler |
[4] | Link |
| Active Sampling | samplers.ActiveSampler |
[5], [6] | Link |
| Cost-Optimal Sampling | samplers.CostOptimalSampler |
[8] | β |
| Cost-Optimal Random Sampling | samplers.CostOptimalRandomSampler |
[8] | β |
| Name | Class | Reference Paper(s) | Original Implementation |
|---|---|---|---|
| Prediction-Powered Inference | estimators.PPIMeanEstimator (with power_tuning=False) |
[1] | Link |
| PPI++ | estimators.PPIMeanEstimator |
[2] | Link |
| Stratified Prediction-Powered Inference | estimators.StratifiedPPIMeanEstimator |
[3] | β |
| Clustered Prediction-Powered Inference | estimators.ClusteredPPIMeanEstimator |
β | Link |
| Multi-Proxy Prediction-Powered Inference | estimators.MultiPPIMeanEstimator |
[9] | Link |
| Active Statistical Inference | estimators.ASIMeanEstimator |
[5], [6] | Link |
| Predict-Then-Debias | estimators.PTDMeanEstimator |
[7] | Link |
| Stratified Predict-Then-Debias | estimators.StratifiedPTDMeanEstimator |
[7] | Link |
| Multi-Proxy Predict-Then-Debias | estimators.MultiPTDMeanEstimator |
[7], [9] | β |
| Clustered Predict-Then-Debias | estimators.ClusteredPTDMeanEstimator |
[7] | Link |
| IPW Predict-Then-Debias | estimators.IPWPTDMeanEstimator |
[7] | Link |
| Name | Class | Reference Paper(s) | Original Implementation |
|---|---|---|---|
| Asymptotic Prediction-Powered Risk Monitoring | monitors.AsymptoticPPRM |
[10], [11], [12] | β |
| Asymptotic Stratified Prediction-Powered Risk Monitoring | monitors.AsymptoticStratifiedPPRM |
[3], [4], [10], [11], [12] | β |
| Asymptotic Clustered Prediction-Powered Risk Monitoring | monitors.AsymptoticClusteredPPRM |
[10], [11], [12] | β |
| Asymptotic Multi-Proxy Prediction-Powered Risk Monitoring | monitors.AsymptoticMultiPPRM |
[9], [10], [11], [12] | β |
[1] Angelopoulos, Anastasios N., Stephen Bates, Clara Fannjiang, Michael I. Jordan, and Tijana Zrnic. "Prediction-powered inference." Science 382, no. 6671 (2023): 669-674.
[2] Angelopoulos, Anastasios N., John C. Duchi, and Tijana Zrnic. "PPI++: Efficient prediction-powered inference." arXiv preprint arXiv:2311.01453 (2023).
[3] Fisch, Adam, Joshua Maynez, R. Alex Hofer, Bhuwan Dhingra, Amir Globerson, and William W. Cohen. "Stratified prediction-powered inference for effective hybrid evaluation of language models." Advances in Neural Information Processing Systems 37 (2024): 111489-111514.
[4] Fogliato, Riccardo, Pratik Patil, Mathew Monfort, and Pietro Perona. "A framework for efficient model evaluation through stratification, sampling, and estimation." In European Conference on Computer Vision, pp. 140-158. Cham: Springer Nature Switzerland, 2024.
[5] Zrnic, Tijana, and Emmanuel J. Candès. "Active statistical inference." In Proceedings of the 41st International Conference on Machine Learning, pp. 62993-63010. 2024.
[6] GligoriΔ, Kristina, Tijana Zrnic, Cinoo Lee, Emmanuel Candes, and Dan Jurafsky. "Can unconfident LLM annotations be used for confident conclusions?" In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pp. 3514-3533. 2025.
[7] Kluger, Dan M., Kerri Lu, Tijana Zrnic, Sherrie Wang, and Stephen Bates. "Prediction-powered inference with imputed covariates and nonuniform sampling." arXiv preprint arXiv:2501.18577 (2025).
[8] Angelopoulos, Anastasios N., Jacob Eisenstein, Jonathan Berant, Alekh Agarwal, and Adam Fisch. "Cost-optimal active ai model evaluation." arXiv preprint arXiv:2506.07949 (2025).
[9] Shan, Jiawei, Zhifeng Chen, Yiming Dong, Yazhen Wang, and Jiwei Zhao. "SADA: Safe and Adaptive Aggregation of Multiple Black-Box Predictions in Semi-Supervised Learning." arXiv preprint arXiv:2509.21707 (2025)..
[10] Zhang, Guangyi, Yunlong Cai, Guanding Yu, and Osvaldo Simeone. "Prediction-Powered Risk Monitoring of Deployed Models for Detecting Harmful Distribution Shifts." arXiv preprint arXiv:2602.02229 (2026).
[11] Podkopaev, Aleksandr, and Aaditya Ramdas. "Tracking the risk of a deployed model and detecting harmful distribution shifts." International Conference on Learning Representations (ICLR), 2022.
[12] Waudby-Smith, Ian, David Arbour, Ritwik Sinha, Edward H. Kennedy, and Aaditya Ramdas. "Time-uniform central limit theory and asymptotic confidence sequences." The Annals of Statistics 52, no. 6 (2024): 2613-2640.
Follow our LinkedIn newsletter for updates on GLIDE and GenAI evaluation.
Developed at Emerton Data.
