Skip to content
samplics-orgPublic

About

Modern Python ecosystem for complex survey design, weighting, estimation, and small area estimation.

Topics

Resources

Contributing

Security policy

Stars

23 stars

Watchers

2 watching

Forks

Latest commit

ย 

History

447 Commits

Folders and files

NameName
Last commit message
Last commit date
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 

Repository files navigation

svy

Design, analyze, and report complex surveys in Python.

PyPI Python License: MIT Docs

svy is a design-based, production-oriented library covering the full survey workflow: sample size, selection, weighting, estimation, testing, and modeling. It is validated against R's survey package to six significant digits and powered by a Rust engine.

๐ŸŒ svylab.com ยท ๐Ÿ“˜ Documentation ยท ๐Ÿš€ Quick Tour


Why svy?

  • One object, whole workflow. A Sample binds your data to its sampling design once; .wrangling, .weighting, .estimation, .categorical, and .glm all hang off it, and the design metadata travels through every step.
  • Correct by construction. Sample is immutable: every transformation returns a new object, and subpopulation analysis (where=) keeps the full design for variance estimation instead of filtering rows, which understates standard errors.
  • Replicate weights done right. Nonresponse adjustment, calibration, raking, and trimming are applied to the replicate weights in the same pass as the full-sample weights, so replication standard errors reflect the whole weighting process.
  • Fast. Variance estimation and other hot paths run in Rust (svy-rs); data operations run on Polars. A full pipeline with 500 bootstrap replicates completes in under a quarter of a second on a laptop.
  • Reproducible and automation-ready. Explicit seeds, typed result objects (Estimate, Table, TTest, GLM fits), and a versioned serialization module make pipelines easy to rerun, audit, and wire into production systems.

Installation

pip install svy             # or: uv add svy
pip install "svy[all]"      # adds rich terminal output and the online dataset catalog

A base install makes no network calls: its example datasets are the subsets packaged with svy. The report extra adds rich terminal output, and remote adds downloads of the full example datasets from the svyLab catalog.

Platforms. Python 3.11 or later. Prebuilt wheels for Linux (x86_64 and aarch64, glibc 2.28 or later), macOS (arm64 and x86_64) and Windows (x86_64), so no compiler or Rust toolchain is needed. Elsewhere pip builds from source, which needs Rust 1.91 or later.

Offline machines. Download the wheels on a connected machine, copy the folder, and install from it. Match --python-version and --platform to the offline machine (for Windows, --platform win_amd64); on a machine of the same kind, leave both out.

pip download "svy[report]" --dest svy-wheels --only-binary=:all: --python-version 3.12 \
  --platform manylinux_2_28_x86_64 --platform manylinux_2_17_x86_64 --platform manylinux2014_x86_64
pip install --no-index --find-links svy-wheels "svy[report]"   # on the offline machine

Coding agents. svy ships agent skills: guidance for coding agents such as Claude Code, matching the installed version. Run this once per project, from the project folder:

uv run svy-skills    # or svy-skills with the environment active; svy.skills.install() from Python

It links the skills into .claude/skills/, so upgrading svy updates them. Rerun it when a release adds a skill or after recreating the environment; --copy copies them instead, e.g. to commit them.

Security and data handling. svy runs locally, sends no telemetry, and never sends your data anywhere. SECURITY.md covers network access, dependencies, how releases are built and published, and how to report a vulnerability.

The whole workflow as one chain

Every transformation returns a new Sample, so an entire workflow reads as one fluent pipeline. Load a bundled survey (runs offline), create an indicator, adjust the weights for nonresponse, trim extreme weights, and estimate the population literacy rate with a design-based standard error:

import svy
import numpy as np

rng = np.random.default_rng(42)

# A bundled example survey; its metadata carries the sampling design
data = svy.datasets.load("ind_sample_wb_2023", source="bundled")
info = svy.datasets.describe("ind_sample_wb_2023", source="bundled")
sample = svy.Sample(data, svy.Design(**info.design))

literacy_rate = (
    sample
    # 1. Wrangle: a 1/0 literacy indicator, plus a simulated response status
    .wrangling.mutate({
        "literate": svy.when(svy.col("literacy") == "Yes").then(1)
                       .when(svy.col("literacy") == "No").then(0)
                       .otherwise(None),
        # simulated for illustration; the example data has 100% response
        "resp_status": rng.choice(
            ["respondent", "non-respondent"], p=[0.85, 0.15],
            size=sample.n_records,
        ),
    })
    # 2. Adjust the weights for nonresponse, within urban/rural classes
    .weighting.adjust(
        resp_status="resp_status",
        cells="urbrur",
        resp_mapping={"rr": "respondent", "nr": "non-respondent"},
        wgt_name="nr_wgt",
    )
    # 3. Trim weights above 1.2 x the median; the excess goes to the others
    .weighting.trim(upper=svy.Threshold("median", 1.2))
    # 4. Estimate the population literacy rate, with a design-based SE
    .estimation.mean("literate", drop_nulls=True)
)

print(literacy_rate)

svy Estimate output: mean literacy rate with standard error, confidence interval, and CV

More analysis

The same Sample reaches every other kind of analysis through an accessor.

Domain estimation with correct within-domain standard errors:

print(sample.estimation.mean("age", by="urbrur"))
โ•ญโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ Estimate: MEAN (TAYLOR) โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฎ
โ”‚ y: age                                                   โ”‚
โ”‚                                                          โ”‚
โ”‚  urbrur       est       se       lci       uci   cv (%)  โ”‚
โ”‚  โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”  โ”‚
โ”‚  Rural    27.4893   3.1681   20.5163   34.4624    11.53  โ”‚
โ”‚  Urban    29.7557   2.5016   24.4236   35.0877     8.41  โ”‚
โ”‚                                                          โ”‚
โ•ฐโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฏ

Tabulation with design-based standard errors (two-way tables add Rao-Scott tests automatically):

print(sample.categorical.tabulate("educ_attain", units="percent"))

Regression with design-adjusted standard errors (Gaussian, binomial, Poisson, and Gamma families):

model = sample.glm.fit(y="yrs_school", x=["age", svy.Cat("sex")], family="gaussian")
print(model)

svy GLM output: Gaussian model of years of schooling with design-adjusted standard errors

The Quick Tour runs this whole workflow in five minutes, every block offline on the bundled data.

Capabilities across the survey lifecycle

Stage What svy provides
Plan Sample size for estimation and comparison objectives, with design effects and nonresponse
Select SRS, systematic, PPS methods; multi-stage designs with automatic probability chaining
Weight Nonresponse adjustment, poststratification, GREG calibration, raking, trimming, normalization
Estimate Means, totals, proportions, ratios, medians; Taylor linearization and replication (BRR, Fay, jackknife, bootstrap, SDR)
Test Tabulations, t-tests, rank tests, Rao-Scott chi-squared tests, all design-adjusted
Model GLMs (linear, logistic, Poisson, Gamma) with design-adjusted standard errors

All methods are grounded in established survey methodology, with domain estimation (by=) and subpopulation analysis (where=) throughout.

Validation

svy is validated against R's survey package across design specification, Taylor linearization, replication methods, calibration, categorical analysis, and GLMs, producing numerically identical results to at least six significant digits. The few intentional differences follow Stata conventions and are documented. See the full comparison with reproducible code.

Ecosystem

Package Purpose Install
svy Core survey design & estimation pip install svy
svy-sae Small Area Estimation pip install svy-sae
svy-io SPSS / Stata / SAS I/O pip install svy-io
svy-rs Rust computational engine used by svy installed automatically

This repository is a monorepo: the Python package lives in packages/svy, the Rust engine in packages/svy-rs, and the I/O library in packages/svy-io.

Documentation

Guides, tutorials, and methodological notes: svylab.com/docs/svy. Start with the Quick Tour.

Citation

A software paper for svy is under review at the Journal of Open Source Software. Until it is published, please cite this repository. The predecessor package, samplics, is described in Diallo (2021), JOSS 6(68):3376.

Feedback

If you work with complex surveys and want to influence the design of a modern Python survey stack, this is the right place to engage.

License

MIT License. Copyright ยฉ 2026 Samplics LLC.


svy is built for practitioners who need statistical rigor that survives contact with reality.

About

Modern Python ecosystem for complex survey design, weighting, estimation, and small area estimation.

Topics

Resources

Contributing

Security policy

Stars

23 stars

Watchers

2 watching

Forks

Releases

Packages

Used by

Contributors

Languages