Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
## Analytic purpose

What business question or metric does this change address?

## Changes

-

## Validation

- [ ] Database builds from a clean directory
- [ ] Data-quality queries return zero failures
- [ ] Integration tests pass
- [ ] Saved outputs were regenerated when logic changed
- [ ] Metric definitions and interpretation remain aligned
- [ ] No real or restricted data are included

## Interpretation impact

Describe any change to a denominator, business rule, finding, or limitation.
24 changes: 24 additions & 0 deletions .github/workflows/validate-sql.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
name: Validate SQL case study

on:
push:
branches: [main]
pull_request:

permissions:
contents: read

jobs:
integration-tests:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- uses: actions/setup-python@v7
with:
python-version: "3.13"
- name: Build database and save query outputs
run: |
python scripts/build_demo_database.py
python scripts/run_queries.py
- name: Run integration tests
run: python -m unittest discover -s tests -v
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
data/*.sqlite
__pycache__/
*.pyc
.coverage
21 changes: 21 additions & 0 deletions LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
MIT License

Copyright (c) 2026 Matthew Jeans

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
15 changes: 15 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
.PHONY: build query test all clean

build:
python scripts/build_demo_database.py

query: build
python scripts/run_queries.py

test:
python -m unittest discover -s tests -v

all: query test

clean:
rm -f data/analytics_demo.sqlite
12 changes: 12 additions & 0 deletions data/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# Generated database

`scripts/build_demo_database.py` creates `analytics_demo.sqlite` in this directory.

The generated database contains:

- 8 fictional sites
- 480 fictional members
- 3,076 deterministic service-event records
- 252 deterministic support tickets

The SQLite file is ignored because it is reproducible from source. Saved aggregate query outputs are committed in `outputs/` for quick review.
39 changes: 39 additions & 0 deletions docs/data-model.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# Data model

## `sites`

One row per service location.

| Column | Type | Description |
|---|---|---|
| site_id | text | Primary key |
| site_name | text | Display name |
| region | text | North, South, East, or West |
| launch_date | date text | Site launch date |
| monthly_target | integer | Completed-service target |

## `members`

One row per member.

| Column | Type | Description |
|---|---|---|
| member_id | text | Primary key |
| site_id | text | Foreign key to `sites` |
| signup_date | date text | Enrollment date |
| acquisition_channel | text | Referral, outreach, web, or partner |
| age_band | text | Nonidentifying analytic band |

## `service_events`

One row per scheduled service event. `completed` equals one when the service occurred. Duration is populated only for completed services.

## `support_tickets`

One row per support contact, with opened date, resolution hours, and optional satisfaction score.

## Grain and joins

Metric definitions preserve table grain explicitly. Member outcomes are first aggregated to one row per member, then joined to dimensions. Site-month metrics aggregate events before window calculations so repeated event rows do not inflate denominators.

Dates use ISO-8601 text because SQLite's built-in date functions operate reliably on that representation.
49 changes: 49 additions & 0 deletions docs/decision-memo.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
# Decision memo

## Executive summary

The synthetic results identify three practical priorities: investigate the sustained activity decline at Summit, protect and learn from the stronger referral channel, and review support-resolution processes because slower resolution coincides with substantially weaker early retention.

These are decision signals from generated data, not claims about a real program.

## Findings

### Summit requires operational review

Summit completed 3 services in September compared with a prior three-month average of 30, a 90% decline. It remained the lowest-ranked site by target attainment in October, November, and December. The persistence of the signal makes a data-timing explanation less likely within this synthetic scenario, but the appropriate next step is still diagnosis rather than immediate corrective attribution.

Recommended review:

- confirm staffing, scheduling capacity, and service availability
- reconcile event capture against source-system counts
- examine cancellations and member mix
- document whether the September change reflects a known operational event

### Referral is the strongest acquisition channel

Across all signup months, referral members had a 65.5% 14-day activation rate and 58.1% 45-day retention rate. Outreach produced 40.0% activation and 31.8% retention. Partner and web channels fell between those groups.

The channel mix should not be changed from these descriptive rates alone. Channel populations may differ. The next analysis should adjust for site, signup month, and available member characteristics, then examine cost per activated and retained member.

### Slow support resolution is a useful risk signal

Members in the fastest support-resolution quartile averaged 15.2 hours to resolution, 4.22 satisfaction, and 66.7% 45-day retention. The slowest quartile averaged 47.3 hours, 2.09 satisfaction, and 26.0% retention.

This is an association, not evidence that faster support causes retention. Still, the gradient is large enough to justify an operational SLA review and a more careful analysis of ticket reason, member risk, site, and timing.

### Longer-term cohort retention remains uneven

Six-month active-member rates range from 40.7% to 54.9% across signup cohorts. The variation does not show a consistent upward pattern. A standardized onboarding and follow-up protocol could be tested prospectively, with retention definitions and analysis timing agreed before launch.

## Recommended next actions

| Priority | Action | Owner | Measure of progress |
|---|---|---|---|
| 1 | Complete a Summit root-cause review | Operations lead | Reconciled counts and documented cause |
| 2 | Model channel performance with adjustment and acquisition cost | Analytics lead | Cost per activated and retained member |
| 3 | Review support SLA and ticket categories | Support lead | Resolution distribution and repeat-contact rate |
| 4 | Design an onboarding improvement test | Program and analytics leads | Prespecified activation and retention outcomes |

## Caveats

The database is synthetic and deliberately encodes some performance differences. Results demonstrate SQL reasoning and communication, not causal inference or evidence about an actual organization.
24 changes: 24 additions & 0 deletions docs/metric-definitions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# Metric definitions

| Metric | Definition | Grain |
|---|---|---|
| 14-day activation | At least one completed service from signup through day 14 | Member |
| 45-day retention | At least one completed service from day 15 through day 45 after signup | Member |
| Monthly active member | At least one completed service during a calendar month | Member-month |
| Completion rate | Completed events divided by all scheduled events | Site-month |
| Target attainment | Completed services divided by the site's monthly target | Site-month |
| Cohort retention | Active members at month N divided by original signup-cohort size | Cohort-month |
| Support-resolution quartile | Member's average resolution hours ranked across members with tickets | Member |
| Activity anomaly | Completed services at least 25% below the prior three-month average, with three prior months available | Site-month |

## Denominator rules

- Activation and retention denominators include every member in the relevant signup group.
- Event completion uses all scheduled events, not only members with completed events.
- Cohort size is fixed at signup and does not shrink when a member becomes inactive.
- Members with no support ticket are excluded from support quartiles and reported separately when needed.
- Months with zero events are materialized through the calendar view so absence is not mistaken for missing data.

## Interpretation

These definitions are operational, not causal. They are written before querying so business logic can be reviewed independently of the code.
7 changes: 7 additions & 0 deletions outputs/02_data_quality.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
check_name,failed_rows
duplicate_site_key,0
duplicate_member_key,0
orphan_service_member,0
orphan_ticket_member,0
service_before_signup,0
invalid_completed_duration,0
25 changes: 25 additions & 0 deletions outputs/03_activation_funnel.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
signup_month,acquisition_channel,signups,activated_members,activation_rate,retained_members,retention_rate_45d
2025-01-01,Partner,18,13,0.722,8,0.444
2025-01-01,Web,20,12,0.6,7,0.35
2025-01-01,Referral,24,14,0.583,14,0.583
2025-01-01,Outreach,18,10,0.556,8,0.444
2025-02-01,Referral,29,20,0.69,18,0.621
2025-02-01,Partner,19,12,0.632,11,0.579
2025-02-01,Web,20,10,0.5,5,0.25
2025-02-01,Outreach,21,6,0.286,3,0.143
2025-03-01,Referral,30,23,0.767,20,0.667
2025-03-01,Partner,14,7,0.5,7,0.5
2025-03-01,Web,25,12,0.48,9,0.36
2025-03-01,Outreach,16,6,0.375,5,0.313
2025-04-01,Partner,14,8,0.571,7,0.5
2025-04-01,Outreach,15,7,0.467,7,0.467
2025-04-01,Referral,16,7,0.438,5,0.313
2025-04-01,Web,14,3,0.214,3,0.214
2025-05-01,Referral,27,19,0.704,16,0.593
2025-05-01,Partner,17,11,0.647,6,0.353
2025-05-01,Outreach,16,8,0.5,5,0.313
2025-05-01,Web,22,9,0.409,8,0.364
2025-06-01,Referral,22,14,0.636,13,0.591
2025-06-01,Web,28,13,0.464,11,0.393
2025-06-01,Partner,11,5,0.455,5,0.455
2025-06-01,Outreach,24,7,0.292,7,0.292
43 changes: 43 additions & 0 deletions outputs/04_cohort_retention.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
cohort_month,month_number,cohort_size,active_members,retention_rate
2025-01-01,0,80,47,0.588
2025-01-01,1,80,58,0.725
2025-01-01,2,80,48,0.6
2025-01-01,3,80,45,0.563
2025-01-01,4,80,41,0.512
2025-01-01,5,80,35,0.438
2025-01-01,6,80,35,0.438
2025-02-01,0,89,36,0.404
2025-02-01,1,89,62,0.697
2025-02-01,2,89,48,0.539
2025-02-01,3,89,46,0.517
2025-02-01,4,89,48,0.539
2025-02-01,5,89,52,0.584
2025-02-01,6,89,42,0.472
2025-03-01,0,85,47,0.553
2025-03-01,1,85,68,0.8
2025-03-01,2,85,47,0.553
2025-03-01,3,85,50,0.588
2025-03-01,4,85,40,0.471
2025-03-01,5,85,50,0.588
2025-03-01,6,85,40,0.471
2025-04-01,0,59,28,0.475
2025-04-01,1,59,48,0.814
2025-04-01,2,59,27,0.458
2025-04-01,3,59,27,0.458
2025-04-01,4,59,29,0.492
2025-04-01,5,59,25,0.424
2025-04-01,6,59,24,0.407
2025-05-01,0,82,43,0.524
2025-05-01,1,82,58,0.707
2025-05-01,2,82,49,0.598
2025-05-01,3,82,44,0.537
2025-05-01,4,82,34,0.415
2025-05-01,5,82,46,0.561
2025-05-01,6,82,45,0.549
2025-06-01,0,85,38,0.447
2025-06-01,1,85,65,0.765
2025-06-01,2,85,46,0.541
2025-06-01,3,85,44,0.518
2025-06-01,4,85,43,0.506
2025-06-01,5,85,40,0.471
2025-06-01,6,85,37,0.435
97 changes: 97 additions & 0 deletions outputs/05_site_performance.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
site_id,site_name,region,month_start,scheduled_events,completed_services,active_members,completion_rate,target_attainment,prior_month_services,month_over_month_change,rolling_3m_services,monthly_site_rank
S001,Harbor,North,2025-01-01,15,15,10,1.0,0.273,,,15.0,1
S005,Meadow,North,2025-01-01,7,7,7,1.0,0.14,,,7.0,2
S007,Summit,East,2025-01-01,7,7,6,1.0,0.13,,,7.0,3
S008,Park,West,2025-01-01,6,6,5,1.0,0.105,,,6.0,4
S002,Riverside,South,2025-01-01,6,6,5,1.0,0.1,,,6.0,5
S003,Cedar,East,2025-01-01,5,5,5,1.0,0.096,,,5.0,6
S006,Juniper,South,2025-01-01,5,5,5,1.0,0.089,,,5.0,7
S004,Lakeside,West,2025-01-01,4,4,4,1.0,0.069,,,4.0,8
S003,Cedar,East,2025-02-01,25,24,16,0.96,0.462,5,3.8,14.5,1
S001,Harbor,North,2025-02-01,27,24,17,0.889,0.436,15,0.6,19.5,2
S005,Meadow,North,2025-02-01,20,18,14,0.9,0.36,7,1.571,12.5,3
S007,Summit,East,2025-02-01,16,14,11,0.875,0.259,7,1.0,10.5,4
S008,Park,West,2025-02-01,14,13,11,0.929,0.228,6,1.167,9.5,5
S004,Lakeside,West,2025-02-01,15,12,10,0.8,0.207,4,2.0,8.0,6
S006,Juniper,South,2025-02-01,12,11,7,0.917,0.196,5,1.2,8.0,7
S002,Riverside,South,2025-02-01,11,11,8,1.0,0.183,6,0.833,8.5,8
S005,Meadow,North,2025-03-01,37,34,21,0.919,0.68,18,0.889,19.7,1
S007,Summit,East,2025-03-01,33,30,18,0.909,0.556,14,1.143,17.0,2
S004,Lakeside,West,2025-03-01,34,32,22,0.941,0.552,12,1.667,16.0,3
S001,Harbor,North,2025-03-01,31,28,21,0.903,0.509,24,0.167,22.3,4
S003,Cedar,East,2025-03-01,28,25,23,0.893,0.481,24,0.042,18.0,5
S008,Park,West,2025-03-01,29,26,20,0.897,0.456,13,1.0,15.0,6
S006,Juniper,South,2025-03-01,23,20,15,0.87,0.357,11,0.818,12.0,7
S002,Riverside,South,2025-03-01,21,17,17,0.81,0.283,11,0.545,11.3,8
S008,Park,West,2025-04-01,39,37,25,0.949,0.649,26,0.423,25.3,1
S007,Summit,East,2025-04-01,36,31,26,0.861,0.574,30,0.033,25.0,2
S006,Juniper,South,2025-04-01,35,32,26,0.914,0.571,20,0.6,21.0,3
S002,Riverside,South,2025-04-01,36,34,26,0.944,0.567,17,1.0,20.7,4
S001,Harbor,North,2025-04-01,33,31,24,0.939,0.564,28,0.107,27.7,5
S004,Lakeside,West,2025-04-01,35,32,24,0.914,0.552,32,0.0,25.3,6
S005,Meadow,North,2025-04-01,29,24,20,0.828,0.48,34,-0.294,25.3,7
S003,Cedar,East,2025-04-01,30,21,18,0.7,0.404,25,-0.16,23.3,8
S003,Cedar,East,2025-05-01,43,38,31,0.884,0.731,21,0.81,28.0,1
S002,Riverside,South,2025-05-01,45,41,35,0.911,0.683,34,0.206,30.7,2
S007,Summit,East,2025-05-01,39,35,29,0.897,0.648,31,0.129,32.0,3
S006,Juniper,South,2025-05-01,35,32,28,0.914,0.571,32,0.0,28.0,4
S001,Harbor,North,2025-05-01,35,30,26,0.857,0.545,31,-0.032,29.7,5
S004,Lakeside,West,2025-05-01,37,31,28,0.838,0.534,32,-0.031,31.7,6
S008,Park,West,2025-05-01,37,29,28,0.784,0.509,37,-0.216,30.7,7
S005,Meadow,North,2025-05-01,30,22,20,0.733,0.44,24,-0.083,26.7,8
S004,Lakeside,West,2025-06-01,61,49,40,0.803,0.845,31,0.581,37.3,1
S001,Harbor,North,2025-06-01,47,41,36,0.872,0.745,30,0.367,34.0,2
S002,Riverside,South,2025-06-01,50,44,34,0.88,0.733,41,0.073,39.7,3
S006,Juniper,South,2025-06-01,45,41,33,0.911,0.732,32,0.281,35.0,4
S003,Cedar,East,2025-06-01,43,37,28,0.86,0.712,38,-0.026,32.0,5
S008,Park,West,2025-06-01,41,38,37,0.927,0.667,29,0.31,34.7,6
S005,Meadow,North,2025-06-01,31,28,24,0.903,0.56,22,0.273,24.7,7
S007,Summit,East,2025-06-01,39,29,24,0.744,0.537,35,-0.171,31.7,8
S001,Harbor,North,2025-07-01,60,47,41,0.783,0.855,41,0.146,39.3,1
S008,Park,West,2025-07-01,50,43,35,0.86,0.754,38,0.132,36.7,2
S004,Lakeside,West,2025-07-01,45,42,35,0.933,0.724,49,-0.143,40.7,3
S006,Juniper,South,2025-07-01,48,40,38,0.833,0.714,41,-0.024,37.7,4
S003,Cedar,East,2025-07-01,43,37,30,0.86,0.712,37,0.0,37.3,5
S007,Summit,East,2025-07-01,44,35,30,0.795,0.648,29,0.207,33.0,6
S002,Riverside,South,2025-07-01,42,36,33,0.857,0.6,44,-0.182,40.3,7
S005,Meadow,North,2025-07-01,38,28,26,0.737,0.56,28,0.0,26.0,8
S001,Harbor,North,2025-08-01,52,36,34,0.692,0.655,47,-0.234,41.3,1
S004,Lakeside,West,2025-08-01,43,34,34,0.791,0.586,42,-0.19,41.7,2
S002,Riverside,South,2025-08-01,43,35,34,0.814,0.583,36,-0.028,38.3,3
S005,Meadow,North,2025-08-01,32,27,27,0.844,0.54,28,-0.036,27.7,4
S003,Cedar,East,2025-08-01,39,28,27,0.718,0.538,37,-0.243,34.0,5
S006,Juniper,South,2025-08-01,41,28,27,0.683,0.5,40,-0.3,36.3,6
S007,Summit,East,2025-08-01,33,26,26,0.788,0.481,35,-0.257,30.0,7
S008,Park,West,2025-08-01,30,25,25,0.833,0.439,43,-0.419,35.3,8
S008,Park,West,2025-09-01,41,33,33,0.805,0.579,25,0.32,33.7,1
S003,Cedar,East,2025-09-01,34,28,28,0.824,0.538,28,0.0,31.0,2
S004,Lakeside,West,2025-09-01,36,31,31,0.861,0.534,34,-0.088,35.7,3
S002,Riverside,South,2025-09-01,38,32,32,0.842,0.533,35,-0.086,34.3,4
S006,Juniper,South,2025-09-01,39,29,29,0.744,0.518,28,0.036,32.3,5
S005,Meadow,North,2025-09-01,28,25,25,0.893,0.5,27,-0.074,26.7,6
S001,Harbor,North,2025-09-01,35,24,24,0.686,0.436,36,-0.333,35.7,7
S007,Summit,East,2025-09-01,8,3,3,0.375,0.056,26,-0.885,21.3,8
S001,Harbor,North,2025-10-01,44,35,35,0.795,0.636,24,0.458,31.7,1
S004,Lakeside,West,2025-10-01,46,35,35,0.761,0.603,31,0.129,33.3,2
S002,Riverside,South,2025-10-01,47,34,34,0.723,0.567,32,0.063,33.7,3
S008,Park,West,2025-10-01,36,29,29,0.806,0.509,33,-0.121,29.0,4
S005,Meadow,North,2025-10-01,32,24,24,0.75,0.48,25,-0.04,25.3,5
S003,Cedar,East,2025-10-01,40,24,24,0.6,0.462,28,-0.143,26.7,6
S006,Juniper,South,2025-10-01,31,24,24,0.774,0.429,29,-0.172,27.0,7
S007,Summit,East,2025-10-01,11,6,6,0.545,0.111,3,1.0,11.7,8
S008,Park,West,2025-11-01,45,34,34,0.756,0.596,29,0.172,32.0,1
S004,Lakeside,West,2025-11-01,38,33,33,0.868,0.569,35,-0.057,33.0,2
S001,Harbor,North,2025-11-01,34,28,28,0.824,0.509,35,-0.2,29.0,3
S003,Cedar,East,2025-11-01,34,26,26,0.765,0.5,24,0.083,26.0,4
S002,Riverside,South,2025-11-01,38,28,28,0.737,0.467,34,-0.176,31.3,5
S005,Meadow,North,2025-11-01,29,22,22,0.759,0.44,24,-0.083,23.7,6
S006,Juniper,South,2025-11-01,29,24,24,0.828,0.429,24,0.0,25.7,7
S007,Summit,East,2025-11-01,5,4,4,0.8,0.074,6,-0.333,4.3,8
S008,Park,West,2025-12-01,37,27,27,0.73,0.474,34,-0.206,30.0,1
S001,Harbor,North,2025-12-01,34,26,26,0.765,0.473,28,-0.071,29.7,2
S006,Juniper,South,2025-12-01,31,25,25,0.806,0.446,24,0.042,24.3,3
S005,Meadow,North,2025-12-01,30,22,22,0.733,0.44,22,0.0,22.7,4
S003,Cedar,East,2025-12-01,25,22,22,0.88,0.423,26,-0.154,24.0,5
S004,Lakeside,West,2025-12-01,31,22,22,0.71,0.379,33,-0.333,30.0,6
S002,Riverside,South,2025-12-01,28,22,22,0.786,0.367,28,-0.214,28.0,7
S007,Summit,East,2025-12-01,17,5,5,0.294,0.093,4,0.25,5.0,8
5 changes: 5 additions & 0 deletions outputs/06_support_experience.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
resolution_quartile,members_with_tickets,mean_resolution_hours,mean_satisfaction,mean_ticket_count,retention_rate_45d
1,51,15.2,4.22,1.25,0.667
2,51,24.5,3.74,1.29,0.588
3,51,34.0,3.15,1.29,0.353
4,50,47.3,2.09,1.12,0.26
Loading