test: exercise public Elixir flags in SDK compliance

[dustinbyrne]

💡 Motivation and Context

The compliance adapter built flag requests, parsed results, and captured called-events itself, so passing assertions did not establish public SDK flag behavior. Route flag actions through PostHog.FeatureFlags.evaluate_flags/2 with singleton flag_keys, then Evaluations.get_flag/2. Forward supplied supported options and leave transport, retries, parsing, defaults, and side effects to the SDK.

Update the harness image to 1.0.0, add an adapter regression-test CI job, and document the existing single-sender V0/default-Req-gzip profile. SDK source, public APIs, and shipped behavior are unchanged; no release changeset is needed.

💚 How did you test it?

• Six adapter regression tests passed against a real local HTTP fixture, covering rich-v2 results/event metadata, explicit and omitted options, remote evaluation, legacy-response errors, and SDK 502/504 retries.
• Production compilation with warnings as errors, scoped formatting, YAML parsing, and diff checks passed.
• Native Elixir 1.19.5/OTP28 run of the pinned harness selected all 47 tests: 40 passed, 7 failed (capture 29/30; flags 11/17). The seven failures match the documented expectations: timestamp override is unsupported, unspecified groups/GeoIP are omitted, and four result-dependent fixtures return legacy featureFlags rather than rich-v2 flags. No cases are filtered or rewritten to pass.
• Linux/Docker execution is left to PR CI. Compliance remains advisory: a green job does not imply every assertion passed. Existing timer-only flush, approximate state counters, and unmapped init transport options remain documented limitations.

📝 Checklist

• I reviewed the submitted code.
• I added tests to verify the changes.
• I updated the docs if needed.
• No breaking change or entry added to the changelog.

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Implemented with the Pi coding agent using file inspection/editing, Bash, Git, Mix, and GitHub CLI; a fresh read-only agent reviewed the candidate. Session provenance: 025f22a4-2b69-4bba-b134-9a7df6409188 (local session, no public transcript). This adapter tests existing public SDK evaluation and preserves SDK-owned wire behavior. Human review is required.

[github-actions]

posthog-elixir Compliance Report

Date: 2026-09-07T20:52:53.327555+00:00
Duration: 108520ms

⚠️ Some Tests Failed

40/47 tests passed, 7 failed

---

Capture Tests

⚠️ 29/30 tests passed, 1 failed

View Details

Test	Status	Duration
Format Validation.Event Has Required Fields	✅	609ms
Format Validation.Event Has Uuid	✅	610ms
Format Validation.Event Has Lib Properties	✅	609ms
Format Validation.Distinct Id Is String	✅	609ms
Format Validation.Token Is Present	✅	608ms
Format Validation.Custom Properties Preserved	✅	609ms
Format Validation.Event Has Timestamp	✅	609ms
Format Validation.Non Utc Event Timestamp Is Converted To Utc	❌	608ms
Retry Behavior.Retries On 503	✅	5614ms
Retry Behavior.Does Not Retry On 400	✅	2611ms
Retry Behavior.Does Not Retry On 401	✅	2611ms
Retry Behavior.Respects Retry After Header	✅	5615ms
Retry Behavior.Implements Backoff	✅	15624ms
Retry Behavior.Retries On 500	✅	5614ms
Retry Behavior.Retries On 502	✅	5615ms
Retry Behavior.Retries On 504	✅	5615ms
Retry Behavior.Max Retries Respected	✅	15624ms
Deduplication.Generates Unique Uuids	✅	619ms
Deduplication.Preserves Uuid On Retry	✅	5613ms
Deduplication.Preserves Uuid And Timestamp On Retry	✅	10619ms
Deduplication.Preserves Uuid And Timestamp On Batch Retry	✅	5616ms
Deduplication.No Duplicate Events In Batch	✅	613ms
Deduplication.Different Events Have Different Uuids	✅	611ms
Compression.Sends Gzip When Enabled	✅	609ms
Batch Format.Uses Proper Batch Structure	✅	609ms
Batch Format.Flush With No Events Sends Nothing	✅	606ms
Batch Format.Multiple Events Batched Together	✅	613ms
Error Handling.Does Not Retry On 403	✅	2610ms
Error Handling.Does Not Retry On 413	✅	2611ms
Error Handling.Retries On 408	✅	5614ms

Failures

format_validation.non_utc_event_timestamp_is_converted_to_utc

Event 0 field 'timestamp' instant '2026-09-07T20:51:09.296266Z' != expected '2025-01-02T03:04:05Z'

Feature_Flags Tests

⚠️ 11/17 tests passed, 6 failed

View Details

Test	Status	Duration
Request Payload.Request With Person Properties Device Id	✅	7ms
Request Payload.Flags Request Uses V2 Query Param	✅	6ms
Request Payload.Flags Request Hits Flags Path Not Decide	✅	6ms
Request Payload.Flags Request Omits Authorization Header	✅	6ms
Request Payload.Token In Flags Body Matches Init	✅	7ms
Request Payload.Groups Round Trip	✅	6ms
Request Payload.Groups Default To Empty Object	❌	6ms
Request Payload.Disable Geoip False Propagates As Geoip Disable False	✅	6ms
Request Payload.Disable Geoip Omitted Defaults To False	❌	6ms
Request Payload.Flag Keys To Evaluate Contains Only Requested Key	✅	6ms
Request Lifecycle.No Flags Request On Init Alone	✅	3ms
Request Lifecycle.No Flags Request On Normal Capture	✅	607ms
Request Lifecycle.Two Flag Calls Produce Two Remote Requests	✅	10ms
Request Lifecycle.Mock Response Value Is Returned To Caller	❌	6ms
Retry Behavior.Retries Flags On 502	❌	310ms
Retry Behavior.Retries Flags On 504	❌	309ms
Side Effect Events.Get Feature Flag Captures Feature Flag Called Event	❌	609ms

Failures

request_payload.groups_default_to_empty_object

Field 'groups' not found in /flags request body at path 'groups'. Available keys: ['distinct_id', 'api_key', 'flag_keys_to_evaluate']

request_payload.disable_geoip_omitted_defaults_to_false

Field 'geoip_disable' not found in /flags request body at path 'geoip_disable'. Available keys: ['distinct_id', 'api_key', 'flag_keys_to_evaluate']

request_lifecycle.mock_response_value_is_returned_to_caller

Last action result missing field 'value'. Keys: ['error', 'success']

retry_behavior.retries_flags_on_502

Last action result missing field 'value'. Keys: ['error', 'success']

retry_behavior.retries_flags_on_504

Last action result missing field 'value'. Keys: ['error', 'success']

side_effect_events.get_feature_flag_captures_feature_flag_called_event

Expected 1 events with name '$feature_flag_called', got 0