Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -53,4 +53,6 @@ android/.kotlin/sessions/
# Android signing secrets — never commit these
android/key.properties
*.jks
*.keystore
*.keystore
# Site validation runs (validation/README.md)
validation/runs/
51 changes: 51 additions & 0 deletions docs/audits/2026-11-site-validatie.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
# Site-validatie: triage (13 nov 2026)

Tien echte sites zijn door de app gehaald (`validation/run.sh`, zie `validation/README.md`). Elke export is vergeleken met een onafhankelijke meting: curl, openssl, Playwright WebKit, dig en Team Cymru, telkens twee keer. De eerste run gaf ~40 afwijkingen. Per afwijking: oordeel, oorzaak en wat ermee is gedaan.

Oordelen: **app-bug** · **harness** (de referentie of vergelijker zat fout) · **site** (de site gedraagt zich zo) · **platform** (een grens van WKWebView of iOS/macOS).

## App-bugs (gefixt, met test)

| # | Bevinding | Site(s) | Oorzaak | Fix | Test |
|---|---|---|---|---|---|
| A1 | Certificaat "CN differs from host" terwijl de SAN-lijst de host bevat | rtvutrecht | Alleen de Common Name werd vergeleken (zie ook audit nov 2026) | SAN (OID 2.5.29.17) uit de PEM gelezen; label zegt nu of CN of SAN matchte | `test/derive/security_audit_test.dart` met het echte certificaat (`test/fixtures/certs/rtvutrecht.pem`) |
| A2 | Dezelfde cookie zonder SameSite heette "None" (zwak, −5 in de cookiescore) als de browser hem ook zag, en "not set (Lax by default)" als alleen de server hem zag | kvk, wikipedia | WebKit meldt een ontbrekende SameSite als `None`; bij het samenvoegen won die waarde | Header zonder SameSite + browser `None` blijft "not set"; een echte browserwaarde (bv. Lax via JS) blijft staan | `test/derive/cookies_test.dart` (groep SameSite) |
| A3 | Null-MX (`0 .`, RFC 7505) verscheen als lege mailserver en werd als host opgezocht | example.com | DNS-parser nam de lege naam op | Null-MX wordt overgeslagen | `test/services/sovereignty_service_test.dart` |
| A4 | De CMS-check in Sovereignty toonde ruwe bronfragmenten als evidence | github | `detail` (bronfragment) in plaats van `title` | `title` ("HTML line 414: …") | — (alleen tekst) |
| A5 | Een tracker die alleen via een later binnengekomen cookie werd gevonden, stond wel op Services maar niet in de privacypijler van Sovereignty | rijksoverheid | Sovereignty rekende alleen opnieuw als het aantal resources veranderde | De detectienamen horen nu bij de invoersleutel | `test/services/sovereignty_state_test.dart` |
| A6 | Privacy-tab ("7 trackers follow you") en Sovereignty ("Number of trackers: 6") telden verschillend | allbirds, techcrunch | Privacy telt social widgets als tracker, Sovereignty niet | Besluit: social widgets zijn trackers op beide pagina's. Sovereignty-check heet nu "Trackers in the site's jurisdiction"; de oude widgets-check is "Payments & e-commerce" (geen dubbeltelling) | `test/derive/privacy_consistency_test.dart` |
| A7 | In de export ontbrak de koppeling cookie → service als de evidence-lijst (max. 12) vol zat | allbirds (`_gcl_au`) | Alleen evidence werd geëxporteerd | `cookies` per detectie staat nu in `analysis/detections.json` | `test/derive/privacy_consistency_test.dart` |
| A9 | Elke kale domeinnaam (`https://www.rijksoverheid.nl`) en elke redirect (`http://kvk.nl` → `https://www.kvk.nl/`) werd na afloop **nog een keer** in de WebView geladen. Trackers vuurden dus twee keer en het tweede document kwam uit de cache. | rijksoverheid, kvk, example | `areUrlsEqual` vond `""` en `"/"` verschillend; `BrowserView` vergeleek letterlijk; de "al geladen"-URL werd gewist zodra de load klaar was | `areUrlsEqual` normaliseert het rootpad; `browserAlreadyHas()` (pure functie) beslist in `BrowserView`; de geladen URL blijft bewaard tot de volgende navigatie | `test/services/navigation_policy_test.dart`; in de echte app 0× "Reloading" (was 2× per site) |
| A8 | Kleurdrempels van de scorekaarten stonden los in de widgets (80/50, 75/45) | — | Dubbele drempels in model en UI | `lib/services/derive/score_band.dart` is de enige bron; `bandColor()` voor de kleur | `test/derive/score_band_test.dart` (grenzen + geen losse drempels in de widgets) |

## Open (niet gefixt)

| # | Bevinding | Oordeel | Voorstel |
|---|---|---|---|
| O1 | bol.com: ook de WebView van de app kreeg Akamai's blokpagina ("IP address is blocked"), terwijl Playwright vanaf hetzelfde IP de echte site kreeg. De app analyseert de blokpagina als de site, en de divergentiemelding zegt ten onrechte "the browser loaded the page". | app-bug | De browser-status van het hoofddocument meten (`PerformanceNavigationTiming.responseStatus`) en een blok-/challengepagina markeren in plaats van analyseren. Dit is review-punt 2 uit het plan van 6 nov. |
| O2 | rijksoverheid: de eerste load vanaf een andere pagina geeft 1 cookie (`stg_traffic_source_priority`); dezelfde URL nog eens geeft er 5 (Piwik PRO `_pk_*`, `stg_*`), net als Playwright. Reproduceerbaar in 6 runs. Uitgesloten: timing (alle exports op ~30–36 s), zichtbaarheid (`visible`), cookie-wis-race (3 s pauze), opgeslagen sitedata (volledige wipe), het wissen zelf (geen wis vóór de load), verschil in requests (`ppms.php` gaat in beide). Wel gevonden: `document.cookie` heeft dan echt maar 1 cookie, dus de app leest correct wat de pagina heeft. Overal waar het misgaat is `responseStart` 0 (cross-origin navigatie). | open | Handmatig in Safari controleren (privévenster, Web Inspector → Storage). Volgende hypothese: de User-Agent van WKWebView (zonder `Version/… Safari/…`). |
| O3 | Paginagewicht wijkt fors af op advertentiesites (techcrunch 25 MB tegen 6 MB) | site | Alleen als signaal (±50%); geen fout |

## Harness-correcties (de app had gelijk)

- HSTS en Referrer-Policy: de app volgt de drempels van Mozilla Observatory (HSTS ≥ 6 maanden; `no-referrer-when-downgrade` lekt). De referentie gebruikt nu dezelfde norm.
- robots.txt: curl volgde de consent-redirect van ad.nl. Daarnaast telde de regex lege `Disallow:`-regels mee omdat `\s` ook een newline matcht.
- NS/MX: gemeten op de CNAME of op een gedelegeerde www-zone in plaats van op het domein zelf.
- Hosting achter een CDN: de app noemt bewust de CDN in plaats van de verborgen origin. De check is nu "CDN = AS-houder".
- Browserdata: de eerste runs exporteerden direct na "settled". Met 20 s extra wachttijd (`VSV_LINGER`, standaard in `run.sh`) komen cookies, storage en hosts overeen.

## Consistentiechecks (nieuw)

Pagina's die hetzelfde beoordelen moeten hetzelfde zeggen. Dat wordt nu op twee niveaus getest:

- **Offline (Dart)**: `test/derive/privacy_consistency_test.dart` controleert vijf dingen. Elke trackingcookie uit de Cookie Lens is een Services-detectie. Cookie en service zijn het eens over het soort tracking. De eigen cookie van de site is nergens een tracker. Privacy en Sovereignty tellen dezelfde trackers. De consent-tool is dezelfde. Daarnaast: `test/derive/score_band_test.dart` en de bestaande `cross_page_consistency_test.dart` (HSTS, nosniff, CDN).
- **Op echte exports**: `compare.py` heeft het domein `consistency` met dezelfde invarianten, plus de controle dat de Sovereignty-grade bij de score hoort.

## Eindstand

| Run | ok | volatiel | ❌ |
|---|---|---|---|
| Eerste (13 nov, 08:34), voor triage | 336 | 6 | 78 (incl. gaten en fantomen) |
| Laatste (13 nov, 09:32), na fixes | 418 | 7 | 16 |

De resterende 16: tien komen door bol.com (O1). De andere zes zijn browservariatie op advertentie- en surveysites (O2, O3). Consistentie is op alle tien sites groen.
151 changes: 151 additions & 0 deletions integration_test/site_validation_test.dart
Original file line number Diff line number Diff line change
@@ -0,0 +1,151 @@
// Loads real sites in the real app and writes each inspection package, for
// the validation harness in validation/ (see validation/README.md).
//
// flutter test integration_test/site_validation_test.dart -d macos \
// --dart-define=VSV_SITES=example=https://example.com,github=https://github.com \
// --dart-define=VSV_OUT=/abs/path/validation/runs/<stamp>
//
// Writes <VSV_OUT>/<slug>/app.zip and <VSV_OUT>/<slug>/app-meta.json.
import 'dart:convert';
import 'dart:io';

import 'package:flutter/widgets.dart';
import 'package:flutter_inappwebview/flutter_inappwebview.dart';
import 'package:flutter_test/flutter_test.dart';
import 'package:integration_test/integration_test.dart';
import 'package:provider/provider.dart';
import 'package:shared_preferences/shared_preferences.dart';
import 'package:view_source_vibe/main.dart' as app;
import 'package:view_source_vibe/models/inspection/observed.dart';
import 'package:view_source_vibe/services/html_service.dart';
import 'package:view_source_vibe/services/inspection_export_service.dart';

const _sites = String.fromEnvironment('VSV_SITES');
const _out = String.fromEnvironment('VSV_OUT');
const _timeout = Duration(seconds: 90);

/// Extra seconds to keep the page open after it settled (to test whether
/// late scripts change what the app reports).
const _linger = int.fromEnvironment('VSV_LINGER');

/// Session-restore keys of AppStateService; removed so the app does not
/// reopen the last page on start. Settings stay untouched.
const _restoreKeys = [
'pending_url',
'last_url',
'last_file',
'last_file_name',
'last_file_path',
'last_file_is_url',
'input_text',
];

/// What is still unfinished, by name; empty when the page is ready.
List<String> _pending(HtmlService s) => [
if (s.isLoading) 'loading',
if (!s.store.current.browserSettled) 'browserSettled',
if (!s.detectionsView.complete) 'detections',
if (s.generalView is Pending) 'general',
if (s.securityView is Pending) 'security',
if (s.cookiesView is Pending) 'cookies',
if (s.sovereigntyState is Pending) 'sovereignty',
];

void main() {
final binding = IntegrationTestWidgetsFlutterBinding.ensureInitialized();
binding.framePolicy = LiveTestWidgetsFlutterBindingFramePolicy.fullyLive;

testWidgets('site validation run', (tester) async {
if (_sites.isEmpty || _out.isEmpty) {
fail('Pass --dart-define=VSV_SITES=slug=url,... and VSV_OUT=/abs/dir');
}
final sites = [
for (final entry in _sites.split(','))
if (entry.contains('='))
(
slug: entry.substring(0, entry.indexOf('=')),
url: entry.substring(entry.indexOf('=') + 1),
),
];

final prefs = await SharedPreferences.getInstance();
for (final key in _restoreKeys) {
await prefs.remove(key);
}

app.main();
for (var i = 0; i < 100 && find.byType(app.MyApp).evaluate().isEmpty; i++) {
await tester.pump(const Duration(milliseconds: 100));
}
await tester.pump(const Duration(seconds: 2));
final html = tester.element(find.byType(app.MyApp)).read<HtmlService>();
// Sovereignty only runs while a view watches it; act as that view.
html.watchSovereignty(true);

for (final site in sites) {
final dir = Directory('$_out/${site.slug}')..createSync(recursive: true);
final started = DateTime.now().toUtc();
final clock = Stopwatch()..start();
String? error;
Object? pageState;
var pending = <String>[];

try {
// Cold, pre-consent measurement: no cookies or cache from earlier
// sites or earlier runs.
await CookieManager.instance().deleteAllCookies();
await InAppWebViewController.clearAllCache();

await html.loadUrl(site.url);
// Give the load a moment to start before polling for "done".
await tester.pump(const Duration(seconds: 2));
while (clock.elapsed < _timeout) {
pending = _pending(html);
if (pending.isEmpty) break;
await tester.pump(const Duration(milliseconds: 500));
}
pending = _pending(html);
if (_linger > 0) {
await tester.pump(Duration(seconds: _linger));
// Collect once more (as when the page reports new resources) so
// the late state is in the snapshot.
html.onBrowserResourcesChanged();
await tester.pump(const Duration(seconds: 8));
}

// What the page itself sees: hidden or unfocused pages hold back
// some scripts (analytics waits for a visible page).
try {
pageState = await html.activeWebViewController?.evaluateJavascript(
source: 'JSON.stringify({visibility: document.visibilityState,'
' focus: document.hasFocus(), cookies: document.cookie'
".split(';').filter(Boolean).length})");
} catch (e) {
pageState = 'unavailable: $e';
}

final bytes = InspectionExportService().buildPackageBytes(html);
File('${dir.path}/app.zip').writeAsBytesSync(bytes);
} catch (e) {
error = '$e';
}

final meta = {
'slug': site.slug,
'url': site.url,
'startedAt': started.toIso8601String(),
'finishedAt': DateTime.now().toUtc().toIso8601String(),
'waitedMs': clock.elapsedMilliseconds,
'timedOut': pending.isNotEmpty,
'pendingAtExport': pending,
'pageState': pageState?.toString(),
if (error != null) 'error': error,
};
File('${dir.path}/app-meta.json')
.writeAsStringSync(const JsonEncoder.withIndent(' ').convert(meta));
debugPrint('VSV ${site.slug}: ${clock.elapsedMilliseconds} ms'
'${pending.isEmpty ? '' : ', still pending: $pending'}'
'${error == null ? '' : ', error: $error'}');
}
}, timeout: const Timeout(Duration(minutes: 30)));
}
22 changes: 21 additions & 1 deletion lib/models/inspection/browser_snapshot.dart
Original file line number Diff line number Diff line change
Expand Up @@ -231,6 +231,11 @@ class BrowserSnapshot {
final Observed<BrowserStorage> storage;
final Observed<NavigationTiming> navigation;

/// HTTP status of the main document as the browser received it
/// (PerformanceNavigationTiming.responseStatus, or the WebView's main-frame
/// HTTP error callback for >= 400 where the page cannot see it).
final Observed<int> mainStatus;

/// performance.now() when the snapshot was taken: resources are a snapshot up
/// to this time.
final Observed<double> snapshotTimeMs;
Expand All @@ -253,6 +258,7 @@ class BrowserSnapshot {
required this.storage,
required this.navigation,
required this.snapshotTimeMs,
this.mainStatus = mainStatusNotReported,
this.resourcesTruncated = false,
this.resourcesDropped = 0,
this.rawPayload,
Expand All @@ -268,18 +274,32 @@ class BrowserSnapshot {
navigation is Present ||
snapshotTimeMs is Present;

static const mainStatusNotReported = Unavailable<int>(
UnavailableReason.notReportedByPlatform,
'the browser did not report the HTTP status of the page');

/// A copy with [cookies] replaced (e.g. by the native cookie manager's).
BrowserSnapshot withCookies(Observed<List<BrowserCookie>> cookies) =>
copyWith(cookies: cookies);

/// A copy with [mainStatus] replaced (e.g. by the WebView's HTTP error).
BrowserSnapshot withMainStatus(Observed<int> mainStatus) =>
copyWith(mainStatus: mainStatus);

BrowserSnapshot copyWith(
{Observed<List<BrowserCookie>>? cookies,
Observed<int>? mainStatus}) =>
BrowserSnapshot(
url: url,
platform: platform,
contentType: contentType,
renderedDom: renderedDom,
resources: resources,
cookies: cookies,
cookies: cookies ?? this.cookies,
storage: storage,
navigation: navigation,
snapshotTimeMs: snapshotTimeMs,
mainStatus: mainStatus ?? this.mainStatus,
resourcesTruncated: resourcesTruncated,
resourcesDropped: resourcesDropped,
rawPayload: rawPayload,
Expand Down
16 changes: 15 additions & 1 deletion lib/screens/home_screen.dart
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import 'package:flutter/material.dart';
import 'package:view_source_vibe/widgets/analysis_disclaimer.dart';
import 'package:view_source_vibe/widgets/block_page_banner.dart';
import 'package:flutter/services.dart';
import 'package:flutter/foundation.dart';
import 'package:flutter/gestures.dart';
Expand Down Expand Up @@ -369,8 +370,21 @@ class HomeScreenState extends State<HomeScreen> with TickerProviderStateMixin {
if (const {'browser-content', 'reader', 'source', 'dom-tree'}.contains(tag)) {
return page;
}
// Tabs that analyse the page content say so when it is a block page.
const contentTabs = {
'metadata',
'media',
'page-size',
'services',
'privacy',
'sovereignty'
};
return Column(
children: [Expanded(child: page), const AnalysisDisclaimer()],
children: [
if (contentTabs.contains(tag)) const BlockPageBanner(),
Expanded(child: page),
const AnalysisDisclaimer(),
],
);
}

Expand Down
32 changes: 32 additions & 0 deletions lib/services/collectors/browser_payload.dart
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,7 @@ BrowserSnapshot parseBrowserPayload(Object? raw, BrowserPlatform platform,
cookies: _section(p['cookies'], _parseJsCookies),
storage: _parseStorage(p['storage']),
navigation: _section(p['navigation'], (m) => _parseNavigation(m)),
mainStatus: parseMainStatus(p['navigation']),
snapshotTimeMs: _section(p['timing'], (m) {
final n = m['now'];
if (n is! num) throw const FormatException('timing.now missing');
Expand Down Expand Up @@ -422,6 +423,24 @@ List<ResourceEntry> mergeResources(

// ---- Navigation ----------------------------------------------------------

/// The main document's HTTP status from the collector's navigation section
/// (PerformanceNavigationTiming.responseStatus). Not every engine has it;
/// 0 means the engine has it but could not tell for this response.
Observed<int> parseMainStatus(Object? navigation) {
const na = UnavailableReason.notReportedByPlatform;
if (navigation is! Map || navigation['ok'] != true) {
return const Unavailable(na, 'no navigation timing entry');
}
final s = navigation['status'];
if (s is num && s >= 100 && s < 600) return _measured(s.toInt());
if (s is num) {
return const Unavailable(
na, 'the browser reported no status for this response');
}
return const Unavailable(
na, 'this browser does not expose the HTTP status to the page');
}

NavigationTiming _parseNavigation(Map<String, dynamic> m) {
Observed<double> t(Object? v) {
if (v is! num) {
Expand All @@ -443,6 +462,19 @@ NavigationTiming _parseNavigation(Map<String, dynamic> m) {
);
}

/// [fromPage] (the page's own report) wins; otherwise the WebView's
/// main-frame HTTP error [httpError] (only reported for >= 400). Without
/// either the status stays unknown, with the hint that no error was seen.
Observed<int> mainStatusWithHttpError(Observed<int> fromPage, int? httpError) {
if (fromPage is Present<int>) return fromPage;
if (httpError != null) return _measured(httpError);
if (fromPage is Unavailable<int>) {
return Unavailable(fromPage.reason,
'${fromPage.detail}; the WebView reported no HTTP error (>= 400)');
}
return fromPage;
}

// ---- Storage -------------------------------------------------------------

Observed<BrowserStorage> _parseStorage(Object? sec) {
Expand Down
4 changes: 3 additions & 1 deletion lib/services/collectors/vsv_collect_js.dart
Original file line number Diff line number Diff line change
Expand Up @@ -378,7 +378,9 @@ window.__vsv_collect = function(opts) {
return {
responseStart: nav.responseStart, domInteractive: nav.domInteractive,
domContentLoaded: nav.domContentLoadedEventEnd, loadEnd: nav.loadEventEnd,
protocol: nav.nextHopProtocol || null
protocol: nav.nextHopProtocol || null,
// HTTP status of the document (absent where the engine lacks it).
status: typeof nav.responseStatus === 'number' ? nav.responseStatus : null
};
});

Expand Down
Loading
Loading