Testing

Testing

How this app is tested, what runs where, and how to add to it.

 tool/quality.sh fast                      (pre-push hook runs this)
 ├─ dart format          changed files only
 ├─ gen_coverage_imports --check           test/coverage_all_files_test.dart current?
 ├─ analyze_ratchet      per-rule counts must not rise
 ├─ design_ratchet       untokenised UI values must not rise
 ├─ flutter test --coverage                unit · widget · contract
 ├─ coverage_check       per-tier floors, never drop
 └─ flutter test --tags golden             pixel comparisons

 tool/quality.sh full  = the above + e2e/run.sh   (simulator + QA session)

Running things

tool/quality.sh                 # the whole local gate
tool/quality.sh --update        # re-record the analyzer, design and coverage baselines
fvm dart run tool/design_ratchet.dart --report    # the screens to move onto tokens next
fvm flutter test                # every test
fvm flutter test test/data      # one folder
fvm flutter test --coverage && fvm dart run tool/coverage_check.dart
fvm flutter test --tags golden --update-goldens   # after an intended UI change
fvm flutter test | tool/testlog.sh                # drop the app's own logging

The app logs a good deal through debugPrint, and a run carries all of it. tool/testlog.sh keeps the reporter’s lines, every framework error block and every compiler error, and drops the rest; a failure comes through whole. It is not part of tool/quality.sh on purpose — a gate that hides something unexpected is worse than a noisy one.

debugPrint cannot simply be muted in test/flutter_test_config.dart: flutter_test runs debugAssertAllFoundationVarsUnset after every test and fails any test that left debugPrint pointing anywhere but the binding’s own sink.

Enable the hooks once per clone:

git config core.hooksPath .githooks

pre-commit formats staged Dart files. pre-push runs the gate. Both can be skipped for one command with --no-verify.

Layout

 test/
 ├─ helpers/          shared test infrastructure (see below)
 ├─ fixtures/api/     real API responses, extracted from the Postman collection
 ├─ core/ data/ domain/ presentation/    mirror lib/ exactly
 ├─ contract/         whole-codebase rules (architecture, l10n, schema)
 ├─ goldens/          pixel comparisons, tagged `golden`
 ├─ injection_container_test.dart        the whole DI graph resolves
 └─ coverage_all_files_test.dart         GENERATED — imports every lib file
 patrol_test/         on-device E2E, driven by e2e/run.sh (see e2e/README.md)

coverage_all_files_test.dart exists so a file no test touches counts as 0% instead of vanishing from the report. Regenerate it with fvm dart run tool/gen_coverage_imports.dart whenever files are added or moved.

The helpers

test/helpers/…	What it gives you
globals.dart	resetGlobals() puts every app-wide ValueNotifier back to its launch value; signIn(), privilegedUser(), blankSettings(). Call tearDown(resetGlobals) in any test that touches them.
di.dart	resetDi(), provide<T>(), provideFactory<T>(), provideCastStores().
platform_mocks.dart	installPlatformMocks() answers every plugin channel a screen touches (Firebase, analytics pigeons, package/device info, connectivity, wakelock, toast, secure storage) and points path_provider at a temp dir. Also primeSizeUtils() and initMarktagForTest().
pump_app.dart	tester.pumpApp(widget, …) mounts a widget the way MyApp does (Sizer, theme, localizations, clamped text scale), with MockStackRouter for context.router; collectOverflows(); TestScreens.
bloc_harness.dart	blocTest, MockBloc, MockCubit, whenListen, statesUntil — the bloc_test API, in-house (see below).
fake_http.dart	installFakeHttp() swaps the app’s Dio transport for a recording fake. apiOk(result) builds the API envelope. No test reaches the network.
fixture.dart	loadFixture('home_page'), fixtureResult(...), withField(...) to break one field of a real response.
builders.dart	aContent(), aSection(), aSubscription(), aFeatures(), aMeta(), aLicense() … with sensible defaults.
hive.dart	openTestCache(), openTestBox<T>() on a throwaway directory.
fake_iap.dart	installFakeInAppPurchase() — otherwise InAppPurchase.instance opens a real billing connection.
golden.dart	loadAppFonts(), expectGoldens(...) at 1x across phone and tablet.

Why blocTest is ours

package:bloc_test cannot be resolved here: every release needs a test version that excludes the analyzer 6.x that hive_generator and auto_route_generator hold the project on. test/helpers/bloc_harness.dart provides the same API (build/act/expect/verify/skip/wait, MockBloc, whenListen), so switching to the package later is a find and replace.

Writing a test

A bloc. Mock the repository, not the data source:

blocTest<SearchScreenBloc, SearchScreenState>(
  'a query is sent and its results shown',
  build: () {
    when(() => repository.getSearchData(any(), any(), any()))
        .thenAnswer((_) async => Right(_results(['hit'])));
    return SearchScreenBloc(getSearchDataUseCase: GetSearchDataUseCase(repository));
  },
  act: (bloc) => bloc.add(SearchItems(query: 'domm')),
  expect: () => [isA<SearchScreenLoadingState>(), isA<SearchScreenSearchedDataState>()],
);

Cover, for every bloc: the initial state, each event’s success path, its failure path, the offline path (NoInternetFailure — the UI shows a retry for it), and the cache-first behaviour if it has one.

A data source. Use the fake transport and assert the URL, not just the result:

final http = installFakeHttp();
http.onGet('/v2/movie', json: apiOk(fixtureResult('home_tab_movie')));

final result = await HomeDataSource().getHomePageData('/movie');

expect(http.requests.single.uri.toString(), apiV2Url('/movie'));

An unmatched request fails the test, so a data source can never quietly reach a URL the test did not expect.

A model. Parse the real fixture, then break it: null fields, missing keys, [] where an object was expected. The API really does send those.

A widget. tester.pumpApp(...) with a MockBloc for its bloc, then assert what is on screen, which event a tap dispatches, and that nothing overflows at 320 wide, in bn, and at text scale 1.1.

The deep-qa agent

/qa <path> runs the deep-qa agent over a target: it builds a risk inventory, writes the tests, proves each one can fail by mutating the code under test, triages what goes red, fixes what it may and escalates what it may not, and leaves the gate green. /qa diff does the same for everything uncommitted; /qa audit <path> writes nothing new and reports which existing tests are vacuous.

It exists because the audit found tests that could not fail — see docs/testing/deep-qa.md for the loop, the boundaries and where the files live.

The three baselines

All live in tool/baselines/ and only ever move in the improving direction.

• analyzer.json — issue count per rule. Any error fails; any rule that goes up fails. The null-check rules are frozen on purpose (unnecessary_null_comparison, unnecessary_non_null_assertion, invalid_null_aware_operator): those guards cover nulls the API really sends, so they are never removed. Cover null handling with a model test instead.
• design.json — how many UI values in lib/presentation bypass the design tokens: raw and Material colours, appTheme.* primitives, TextStyle(, numbered radii and numbered EdgeInsets. theme/ and widgets/glass/ are exempt, since that is where the tokens live. What counts is pinned by test/tool/design_ratchet_test.dart; the rules are in docs/design/README.md.
• coverage.json — percentage per tier (logic, player, ui, overall), with the tier definitions in tool/coverage_tiers.json. A drop fails the gate. Raise it with tool/quality.sh --update in the same commit that earns it.

Fixtures

test/fixtures/api/*.json are real responses, extracted from docs/api/chorki-api.postman_collection.json by fvm dart run tool/extract_api_fixtures.dart. The collection clips large bodies at 60 KB, so a clipped body is cut back to its last complete list element and closed. CDN-auth values and the caller’s network identity are replaced. Re-run the tool after re-capturing the collection.

Bugs found by tests

docs/testing/bug-log.md has one row per real bug a test caught, with the test that caught it and the commit that fixed it. Add to it whenever a new test finds something.

On-device tests

patrol_test/ runs the real app on iOS simulators — see e2e/README.md for sessions, the device matrix and the bug report it produces. It needs a QA session token that only you can mint.

docs/testing/phase-4-e2e-plan.md is the plan for the flows still to be written there (search, watchlist, offline downloads, deep links, subscription, language, shorts, the offline guard) and for putting the Android emulator into the matrix. It starts with the three things that reliably waste an afternoon: the token rotating out from under a run, the Xcode 27 simulator build failure, and the fact that patrol is injected rather than committed.