8. The Stale KMS Token Fix

8. The Stale KMS Token Fix

Commit 40b26c2a (2026-09-27), “Implement content access token service and JWT expiry handling”. This doc explains how KMS authorization works today, which files take part, what the bug was, and what the fix changed.

The root cause below comes from the commit’s code comments and message. The failure was not reproduced on a device, and the bug has no entry in docs/testing/bug-log.md.

How KMS works now

Chorki’s video is HLS encrypted with AES-128. Before the player can decode a segment, it has to fetch the decryption key from kms.chorki.net. That key request needs five headers, all built in one place: EncryptionHelper.playerHeader() in lib/core/helper/encryption_helper.dart.

 referer          ← decrypted from /v2/settings
 vpsid            ← HMAC-SHA1("android|<referer>", cdn secret)   ← same for every user
 platform         ← android / ios
 x-entitlement    ← "this user is allowed"   (one per user)
 x-content-access ← "this title is allowed"  (one per title)

The last two are the newer tokens (docs 6 and 7). They are sent only when the kms_token_headers_enabled toggle in profile settings is on, and only when they are not empty.

Where the two tokens come from

 x-entitlement                              x-content-access
 ─────────────                              ────────────────
 GET /v1/user/entitlements                  "content_access_token" field on
   ↓                                          GET /v1/:type/:slug (content page)
 EntitlementService                           ↓
   • cached in the global                   ContentModel / ContentPageModel parse it
     `entitlement_token` ValueNotifier        ↓
   • saved per user in SharedPreferences    ContentEntity.content_access_token
   • dropped when the user/profile changes    ↓ (NEW) ContentAccessTokenService

The three playback paths

 1. Online play   The native ExoPlayer/AVPlayer sends the headers on every request.
                  → Dart never learns whether KMS answered 401/403.

 2. Cast/AirPlay  The app's own LAN proxy fetches the key.
                  → Dart sees a 403, refreshes the entitlement token and retries once.

 3. Downloads     The key is fetched once at download time and saved as key.bin.
                  → KMS is never contacted again for that title.

Files involved

Role	File
Builds the headers	lib/core/helper/encryption_helper.dart
Global token + on/off toggle	lib/core/constant/const.dart (entitlement_token, kms_token_headers_enabled), lib/main.dart
Entitlement token	lib/data/datasources/entitlement_data_source.dart, lib/data/services/entitlement_service.dart
Per-title token (new)	lib/data/services/content_access_token_service.dart, lib/data/repositories/content_repository_impl.dart
Token expiry check (new)	lib/core/helper/token_expiry.dart (was jwt_expiry.dart; reads PASETO too, see doc 10)
Parsing	lib/data/models/content_model.dart, lib/data/models/content_page_model.dart, lib/domain/entities/content.dart
Online player	lib/presentation/widgets/player/player_media_builder.dart, lib/presentation/pages/content_pages/video_player_screen.dart, lib/presentation/widgets/player/player_controls.dart
Trailer / shorts	lib/presentation/pages/content_pages/single_content_screen.dart, lib/presentation/pages/shorts/widgets/byte_ad_page.dart, lib/presentation/widgets/shorts_widget/shorts_player_pool.dart
Cast / AirPlay	lib/data/services/cast/cast_lan_proxy_server_impl.dart, lib/presentation/widgets/player/cast_controller.dart, lib/presentation/widgets/player/airplay/airplay_controller.dart
Downloads	lib/data/services/download_manager_impl.dart, lib/data/services/hls/download_orchestrator.dart, lib/domain/entities/cache_content_entity.dart (Hive field 22)
Dependency wiring	lib/injection_container.dart

What the bug was: stale tokens

Playback failed after the app had been in the background for a while, or when a title was opened from cached data. Three problems combined to cause it.

1. The headers were a copy made too early. See The headers were a copy made too early below.
2. The per-title token came from the disk cache. Content pages are cached for up to 7 days and are shown from the cache first. Right after the app resumes, the live fetch often fails, so the repository falls back to the cached copy. The content_access_token inside that copy can be days old.
3. The entitlement token was never checked for expiry. ensureToken() returned any non-empty cached token. It only fetched a new one when the user changed.

As a result, the native player sent an expired token and KMS refused to hand out the key. On the native path Dart cannot see HTTP status codes, so this looked like an ordinary playback error. The user saw the “playback failed” card, and nothing retried.

The headers were a copy made too early

Think of the headers as a note the app hands to the player: “when you ask KMS for the key, show them these passes.” The question is when that note gets written.

In the old code the note was written the moment the app prepared a video entry, not the moment the video started playing:

// player_media_builder.dart, before the fix
return OSMediaModel(
  ...
  headers: EncryptionHelper().playerHeader(
    contentAccessToken: content.content_access_token,
  ),
);

playerHeader() runs right there, while the entry is being created. It returns a plain map of text: the tokens exactly as they were at that second. The map is a copy, not a live link. If the app fetches a new token a minute later, the copy inside the entry still holds the old one, and nobody updates it.

The player library only uses that note later, when it actually loads the video (_loadMediaEntry in ottstack_player). It passes the stored map straight to the native player:

 the app prepares an entry           the player loads the entry
 ─────────────────────────           ──────────────────────────
 old: headers written HERE    ───→   sent to KMS as they are, however old
 new: headers written later   ───→   prepare() writes them HERE, from tokens
                                     that are valid right now

Is the gap between those two moments long? For an ordinary tap on Play, no; it is a fraction of a second. On that path the real damage came from problem 2: the token was already old when it was copied, because it came from the cached page. The early copy becomes a problem of its own when an entry is kept and played later:

• Playlists and the next episode. The player keeps a list of entries built up front and plays the next one when the current one ends. That can be a whole episode (an hour or more) after its headers were written.
• Playing the same entry again. A retry, or picking the video up again after the app returns from the background, reuses the entry and its old note.
• The native player. kVideo receives the headers once per load and uses them for every request in that video, including key requests. There is no way to hand it a new token partway through.

A second, smaller problem: the “am I online?” flag

The old code also decided whether to send KMS headers at all by looking at the global internet_available flag, rather than asking whether the video was stored on the phone:

headers: internet_available.value
    ? EncryptionHelper().playerHeader(...)
    : const {},            // ← no referer, no vpsid, no tokens

If the flag said “offline” while the phone was actually online, a normal streaming video started with an empty header map. KMS then refused the key, because not even referer and vpsid were there.

Was the flag actually false after a resume? Nobody has confirmed it. There are no logs showing it. But coming back from the background is exactly when the flag is most likely to be wrong, so the risk was real.

What sets internet_available

Only Reachability._publish() in lib/core/network/reachability.dart changes it:

internet_available = the phone has a network  AND  it was not marked unreachable

Two things can make it false:

1. The phone reports no network. ConnectivityCubit listens to the connectivity_plus plugin. When the plugin reports none, the cubit calls interfaceChanged(up: false).
2. The API keeps failing to connect. Every connection error in ApiClient calls recordConnectionFailure(). After two failures at least 5 seconds apart, within 60 seconds, Reachability sends a quick HEAD request to the API host. If that fails too, it marks the network unreachable.

Why coming back from the background can trip it

 app in background                   app comes back
 ─────────────────                   ──────────────
 the OS freezes the app              first API calls go out on dead connections
 open connections go stale           → connection errors → recordConnectionFailure()
 the radio may power down            radio / DNS / Wi-Fi still waking up
                                     → the HEAD check fails too
                                     → internet_available = false  (for a while)

• Dead connections. The app keeps connections to the API open so it can reuse them (the HTTP/2 connection pool). While the app is frozen, the phone or the router quietly closes them. The first requests after resume fail with connection errors, and those are exactly what Reachability counts.
• The radio is still waking up. Just after resume, mobile data or Wi-Fi may not be fully back. The HEAD check can fail during that window, so the “unreachable” verdict gets confirmed.
• A wrong “offline” can outlast the whole background stay. If the flag went false just before the app went to the background (say the network dropped as the phone locked), the timer that re-checks the network (after 5, 10, 20, then 30 seconds) does not run while iOS has the app suspended. After resume the flag stays false until that timer fires or any request succeeds (recordSuccess()).
• connectivity_plus may briefly report none while the phone brings networking back. This has not been checked on either platform, so treat it as a possibility only.

The fix’s own code comments make the same assumption: “right after resume the network is often not back yet” (EntitlementService.renew) and “a live fetch fails (typical right after the app resumes)” (ContentAccessTokenService).

It was a known trap

The player screen had already been moved off internet_available onto network_interface_available (which only asks “is there a network at all?”) for this reason: Reachability can mark a slow or throttled link as offline. See video_player_screen.dart, the three network_interface_available checks. buildPlayerMedia was one of the places still reading the old flag.

The fix removes that dependency. Headers now depend only on whether the URL is local (isLocalPlaybackUrl), so a wrong “offline” flag can no longer strip the KMS headers from a streaming video.

How to confirm it on a device

Run a debug build, put the app in the background for a few minutes, bring it back and play something. Then look in the console for:

• [reachability] N separate requests could not connect … treating the network as unreachable, which means cause 2 happened;
• [connectivity] … lines, and whether a none event shows up around the resume, which means cause 1 happened.

If neither appears, the flag stayed true, and this part never contributed to the bug.

What the fix changed

 BEFORE                                  AFTER
 ──────                                  ─────
 build entry → headers copied now        build entry → prepare() hook attached
 play → KMS rejects old token            play → prepare() fetches fresh tokens (≤ 6 s)
 → error card                                 → headers rebuilt → KMS accepts
                                         still errors? → one silent retry

1. Headers are rebuilt at load time

The fix doesn’t try to keep the stored copy up to date. Instead it writes the note at the last possible moment. buildPlayerMedia in player_media_builder.dart now attaches a prepare hook, which the player library runs just before it loads the stream. The hook keeps itself attached to the new entry (via withHeaders()), so every later load of that entry writes the headers again. The hook calls _resolveKmsHeaders(), which does two things in parallel:

• asks EntitlementService.ensureToken() for a valid entitlement token;
• asks ContentAccessTokenService.resolve() for a per-title token it trusts.

Both calls share a 6-second timeout. Each falls back to the token already known, so a slow or dead network never ends up with fewer headers than the old code sent.

Trailers and shorts ads build their media outside buildPlayerMedia. They get the same behaviour through refreshKmsHeadersOnLoad().

2. ContentAccessTokenService (new)

This service trusts a per-title token only when both of these hold:

• the token arrived from the network in the last 10 minutes (the repository calls record() after every live content-page fetch);
• it is not JWT-expired.

Any other token is re-fetched at play time. If two callers ask for the same title at the same moment, they share one fetch. markStale(route) makes the service forget a title’s token after a failed playback.

3. jwt_expiry.dart (new)

This reads the token’s exp claim, with 60 seconds of leeway. A token that is not a JWT counts as never expiring, so the server stays the judge for those.

4. EntitlementService changes

Method	What it does	Used when
ensureToken()	Now drops an expired JWT and fetches a new one	Before every load
refresh() (new)	Throws the token away, then fetches	KMS definitely rejected it (cast proxy 403)
renew() (new)	Fetches a new token but keeps the old one until the fetch succeeds	A suspicion: a playback error or a long stay in the background

Superseded by doc 10 (cache first; renew() and the 30-minute rule are gone). Previously superseded by doc 9. Loads now go through PlaybackAuthResolver, which calls renew() on every play (15 s de-dupe window). ensureToken() is only used for trailers and shorts ads now.

The service now also watches the app lifecycle. On resume after 30 minutes or more in the background it calls renew(); after a shorter stay it calls ensureToken().

5. Automatic recovery in the player

When playback fails, video_player_screen.dart:

1. marks the title’s token stale,
2. renews the entitlement token,
3. reloads the stream at the same position.

This runs once per title. It is skipped for offline playback, local files and cast sessions. While the retry is running, player_controls.dart shows the normal buffering spinner instead of the error card, so a recovered failure looks like a short rebuffer.

6. Local vs remote decides the headers

isLocalPlaybackUrl() (a file path, a file:// URL, 127.0.0.1 or localhost) now decides whether KMS headers are sent. Downloads and the local playback server get none; remote streams always get them.

7. Startup order

main.dart now calls WidgetsFlutterBinding.ensureInitialized() before init(). The entitlement service’s lifecycle observer needs the binding. Without that call the app throws during startup and stays on the splash screen.

Tests added

• test/core/helper/jwt_expiry_test.dart
• test/data/services/content_access_token_service_test.dart
• test/data/services/entitlement_service_test.dart (expiry and lifecycle cases)
• test/presentation/widgets/player/player_controls_error_test.dart
• test/presentation/widgets/player/player_media_builder_test.dart

Still not covered

• Offline downloads. (Entitlement half fixed in doc 9: re-queues now fetch a fresh x-entitlement. The per-title token is still the one saved on the row.) The per-title token saved on the download row is frozen at download time, so a download the OS re-queues days later can send an expired token. Once key.bin is on disk, playback never goes back to KMS, so a lapsed subscription does not stop a downloaded title from playing.
• Expiry mid-playback. Nothing refreshes a token that expires during a title, apart from the one-shot retry after an error.
• Tokens reach the CDN. On the native path the player applies one header set to every request. Both tokens therefore go out on every segment request, not only on the key request (doc 6, drawback 1).