Skip to content
Chorki Engineering

7. State, Storage and Startup

7. State, Storage and Startup

Section titled “7. State, Storage and Startup”

Where we left off

Section titled “Where we left off”

The earlier docs described behaviour. This one is the reference for where state lives and how it is rebuilt:

  • every store that holds a piece of a download
  • the status state machine
  • what happens, in order, when the app process starts

Five places state lives

Section titled “Five places state lives”
 ┌──────────────────────── the app's Documents directory ──────────────────────┐
 │  downloads.hive            Hive box "downloads": one row per title  (1)      │
 │  <slug>/…                  the files themselves                     (2)      │
 └──────────────────────────────────────────────────────────────────────────────┘
 ┌──────────────────── the app's Application Support directory ────────────────┐
 │  backgroundDownloaderTaskRecords/   the package's task database     (3)      │
 └──────────────────────────────────────────────────────────────────────────────┘
 ┌─────────────────────────────── preferences ─────────────────────────────────┐
 │  app prefs (shared_preferences): schema version, settings, one-time flags (4)│
 │  package prefs (native): task map, undelivered updates              (5)      │
 └──────────────────────────────────────────────────────────────────────────────┘
 plus in-memory state inside DownloadManagerImpl, rebuilt on every launch
# Store Owner Survives process death Authoritative for
1 Hive box downloads the app yes which titles exist, their status, and the saved plan
2 <Documents>/<slug>/ package writes, app reads yes which files are actually done
3 Package task database package yes per-task outcomes the package observed (only used to seed progress)
4 App preferences the app yes settings and one-time prompts
5 Package native preferences package yes tasks in WorkManager (Android), updates not yet delivered to Dart

When stores disagree, the code trusts them in this order: the disk for “is this file done”, the Hive row for “what does this title need, and what did the user ask for”. The package database only gives progress a head start at launch.

1. The Hive row: CacheContentEntity

Section titled “1. The Hive row: CacheContentEntity”

lib/domain/entities/cache_content_entity.dart, Hive typeId: 1, box downloads:

# Field Type Meaning
0 contentId String Key used everywhere.
1 title String Display title.
2 slug String Folder name under Documents.
3 masterUrl String Remote master playlist URL.
4 localMasterRelativePath String? <slug>/master.m3u8 once finalized; null before.
5 posterUrl String Remote poster.
6 localPosterRelativePath String? Set when the poster task completes.
7 thumbnailUrl String Remote thumbnail.
8 localThumbnailRelativePath String? Set when the thumbnail task completes.
9 status String queued, downloading, paused, completed or failed (see the state machine).
10 progress double 0.0–1.0, completedSegments / totalSegments.
11 totalSegments int plan.totalRequiredCount: video + audio segments + key. 0 until planned.
12 completedSegments int Counter value last saved.
13 errorMessage String? Why the row failed.
14 createdAtMs int When the download was first requested.
15 updatedAtMs int Last change; also the watchdog’s fallback “last progress” time.
16 episodeMetaJson String ContentEntitySerde.encode(content): the fields the offline UI needs.
17 userProfileId String The account that downloaded the title. Empty on rows written before owners were recorded. Drives who can see the download.
18 subtitles List<CacheSubtitleEntity>? Remote URL, label, language and local path per track.
19 assetTitleForNotification String Title used in notifications.
20 cachedPlanJson String? The whole HlsParsedPlan as JSON (below).
21 settlementAttempts int Fruitless recovery rounds used (doc 4). Default 0.

CacheSubtitleEntity is Hive typeId: 4, with fields url (0), label (1), language (2) and localRelativePath (3).

Who can see a download

Section titled “Who can see a download”

The files belong to the device, but a row is only shown to the account and profile allowed to see it. DownloadVisibility.allows decides, and DownloadManagerImpl applies it to list, get, watch and the all-records stream:

 nobody signed in                         → show nothing
 row's owner is another account           → hide it
 row has no owner (downloaded before this
   rule existed)                          → show it to any signed-in account
 profile's maturityLevel is 0             → show everything of this account
 profile is restricted (maturityLevel > 0)→ show a title only when its
                                            stored contentRatingMinAge
                                            exists and is ≤ maturityLevel

The maturity rule mirrors the one the catalog uses, including hiding titles with no rating from a restricted profile. The age comes from contentRatingMinAge inside the row’s episodeMetaJson, cached in memory per title.

Two consequences worth knowing:

  • Re-downloading adopts a row. When an account starts a title whose row belongs to someone else, the row’s owner becomes the current account and the files already on disk are reused.
  • Hiding is not deleting. A hidden download keeps its files, and one that is still running keeps downloading; it reappears when its account signs back in. The manager re-publishes every row whenever the signed-in user or profile changes.

Changing the schema.

  • Adding a field with a new index is safe: old rows read the new field as its default.
  • Reusing or retyping an index is not: rows become unreadable. Bump the migration version (below) instead, and regenerate the adapters with dart run build_runner build --delete-conflicting-outputs.

The saved plan (cachedPlanJson)

Section titled “The saved plan (cachedPlanJson)”

Short keys keep a large plan compact, since the raw playlists are stored too:

{
  "mu": "master URL",
  "mc": "master playlist text",
  "sil": "#EXT-X-STREAM-INF line of the chosen rendition",
  "v": {
    "u": "variant playlist URL", "r": "1280x720", "c": "variant playlist text",
    "s": [ {"u": "segment URL", "f": "segment_000.ts"}, … ],
    "k": {"u": "key URL"}
  },
  "a": [
    { "gid": "group id", "n": "name", "lang": "bn", "def": true, "fold": "audio_bn",
      "u": "audio playlist URL", "c": "audio playlist text",
      "s": [ {"u": "…", "f": "…"}, … ] }
  ]
}

Everything after the first parse reads this, never the network:

  • the task factory
  • reenqueueMissing
  • missingOnDisk
  • the playlist writer

Its size grows with the title: a few hundred bytes per segment plus the playlist text.

The status state machine

Section titled “The status state machine”
                 start (new row)
                       │
                       ▼
                  ┌─────────┐   start/retry threw before enqueue
                  │ queued  │──────────────────────────────┐
                  └────┬────┘                              │
       enqueue finished│ or first file counted             │
                       ▼                                   ▼
  resume ┌──────────────────────┐  settlement gave up  ┌────────┐
 ┌──────►│     downloading      │─────────────────────►│ failed │
 │       └──┬────────────┬──────┘  or finalize threw   └───┬────┘
 │   pause  │            │ disk check passed,              │
 │ (Wi-Fi   │            │ playlists written               │ retry → queued
 │  only)   ▼            ▼                                 │ cold start → downloading
 │     ┌────────┐   ┌───────────┐                          │
 └─────┤ paused │   │ completed │                          │
       └───┬────┘   └─────┬─────┘                          │
           │              │ disk check finds files missing │
           │              └──► downloading (rare)          │
           │ app resumes and the disk count is complete    │
           └──► completed                                  │
                                                           │
   cancel / remove, from ANY state ──► row deleted (no status)
Transition Where
none → queued start (new row)
queued → downloading start after enqueueing; _onTrackerChange on first progress; _reconcileSingleRow
queued → failed _failIfStillQueued (exception in start/retry)
downloading → paused pause (called by ConnectivityCubit)
paused → downloading resume
paused → completed _onAppResume → disk complete → _safeFinalize
downloading → failed _settleImpl budget exhausted; _safeFinalize exception
failed → queued → downloading retry
failed → downloading _reconcileSingleRow on cold start (auto-resume)
downloading → completed orchestrator.finalize via _safeFinalize
completed (no local master) → downloading _resumeIfIncomplete finds files missing
any → deleted cancel

The enum also has cancelling. No code assigns it. It survives only in switch statements and guards, as a leftover from an earlier design. downloadStatusFromName maps unknown strings to queued.

What the UI receives

Section titled “What the UI receives”

DownloadRecord (lib/domain/entities/download_record.dart) is an immutable, equatable copy of a row. Timestamps become DateTimes, the status becomes DownloadStatusV2, and subtitles become DownloadSubtitleRefs. Older widgets still speak the legacy DownloadStatus enum in lib/core/enum.dart, so the bloc and the page widgets translate:

DownloadStatusV2 legacy DownloadStatus legacy string (ContentEntity.downloadStatus)
queued download_enqueued download_enqueued
downloading downloading downloading
paused download_paused download_paused
completed downloaded downloaded
failed download_failed download_failed
cancelling none download_paused

2. Files on disk

Section titled “2. Files on disk”
Platform <Documents> Temp files while downloading
Android /data/user/0/com.prothomalo/app_flutter /data/user/0/com.prothomalo/files (Config.useCacheDir: never)
iOS the app container’s Documents/ managed by nsurlsessiond, moved in on completion

The Hive box file downloads.hive sits in the same Documents directory: Hive.init uses it, and each title’s folder is created alongside. Layout of a title folder: doc 1. All stored paths are relative to <Documents>, and the Downloads screen and player rebuild absolute paths at runtime.

3. The package’s task database

Section titled “3. The package’s task database”

FileDownloader().start() turns on task tracking. The package then records every task’s latest status in a small file-per-record store under Application Support, in backgroundDownloaderTaskRecords/.

  • Purpose. It seeds progress at cold start (_completedSegmentsByContentFromPkgDb counts complete records per title).
  • Cleanup. autoCleanDatabase: true keeps it bounded: at most 500 records and nothing older than 10 days, deleted about five per second.
  • Removal. cancel deletes a title’s records, and the migration deletes all of them.

Because cleanup keeps only 500 records, the seed for a large title is usually partial. The disk count completes the picture immediately afterwards.

4. App preferences

Section titled “4. App preferences”

Stored through shared_preferences:

Key Type Written by Meaning
downloads_schema_v int _runDownloadsMigrationIfNeeded Download storage version (currently 2).
downloads_wipe_notice_pending bool migration / notice widget Show “Downloads upgraded” once.
downloadQuality int Download settings 0 → 1080, 1 → 720 (default), 2 → 360.
downloadOverWifi bool Download settings “Download over Wi-Fi only”.
downloads.battery_opt_prompted bool DownloadManagerImpl Battery-optimization prompt already shown (Android).
downloads.oem_protection_tip_shown bool download_protection_tip.dart OEM autostart tip already shown.

At startup AppBootstrapImpl._restoreDownloadPreferences loads downloadQuality and downloadOverWifi into the globals selected_download_quality and download_over_wifi_only.

5. Package native preferences

Section titled “5. Package native preferences”

Not written by the app, but useful when debugging:

Platform Store Keys (prefix com.bbflight.background_downloader.)
Android the default SharedPreferences file (com.prothomalo_preferences.xml) taskMap.v2 (every WorkManager-enqueued task), statusUpdateMap.v2, progressUpdateMap.v2, resumeDataMap.v2, plus config.*
iOS UserDefaults.standard statusUpdateMap.v2, progressUpdateMap.v2, resumeDataMap.v2, plus config values

The …UpdateMap entries hold updates Dart could not receive. They are emptied when FileDownloader().start() replays them.

In-memory state (rebuilt on every launch)

Section titled “In-memory state (rebuilt on every launch)”
Holder State
DownloadManagerImpl _rowById (rows by ID), _perContent and _all (record streams), _missedTaskIds, _lastAdvanceMs, _finalizing and _settling (per-title locks), _saveTimers, the watchdog and emit timers
AssetProgressTracker per-title counters, counted task IDs, disk-scan timers
DownloadOrchestrator per-title plan cache, in-flight start locks
Android holding queue tasks not yet handed to WorkManager

None of it is needed after a restart. Cold start (doc 4) rebuilds it from stores 1–3.

Startup, in order

Section titled “Startup, in order”
main()
 ├─ init()                       GetIt registrations (lib/injection_container.dart):
 │                                 HlsManifestParser, LocalPlaylistWriter,
 │                                 DownloadTaskFactory, DownloadOrchestrator,
 │                                 DownloadManager → lazy singletons;
 │                                 DownloadScreenBloc → factory
 └─ initial()
     ├─ … Firebase, language, version (not download-related)
     ├─ Hive.init(<Documents>)
     ├─ register adapters: CacheContentEntity, CacheSubtitleEntity, …
     ├─ _runDownloadsMigrationIfNeeded()          (below)
     ├─ _initHive(): open "downloads"; if that throws, delete the box and
     │               open a fresh one
     └─ after the first frame (not before runApp):
          _initDownloadsSubsystem()
            ├─ _initBackgroundDownloader()        configure(), max 6 s
            │     global:  requestTimeout 60 s
            │     android: useCacheDir never, holdingQueue (8, –, –)
            │     iOS:     resourceTimeout 7 days, excludeFromCloudBackup
            └─ DownloadManager.initialize(), max 15 s
                  (doc 4, layer 6: listen → start → permission → reconcile
                   → watchdog → lifecycle listener)
                  `ready` completes even if initialization fails or times out

Initialization waits until after the first frame for two reasons: it can take seconds, and it may show the notification-permission dialog. Anything that needs a reconciled list must await DownloadManager.ready. DownloadScreenBloc’s GetDownloadList does exactly that.

The storage migration

Section titled “The storage migration”

_runDownloadsMigrationIfNeeded exists because the previous downloader (built on dio) used the same Hive typeId 1 with a different field layout. Opening old rows with the new adapter would crash. When downloads_schema_v is below 2:

  1. Delete the downloads and downloadQueue boxes.
  2. Delete all of the package’s task database records.
  3. Delete every top-level folder under Documents that looks like an old download: it contains master.m3u8, or any top-level .ts/.m4s. Those stray segments would otherwise inflate the new disk counts and trigger early finalization.
  4. Only if the box deletion succeeded: save version 2 and set downloads_wipe_notice_pending. Otherwise it retries on the next launch.

On the next screen a small widget in main.dart shows a one-time “Downloads upgraded” dialog and clears the flag. Any future breaking change to the row format or disk layout needs a new version number here.

Where this lives in the code

Section titled “Where this lives in the code”
  • lib/domain/entities/cache_content_entity.dart, cache_subtitle_entity.dart: Hive schema (regenerate adapters after edits).
  • lib/domain/entities/download_record.dart, download_status.dart, lib/core/enum.dart: UI-facing types.
  • lib/data/services/content_entity_serde.dart: episodeMetaJson.
  • lib/data/services/hls/hls_manifest_parser.dart: toJson/fromJson of the plan.
  • lib/main.dart: _runDownloadsMigrationIfNeeded, _isLegacyDownloadDir, _initHive, _initDownloadsSubsystem, _initBackgroundDownloader, the wipe-notice widget.
  • lib/injection_container.dart: registrations.
  • lib/data/services/app_bootstrap_impl.dart: _restoreDownloadPreferences.

The takeaway

Section titled “The takeaway”

A download is spread across the Hive row, the disk, the package’s records and two sets of preferences. The design keeps that manageable by giving each store one job:

  • the row says what the title is and what the user wants
  • the disk says what is done
  • everything else is a cache or a hint

Cold start can always rebuild the in-memory picture from the first two.