iPhone backups: import, chunked upload and background processing

Data model of the iPhone backup module, adding backups and the resumable chunked upload, starting processing and data types, the steps of the background job, deletion and content scope.

The module imports and processes iPhone backup archives (ZIP). The source is a document already uploaded to the document centre or the module's own resumable chunked uploader. Processing runs in the background in time-boxed scheduler rounds with checkpoints: open, zip index, Manifest catalogue, export to the target storage, also per data type. Company-level content scope applies on every endpoint.

Data model

Table names carry the installation's own prefix in the database.

Table	Role
`backup`	backup: company, source document, target, status (14 states from `new` to `deleting`), sizes, `active_scope`
`backup_file`, `backup_record`	the archive's files and the extracted records
`backup_scope_run`	runs per data type
`upload_session`	chunked upload sessions
`doc`	source document in the document centre
`cron_queue`	background jobs

Adding and uploading

save: client + selected document centre document(s) + target (local, gdrive, onedrive; folder required for cloud); one backup record per document in state new, target folder registered; sources outside the scope are skipped and reported.
Chunked upload: uploadinit → uploadchunk → uploadstatus → uploadcommit. The server issues the session ID, the server-side offset decides (409 with the correct offset on mismatch), partial writes are reported, size and SHA-256 are checked on commit.
After commit the archive moves read-only into the backup's source/ folder; a backup record (uploaded), a document centre folder through the folders model and a source document through the document model's addDoc (without text extraction), then an iphone_backup_extract row.
Helpers: docsearch (source search), targetdrives, targetfolders (target cascade).

Processing

process (optional scope): error without a source document; if it obviously cannot start, the gap is named. A data type only after indexing (error_scope_not_indexed); then with the streampublish step, active_scope set and waiting rows deleted first. Queues an iphone_backup_stream or iphone_backup_extract row, status queued.
runtype: starts one data type (message, contact, attachment, calendar, device, callhistory, geo, file scopes) separately, deleting identical waiting rows. Errors: error_scope_unknown, error_type_no_extractor.
Background: the catalog task/iphonebackup worker follows the payload step: extract, stream, zipindex, manifest, streampublish, index, publish, delete, purge (drop the source archive according to the retention rule), plus the data type steps. Every round works within a time budget, resumes from a checkpoint and is idempotent.
delete: selected backups within scope go to deleting with an iphone_backup_delete row; those without permission are skipped and counted.

Permissions

Own keys: list (unclassified), import, process, delete, debug (not in the seeds). The permission seed adds download, but the controller does not check it; its behaviour cannot be determined.
Content scope: backupInScope and companyInScope for source and target company, and the source document's company.

Related topics: iphone backup background job content scope

Request consultation Back to home