Uploading documents – system description

The upload paths (normal, chunked, large file, direct Drive), resolving the storage target, content de-duplication and the background jobs after upload.

Uploading runs through the document centre's save endpoint (FormData) and several supplementary endpoints. Without an explicit target the company's object storage rule decides; after the file come type, labels, sharing, object link and background jobs.

Data model

Table names carry the installation's own prefix in the database.

TableRole
doc, doc_locationrecord and placements
doc_link, label_link, doc_share_itemobject, label and share links
cron_queuedoc_text, doc_recognise, doc_zip_unpack jobs
upload_sessionsession of large and direct Drive uploads (owned by the phone backup model)

Process

  1. An empty POST with no file gives a post_max_size error. A client company in scope is required (error_company).
  2. Target: explicit choice, or the company rule (getObjStorage(object_type | document)); for a gdrive target the company's linked account/drive is enforced; a remote target requires a folder (error_storage_configuration); account scope (storageAccountAllowed, getAllowedStorageAccountIds).
  3. Several files per run; a content duplicate (SHA-256 + size) to the same place is skipped; to another folder a new placement (shortcut) is created with one physical copy.
  4. Invoice package ZIP (PDF only, ≤2000 entries, ≤300 MB, temporary ZIPs together ≤1 GB) → doc_zip_unpack job.
  5. Then: type (doc_role), labels, optional share folder, object link for every document (addDocLink; the answer tells new link or already linked), doc_text job for PDFs; if not for an object and not a non-invoice type, a recognition row and doc_recognise job; for an image uploaded to an invoice, empty invoice fields are filled (autoFillInvoiceFromUpload: text, OCR, reading, fillEmptyFromDocument); thumbnail.
  6. Chunked: uploadchunk. Large file: largeinit / largechunk / largestatus – server-side session (id, offset, size + SHA-256 at the end), free space pre-check.
  7. Direct Drive: drivesession / drivestatus / drivedone – Google resumable session (gdrive only); closing asks the Drive for size and parent folder, and the record is created only if they match.

Settings and rights

  • Settings: upload mode (upload_mode) and the direct upload threshold (upload_direct_min_mb) in the module settings; the mode selector appears only in user mode and with the uploadmode right.
  • Function rights: save canSave(document, 0) (add is not in the catalogue; the Add button is tied to edit), edit on the chunked/large/Drive endpoints, uploadmode.
  • Error codes: error_file, error_file_size, error_large_*, error_drive_*.
Related topics: upload storage background job