Incoming e-mails – operation and data model

How read messages are stored, the import and processing chain, pipeline transitions, background jobs and permissions, for operators and auditors.

Overview

The module stores the messages read from Gmail mailboxes, processes them with rules and AI classification and links them to invoices, tasks or partners. The raw message and the attachments go to the document centre. The screen checks the old base route permission (access, modify); a function permission exists only for the SQL debug.

Data model

Table names carry the installation's own prefix in the database.

Table	Role
`email_message`	message header, business status and pipeline state
`email_content`	message body
`email_attachment`, `email_document`	attachments (legacy store) and the document-centre link
`email_link`	links to invoices, tasks, partners
`email_pipeline_log`	timeline of pipeline transitions
`email_proposal`	AI suggestions with confidence
`doc`, `doc_link`, `document`	document centre and the legacy document store

Import chain

The manual import (syncGmail) and the background job run the same chain for every active source with the Gmail provider; other providers are skipped with a log entry. Without a source, a fallback configuration built from the installation settings is used. Per message:

Fetch the Gmail message, hash-based duplicate check.
E-mail filter rules (skip, ignore, link).
Raw .eml, message row, link by contact filter, contact master upsert.
The .eml and the attachments go to the document centre (Dlbtdocstore::addDocument; company from the source or resolveCompanyForEmail) or to the legacy Drive or local store.
PDF text extraction, PDF-based post-filter.
AI classification if enabled: automatic skip or ignore, automatic link above the threshold, otherwise a suggestion.
Gmail label swap to the processed or error suffix.

Pipeline transitions and manual actions

The manual business status only switches between new and ignored.
Allowed manual transitions: PENDING_APPROVAL → PROCESSED, IGNORED, SKIPPED; INGESTED, FILTERED, AI_*, ERROR → IGNORED, SKIPPED. A terminal state allows nothing. Switching to PROCESSED runs the pending command (task creation via task/task::addTask, linking).
approveProposal, rejectProposal: approve or reject a suggestion with a transition. aiSuggestionApprove creates the link and marks the message processed; aiSuggestionReject deletes the automatic links and resets it to new.
createTaskFromEmail: title up to 255 characters, deadline as YYYY-MM-DD; link, comment, attachment binding, processed status.
linkEmail: manual linking, several targets allowed; searchTargets: invoice, task, partner search; listCompaniesForSelect: active companies without content filtering.
Maintenance batches: extractPdfsExisting and aiClassifyExisting (1–50), cleanupLegacyAttachments (rate limited), applyRulesToExisting (up to 1000 new messages), backfillContacts (up to 5000).
createFilterRule: rule from a message; skip deletes the message at once, ignore sets the status. createFilterAndDelete is a legacy alias.
delete: deletes messages with their files (Drive files with retry, local files, label restore, database cascade).

Background jobs

Job	What it does
`email`	import and local housekeeping: staging directory capped at 256 MB, orphan files older than 3 days removed, up to 25 local rows uploaded to Drive per run
`email_commands`	searches `INGESTED` messages for the system's task prefix, instruction block or marker tag → `PENDING_APPROVAL` or `FILTERED`
`email_ai`	AI suggestion into `email_proposal`; above the threshold `PROCESSED`, below it `PENDING_APPROVAL`; on failure three times `AI_RETRY`, then `ERROR`

All three are scheduled globally and can be switched on or off in the module's system settings. Their entry point is the catalog-side email/email controller.

Permissions

Layer 1: list and details with base access permission; every changing endpoint with base modify permission (the company selector and the target search with access).
Layer 2: only the debug key is checked; the list key is not checked with canAction.
Layer 3: there is no content (company) scope in the code.

Known limitations

In the email_ai step the AI call is a placeholder according to the code; it needs an API key to work.
The list and the endpoints do not filter by content permission, and the function is not classified in the function catalogue.

Related topics: incoming e-mail background job AI suggestion permissions

Request consultation Back to home