==== Fixing mailboxes inconsistencies

....
curl -XPOST /mailboxes?task=SolveInconsistencies
....

Will schedule a task for fixing inconsistencies for the mailbox
deduplicated object stored in Cassandra.

link:#_endpoints_returning_a_task[More details about endpoints returning
a task].

The `I-KNOW-WHAT-I-M-DOING` header is mandatory (you can read more
information about it in the warning section below).

The scheduled task will have the following type
`solve-mailbox-inconsistencies` and the following
`additionalInformation`:

....
{
  "type":"solve-mailbox-inconsistencies",
  "processedMailboxEntries": 3,
  "processedMailboxPathEntries": 3,
  "fixedInconsistencies": 2,
  "errors": 1,
  "conflictingEntries":[{
    "mailboxDaoEntry":{
      "mailboxPath":"#private:user:mailboxName",
      "mailboxId":"464765a0-e4e7-11e4-aba4-710c1de3782b"
    }," +
    "mailboxPathDaoEntry":{
      "mailboxPath":"#private:user:mailboxName2",
      "mailboxId":"464765a0-e4e7-11e4-aba4-710c1de3782b"
    }
  }]
}
....

Note that conflicting entry inconsistencies will not be fixed and will
require to explicitly use link:#_correcting_ghost_mailbox[ghost mailbox]
endpoint in order to merge the conflicting mailboxes and prevent any
message loss.

*WARNING*: this task can cancel concurrently running legitimate user
operations upon dirty read. As such this task should be run offline.

A dirty read is when data is read between the two writes of the
denormalization operations (no isolation).

In order to ensure being offline, stop the traffic on SMTP, JMAP and
IMAP ports, for example via re-configuration or firewall rules.

Due to all of those risks, a `I-KNOW-WHAT-I-M-DOING` header should be
positioned to `ALL-SERVICES-ARE-OFFLINE` in order to prevent accidental
calls.

==== Recomputing mailbox counters

....
curl -XPOST /mailboxes?task=RecomputeMailboxCounters
....

Will recompute counters (unseen & total count) for the mailbox object
stored in Cassandra.

Cassandra maintains a per mailbox projection for message count and
unseen message count. As with any projection, it can go out of sync,
leading to inconsistent results being returned to the client.

link:#_endpoints_returning_a_task[More details about endpoints returning
a task].

The scheduled task will have the following type
`recompute-mailbox-counters` and the following `additionalInformation`:

....
{
  "type":"recompute-mailbox-counters",
  "processedMailboxes": 3,
  "failedMailboxes": ["464765a0-e4e7-11e4-aba4-710c1de3782b"]
}
....

Note that conflicting inconsistencies entries will not be fixed and will
require to explicitly use link:#_correcting_ghost_mailbox[ghost mailbox]
endpoint in order to merge the conflicting mailboxes and prevent any
message loss.

*WARNING*: this task do not take into account concurrent modifications
upon a single mailbox counter recomputation. Rerunning the task will
_eventually_ provide the consistent result. As such we advise to run
this task offline.

In order to ensure being offline, stop the traffic on SMTP, JMAP and
IMAP ports, for example via re-configuration or firewall rules.

`trustMessageProjection` query parameter can be set to `true`. Content
of `messageIdTable` (listing messages by their mailbox context) table
will be trusted and not compared against content of `imapUidTable` table
(listing messages by their messageId mailbox independent identifier).
This will result in a better performance running the task at the cost of
safety in the face of message denormalization inconsistencies.

Defaults to false, which generates additional checks. You can read
https://github.com/apache/james-project/blob/master/src/adr/0022-cassandra-message-inconsistency.md[this
ADR] to better understand the message projection and how it can become
inconsistent.

=== Fixing message inconsistencies

This task is only available on top of Guice Cassandra products.

....
curl -XPOST /messages?task=SolveInconsistencies
....

Will schedule a task for fixing message inconsistencies created by the
message denormalization process.

Messages are denormalized and stored in separated data tables in
Cassandra, so they can be accessed by their unique identifier or mailbox
identifier & local mailbox identifier through different protocols.

Failure in the denormalization process will lead to inconsistencies, for
example:

....
BOB receives a message
The denormalization process fails
BOB can read the message via JMAP
BOB cannot read the message via IMAP

BOB marks a message as SEEN
The denormalization process fails
The message is SEEN via JMAP
The message is UNSEEN via IMAP
....

link:#_endpoints_returning_a_task[More details about endpoints returning
a task].

An admin can specify the concurrency that should be used when running
the task:

* `messagesPerSecond` rate of messages to be processed per second.
Default is 100.

This optional parameter must have a strictly positive integer as a value
and be passed as query parameter.

An admin can also specify the reindexing mode it wants to use when
running the task:

* `mode` the reindexing mode used. There are 2 modes for the moment:
** `rebuildAll` allows to rebuild all indexes. This is the default mode.
** `fixOutdated` will check for outdated indexed document and reindex
only those.

This optional parameter must be passed as query parameter.

It’s good to note as well that there is a limitation with the
`fixOutdated` mode. As we first collect metadata of stored messages to
compare them with the ones in the index, a failed `expunged` operation
might not be well corrected (as the message might not exist anymore but
still be indexed).

Example:

....
curl -XPOST /messages?task=SolveInconsistencies&messagesPerSecond=200&mode=rebuildAll
....

Response codes:

* 201: Success. Corresponding task id is returned.
* 400: Error in the request. Details can be found in the reported error.

The scheduled task will have the following type
`solve-message-inconsistencies` and the following
`additionalInformation`:

....
{
  "type":"solve-message-inconsistencies",
  "timestamp":"2007-12-03T10:15:30Z",
  "processedImapUidEntries": 2,
  "processedMessageIdEntries": 1,
  "addedMessageIdEntries": 1,
  "updatedMessageIdEntries": 0,
  "removedMessageIdEntries": 1,
  "runningOptions":{
    "messagesPerSecond": 200,
    "mode":"REBUILD_ALL"
  },
  "fixedInconsistencies": [
    {
      "mailboxId": "551f0580-82fb-11ea-970e-f9c83d4cf8c2",
      "messageId": "d2bee791-7e63-11ea-883c-95b84008f979",
      "uid": 1
    },
    {
      "mailboxId": "551f0580-82fb-11ea-970e-f9c83d4cf8c2",
      "messageId": "d2bee792-7e63-11ea-883c-95b84008f979",
      "uid": 2
    }
  ],
  "errors": [
    {
      "mailboxId": "551f0580-82fb-11ea-970e-f9c83d4cf8c2",
      "messageId": "ffffffff-7e63-11ea-883c-95b84008f979",
      "uid": 3
    }
  ]
}
....

User actions concurrent to the inconsistency fixing task could result in
concurrency issues. New inconsistencies could be created.

However the source of truth will not be impacted, hence rerunning the
task will eventually fix all issues.

This task could be run safely online and can be scheduled on a recurring
basis outside of peak traffic by an admin to ensure Cassandra message
consistency.


==== Fixing mailboxes flag inconsistencies

===== Fixing mailbox messages deleted flag inconsistencies

This task is only available on top of Guice Cassandra products.

....
curl -XPOST /mailboxes?task=SolveMessageDeletedInconsistencies
....

Will schedule a task for fixing mailbox messages `deleted` flag inconsistencies created by the
mailbox denormalization process.

Response codes:

* 201: Success. Corresponding task id is returned.
* 400: Error in the request. Details can be found in the reported error.

The scheduled task will have the following type
`solve-mailbox-flag-inconsistencies` and the following
`additionalInformation` example:

....
{
    "timestamp": "2024-09-17T04:58:33.683813161Z",
    "type": "solve-mailbox-flag-inconsistencies",
    "processedMailboxEntries": 1,
    "errors": ["551f0580-82fb-11ea-970e-f9c83d4cf8c2"],
    "targetFlag": "DELETED"
}
....


===== Fixing mailbox messages recent flag inconsistencies

This task is only available on top of Guice Cassandra products.

....
curl -XPOST /mailboxes?task=SolveMessageRecentInconsistencies
....

Will schedule a task for fixing mailbox messages `recent` flag inconsistencies created by the
mailbox denormalization process.

Response codes:

* 201: Success. Corresponding task id is returned.
* 400: Error in the request. Details can be found in the reported error.

The scheduled task will have the following type
`solve-mailbox-flag-inconsistencies` and the following
`additionalInformation` example:

....
{
    "timestamp": "2024-09-17T04:59:10.042097161Z",
    "type": "solve-mailbox-flag-inconsistencies",
    "processedMailboxEntries": 2,
    "errors": ["551f0580-82fb-11ea-970e-f9c83d4cf8c2"],
    "targetFlag": "RECENT"
}
....