IMAP MESSAGELIMIT Extension
Alexey Melnikov
Isode Limited
alexey.melnikov@isode.com
https://www.isode.com
Arun Prakash Achuthan
Yahoo!
arunprakash@myyahoo.com
Vikram Nagulakonda
Yahoo!
nvikram_imap@yahoo.com
Luis Alves
luis.alves@lafaspot.com
2022
The MESSAGELIMIT extension of the Internet Message Access Protocol (RFC 3501/RFC 9051) allows servers to announce a limit on the number of messages that can be processed in a single FETCH/SEARCH/STORE/COPY/MOVE command. This helps servers to control resource usage when performing various IMAP operations.

Introduction and Overview
This document defines an extension to the Internet Message Access Protocol [RFC3501] for announcing a server limit on the number of messages that can be processed in a single FETCH/SEARCH/STORE/COPY/MOVE command. This extension is compatible with both IMAP4rev1 [RFC3501] and IMAP4rev2 [RFC9051].

Document Conventions
In protocol examples, this document uses a prefix of "C: " to denote lines sent by the client to the server, and "S: " for lines sent by the server to the client. Lines prefixed with "// " are comments explaining the previous protocol line. These prefixes and comments are not part of the protocol. Lines without any of these prefixes are continuations of the previous line, and no line break is present in the protocol unless specifically mentioned.
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.
Other capitalised words are IMAP keywords [RFC3501][RFC9051] or keywords from this document.

The MESSAGELIMIT extension
An IMAP server advertises support for the MESSAGELIMIT extension by including "MESSAGELIMIT=<limit>" capability in the CAPABILITY response/response code, where "<limit>" is a positive integer that conveys the maximum number of messages that can be processed in a single SEARCH/FETCH/STORE/COPY/MOVE command.
Does this extension need use of ENABLE by clients to opt-in? In particular see below the open issue about COPY command atomicity.

Returning limits on the number of messages processed in a single SEARCH/FETCH/STORE/COPY/MOVE command
Do we need a way to specify SEARCH criterion for "all UIDs after" or "all UIDs before" a specific UID?
If a server implementation doesn't allow more than <N> messages to be operated on by a single SEARCH/FETCH/STORE/COPY/MOVE command, it MUST return the MESSAGELIMIT response code defined below:
MESSAGELIMIT:
  The server doesn't allow more than <N> messages to be operated on by a single SEARCH/FETCH/STORE/COPY/MOVE command. The lowest processed UID is <LastUID>. The client needs to repeat the operation for remaining messages, if required.
  In the following example the <N> value is 1000 and the lowest processed UID <LastUID> is 23221.

    C: 03 FETCH 10000:14589 (UID FLAGS)
  S: * 14589 FETCH (FLAGS (\Seen) UID 25000)
  S: * 14588 FETCH (FLAGS (\Answered) UID 24998)
  S: ... further 997 fetch responses
  S: * 13590 FETCH (FLAGS () UID 23221)
  S: 03 OK [MESSAGELIMIT 1000 23221] FETCH completed with 1000 partial
      results

  In the following example the client searches for UNDELETED UIDs between 22000:25000. The total number of matching messages exceeds the server's published 1000 messages limit.

    C: 04 UID SEARCH UID 22000:25000 UNDELETED
  S: * SEARCH 25000 24998 (... 997 UIDs ...) 23221
  S: 04 OK [MESSAGELIMIT 1000 23221] SEARCH completed with 1000 partial results

  The following example demonstrates copy of messages with UIDs between 18000:21000. The total message count exceeds the server's published 1000 messages limit.

    C: 05 UID COPY 18000:21000 "Trash"
  S: * NO [MESSAGELIMIT 1000 20001] Too many messages to copy
  S: 05 OK [COPYUID 1397597919 20001:21000 21363:22362] COPY completed for the last 1000 messages

  Open Issue: Note that the above example shows a UID COPY that partially fails. This is assumed to be better for clients that don't understand the MESSAGELIMIT response code. However this might cause naive clients to believe that the COPY fully completed and that all messages were copied. (An alternative would be to return MESSAGELIMIT in the tagged NO response, meaning that no messages could be copied. However this wouldn't work well with clients that don't support MESSAGELIMIT response code.)
As a possible solution to this we can make clients opt-in into this behaviour using the ENABLE command. Would this work? What is the incentive for clients to do this?
  The following example shows MOVE of messages with UIDs between 18000:21000. The total message count exceeds the server's published 1000 messages limit. The client that wants to move all messages in the range and observes a MESSAGELIMIT response code,
can repeat the command by updating the UID set parameter specified in the command. The client needs to keep doing this until MESSAGELIMIT response is not returned (or until a tagged NO/BAD is returned).

    C: 06 UID MOVE 18000:21000 "Archive/2021/2021-12"
  S: * OK [COPYUID 1397597919 20001:21000 22363:23362] Some messages were not moved
  S: * 12336 EXPUNGE
  S: * 12335 EXPUNGE
  ...
  S: * 11335 EXPUNGE
  S: 06 OK [MESSAGELIMIT 1000 20001] MOVE completed for the last 1000 messages

  The following example shows update of flags for messages with UIDs between 18000:20000. The total message count exceeds the server's published 1000 messages limit. The client that wants to change flags for all messages in the range and observes a MESSAGELIMIT response code, can repeat the command by updating the UID set parameter specified in the command. The client needs to keep doing this until MESSAGELIMIT response is not returned (or until a tagged NO/BAD is returned).

    C: 07 UID STORE 18000:20000 +FLAGS (\Seen)
  S: * 11215 FETCH (FLAGS (\Seen \Deleted) UID 20000)
  S: * 11214 FETCH (FLAGS (\Seen \Answered \Deleted) UID 19998)
  ...
  S: * 10216 FETCH (FLAGS (\Seen) UID 19578)
  S: 07 OK [MESSAGELIMIT 1000 19578] STORE completed for the last 1000 messages

  The following example shows use of MESSAGELIMIT response code together with the PARTIAL [draft-ietf-extra-imap-partial] extension. The total message count exceeds the server's published 1000 messages limit.

    C: 08 UID FETCH 22000:25000 (UID FLAGS MODSEQ) (PARTIAL -1:-1500)
  S: 08 NO [MESSAGELIMIT 1000] FETCH exceeds the maximum 1000 message limit

Note that when the server needs to return both EXPUNGEISSUED ([RFC9051]) and MESSAGELIMIT response codes, the former MUST be returned in the tagged OK response, while the latter MUST be returned in an untagged NO response. The following example demonstrates that:

  C: 031 FETCH 10000:14589 (UID FLAGS)
  S: * 14589 FETCH (FLAGS (\Seen) UID 25000)
  S: * 14588 FETCH (FLAGS (\Answered) UID 24998)
  S: ... further 997 fetch responses
  S: * 13590 FETCH (FLAGS () UID 23221)
  S: * NO [MESSAGELIMIT 1000 23221] FETCH completed with 1000 partial
      results
  S: 031 OK [EXPUNGEISSUED] Some messages were also expunged

Interaction with SORT and THREAD extensions
Servers that advertise MESSAGELIMIT N will be unable to execute a THREAD [RFC5256] command in a mailbox with more than N messages.
Servers that advertise MESSAGELIMIT N might be unable to execute a SORT [RFC5256] command in a mailbox with more than N messages, unless they maintain indices for different SORT orders they support. In absence of such indeces server implementors will need to decide whether their server advertises SORT or MESSAGELIMIT capability.

Formal syntax
The following syntax specification uses the Augmented Backus-Naur Form (ABNF) notation as specified in [ABNF].
Non-terminals referenced but not defined below are as defined by IMAP4.
Except as noted otherwise, all alphabetic characters are case-insensitive. The use of upper or lower case characters to define token strings is for editorial clarity only. Implementations MUST accept these strings in a case-insensitive fashion.

capability          =/ "MESSAGELIMIT=" message-limit
                       ;; <capability> from [RFC3501]

message-limit       = nz-number

resp-text-code      =/ "MESSAGELIMIT" SP message-limit [SP uniqueid]
    ;; No more than nz-number messages can be processed
    ;; by any command at a time. The last (lowest) processed
    ;; UID is uniqueid.
    ;; The last parameter is omitted, when not known.

Security Considerations
This document defines an additional IMAP4 capability. As such, it does not change the underlying security considerations of [RFC3501] and IMAP4rev2 [RFC9051].
This document defines an optimization that can both reduce the amount of work performed by the server, as well at the amount of data returned to the client. Use of this extension is likely to cause the server and the client to use less memory than when the extension is not used. However, as this is going to be new code in both the client and the server, rigorous testing of such code is required in order to avoid introducing of new implementation bugs.

IANA Considerations

Changes/additions to the IMAP4 capabilities registry
IMAP4 capabilities are registered by publishing a standards track or IESG approved Informational or Experimental RFC. The registry is currently located at:

   https://www.iana.org/assignments/imap4-capabilities

IANA is requested to add definition of the MESSAGELIMIT extension to point to this document.

Acknowledgments
This document was motivated by Yahoo! team and their questions about best client practices for dealing with large mailboxes.
Editor of this document would like to thank the following people who provided useful comments or participated in discussions of this document: Timo Sirainen and Barry Leiba.

Normative References
[ABNF]

Augmented BNF for Syntax Specifications: ABNF
Dave Crocker, Ed.
Paul Overell, Ed.
January 2008
RFC 5234

Informative References
[draft-ietf-extra-imap-partial]

IMAP Paged SEARCH & FETCH Extension
Alexey Melnikov
Isode Limited
Arun Prakash Achuthan
Yahoo!
Vikram Nagulakonda
Yahoo!
Luis Alves
12 August 2022
Internet-Draft draft-ietf-extra-imap-partial-02
[RFC3501]
[RFC9051]
[RFC2119]
[RFC8174]
[RFC5256]