| Internet-Draft | IMAP MESSAGELIMIT | July 2024 |
| Melnikov, et al. | Expires 30 January 2025 | [Page] |
The MESSAGELIMIT extension of the Internet Message Access Protocol (RFC 3501/RFC 9051) allows servers to announce a limit on the number of messages that can be processed in a single FETCH/SEARCH/STORE/COPY/MOVE (or their UID variants), APPEND or UID EXPUNGE command. This helps servers to control resource usage when performing various IMAP operations. This helps clients to know the message limit enforced by corresponding IMAP server and avoid issuing commands that would exceed such limit.¶
This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.¶
Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.¶
Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."¶
This Internet-Draft will expire on 30 January 2025.¶
Copyright (c) 2024 IETF Trust and the persons identified as the document authors. All rights reserved.¶
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License.¶
This document may contain material from IETF Documents or IETF Contributions published or made publicly available before November 10, 2008. The person(s) controlling the copyright in some of this material may not have granted the IETF Trust the right to allow modifications of such material outside the IETF Standards Process. Without obtaining an adequate license from the person(s) controlling the copyright in such materials, this document may not be modified outside the IETF Standards Process, and derivative works of it may not be created outside the IETF Standards Process, except to format it for publication as an RFC or to translate it into languages other than English.¶
This document defines an extension to the Internet Message Access Protocol [RFC3501] for announcing a server limit on the number of messages that can be processed in a single FETCH/SEARCH/STORE/COPY/MOVE (or their UID variants), APPEND or UID EXPUNGE command. This extension is compatible with both IMAP4rev1 [RFC3501] and IMAP4rev2 [RFC9051].¶
In protocol examples, this document uses a prefix of "C: " to denote lines sent by the client to the server, and "S: " for lines sent by the server to the client. Lines prefixed with "// " are comments explaining the previous protocol line. These prefixes and comments are not part of the protocol. Lines without any of these prefixes are continuations of the previous line, and no line break is present in the protocol unless specifically mentioned.¶
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.¶
Other capitalised words are IMAP key words [RFC3501][RFC9051] or key words from this document.¶
An IMAP server advertises support for the MESSAGELIMIT extension by including "MESSAGELIMIT=<limit>" capability in the CAPABILITY response/response code, where "<limit>" is a positive integer that conveys the maximum number of messages that can be processed in a single [UID] SEARCH/FETCH/STORE/COPY/MOVE, APPEND or UID EXPUNGE command.¶
An IMAP server that only enforces message limit on [UID] COPY/APPEND commands would include the "SAVELIMIT=<limit>" capability (instead of the "MESSAGELIMIT=<limit>") in the CAPABILITY response/response code.¶
The limit advertised in the MESSAGELIMIT or SAVELIMIT capability SHOULD NOT be lower than 1000 messages.¶
If a server implementation doesn't allow more than <N> messages to be operated on by a single COPY/UID COPY command, it MUST fail the command by returning a tagged NO response with the MESSAGELIMIT response code defined below. No messages are copied in this case. If a server implementation doesn't allow more than <N> messages to be operated on by a single SEARCH/FETCH/STORE/MOVE (or their UID variants), APPEND or UID EXPUNGE command, it MUST return the MESSAGELIMIT response code defined below:¶
C: 03 FETCH 10000:14589 (UID FLAGS)
S: * 14589 FETCH (FLAGS (\Seen) UID 25000)
S: * 14588 FETCH (FLAGS (\Answered) UID 24998)
S: ... further 997 fetch responses
S: * 13590 FETCH (FLAGS () UID 23221)
S: 03 OK [MESSAGELIMIT 1000 23221] FETCH completed with 1000 partial
results
¶
C: 04 UID SEARCH UID 22000:25000 UNDELETED S: * SEARCH 25000 24998 (... UIDs ...) 23221 S: 04 OK [MESSAGELIMIT 1000 23221] SEARCH completed with 1000 partial results¶
C: 05 UID COPY 18000:21000 "Trash" S: 05 NO [MESSAGELIMIT 1000 20001] Too many messages to copy, try a smaller subset¶
C: 06 UID MOVE 18000:21000 "Archive/2021/2021-12" S: * OK [COPYUID 1397597919 20001:21000 22363:23362] Some messages were not moved S: * 12336 EXPUNGE S: * 12335 EXPUNGE ... S: * 11337 EXPUNGE S: 06 OK [MESSAGELIMIT 1000 20001] MOVE completed for the last 1000 messages¶
C: 07 UID STORE 18000:20000 +FLAGS (\Seen) S: * 11215 FETCH (FLAGS (\Seen \Deleted) UID 20000) S: * 11214 FETCH (FLAGS (\Seen \Answered \Deleted) UID 19998) ... S: * 10216 FETCH (FLAGS (\Seen) UID 19578) S: 07 OK [MESSAGELIMIT 1000 19578] STORE completed for the last 1000 messages¶
C: 08 UID EXPUNGE 11000:13000 S: * 4306 EXPUNGE S: * 4305 EXPUNGE ... S: * 3307 EXPUNGE S: 08 OK [MESSAGELIMIT 1000 11627] UID EXPUNGE completed for the last 1000 messages¶
C: 09 EXPUNGE S: * 4306 EXPUNGE S: * 4305 EXPUNGE ... S: * 3307 EXPUNGE S: * 112 EXPUNGE S: 09 OK EXPUNGE completed¶
C: 10 UID FETCH 22000:25000 (UID FLAGS MODSEQ) (PARTIAL -1:-1500) S: 10 NO [MESSAGELIMIT 1000] FETCH exceeds the maximum 1000 message limit¶
C: 10 UID FETCH 22000:25000 (UID FLAGS MODSEQ) S: * 12367 FETCH (FLAGS (\Seen \Deleted) UID 23007) S: * 12366 FETCH (FLAGS (\Seen \Answered \Deleted) UID 23114) ... S: * 13366 FETCH (FLAGS (\Seen) UID 24598) S: 10 OK [MESSAGELIMIT 1000 23007] FETCH exceeds the maximum 1000 message limit¶
Note that when the server needs to return both EXPUNGEISSUED ([RFC9051]) and MESSAGELIMIT response codes, the former MUST be returned in the tagged OK response, while the latter MUST be returned in an untagged NO response. The following example demonstrates that:¶
C: 11 FETCH 10000:14589 (UID FLAGS)
S: * 14589 FETCH (FLAGS (\Seen) UID 25000)
S: * 14588 FETCH (FLAGS (\Answered) UID 24998)
S: ... further 997 fetch responses
S: * 13590 FETCH (FLAGS () UID 23221)
S: * NO [MESSAGELIMIT 1000 23221] FETCH completed with 1000 partial
results
S: 11 OK [EXPUNGEISSUED] Some messages were also expunged
¶
When IMAP MULTIAPPEND [RFC3502] extension is also supported by the server, the message limit also applies to the APPEND command. As MULTIAPPEND APPEND needs to atomic (as per [RFC3502]), no messages are appended when the server MESSAGELIMIT is exceeded.¶
The MESSAGELIMIT extension also defines 2 extra SEARCH keys: UIDAFTER and UIDBEFORE, which make it easier to convert a single UID to a range of UIDs.¶
"UIDAFTER <uid>" - Messages that have a UID greater than the specified UID. This is semantically the same as "UID <uid>+1:*".¶
"UIDBEFORE <uid>" - Messages that have a UID less than the specified UID. This is semantically the same as "UID 1:<uid>-1" (or if <uid> has the value 1, then the empty set).¶
These 2 SEARCH keys are particularly useful when the SEARCHRES [RFC5182] extension is also supported, but they can be used without it. For example, this allows a SEARCH that sets the "$" marker to be converted to a range of messages in a subsequent SEARCH, and both SEARCH requests can be pipelined.¶
C: 12 UID SEARCH UIDAFTER 25000 UNDELETED S: * SEARCH 27800 27798 (... 250 UIDs ...) 25001 S: 12 OK SEARCH completed¶
Servers that advertise MESSAGELIMIT N will be unable to execute a THREAD [RFC5256] command in a mailbox with more than N messages.¶
Servers that advertise MESSAGELIMIT N might be unable to execute a SORT [RFC5256] command in a mailbox with more than N messages, unless they maintain indices for different SORT orders they support. In absence of such indeces server implementors will need to decide whether their server advertises SORT or MESSAGELIMIT capability.¶
Servers that support both MESSAGELIMIT and SEARCHRES [RFC5182] extensions MUST truncate SEARCH SAVE result stored in the $ variable when the SEARCH command succeeds, but the MESSAGELIMIT response code is returned. For example, if the following SEARCH would have returned 1200 results in absence of MESSAGELIMIT, and the MESSAGELIMIT is 1000, only 1000 matching results will be saved in the $ variable:¶
C: D0004 UID SEARCH RETURN (SAVE) SINCE 1-Jan-2004 NOT FROM "Smith" UID 22000:25000 UNDELETED S: D0004 OK [MESSAGELIMIT 1000 1179] SEARCH completed with 1000 partial results saved¶
A server that advertises the MESSAGELIMIT=N capability would have the following effect on clients that don't support this capability:¶
Operations on a mailbox that has <= N messages are not affected.¶
In a mailbox with more than N messages:¶
An attempt to COPY/UID COPY more than N messages will always fail.¶
EXPUNGE and CLOSE will always operate on the full mailbox, so they are not affected.¶
Other commands like FETCH, SEARCH and MOVE will be effectively restricted to the last N messages of the mailbox. In particular unextended SEARCHes intended for counting of messages with or without a particular set of flags would return incorrect counts.¶
It is important to understand that the above effects essentially abandon existing clients with respect to operation on large mailboxes. Suppose, for example, that a user is accessing a large and active mailing list via IMAP - the mailing list gets on the order of 1500 posts per week. When the user returns from a week-long vacation and catches up on the mailing list, the user’s client will be fetching information about 1500 messages. If the server has a MESSAGELIMIT of 1000, the client will only be able to download 1000 of most recent messages; the client will not understand why, will not be prepared to recover from the situation, and will act as if it is interacting with a broken server.¶
In order to give clients time to implement this extension, servers should not be strict about applying the MESSAGELIMIT at first. One possible approach is to advertise a MESSAGELIMIT but not enforce it at all for a while. Clients that understand this extension will comply, reducing load on the server, but clients that do not understand the limit will continue to work in all situations.¶
Another approach, perhaps phased in later, is to advertise one limit but to treat it as a soft limit and to begin enforcement at a higher, unadvertised hard limit. In the above example, perhaps the server might advertise 1000 but actually enforce a limit of 10,000. Again, clients that understand MESSAGELIMIT will comply with the limit of 1000, but other clients will still interoperate up to the higher threshold.¶
Attempts to go beyond the advertised limit can be logged so that client understanding of MESSAGELIMIT can be tracked. If implementation and deployment of this extension becomes common, it may at some point be acceptable to strictly enforce the advertised limit and to accept that the remaining clients will, indeed, no longer work properly with mailboxes above that limit.¶
The following syntax specification uses the Augmented Backus-Naur Form (ABNF) notation as specified in [ABNF].¶
Non-terminals referenced but not defined below are as defined by IMAP4 [RFC3501].¶
Except as noted otherwise, all alphabetic characters are case-insensitive. The use of upper or lower case characters to define token strings is for editorial clarity only. Implementations MUST accept these strings in a case-insensitive fashion.¶
capability =/ "MESSAGELIMIT=" message-limit /
"SAVELIMIT=" message-limit
;; <capability> from [RFC3501]
message-limit = nz-number
resp-text-code =/ "MESSAGELIMIT" SP message-limit [SP uniqueid]
;; No more than nz-number messages can be processed
;; by any command at a time. The last (lowest) processed
;; UID is uniqueid.
;; The last parameter is omitted, when not known.
¶
This document defines an additional IMAP4 capability. As such, it does not change the underlying security considerations of [RFC3501] and IMAP4rev2 [RFC9051].¶
This document defines an optimization that can both reduce the amount of work performed by the server, as well at the amount of data returned to the client. Use of this extension is likely to cause the server and the client to use less memory than when the extension is not used, which can in turn help to protect from Denial-of-Service attacks. However, as this is going to be new code in both the client and the server, rigorous testing of such code is required in order to avoid introducing of new implementation bugs.¶
IMAP4 capabilities are registered by publishing a standards track or IESG approved Informational or Experimental RFC. The registry is currently located at:¶
https://www.iana.org/assignments/imap-capabilities/¶
IANA is requested to add registrations of "MESSAGELIMIT=" and "SAVELIMIT=" capabilities to this registry, both pointing to this document.¶
This document was motivated by the Yahoo! team and their questions about best client practices for dealing with large mailboxes.¶
Editor of this document would like to thank the following people who provided useful comments, contributed text or participated in discussions of this document: Timo Sirainen, Barry Leiba, Ken Murchison and Arnt Gulbrandsen.¶