Current File : //proc/thread-self/root/usr/share/doc/imapsync/FAQ.d/FAQ.Messages_Selection.txt
#!/bin/cat
$Id: FAQ.Messages_Selection.txt,v 1.18 2021/07/04 23:05:24 gilles Exp gilles $
This document is also available online at
https://imapsync.lamiral.info/FAQ.d/
https://imapsync.lamiral.info/FAQ.d/FAQ.Messages_Selection.txt
=======================================================================
Imapsync tips to select messages.
=======================================================================
Questions answered in this FAQ are:
Q. What messages imapsync syncs by default?
Q. Is there a way we can specify a date range to sync emails?
If yes, can you please share an example?
Q. Is there a way we can specify an age to sync emails?
If yes, can you please share some examples?
Q. I want to sync messages based on their UID.
Q. Can I migrate only mails with attachments?
Q. How can I move messages marked \Deleted from all folders to
a dedicated folder?
Q. What are the selection criteria available with --search option?
=======================================================================
Q. What messages imapsync syncs by default?
R1. By default, Imapsync syncs all messages, except duplicates.
If you want to sync duplicates, add the option --syncduplicates
R2. You can alter the default "all messages selected" by using one or
more of the following options:
Extract from the README https://imapsync.lamiral.info/README
OPTIONS/message selection
--maxsize int : Skip messages larger (or equal) than int bytes
--minsize int : Skip messages smaller (or equal) than int bytes
--maxage int : Skip messages older than int days.
final stats (skipped) don't count older messages
see also --minage
--minage int : Skip messages newer than int days.
final stats (skipped) don't count newer messages
You can do (+ zone are the messages selected):
past|----maxage+++++++++++++++>now
past|+++++++++++++++minage---->now
past|----maxage+++++minage---->now (intersection)
past|++++minage-----maxage++++>now (union)
--search str : Selects only messages returned by this IMAP SEARCH
command. Applied on both sides.
For a complete set of what can be search see
https://imapsync.lamiral.info/FAQ.d/FAQ.Messages_Selection.txt
--search1 str : Same as --search but for selecting host1 messages only.
--search2 str : Same as --search but for selecting host2 messages only.
So --search CRIT equals --search1 CRIT --search2 CRIT
--maxlinelength int : skip messages with a line length longer than int bytes.
RFC 2822 says it must be no more than 1000 bytes but
real life servers and email clients do more.
--useheader str : Use this header to compare messages on both sides.
Ex: Message-ID or Subject or Date.
--useheader str and this one, etc.
--usecache : Use cache to speed up next syncs. Not set by default.
--nousecache : Do not use cache. Caveat: --useuid --nousecache creates
duplicates on multiple runs.
--useuid : Use UIDs instead of headers as a criterion to recognize
messages. Option --usecache is then implied unless
--nousecache is used.
=======================================================================
Q. Is there a way we can specify a date range to sync emails?
If yes, can you please share an example?
R. Yes, a date range is possible with the --search option.
imapsync ... --search "SENTSINCE 1-Jan-2010"
or
imapsync ... --search "SENTBEFORE 31-Dec-2010"
or
imapsync ... --search "SENTSINCE 1-Jan-2010 SENTBEFORE 31-Dec-2010"
Months are specified like this:
Jan
Feb
Mar
Apr
May
Jun
Jul
Aug
Sep
Oct
Nov
Dec
=======================================================================
Q. Is there a way we can specify an age to sync emails?
If yes, can you please share some examples?
R. Yes, with the --maxage or the --minage option.
Have in mind that the options --maxage and --minage can NOT be
combined successfully with the --search command. If the --search
option is present on the command line then --maxage and --minage
are ignored.
Example.1 Sync only messages less than 2 days old:
imapsync ... --maxage 2
Example.2 Sync only messages more than 2 days old:
imapsync ... --minage 2
Example.3 Sync only messages more than 30 days old and less than 365 days old:
imapsync ... --minage 30 --maxage 365
Example.4 Sync only messages less than 30 days old or more than 365 days old:
imapsync ... --maxage 30 --minage 365
Full explanation:
--maxage <int> : Skip messages older than <int> days.
final stats (skipped) don't count older messages
see also --minage
--minage <int> : Skip messages newer than <int> days.
final stats (skipped) don't count newer messages
You can do (+ are the messages selected):
past|----maxage+++++++++++++++>now
past|+++++++++++++++minage---->now
past|----maxage+++++minage---->now (intersection)
past|++++minage-----maxage++++>now (union)
Caveat.1 By default,
Option --maxage is implemented as a --search SENTSINCE
RFC 3501 says: SENTSINCE <date>
Messages whose [RFC-2822] Date: header (disregarding time and
timezone) is within or later than the specified date.
Option --minage is implemented as a --search SENTBEFORE
RFC 3501 says: SENTBEFORE <date>
Messages whose [RFC-2822] Date: header (disregarding time and
timezone) is earlier than the specified date.
If --noabletosearch is on then --minage and --maxage deal
with the internal dates given by a FETCH imap command but
not the Date: header. Internal date is the arrival date
in the mailbox. Same remark for --noabletosearch1 and
--noabletosearch2 but only for one side then.
=======================================================================
Q. I want to sync messages based on their UID.
R. First have in mind that UIDs are unique only per folder, so work this
way only with one folder at a time, with --folder option.
To show UIDs, there is the --debugLIST parameter.
imapsync ... --debugLIST
To sync only a part of all messages, selected by UIDs
from 10000 to 11000:
imapsync ... --search1 "UID 10000:11000"
To sync from INBOX only 3 messages UIDs 20000 20002 20004:
imapsync ... --search1 'OR OR UID 20000 UID 20002 UID 20004' --folder INBOX
To sync all messages from INBOX except 3 messages
UIDs 20000 20002 20004:
imapsync ... --search1 'NOT OR OR UID 20000 UID 20002 UID 20004' --folder INBOX
If you search n UIDs then you have to put n-1 OR in the search line,
that's IMAP.
=======================================================================
Q. Can I migrate only mails with attachments?
R. Use:
imapsync ... --search "HEADER Content-Disposition attachment"
or
imapsync ... --search "HEADER Content-Type multipart/mixed"
or more generally:
imapsync ... --search "OR HEADER Content-Disposition attachment HEADER Content-Type multipart/mixed"
The last one doesn't work on all imap servers, not on Gmail at least (tested 2020/06/16)
=======================================================================
Q. How can I move messages marked \Deleted from all folders to
a dedicated folder?
R. To move \Deleted messages from all folders to a specific folder,
let's call it Trash, use:
imapsync ... --search DELETED --regextrans2 "s/.*/Trash/"
=======================================================================
Q. What are the selection criteria available with --search option?
R. The list of search criteria are listed below, an excerpt from RFC3501.
https://tools.ietf.org/html/rfc3501#section-6.4.4
...
The SEARCH command searches the mailbox for messages that match
the given searching criteria. Searching criteria consist of one
or more search keys. The untagged SEARCH response from the server
contains a listing of message sequence numbers corresponding to
those messages that match the searching criteria.
When multiple keys are specified, the result is the intersection
(AND function) of all the messages that match those keys. For
example, the criteria DELETED FROM "SMITH" SINCE 1-Feb-1994 refers
to all deleted messages from Smith that were placed in the mailbox
since February 1, 1994. A search key can also be a parenthesized
list of one or more search keys (e.g., for use with the OR and NOT
keys).
In all search keys that use strings, a message matches the key if
the string is a substring of the field. The matching is
case-insensitive.
The defined search keys are as follows. Refer to the Formal
Syntax section for the precise syntactic definitions of the
arguments.
<sequence set>
Messages with message sequence numbers corresponding to the
specified message sequence number set.
ALL
All messages in the mailbox; the default initial key for
ANDing.
ANSWERED
Messages with the \Answered flag set.
BCC <string>
Messages that contain the specified string in the envelope
structure's BCC field.
BEFORE <date>
Messages whose internal date (disregarding time and timezone)
is earlier than the specified date.
BODY <string>
Messages that contain the specified string in the body of the
message.
CC <string>
Messages that contain the specified string in the envelope
structure's CC field.
DELETED
Messages with the \Deleted flag set.
DRAFT
Messages with the \Draft flag set.
FLAGGED
Messages with the \Flagged flag set.
FROM <string>
Messages that contain the specified string in the envelope
structure's FROM field.
HEADER <field-name> <string>
Messages that have a header with the specified field-name (as
defined in [RFC-2822]) and that contains the specified string
in the text of the header (what comes after the colon). If the
string to search is zero-length, this matches all messages that
have a header line with the specified field-name regardless of
the contents.
KEYWORD <flag>
Messages with the specified keyword flag set.
LARGER <n>
Messages with an [RFC-2822] size larger than the specified
number of octets.
NEW
Messages that have the \Recent flag set but not the \Seen flag.
This is functionally equivalent to "(RECENT UNSEEN)".
NOT <search-key>
Messages that do not match the specified search key.
OLD
Messages that do not have the \Recent flag set. This is
functionally equivalent to "NOT RECENT" (as opposed to "NOT
NEW").
ON <date>
Messages whose internal date (disregarding time and timezone)
is within the specified date.
OR <search-key1> <search-key2>
Messages that match either search key.
RECENT
Messages that have the \Recent flag set.
SEEN
Messages that have the \Seen flag set.
SENTBEFORE <date>
Messages whose [RFC-2822] Date: header (disregarding time and
timezone) is earlier than the specified date.
SENTON <date>
Messages whose [RFC-2822] Date: header (disregarding time and
timezone) is within the specified date.
SENTSINCE <date>
Messages whose [RFC-2822] Date: header (disregarding time and
timezone) is within or later than the specified date.
SINCE <date>
Messages whose internal date (disregarding time and timezone)
is within or later than the specified date.
SMALLER <n>
Messages with an [RFC-2822] size smaller than the specified
number of octets.
SUBJECT <string>
Messages that contain the specified string in the envelope
structure's SUBJECT field.
TEXT <string>
Messages that contain the specified string in the header or
body of the message.
TO <string>
Messages that contain the specified string in the envelope
structure's TO field.
UID <sequence set>
Messages with unique identifiers corresponding to the specified
unique identifier set. Sequence set ranges are permitted.
UNANSWERED
Messages that do not have the \Answered flag set.
UNDELETED
Messages that do not have the \Deleted flag set.
UNDRAFT
Messages that do not have the \Draft flag set.
UNFLAGGED
Messages that do not have the \Flagged flag set.
UNKEYWORD <flag>
Messages that do not have the specified keyword flag set.
UNSEEN
Messages that do not have the \Seen flag set.
...
search = "SEARCH" [SP "CHARSET" SP astring] 1*(SP search-key)
; CHARSET argument to MUST be registered with IANA
...
search-key = "ALL" / "ANSWERED" / "BCC" SP astring /
"BEFORE" SP date / "BODY" SP astring /
"CC" SP astring / "DELETED" / "FLAGGED" /
"FROM" SP astring / "KEYWORD" SP flag-keyword /
"NEW" / "OLD" / "ON" SP date / "RECENT" / "SEEN" /
"SINCE" SP date / "SUBJECT" SP astring /
"TEXT" SP astring / "TO" SP astring /
"UNANSWERED" / "UNDELETED" / "UNFLAGGED" /
"UNKEYWORD" SP flag-keyword / "UNSEEN" /
; Above this line were in [IMAP2]
"DRAFT" / "HEADER" SP header-fld-name SP astring /
"LARGER" SP number / "NOT" SP search-key /
"OR" SP search-key SP search-key /
"SENTBEFORE" SP date / "SENTON" SP date /
"SENTSINCE" SP date / "SMALLER" SP number /
"UID" SP sequence-set / "UNDRAFT" / sequence-set /
"(" search-key *(SP search-key) ")"
seq-number = nz-number / "*"
; message sequence number (COPY, FETCH, STORE
; commands) or unique identifier (UID COPY,
; UID FETCH, UID STORE commands).
; * represents the largest number in use. In
; the case of message sequence numbers, it is
; the number of messages in a non-empty mailbox.
; In the case of unique identifiers, it is the
; unique identifier of the last message in the
; mailbox or, if the mailbox is empty, the
; mailbox's current UIDNEXT value.
; The server should respond with a tagged BAD
; response to a command that uses a message
; sequence number greater than the number of
; messages in the selected mailbox. This
; includes "*" if the selected mailbox is empty.
seq-range = seq-number ":" seq-number
; two seq-number values and all values between
; these two regardless of order.
; Example: 2:4 and 4:2 are equivalent and indicate
; values 2, 3, and 4.
; Example: a unique identifier sequence range of
; 3291:* includes the UID of the last message in
; the mailbox, even if that value is less than 3291.
sequence-set = (seq-number / seq-range) *("," sequence-set)
; set of seq-number values, regardless of order.
; Servers MAY coalesce overlaps and/or execute the
; sequence in any order.
; Example: a message sequence number set of
; 2,4:7,9,12:* for a mailbox with 15 messages is
; equivalent to 2,4,5,6,7,9,12,13,14,15
; Example: a message sequence number set of *:4,5:7
; for a mailbox with 10 messages is equivalent to
; 10,9,8,7,6,5,4,5,6,7 and MAY be reordered and
; overlap coalesced to be 4,5,6,7,8,9,10.
See also the excellent "Introduction to IMAP" by Nick Babcock
https://nbsoftsolutions.com/blog/introduction-to-imap
See also "WITHIN Search Extension to the IMAP Protocol"
https://tools.ietf.org/html/rfc5032
...
search-key =/ ( "OLDER" / "YOUNGER" ) SP nz-number
; search-key defined in RFC 3501
4. Example
C: a1 SEARCH UNSEEN YOUNGER 259200
S: a1 * SEARCH 4 8 15 16 23 42
Search for all unseen messages within the past 3 days, or 259200
seconds, according to the server's current time.
=======================================================================
=======================================================================
Mini Shell