Tricks + things to know when copying/moving large number of mails in Thunderbird

Storage format and location

1. At the time of this writing, mails are stored on a machine called lipari

2. The home directoy on this machine is private (not the one shared via nfs on most other machines)
3. The format is mbox: one file per folder, each mail starting with "From " (without the quotes)

4. The INBOX is called .mail and is stored in the home directory

5. The imap folders are in a subdirectory called imap (this can be configured in Thunderbird)

Tricks

1. For folders != INBOX and not currently used, you can just copy/move the respective files on lipari
2. When moving folders, after the previous step: unsubscribe from the old folder and subscribe to the new one
3. It's probably best to close Thunderbird while doing this, but I don't think it's necessary

4. For bulk moves from INBOX (.mail), manually create a lock file .mail.lock in the director of the .mail file: during that time, no other process should be able to deliver mail (append) to the file

Mailbox full on 11.02.2020, LOG form Hannah's actions

I closed Thunderbird and then (more or less) did the following

ssh lipari
touch .mail.lock # after checking that the file does not exist
mv .mail imap/INBOX.MOVED_20200211
touch .mail
rm .mail.lock

The file imap/INBOX.MOVED_20200211 contained mails from 2018 and 2019. I found out via grep -n "^From " imap/INBOX.MOVED_20200211 | less and within less search for /Jan 01 that the line number of the first mail from 2019 was 29347136. Then the following command splits the file in two files, one for the mails from 2018 and one for the files from 2019. The next three lines are just for checking that no lines got lost in the process (the -q option only checks whether the file differ or not, without that option, an error memory exhausted was thrown).

(head -29347135 > 2018.FROM_INBOX_20200211; cat > 2019.FROM_INBOX_20200211) < INBOX.MOVED_20200211
cat 2018.FROM_INBOX_20200211 2019.FROM_INBOX_20200211 > xxx
diff INBOX.MOVED_20200211 xxx
rm xxx

I then appended the contents to the yearly folders in Archives. Before that, I made backup copies of the affected folders.

cp 2018 2018.BACKUP_20200211
cp 2019 2019.BACKUP_20200211
cat 2018.FROM_INBOX_20200211 >> 2018
cat 2019.FROM_INBOX_20200211 >> 2019