Productivity Automation Sync Mail With MbSync And Tagged With NotMuch

Well, while waiting for my system to get synced with updated software on specific Linux distribution (Gentoo), where I am sitting right now, so thought to share one of the productivity and kinda primary ones to get over the overhaul.

My productivity heavily relies on the mail and keeping them the proper way to enhance my knowledge of information on the right track. I do use various tools to keep them in a proper way, as you can see in the title itself I have mentioned two. Plus, I might be sitting in a specific editor then I could be using several other pieces of software to get it a more streamlined way.

I have a little bash script that does the job for the entire process, in fact, two. One is dedicated to syncing with the IMAP server and the other one tags as per the rule I specify. And I combine them both in the same script.

Here is the script in its entirety :

#!/bin/bash

source ~/.bashrc >/dev/null  2>&1

mail_backup=/home/bhaskar/gmail-backup/

# Clean up the old backup dirs
cd $mail_backup

echo Cleaning old backup dirs ......
find $PWD -type d -name  ".notmuch" -prune -o -name ".notmuch*"  -ls

find $PWD -type d -name ".notmuch" -prune -o -name ".notmuch*"  -exec rm -rf {} \; >/dev/null  2>&1

# Take backup of the existing database
echo Taking backup of the existing database.....

mv /home/bhaskar/gmail-backup/.notmuch{,$(date +'%F_%T')}
find $mail_backup -type d -name ".notmuch*" -newermt $(date '+%F') -ls

# Fetch mails from remote IMAP server to local mail directory
printf "\n\n Getting sync with remote imap server and pull down new mails...\n\n"

mbsync gmail

# Start indexing the fetched mails
printf "\n\n Hang on! It is building the database with new mails.......\n\n"
notmuch new
#> /dev/null 2>&1


# Various tags for refinement
printf "\n\n Wait....building tags based inboxes for easy searchability.....\n"

notmuch tag -inbox from:"googlealerts-noreply@google.com" and subject:"Google Alert"
notmuch tag -inbox from:"lwn@lwn.net" and subject:"LWN.net"
notmuch tag +linux-kernel -inbox -kernel-patch -xfs -ext4 to:"linux-kernel@vger.kernel.org" or cc:"linux-kernel@vger.kernel.org" and not subject:"[PATCH]"
notmuch tag +EmacsDiffBugs -inbox -attachment from:"emacs-diffs-request@gnu.org" or from:"bug-gnu-emacs-request@gnu.org"
notmuch tag +emacs_orgmode -inbox to:"emacs-orgmode@gnu.org"
notmuch tag +geek-stuff -inbox subject:"The Geek Stuff"
notmuch tag +git-pull -inbox subject:"[GIT PULL]"
notmuch tag +kernel-patch -inbox -linux-kernel subject:"[PATCH]" or from:"bhaskarlinux73@gmail.com"
notmuch tag +mutt -inbox -attachment  to:"mutt-users@mutt.org"
notmuch tag +git -inbox -attachment to:"git-users@googlegroups.com"
notmuch tag +vim -inbox -attachment to:"vim_use@googlegroups.com"
notmuch tag +i3 -inbox -attachment  to:"i3-discuss@freelists.org"
notmuch tag +mutt-dev -inbox -attachment to:"mutt-dev@mutt.org"
notmuch tag +Junio -inbox -attachment -kernel-patch from:"gitster@pobox.com"
notmuch tag +git-dev -inbox to:"git@vger.kernel.org"
notmuch tag +ext4 -inbox -kernel-patch to:"linux-ext4@vger.kernel.org" and cc:"linux-fsdevel@vger.kernel.org"
notmuch tag +xfs -inbox -kernel-patch  to:"djwong@kernel.org" and cc:"linux-xfs@vger.kernel.org" and cc:"linux-fsdevel@vger.kernel.org"
notmuch tag +kernel-build -inbox -kernel-patch to"linux-kbuild@vger.kernel.org"
notmuch tag +llvm-clang -inbox -kernel-patch to:"llvm@lists.linux.dev"
notmuch tag +linux-firmware to:"linux-firmware@kernel.org"
notmuch tag +gentoo -inbox -attachment to:"gentoo-user@lists.gentoo.org"
notmuch tag +wireguard -inbox -attachment to"wireguard@lists.zx2c4.com"
notmuch tag +nyxt -inbox -attachment  subject:"[atlas-engineer/nyxt]"
notmuch tag +gcc -inbox -attachment  to:"gcc-patches@gcc.gnu.org"
notmuch tag +gcc -inbox -attachment  to:"gcc@gcc.gnu.org"
notmuch tag +gcc -inbox -attachment to:"gcc-help@gcc.gnu.org"
notmuch tag +UNIX_Tips -inbox subject:"Unix Tip:"
notmuch tag +Al_Viro -inbox -attachment  to:"viro@zeniv.linux.org.uk"
notmuch tag +Linus  -inbox -attachment   to:"torvalds@linux-foundation.org"
notmuch tag +Andrew  -inbox -attachment  from:"akpm@linux-foundation.org" and to:"akpm@linux-foundation.org"
notmuch tag +Paul_E_McKenney -inbox -attachment  from:"paulmck@kernel.org"
notmuch tag +Greg -inbox -attachment   from:"gregkh@linuxfoundation.org" and to:"gregkh@linuxfoundation.org"
notmuch tag +David_Millar -inbox -attachment   to:"davem@davemloft.net"
notmuch tag +Jonathan_Corbet -inbox -attachment to:"corbet@lwn.net"
notmuch tag +Thomas_Gleixner -inbox -attachment to:"tglx@linutronix.de"
notmuch tag +syzbot -inbox subject:"[syzbot]"
notmuch tag +gentoo -inbox subject:"[gentoo-kernel]" and from:"mpagano@gentoo.org"
notmuch tag +github-linux-repo -inbox subject:"[unixbhaskar/linux]"
notmuch tag +Ted_Tso -inbox -attachment to:"tytso@mit.edu"


# Print the message only if everything above went well.

if [[ $? != 0 ]];then
        echo not done.
else
notify-send --expire-time=50000 --urgency=normal "Mailboxes have synced at $(date +'%R')"
fi

It is run by the cron after certain intervals automatically and that too in a dedicated workspace allotted to it. I only run it by hand, when I want to debug something in it. I have put a mechanism in it to inform me when it is done., if I am working in a different workspace, I can see the popup.

This script gives what I want from my mail folders, as you can, I am using NotMuch for tagging various people from my mail.

Here is the mail-syncing script in its entirety :

IMAPAccount      gmail
Host             imap.gmail.com
User             your_mail
Pass             PassCmd "gpg2 --quiet --for-your-eyes-only --no-tty --decrypt ~/.password-store/mbsynccred.gpg"
#UseIMAPS         yes
CertificateFile  ~/gmail-backup/gmail.crt
CertificateFile  ~/gmail-backup/google.crt
#CertificateFile  /usr/share/ca-certificates/mozilla/Equifax_Secure_CA.crt
SSLType IMAPS
SSLVersion TLSv1.2

IMAPStore  gmail-remote
Account    gmail

MaildirStore  gmail-backup
Path          ~/gmail-backup/
Inbox         ~/gmail-backup/Inbox

Channel   gmail
Master    :gmail-remote:
Slave     :gmail-backup:
Create    Slave
Expunge   Slave
SyncState *
Sync All
#Sync      Pull
# Exclude everything under the internal [Gmail] folder, except archived mails
Patterns  * ![Gmail]*  INBOX ARCHIVED

What it does, is it syncs my Gmail account with a local store directory in my system and keeps all the emails. The above script should be residing in the home directory and named .mbsync.

Pretty simple, right? It should be. My requirements are pretty ordinary, so the process should adhere to that.

Reference Tools:

NotMuch : https://notmuchmail.org/

MbSync : https://isync.sourceforge.io/mbsync.html

Why Linux Native Tools Are Better

Well, this reflects my understanding or experience with tools that come bundled with Linux. First and foremost, I must confess that I haven’t used or sat on other OS long period of time. So, I might be lacking their offering and ploy.

Tools that come bundled with Linux are good enough for most of the people using this operating system, if not all. Okay, that statement drives out the notion, that people like me, who does normal work with their operating system. But is that actually true? No, it is not true. People do weird things all the time with their understanding and ability. So, it boils down to, two kinds of people, one who is happy and enjoys what comes bundled with their OS, and others who are more adventurous and ready to try out their luck with external stuff.

I, probably fall in the former group of people, who are happy and use the stuff that comes bundled with it. The reasoning could be in this line, that most of the tools are time tested and do at least ninety percent of the work people do, sitting on Linux.

Few things play a huge role to get along with this thinking. Those are :

1. It is all about the specific tools that are there by default.
2. You don’t have to maintain those separately, because some smart folks take care of it.
3. Those tools(read native tools) are well integrated with the rest of the OS.
4. No surprises, if the operating system provider does not make a deliberate attempt to do something else with those tools.
5. One doesn’t have to go out in the wild i.e. on the internet, to find help for use, man pages are good enough to get you going.PS: But having said that, man pages are not always helpful, especially those lacking explicit examples.

These all ethos too, if I say so, have to do with reliability and that has been proven over the decades by various different levels of use across different sectors. Not everyone uses those tools and comes from the same background to have a clear understanding or know-how to find those. Moreover, it saves a tremendous amount of time, when people know things are in the default location to use.

It is boring or least surprising in a good way, reliability stems out of it. Doesn’t it?

Okay, I understand the excitement of doing exciting things with so-called new tools. But are they really?? I have found out that most of the new mechanisms are somehow built on age-old stuff.

Now, if a so-called new tool depends on several other tools to get along, that will be a serious pain point. A good tool should be self-sufficient. Believe me or not, most of the time tested tools are independent and have no such reliability as other tools.

It has twofold advantages, one, you can use it individually, and it will provide you only the thing you want, or the tool is capable of. Second, you can easily integrate with other tools to achieve a more complex task. In some way, my brain is wired that way, probably doing in that fashion over and over again.

Make no mistake, that process is no hunky-dory and had some take to get used to. Lack of internet access taught me so many things which I couldn’t have if I had the exposure to have it. Why? Because it took time to understand and use those tools in a proper way. For me, it took way more time due to my lack of bend of mind(strictly no pun intended). So, I can not preach like an “expert” with ease and smoothness.

Moreover, the speed benefit can not be ignored. A native tool does things way faster than something foreign brings into the system and requires more tools to perform something simple. Linux toolchains are way more efficient when used locally available stuff shipped with the operating system they are supposed to use.

The overhead of maintaining such tools is absolutely minimal. Think about it, you don’t have to care about maintaining tools like make, GCC, cat, sed, and awk to name a few. These are all there by default. You don’t have to lose sleep over where to get those and how to maintain those. You probably haven’t realized, that some people are toiling to provide the comfort of doing that on our behalf. But that is positive, that it frees you to think and tap the power of the already well-maintained tool to do the actual problem-solving.

The intention behind all these previous vignettes is, to get yourself comfortable or accustomed to the native stuff Linux offers out of the box. The more you are getting familiar with it the better you reap benefits for yourself.