An unobtrusive email monitor for mu4e on the Mac

This is not exactly an Emacs post, but some users of the mighty mu4e might find it useful. I don’t like big alerts to new emails that distract me from what I am doing, but I also don’t want to keep switching to my mu4e buffer to see if new emails have arrived. My solution is to have a small notification item in my Mac status bar to tell me how many emails I have.

It looks like this (the bit underlined in red):


The text @ 1/3/0/0 tells me that I have, respectively,

  • 1 unread email in my inbox.
  • 3 total emails in my inbox (i.e. 3 too many!).
  • 0 emails in my drafts folder. This is useful to keep an eye on in case I save a draft and then forget about it.
  • 0 emails in my outbox. This is my postfix email outbox and is useful to keep an eye on in case emails sent when I’m offline don’t get automatically sent when I reconnect.

To set this up I use shellwrangler which is a small app that embeds the output of a script in the status bar. Update 2017-11-10 shellwrangler appears dead and bitbar looks like a superior alternative. This then calls the following perl script which generates the text. Shellwrangler Bitbar updates at a specified interval so that is all there is to it.

#!/usr/bin/perl -w

# Count number of emails in inbox, drafts and outbox for mail indexed
# with mu and sent with sendmail or equivalent
# by Ben Maughan
# Feel free to distribute, modify, whatever

## total mails in inbox
chomp(my @tot=`/usr/local/bin/mu find maildir:/work/INBOX 2>/dev/null`);
## unread mails in inbox
chomp(my @unread=`/usr/local/bin/mu find maildir:/work/INBOX AND flag:unread 2>/dev/null`);
## drafts
chomp(my @drafts=`/usr/local/bin/mu find maildir:/work/[Gmail].Drafts 2>/dev/null`);

## number in outbox
chomp(my @mailq=`/usr/bin/mailq`);
my @outbox = grep /^[0-9A-F]{12}/, @mailq;

printf "@ %d/%d/%d/%d",$#unread+1,$#tot+1,$#drafts+1,$#outbox+1;


Using email address lists in mu4e

To set up lists of multiple email multiple recipients with mu4e, you can make use of mail aliases. To do this, add lists to a file with the following format, with the word “alias” followed by the name you want for the list, followed by a space-separated list of email addresses:

alias jsmiths "John Q. Smith <>" "Jane Q. Smith <>"

You can either save this file as ~/.mailrc or put it anywhere and tell Emacs where to find it using e.g.

;;mail aliases for lists (address lists)
(setq mail-personal-alias-file (expand-file-name "/homeb/bjm/docs/mail-aliases"))

Now, when composing an email you can type the alias (“jsmiths” in our example) and hit space, and it will expand to the list of email addresses.

If you are using my improved address completion, then you can add those aliases to your bjm/contact-file to have them appear in your completion lists.

Customise the reply quote string in mu4e

You know the bit of text that appears before the quoted email when you reply to someone? Of course you can customise this in mu4e. By default a quoted reply will be preceded with something like

"On Mon, 30 Jan 2017, Joe Bloggs wrote:"

I wanted to include the time and email address of the sender, so I customised the variable message-citation-line-format as follows

;; customize the reply-quote-string
(setq message-citation-line-format "On %a %d %b %Y at %R, %f wrote:\n")
;; choose to use the formatted string
(setq message-citation-line-function 'message-insert-formatted-citation-line)

which translates to something like this

On Mon 30 Jan 2017 at 19:17, Joe Bloggs <> wrote:

The formatting options can be found by looking up the help for the variable with C-h v message-citation-line-format.

Sadly, this change broke my function to extract the sender name for my email template. Here is an updated version:

;; function to return first name of email recipients
;; used by yasnippet
;; inspired by
(defun bjm/mu4e-get-names-for-yasnippet ()
  "Return comma separated string of names for an email"
  (let ((email-name "") str email-string email-list email-name2 tmpname)
      (goto-char (point-min))
      ;; first line in email could be some hidden line containing NO to field
      (setq str (buffer-substring-no-properties (point-min) (point-max))))
    ;; take name from TO field - match series of names
    (when (string-match "^To: \"?\\(.+\\)" str)
      (setq email-string (match-string 1 str)))
    ;;split to list by comma
    (setq email-list (split-string email-string " *, *"))
    ;;loop over emails
    (dolist (tmpstr email-list)
      ;;get first word of email string
      (setq tmpname (car (split-string tmpstr " ")))
      ;;remove whitespace or ""
      (setq tmpname (replace-regexp-in-string "[ \"]" "" tmpname))
      ;;join to string
      (setq email-name
            (concat email-name ", " tmpname)))
    ;;remove initial comma
    (setq email-name (replace-regexp-in-string "^, " "" email-name))

    ;;see if we want to use the name in the FROM field
    ;;get name in FROM field if available, but only if there is only
    ;;one name in TO field
    (if (< (length email-list) 2)
        (when (string-match "^On.+, \\([^ ,\n]+\\).+wrote:$" str)
            (setq email-name2 (match-string 1 str))
            ;;prefer name in FROM field if TO field has "@"
            (when (string-match "@" email-name)
              (setq email-name email-name2))

Use org-mode tables and structures in emails and elsewhere

I love the way that org-mode allows you to add simple clean structures to your text, with lists and tables. You can get some of that functionality in other modes by using orgstruct-mode and orgtbl-mode, which are part of org-mode.

Enable these minor modes in any major mode for one-off use with M-x orgstruct++-mode or M-x orgtbl-mode and you can use the normal org-mode commands to create lists and tables. I find this especially useful in emails, so I use this code in my emacs config file to automatically enable these for message-mode

;; use org structures and tables in message mode
(add-hook 'message-mode-hook 'turn-on-orgtbl)
(add-hook 'message-mode-hook 'turn-on-orgstruct++)

mu4e-delay is dead, long live mu4e-send-delay

A while ago I wrote mu4e-delay, a package (based heavily on gnus-delay), to add a customisable delay to outgoing mail so that a sent email could be “undone” before the delay period had passed. This was spectacularly useful for me, but now Benny Andresen has put together the superior mu4e-send-delay, which improves on my package in several ways.

The key features of mu4e-send-delay are (from the github page):

  • mu4e context support
  • Saves scheduled mails to mu4e-drafts-folder
  • Uses an emacs timer to check Drafts if a mail is scheduled to be sent now
  • Allows easy edit of the X-Delay header in mu4e-compose-mode
  • Displays scheduled time in mu4e-view
  • Doesn’t send if mail is currently being edited
  • Works with attachments

The last four points are all improvements over the original mu4e-delay, with the last two being the most important. I’d encourage any mu4e-delay users to switch over to mu4e-send-delay. I have, and I’ve not looked back!

Even better email contact completion in mu4e

I have written previously about my tweaks to improve contact completion when composing emails with mu4e. Thanks to some help from abo-abo, the author of the fantastic ivy completion library, with the code below you can hit a comma to complete the current choice of email address and start searching for the next one. This matches the behaviour of many other email clients like Gmail or Thunderbird.

This won’t change anybody’s world, but gives you a nice little thrill of efficiency when entering several recipients to an email!

Here is the updated code (see my previous post for more details):

;;need this for hash access
(require 'subr-x)

;;my favourite contacts - these will be put at front of list
(setq bjm/contact-file "/homeb/bjm/docs/fave-contacts.txt")

(defun bjm/read-contact-list ()
  "Return a list of email addresses"
    (insert-file-contents bjm/contact-file)
    (split-string (buffer-string) "\n" t)))

;; code from
(defun bjm/counsel-email-action (contact)
    (insert contact)))

;; bind comma to launch new search
(defvar bjm/counsel-email-map
  (let ((map (make-sparse-keymap)))
    (define-key map "," 'bjm/counsel-email-more)

(defun bjm/counsel-email-more ()
  "Insert email address and prompt for another."
    (insert ", "))
  (setq ivy-text ""))

;; ivy contacts
;; based on
(defun bjm/ivy-select-and-insert-contact (&optional start)
  ;; make sure mu4e contacts list is updated - I was having
  ;; intermittent problems that this was empty but couldn't see why
  (let ((eoh ;; end-of-headers
           (goto-char (point-min))
           (search-forward-regexp mail-header-separator nil t)))
        ;; append full sorted contacts list to favourites and delete duplicates
         (delq nil (delete-dups (append (bjm/read-contact-list) (mu4e~sort-contacts-for-completion (hash-table-keys mu4e~contacts)))))))

    ;; only run if we are in the headers section
    (when (and eoh (> eoh (point)) (mail-abbrev-in-expansion-header-p))
      (let* ((end (point))
            (or start
                  (re-search-backward "\\(\\`\\|[\n:,]\\)[ \t]*")
                  (goto-char (match-end 0))
           (initial-input (buffer-substring-no-properties start end)))

      (delete-region start end)

      (ivy-read "Contact: "
                :re-builder #'ivy--regex
                :sort nil
                :initial-input initial-input
                :action 'bjm/counsel-email-action
                :keymap bjm/counsel-email-map)

Undo-send and schedule email in mu4e

Note that this package has been superseded by the superior mu4e-send-delay.

One of the things I missed when I switched from thunderbird to mu4e was the ability to undo sent emails. I’m sure I’m not the only one that has spotted a glaring error, or a CC’d person that shouldn’t be CC’d the moment after I hit send! Of course you can’t truly undo a sent email, but if you buffer emails for a short period of time before actually sending them then you can retrieve them and edit or cancel them before sending again. This is what the send later extension in thunderbird does, and also the “undo send” function in gmail.

I looked into whether something like this existed for mu4e, I found gnus-delay, but it turned out not to work in mu4e. However it did not take too much effort to modify it to work, and hence mu4e-delay was born. Credit to the authors of gnus-delay who wrote most of the code this is based on.

Once you initialise the package, you can use C-c C-l (mnemonic l for send later) in place of C-c C-c to send mail. This moves the mail to the drafts folder and adds a header keyword specifying the time at which a mail should be sent, defaulting to 2 minutes in the future. A timer runs every minute (default) checking all mails in the drafts folder and sending those that are due. If you wish to “undo send” just find it in the drafts folder and then edit it to remove the delay header and save it. This will prevent it from being sent until you re-send (or delay) it.

The package also provides the function mu4e-delay-add-delay-header which can be called interactively to add any date and time to the delay header keyword, scheduling an email for an arbitrary point in the future.

The mail is sent with smtpmail or sendmail/postfix depending on which you have configured with send-mail-function. Since smtpmail causes emacs to freeze while mail is sent, you are strongly encouraged to use postfix (or similar) to handle your email sending.

A known limitation of mu4e-delay is that attachments are not sent properly when the mail is delayed. I’ve not worked out why this is yet (suggestions welcome!). The package checks to see if you have an attachment and warns you about this. With attachments you currently need you use C-c C-c to send as normal.

I’ll finish with a word of caution. Emails are important and I’d hate you to get into trouble because something went wrong an email did not get sent. I strongly recommend (at least at first) setting the variables mu4e-delay-bcc-address to bcc all outgoing mail to a spare email address so you can check things are being sent correctly. You can also set mu4e-delay-backup-directory to backup all delayed mail to a given directory rather than deleting it. I have used this package for several months without problems, but be careful!

To install the package, download it from github and put the mu4e-delay.el file somewhere on your system and then add the following to your init.el file

(add-to-list 'load-path "/path/to/directory")
(require 'mu4e-delay)

I’m keeping this package off MELPA for now until I get feedback that other people are using it without problems, so please let me know in the comments or on github whether it is working for you (or not).

My other articles on mu4e are collected here.

Using postfix instead of smtpmail to send email in mu4e

The standard setup for mu4e uses smtpmail to actually send the messages. This works fine for the most part, but by default it does not run asynchronously, so emacs pauses when a mail is sent. Usually this is very brief, but for messages with large attachments it can be very noticeable. This was particularly annoying to me when I started developing a package to add a two minute delay to all outgoing mail to give an “undo send” functionality (I’ll post more about this package soon). When the two minute timer finished and the mail was sent, I would be working on something else, and emacs would hang briefly, which was a bit annoying.

It is possible to configure mu4e to send the mail asynchronously, but apparently this can be unreliable. A better solution is to run your own mail server to handle outgoing mail. This is supported in mu4e, and mu4e hands off the mail to the server to send, so it happens instantly from the user’s point of view, regardless of the size of the mail. Another benefit is that mail sent while you are offline is handled automatically; the server queues the mail and then sends it when you reconnect.

I am using OS X Yosemite, and setting up a mail server was quite easy. The postfix mail server is installed by default on OS X, so it was just a matter of configuring it to work with my gmail account. I followed these helpful instructions, noting the advice in the comments that for Yosemite, the extra line

smtp_sasl_mechanism_filter = login

is needed.

Now we tell mu4e to use postfix to handle sending mail by adding the following to our emacs config file:

;;send mail using postfix
(setq send-mail-function 'sendmail-send-it)
(setq message-send-mail-function 'message-send-mail-with-sendmail)

(Note the references to “sendmail” are because this was the name of one of the original mail servers on unix/linux. Postfix is a more modern mail server that is compatible with sendmail.)

This should be all you need to switch to using postfix to send your mail. Some useful commands (to run at the command line, not using M-x in emacs) are

sudo postfix start
sudo postfix stop

to start and stop the server, and


to list the current queue of unsent mail. This will generally be empty, but if you have been offline you might see some mails in there. They should send automatically once the server notices you are back online, but if not, use

mailq -q

to send all mail in the queue. See the man pages for more functionality.

So there you are. This might not be for everyone, but if you don’t mind fiddling with some config files, I highly recommend moving over to postfix for sending your mail.

Migrating from offlineimap to mbsync for mu4e

I have just switched over from offlineimap to mbsync (part of isync) for synchronising my local email with Gmail for use with mu4e. I found offlineimap worked well enough but it would hang frequently if I had connection problems (e.g. on my laptop) and mbsync is much faster. I couldn’t find a guide for migrating, and I encountered a few issues along the way, so I thought I’d document the process here.

The isync mailing list has some discussion of how to convert the maildir folders and related files from offlineimap to mbsync format, but I didn’t see any examples of this being done successfully so I decided to play it safe and sync a new maildir directory with mbsync, downloading a new copy of all of my emails from Gmail’s servers.

I’m using a Mac, and built the newest version of isync from source without problems (the version on Mac Ports is older). I based my mbsyncrc file on the example on this page, and set up the structure of the maildir to match the one created by offlineimap (not really needed but makes things simpler). Here is my mbsyncrc

# mbsyncrc based on
IMAPAccount gmail
# Address to connect to
Pass ************
AuthMechs LOGIN
SSLVersions SSLv3
CertificateFile /opt/local/share/curl/curl-ca-bundle.crt


IMAPStore gmail-remote
Account gmail

# LOCAL STORAGE (CREATE DIRECTORIES with mkdir -p Maildir/gmail)
MaildirStore gmail-local
Path ~/mbsync/
Inbox ~/mbsync/INBOX


Channel gmail-inbox
Master :gmail-remote:
Slave :gmail-local:
Patterns "INBOX"
Create Both
Expunge Both
SyncState *

Channel gmail-trash
Master :gmail-remote:"[Gmail]/Bin"
Slave :gmail-local:"[Gmail].Bin"
Create Both
Expunge Both
SyncState *

Channel gmail-sent
Master :gmail-remote:"[Gmail]/Sent Mail"
Slave :gmail-local:"[Gmail].Sent Mail"
Create Both
Expunge Both
SyncState *

Channel gmail-all
Master :gmail-remote:"[Gmail]/All Mail"
Slave :gmail-local:"[Gmail].All Mail"
Create Both
Expunge Both
SyncState *

Channel gmail-starred
Master :gmail-remote:"[Gmail]/Starred"
Slave :gmail-local:"[Gmail].Starred"
Create Both
Expunge Both
SyncState *

# "gmail-inbox", "gmail-sent", and "gmail-trash"
Group gmail
Channel gmail-inbox
Channel gmail-sent
Channel gmail-trash
Channel gmail-all
Channel gmail-starred

Now I synced with Gmail using

mbsync -V gmail

which worked fine for a while but then kept returning errors from Gmail about throttling and my quota being exceeded. My email is about 6GB in size, so large but not ridiculous, but it seems google didn’t like me trying to download it all in one go (possibly a limitation of my employer’s corporate google service). Anyway, I checked with the isync mailing list and there should be no problems with syncs being interrupted and resumed so it would be possible to keep repeating the sync until it completed. Instead of that I used Apple’s network link conditioner (part of Hardware IO Tools for Xcode) to limit my bandwidth to 750 kbps and left the sync to run overnight, which completed without problems.

Once this was done, I had my offlineimap copy of my maildirs in ~/offlineimap and my mbsync maildirs in ~/mbsync. To make it easy to switch between them if needed, I made a symbolic link from ~/Maildir to the mbsync maildir.

Next I deleted my mu index and re-indexed

rm -rf ~/.mu
mu index

I then set some mu4e variables to point to my new maildir

;;location of my maildir
(setq mu4e-maildir (expand-file-name "~/Maildir"))

;;command used to get mail
;; use this for testing
(setq mu4e-get-mail-command "true")
;; use this to sync with mbsync
;;(setq mu4e-get-mail-command "mbsync gmail")

;;rename files when moving
(setq mu4e-change-filenames-when-moving t)

;;set up queue for offline email
;;use mu mkdir  ~/Maildir/queue to set up first
(setq smtpmail-queue-mail nil  ;; start in normal mode
      smtpmail-queue-dir   "~/Maildir/queue/cur")

Now, after restarting mu4e, I was using the mbsync maildir.

Note that I initially used a dummy command to sync the mail in mu4e so that I could run the sync manually on the command line at first. I really only needed this because I kept hitting problems with duplicate UIDs, which are solved by getting mu4e to rename files when moving them, as in the code above.

The switch over has been very smooth. In fact I was surprised to find that the message IDs were preserved so that the org-links I stored to emails when I was using offlineimap still take me to the correct email!