Super spotlight search with counsel

Inspired by abo-abo‘s post on using his excellent tools counsel, ivy and swiper to search the contents of files indexed by the recoll search tool, I tried to make something similar for spotlight on the Mac. My attempt is below.

Using counsel gives us incremental updates of the spotlight search results (accessed using its command line interface mdfind). When a match is selected, it is opened in emacs, and (unless it is a pdf) a swiper search is launched on the search string.

This works really nicely. The only problem I’ve had is that I wanted to sort the results to prioritise .org and .tex files, but my sort function is not being used correctly in ivy, but I can’t tell why. It gets passed the counsel prompt for more characters, but not the set of mdfind matches for sorting. My lisp skills have been exhausted at this point, but maybe someone else can see what I’ve done wrong! UPDATE: this problem was fixed with an update to ivy and counsel, and the sorting command below now works.

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; counsel-spotlight                                                      ;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; Incrementally search the Mac spotlight database and open matching
;; files with a swiper search on the query text.
;; Based on http://oremacs.com/2015/07/27/counsel-recoll/

(require 'counsel)

;; Function to be called by counsel-spotlight
;; The onlyin option limits results to my home directory
;; and directories below that
;; mdfind is the command-line interface to spotlight
(defun counsel-mdfind-function (string &rest _unused)
  "Issue mdfind for STRING."
  (if (< (length string) 4)
      (counsel-more-chars 4)
    (counsel--async-command
     (format "mdfind -onlyin ~/ '%s'" string))
    nil))

;; Main function
(defun counsel-spotlight (&optional initial-input)
  "Search for a string in the mdfind database.
You'll be given a list of files that match.
Selecting a file will launch `swiper' for that file.
INITIAL-INPUT can be given as the initial minibuffer input."
  (interactive)
  (ivy-read "spotlight: " 'counsel-mdfind-function
            :initial-input initial-input
            :dynamic-collection t
            :sort t
            :action (lambda (x)
                      (when (string-match "\\(\/.*\\)\\'" x)
                        (let ((file-name (match-string 1 x)))
                          (find-file file-name)
                          (unless (string-match "pdf$" x)
                            (swiper ivy-text)))))))

;; Define my sort function
(defun bjm-counsel-mdfind-sort-function (x y)
  "Compare two files X and Y. Prioritise org then tex."
  (if (string-match "org$" x)
      t
    (if (string-match "tex$" x)
        (if (string-match "org$" y)
            nil
          t)
      nil)))

;; Add to list of ivy sorting functions
(add-to-list 'ivy-sort-functions-alist
             '(counsel-mdfind-function . bjm-counsel-mdfind-sort-function))

In an ideal world, I’d like to be able to interactively narrow the matches from mdfind with a second counsel on the filenames. The selected file would then open with a swiper search for the first search term. For example, I would like to

1. Call M-x counsel-spotlight and enter caustic (or enough characters to give me useful results) to get a list of names of files which contain the text caustic somewhere inside them.
2. Hit some keybinding and get a new counsel prompt and enter chandra to incrementally filter the list of filenames to those containing the string chandra in the filename.
3. Select the file I want and hit RET to have emacs open that with a swiper search of my original query caustic.

Given the state of my lisp skills, this might take me a while, but it is nice to dream!

Update: A cheat to sort/filter results

Given that my perl skills are much better than my lisp skills, I cheated and wrote a perl wrapper for mdfind that filters and sorts the results for me. The code is below – feel free to use and modify and share. To use this, save the code to a file called bjm-mdfind in your $PATH, make sure it has executable permissions set, and modify the lisp code above to use it:

;; Function to be called by counsel-spotlight
;;
;; mdfind is the command-line interface to spotlight
;;
;; The onlyin option limits results to my home directory
;; and directories below that
;;
;; N.B. below this is replaced with a custom perl wrapper to sort
;; and filter the mdfind results
(defun counsel-mdfind-function (string &rest _unused)
  "Issue mdfind for STRING."
  (if (< (length string) 4)
      (counsel-more-chars 4)
    (counsel--async-command
     (format "bjm-mdfind '%s'" string))
     ;;(format "mdfind -onlyin ~/ '%s'" string))
    nil))

Here is the perl code:

#!/usr/bin/perl -w

###############################################################################
# bjm-mdfind
#
# wrapper for mdfind to filter and sort results
# written by Ben Maughan http://pragmaticemacs.com/
#
# $Id: bjm-mdfind,v 1.2 2015/08/06 11:21:37 bjm Exp $
###############################################################################

use strict;
use File::Basename;
use Getopt::Long;

my $scriptname = basename($0); # Strip away the leading directory names
my $runtime = localtime;

##########################################################################
# customise these options                                                #
##########################################################################
# limit search to this dir (recursively)
my $dir="~/";
# preferred order of file extensions
# others will appear later
my @order=(".org",".tex",".el",".txt",".dat",".pdf");
# exclude files matching these strings
my @exclude=("Library/Caches","Application Support");

###############################################################################
# Handle command line arguments

my $help;
my $man;
my $v=0; #set default verbosity
my $version = defined ((split / /, q/$Revision: 1.2 $/)[1]) ? (split / /, q/$Revision: 1.2 $/)[1] : 0;
my @args=@ARGV;
my $nargs=1; #number of required command line args

&Getopt::Long::Configure( 'bundling' );
GetOptions(
           'help|h' => \$help,
           'man|m'  => \$man,
           'verbose|v=i' => \$v,
) or die "ERROR: Invalid command line option $!";

if ($help||$man||$#ARGV<$nargs-1) { #print help
  # Load Pod::Usage only if needed.
  require "Pod/Usage.pm";
  import Pod::Usage;
  pod2usage(VERBOSE => 1) if $help;
  pod2usage(VERBOSE => 2) if $man;
  pod2usage(VERBOSE => 0, -message => "ERROR: not enough arguments use --help or --man for more help") if $#ARGV<0;
  pod2usage(VERBOSE => 0, -message => "ERROR: not enough arguments - your input was:\n    $scriptname @args") if $#ARGV<$nargs-1;
}

#check input
my $string=$ARGV[0];

###############################################################################
# Main part of program

if ($v > 0) {
  print <<EOF

------------------------$scriptname version $version------------------

Invocation was:
$scriptname @args

Runtime $runtime

EOF
}

chomp(my @out=`mdfind -onlyin $dir $string`);

## sort and filter
##print "$order[2]\n\n";

##filter
my $exc=join "|", @exclude;
@out = grep !/$exc/, @out;

##sort
my @out2;
foreach my $ext (@order) {
    my @tmp = grep /$ext$/, @out;
    print "###$ext\n###@tmp\n\n" if $v>1;
    push @out2, @tmp;
}
##everything else
my $inv=join "\$|", @order;
print "###$inv\n" if $v>1;
my @rest = grep !/$inv$/, @out;
print "###@rest\n" if $v>1;
push @out2, @rest;

##join
my $out=join "\n", @out2;

##print
print "$out\n";

#report successful completion
print "$scriptname completed successfully\n\n" if $v>0;

###############################################################################
# POD documentation

=head1 NAME

bjm-mdfind

=head1 SYNOPSIS

B<bjm-mdfind> [options] string

=head1 DESCRIPTION

wrapper for mdfind to filter and sort results

  string        - search query

=head1 OPTIONS

=over 4

=item B<-h, --help>

Prints out a brief help message.

=item B<-m, --man>

Prints out detailed help.

=item B<-v, --verbosity>

Control the amount of output B<(Default = 1)>

=back

=head1 VERSION

This is $RCSfile: bjm-mdfind,v $ $Revision: 1.2 $

=head1 AUTHOR

Ben Maughan <benmaughan@gmail.com>

=cut

###############################################################################
# $Log: bjm-mdfind,v $
# Revision 1.2  2015/08/06 11:21:37  bjm
# Summary: added URL
#
# Revision 1.1  2015/08/06 11:17:37  bjm
# Initial revision
#