Using macOS keychains from Emacs

Over the past few days, I have been exploring different methods to store my personal information management system outside of my various machines. The motivation for this is to consolidate my notes into a single place that I can reference without worrying about a single drive failure wiping away a large body of information. As an initial attempt at consolidation, I moved the Org agenda files that make up the system to a SMB mount on a local NAS, then tried connecting to it from Emacs using TRAMP with a password stored in the default macOS keychain. In doing so, I ran into two issues. The first is that Emacs integration with the macOS keychain is somewhat incomplete, and the second is that Org is not able to search agenda files over TRAMP if they have not been opened. The Org-specific issues made TRAMP infeasible for my specific use case, but the process of setting up Emacs integration with the macOS keychain was interesting (and challenging) enough to warrant documenting the steps required to get it all working.

Future releases of Emacs should have much of the information in this post included in the auth-source manual, which has been updated within the last couple of weeks as of the writing of this post. However, given that TRAMP password management is not particularly intuitive, I believe the following primer and code snippets are still of use for those who need them.

To manage credentials, TRAMP uses the auth-source library, which organizes credentials into backends such as plaintext files, the Freedesktop.org Secret Service API, and macOS keychains. When opening a file using a protocol that requires a password, TRAMP calls the auth-source-search function to search these backends for a password based on the desired method, username, and server host. A file name of /smb:user@nas.local:foo/bar, for example, will result in the following function call:

(auth-source-search :max 1
                    :user "user"
                    :host "nas.local"
                    :port "smb"
                    :require '(:secret :user)
                    :create t)

auth-source defines two macOS-specific backend types: macos-keychain-generic and macos-keychain-internet. Both of these interact with the security command-line tool using find-generic-password and find-internet-password respectively. The main difference between them is that macos-keychain-generic maps hostnames to the -c/creator argument in find-generic-password, while macos-keychain-internet uses -s/server. Because a generic password creator must be exactly 4 characters long, it is highly likely that only macos-keychain-internet will work for TRAMP.

To configure backends for auth-source, set the auth-sources user option to an appropriate list of backends. The syntax for sources is not well-defined, so as an initial pass you may want to consider using custom-set-variable and the Customize interface to get valid backend definitions. In my case, I wanted to use the Secret Service API when using Linux and the macOS default keychain when using macOS. The following snippet accomplishes this, setting auth-sources based on system-type, a built-in variable describing type of operating system hosting Emacs:

(setq system-auth-sources
      '(gnu/linux
        ((:source (:secrets "Login")))
        darwin
        ((:source (:macos-keychain-internet "default")))))
(setopt auth-sources
  (or (plist-get system-auth-sources system-type) 
      '("~/.authinfo")))

While the Secret Service API backend supports credential creation, the macOS keychain backends do not, and thus auth-source-search raises an error when using macOS keychains to create credentials. (TRAMP always defaults to creating credentials and silently swallows errors while doing so, which can be a source of frustration if you are trying to understand why saving a password with a macOS keychain does not seem to work.) To get around this, store credentials using security directly, passing the user with -a/account, the server with -s/server, and the TRAMP method with -r/protocol. Because protocols must be exactly 4 characters, you must pad any remaining characters with spaces if your TRAMP method is too short. For example, assuming a file name of /smb:user@nas.local:foo/bar, use the following security invocation, with -w to prompt for the password:

$ security add-internet-password \
           -a user \
           -s nas.local \
           -r 'smb ' \
           -w

After doing this, verify in Emacs that the password is visible to the auth-source backend:

(let ((auth-source-do-cache nil))
  (auth-source-search :user "user"
                      :host "nas.local"
                      :port "smb"
                      :require '(:secret :user)))

If the above expression yields a non-nil result, auth-source successfully found the credential, and opening /smb:user@nas.local:foo/bar with TRAMP should work as expected.