My Tech Notes

It always feels kind of bad when after upgrading beta-version to a next stable release something stops working.

For nearly 6 years after release of Emacs 21.1 I used to compile my own "beta" version of upcoming Emacs 22; this is primarily because new Emacs finally promised to implement proper support of Unicode. For about 4 or 5 years these releases always were sufficiently stable to use in everyday work; one of the most exciting features of Emacs 22 (unrelated to Unicode) was TRAMP, which came to replace outdated ange-ftp to provide transport-independent support for remote file editing.

I must say that in my situation when I almost always work with 2-3 (or more) computers with different OS's available through different means (ftp, ssh, telnet) this feature is extremely useful for me; no other utility I am aware of provides such extensible and configurable transparent access to remote file systems....

So you can understand my disappointment when after installing now-"stable" version Emacs 22.1 (released in June 2007), I found TRAMP to be seriously broken, in more than one way.

First problem I have encountered was lack of file completion; normally, pressing TAB button (once or twice) while entering file name shows possible completions or completes file name right away (if completion is unambiguous). This stopped working for good in Emacs 22.1 / TRAMP 2.0.55 .

Here is what I found out...

The gist of this specific problem is in fact Windows-specific (this is probably why no one noticed). Lisp function tramp-file-name-handler (file tramp.el) has the following logic:

     ;; When we are in completion mode, some operations shouldn't be
    ;; handled by backend.
    ((and completion (memq operation '(expand-file-name)))
     (tramp-run-real-handler operation args))
    ((and completion (zerop (length localname))
          (memq operation '(file-exists-p file-directory-p)))
     t)

This says that when we invoke tramp-file-name-handler with method = expand-file-name, TRAMP delegates to "default" expand-file-name. This was perhaps done as an innocent attempt to improve performance and it does not as and of itself create problems under Unix; however, under Windows "regular" TRAMP names like "/user@machine:path" get "expanded" to "Windows" names like "c:/user@machine:path" (or other drive letter). This "expanded" non-existent path then gets passed to subsequent calls blocking successful file completion.

One solution is simply to block this "delegation":

     ((and nil completion (memq operation '(expand-file-name)))
     (tramp-run-real-handler operation args))
    ((and completion (zerop (length localname))
          (memq operation '(file-exists-p file-directory-p)))
     t)

This makes TRAMP to work, but it is indeed quite slow.

Better solution is to modify file completion. We create new function

(defun expand-file-name-noprefix (name &optional default-directory)
 (let ((std-result (tramp-run-real-handler 'expand-file-name (list name default-directory)))
   (win-prefix-re "^[a-zA-Z]:"))
   (if (or (not (string-match "^/" name))
       (not (string-match win-prefix-re std-result)))
   std-result
     (substring std-result 2))))

which deletes "<drive_letter>:" prefix from "expanded" name, if it seems unwarranted; then we replace code in tramp-file-name-handler with :

     ((and completion (memq operation '(expand-file-name)))
     (tramp-run-real-handler 'expand-file-name-noprefix args))
    ((and completion (zerop (length localname))
          (memq operation '(file-exists-p file-directory-p)))
     t)

This solves problem #1, and makes Unix and Windows versions behave consistently.

However, there is another problem.

File "handlers" can receive third argument (first two are file name and directory name), which is a "filter" to apply (e.g. to feasible completions). Previously in beta versions of Emacs 22 this argument was empty, but in 22.1 file-name-completion method gets passed a filter "file-exists-p", which seems quite meaningless in my opinion: when we are asked to generate list of "completions", doesn't it go without saying we are only talking about existing files?

The result of this "enhancement" in Emacs 22.1 is that after TRAMP backend generates list of possible completions, it is obliged to separately test each of these files for "existence". This substantially slows down the performance, most noticeably if trying to complete in a directory with many entries.

The good solution for this perhaps would be to keep "cache" of "known" files so that test for "existence" wouldn't involve any communication to the server. A quick but working patch is to simply suppress this third argument (code is given below).

One more item of interest (though not technically a bug) is that previously TRAMP could not "complete" very first name after host name (that is, in a name "/user@host:toplevel/nextlevel", we could complete nextlevel but not firstlevel). In Emacs 22.1, this has been fixed, but interestingly enough this fix is not enabled by default, but rather depends on minor mode "partial-completion". This mode is otherwise harmless, so it is a good idea to always use "(setq partial-completion-mode t)" with TRAMP.

With all these points taken into account, this is how my local .emacs file looks like now to make TRAMP work in Emacs 22.1 :

  (require 'tramp)
 (if (equal tramp-version "2.0.55") ;; bundled with Emacs 22.1.1
     (progn
   (setq partial-completion-mode t)
(defun tramp-file-name-handler (operation &rest args)
 "Invoke Tramp file name handler.
Falls back to normal file name handler if no tramp file name handler exists."
;;  (setq edebug-trace t)
;;  (edebug-trace "%s" (with-output-to-string (backtrace)))

 (let ((args
    (if (memq operation '(file-name-completion))
        (list (car args) (car (cdr args)))
      args)))
 (save-match-data
   (let* ((filename (apply 'tramp-file-name-for-operation operation args))
      (completion (tramp-completion-mode filename))
      (foreign (tramp-find-foreign-file-name-handler filename)))
     (with-parsed-tramp-file-name filename nil
   (cond
    ;; When we are in completion mode, some operations shouldn' be
    ;; handled by backend.
    ((and completion (memq operation '(expand-file-name)))
     (tramp-run-real-handler 'expand-file-name-noprefix args))
    ((and completion (zerop (length localname))
          (memq operation '(file-exists-p file-directory-p)))
     t)
    ;; Call the backend function.
    (foreign (apply foreign operation args))
    ;; Nothing to do for us.
    (t (tramp-run-real-handler operation args))))))))
))  ;; if tramp-version == "2.0.55"

(don't forget to also include somewhere function expand-file-name-noprefix as given above)

Appendix 1. The reason why the first problem was not caught in time might have something to do with TRAMP not working by default in Windows at all, not without some Windows-specific customization (this is unchanged from previous versions); even if Cygwin ssh is available, this won't work (not sure why).

Proper approach is to use free Windows utility plink (distributed along with popular free Windows SSH/telnet client PuTTy). Put it in a directory from Emacs's exec-path, and use this code in your .emacs :

(defvar w32
 (not (not (string-match "w32\\|win32\\|mswindows"
             (symbol-name window-system))))
"t if running under Windows GUI, nil otherwise"
)

(when w32
 (setq tramp-methods
   '(
     ("ssh"
      (tramp-connection-function  tramp-open-connection-rsh)
      (tramp-login-program        "plink")
      (tramp-copy-program         nil)
      (tramp-remote-sh            "/bin/sh")
      (tramp-login-args           ("-ssh")) ;optionally add "-v"
      (tramp-copy-args            nil)
      (tramp-copy-keep-date-arg   nil)
      (tramp-password-end-of-line "xy"))
     ("telnet" ;; Sorry, this is NOT working yet....
      (tramp-connection-function  tramp-open-connection-rsh)
      (tramp-login-program        "plink")
      (tramp-copy-program         nil)
      (tramp-remote-sh            "/bin/sh")
      (tramp-login-args           ("-telnet")) ;optionally add "-v"
      (tramp-copy-args            nil)
      (tramp-copy-keep-date-arg   nil)
      (tramp-password-end-of-line "xy")) ;; this is questionable
     ("rlogin"
      (tramp-connection-function  tramp-open-connection-rsh)
      (tramp-login-program        "plink")
      (tramp-copy-program         nil)
      (tramp-remote-sh            "/bin/sh")
      (tramp-login-args           ("-rlogin")) ;optionally add "-v"
      (tramp-copy-args            nil)
      (tramp-copy-keep-date-arg   nil)
      (tramp-password-end-of-line nil))))
         ;; (setq  tramp-debug-buffer t)
          (setq tramp-default-method "rlogin"))

Appendix 2. TRAMP Manual given extensive information on how to debug TRAMP. The bottom line is this: create a special Lisp file, let's say, "debugtramp.el" as follows:

(setq debug-on-error t)
(setq debug-on-signal t)

(require 'tramp)
(require 'trace)

(mapcar 'trace-function-background
 (mapcar 'intern
  (all-completions "tramp-" obarray 'functionp)))
(untrace-function 'tramp-read-passwd)
(untrace-function 'tramp-gw-basic-authentication)

(setq tramp-verbose 10)

And load it before problem might occur; afterward, save buffer output-trace (huge!) to a new file and investigate the problem.

Labels: emacs, tramp, windows