My Tech Notes

Well-respected and popular blog TechCrunch recently published a review of new version of Zimbra desktop e-mail client; as blog authors put it, Zimbra Releases Version 3 Of Open Source Email Client, And It’s Awesome.

OK, I tried playing a little bit with Zimbra and I think that the concept is OK, application is responsive and useful. However, there are a few things that must be fixed or improved before it could indeed outplace Thunderbird on my desktop. In no particular order, these are:

1. Account setup is strange. Software insists that SMTP server must be specified and available at the time of the setup, or else it refuses to allow you to proceed. Why is that? What's wrong with using Zimbra to simply read e-mail from one of my IMAP servers, or e.g. setup SMTP server later?
2. It refuses to show you any information at all about a folder unless it is "synchronized", that is, completely copied to the local machine and indexed. Without doing that (which can take hours and gigabytes in traffic) you can't even (in Zimbra) figure out if there is anything in this folder which is worth your time (you can of course start synchronization and then cancel it);
3. Spellchecker options are nowhere no be found, but this does not actually mean that spellchecking is completely unavailable: since Zimbra UI is really nothing more than a "headless" Mozilla browser (they call this technology "Mozilla Prism") and Mozilla now has built-in spellchecker, you will see incorrectly spelled words underlined in red, except that you can neither see list of corrections (since Zimbra overrides RMB) nor change the language (for the same reason);
4. Zimbra is trying to use (by default) "conversation" mode, which is to say, combine together in the list of messages deemed (based on subject) to belong to the same thread. This is actually done very nicely from the UI standpoint (although GMail is still much better), but background engine which generates "conversations" is faulty; it constantly fails to realize that messages with subjects "foo" and "Re: foo" belong to the same conversation;
5. Zimbra is allowing custom-made tags, but for some reason only one tag per message. Why?
   Update. This is incorrect, you can assigned as many tags as you want to a message.
6. When retrieving message headers from IMAP server, in the message list Zimbra shows column "Received" which is actually the time when IMAP server registered the message, and not date of the message itself. Most people probably won't notice the difference, but since I am using my IMAP server as a backup for various other sources (e.g., GMail) and copy email there only occasionally, it is highly inconvenient for me;
7. I really like the concept of searches and conversations working across the IMAP folders. However, why not across separate "accounts"?
8. Unlike Thunderbird, "search" folders and (probably using same internal implementation) "tag" folders do not display number of unread messages inside them in the folder tree; I have to manually click on each of them to see if there are any new messages;
9. Also regarding search folders, and also unlike Thunderbird, there is no way to edit existing search folder, you can only remove it and re-create;
10. Working with contacts and address books is painful. Zimbra does not support any kind of LDAP queries and uploading complete corporate address book is buggy; even after I wrote a special tool to re-order and re-name properly fields in CSV file, ZImbra import missed about half of e-mails (why only e-mails?) from the import. Worse, I could not manually edit these contacts to add missing emails; the only possibility was to delete these contacts and then manually add (identical) new ones. This kind of worked....
11. For the life of me, I can't figure out where sent messages disappear to; I can't see anything in "Sent" folder (and I am not even mentioning a capability to push sent messages to server). I guess the only available workaround is to always sent a copy to myself...
12. And last but not the least, after a while Zimbra simply stopped updating some of my folder as new messages continued to show up on server. Till now, I couldn't say what specifically triggered this behavior.

Honestly, I can probably live with all these bugs and shortcomings - on a technical side, that it. But there is a bigger issue here of a psychological conformability. When I select a IMAP folder without any filters or adjustment - I expect my mail client to show me just that, folder, precisely as IMAP server reports it to be. If, instead, my e-mail client inserts some intermediate logical layers between me and my folder, even if these layers are supposed to be completely transparent, I know that in practice I can never be really sure that what I see is indeed my folder "as is" and not some internally generated skewed view of it. And that's the problem - for me, anyway.

P.S. Some clarification on the last problem of Zimbra not auto-updating some or all folders.

First, problem was likely triggered by an attempt to synchronize with some "weird" Microsoft Exchange folders which did not really have "messages" as such;

Second, you can get dome useful debugging info (mostly Java tracebacks) from <...>/zimbra/zdesktop/log/mailbox.log, (especially) if you checked "enable debug trace" in account setup.

Labels: mail, web-services, yahoo

# posted by Kostya @ 12:50 PM 0 comments

There are, generally speaking, five "standard" ways to make files on (Linux) server available to clients:

1. Using FTP server;
2. If sshd is running, files could be accessed with SFTP;
3. If Web server is running, WebDAV could be used;
4. Using NFS (see earlier port);
5. Using SMB-shares.

In principle, last choice - using Samba shares - is supposed to be most "native" with respect to Windows clients; let's consider how difficult it is to use it practically...

For the following, we assume that SMB server is running on "SERVER" and we'll be using it to access file of local regular user "user".

1. Install and configure samba

1.1. Install

# apt-get install samba smbclient

1.2. Modify config file.

Here we zero in on "minimalistic" approach, which only requires minimal changes to default config file (as distributed with Debian, anyway). It has one built-in share "homes" which provides access to each user's home directory (it is enabled by default, but in read-only mode)

1.2.1. Backup default config file:

cp /etc/samba/smb.conf /etc/samba/smb.conf.original

1.2.2. Modify global settings:

workgroup = <Enter some name>
interfaces = <enter some interfaces from /sbin/ifconfig, e.g. lo venet0:0>
printcap name = /dev/null (shut down all complaints in logs about printers)
encrypt passwords = yes (or else you won't connect from Windows NT and up)
security = user (this should be the default anyway)
smb passwd file = /etc/samba/smbpasswd (see below why/when this is necessary)

1.2.3. Modify setting for share.

If you are satisfied with read-only access to user's directory, there is nothing more to change. If you want read-write access, there are some settings to adjust:

writable = yes
create mask = 0644
directory mask = 0755

Note on "encrypt password": if encrypt password = false, you don't need "smb passwd file", system password file will be used. For some reason it did NOT work for me if "encrypt password = true". As suggested in [8], I did this:

cat /etc/passwd | mksmbpasswd > /etc/samba/smbpasswd
smbpasswd user (for each use who needs his home dir access via SMB)

1.2.4. Restart server:

/etc/init.d/samba restart

1.2.5. Test installation (on the server as user)

smbclient //localhost/homes

If you can, test from another location which does not block outgoing ports 139, 445

smbclient //SERVER/homes -U user

2. Setup Windows computer

In principle, the following command

net use [<drive letter>:]  \\SERVER\homes /user:user /persistent:no

should be able to mount corresponding share. However, for this to work it is necessary that client computer had direct access to server using ports 139 and/or 445. If server is to be used in local subnet, this is undoubtedly so and no more setup is required. However, if you are accessing server from the Internet, and your ISP is blocking these ports (like RCN), read on.

The idea is to try to mount SMB shares on SERVER as if they were available on localhost; intercept requests made on (local) ports 139 and 445 and somehow forward them to SERVER.

This however appears to be more difficult than it sounds. The problem is, Windows by default binds all adapters on port 445 :

> netstat -ano | grep 445
  TCP    0.0.0.0:445      0.0.0.0:0     LISTENING       4

and the only way to make this port available for binding is to disable NetBios completely, which is rather pointless, since then you won't be able to amount anything at all (if, however, you want to play with this, refer to [5] and [6]).

Fortunately, however, usually Windows, after failing to mount using port 445, falls back to port 139, which apparently you can bind, albeit not on "standard" loopback adapter 127.0.0.1; (well, you can bind to it, but it won't work for whatever reason), thus necessity to create new loopback connection. This "fallback" logic is in no way guaranteed and moreover has reportedly been broken by a recent Vista patch; but at least on XP this seems to work, as long as you implement 2.1 and 2.2 below.

2.1. If you are using Windows XP, install this Windows patch [3].

2.2. For your existing Internet connection, enable option "EnableBIOS over TCP/IP" (in fact, it may be sufficient to do so for any Internet connection. Quoting from [1]: "It also appears that if there are no valid interfaces with NetBIOS over tcp enabled, then windows will not attempt to use port 139")

2.3. Add new Microsoft loopback interface and bind it to port 10.0.0.1, see [4] and [1] ([4] has some screenshots from Vista; XP installation is similar).

2.4. It may or may not be necessary, but I also (a) disabled all "items" for new 10.0.0.1 loopback connection (except TCP/IP, see first screenshot above) ; (b) enabled LMHOSTS lookup; (c) disabled NetBIOS over TCP/IP (see second screenshot) ; (d) disabled "File and Printer Sharing for Microsoft Networks" for ALL connections;

2.5. Reboot and you are all set!

3. Setup port forwarding

This can be done in one of the two ways.

You can use of the many existing utilities for port forwarding. (It is more correct in fact to speak of reversed port forwarding, but people usually call it port forwarding all the same). In this case you need a (non-blocked) port to forward to, and your Samba server should be told to listen on this port.

Alternatively, you can use ssh port tunneling capability and tunnel port 139 through ssh. This requires more complicated setup and is more difficult to automate, but does not require any additional port to be used and has an additional benefit of securing your Samba traffic via ssh.

3.1. Using ssh tunneling.

3.1.1. With Cygwin (or similar CLI) ssh:

ssh -L 10.0.0.1:139:localhost:139 SERVER

3.1.2. Using PuTTY

(see screenshot in [4]). There are various ways to automate this using pageant, but I won't get into this here.

3.2. Using port forwarding.

3.2.1. Add another option to smb.conf :

smb ports = 445 139 8445 8139

3.2.2. Configure AUTAPF (shareware) or PassPort (Open Source) to forward port 139 on local adapter 10.0.0.1 to port 8139 on SERVER.

AUTAPF is more convenient as it immediately tells you if it can't bind a port thus it is better for testing; once you are comfortable with the setup, you can switch to free PassPort.

4. Using SMB shares

4.1. Under Windows

net use [<drive letter>:]  \\10.0.0.1\homes /user:user /persistent:no

(Unless user if Windows user name and password matches, you'll be asked to enter server password at this point)

4.2. Under Linux

Make sure you have "smbfs" installed ("apt-get install smbfs") and issue this command as super-user:

mount -t smbfs -o username=user%<user password>,uid=<local username>,port=8139 //SERVER/homes <local mount point>

(You can use option -credential in place of plain text user name/password, see "man smbmount" and this page)

4.3. Using smbclient, any platform

For any platform which has utility "smbclient" or equivalent, and provided that you've configured SMB server to listen to port 8138, you can use this

smbclient //SERVER/homes -U user -p 8139

to access your files.

You can get MinGW-based smbclient.exe for Windows from here (it works fine except that you have to specify password in the command line); alternatively, you can build your own Cygwin-based version using one of the patched published here (you'll need to disable first test in Samba source file source/tests/summary.c)

References:

[1] Sharing (tunneling) Samba/CIFS/SMB file systems over SSH

[2] How to tunnel Samba via ssh from Windows XP without having to disable local NetBIOS

[3] Programs that connect to IP addresses that are in the loopback address range may not work as you expect in Windows XP Service Pack 2

[4] Vista or XP Accessing Samba shares securely over SSH

[5] Disabling Port 445 (SMB) Entirely

[6] After you disable the "Client for Microsoft Networks" option for a dial-up connection, the dial-up connection is still active on a Windows XP-based computer

[7] "Network Location Cannot be Reached" when accessing shares

[8] SMBCLIENT CONNECTION ERROR

Labels: debian, linux, Samba, server, SMB, windows

# posted by Kostya @ 9:08 PM 1 comments

I am still not sure I am fully understand the inner workings of fonts under Linux; using hints from this page, here is how I recently installed RedHat Liberations fonts under Debian:

cd /tmp
# using link from https://fedorahosted.org/liberation-fonts
wget https://fedorahosted.org/releases/l/i/liberation-fonts/liberation-fonts-1.04.tar.gz
tar -xvzf liberation-fonts-1.04.tar.gz
cd liberation-fonts-1.04
# remove everything except font *.ttf files
rm `ls -1 | grep -iv \.ttf$`
# I prefer to install under /usr/local, but it's irrelevant
mkdir -p /usr/local/share/fonts/truetype
cp -pr /tmp/liberation-fonts-1.04  /usr/local/share/fonts/truetype/liberation-fonts
# defoma is Perl-based; for some reason, some required Perl dependencies weren't
# installed by default
apt-get install libft-perl
defoma-hints -c --no-question truetype \
            /usr/local/share/fonts/truetype/liberation-fonts/* > 
            /etc/defoma/hints/liberation-fonts.hints
defoma-font register-all /etc/defoma/hints/liberation-fonts.hints
defoma-reconfigure 
xset fp rehash 

I just hope one day I'll understand what all of that really means....

Labels: debian, fonts, linux, redhat

# posted by Kostya @ 10:07 AM 0 comments