My Tech Notes

My number finally came up, I was able to register for new revolutionary file sharing service by Ubuntu Team called UbuntuOne. The service is still in closed-beta; see some reviews here, here and here.

(There isn’t much information (currently) on the official site, other than installation instructions, mailing list and bugs database)

This is a potentially brilliant idea, because it gives people a convenient way to support financially their favorite distribution, and with solid financial support this could become a reliable backup solution. Correct, there may be cheaper options available on the market, but you never know how long they’ll last (Streamload being a good example of that)

On the other hand, as of right now the service is hardly usable.

• It is clearly inferior compared to the popular (among Linux crowd) DropBox service, which provided 5 times more of paid storage, supports versions, has be better UI (web-based and client-side) and is compatible with every modern Linux distributions, Windows and Mac;
• It is still very buggy and incomplete. For example, when I added a file through Web UI, it never showed up in my Ubuntu folder, though synchronization the other way around worked fine;
• There are also browser-compatibility issues, e.g. with Google Chrome;
• Is this service supposed to be open source? I could not find any reference to the server source code anywhere, nor to any plans for future client-side API.

Mailing list appears to be relatively low-volume, and I am not sure many people are actually using this – which, given the modest, at best, progress of development and plenty of competitors, does not promise a bright future for this project any time soon, in my opinion.

But, whatever this is going to turn out, the idea was really a good one.

Labels: internet, linux, Ubuntu, web-services

# posted by Kostya @ 5:18 PM 0 comments

Procedures to mount Windows shares from remote machines changed so many times during lifespan of Linux, that an attempt to do a search in Google gives a mix of solutions from using "smbmount" to "mount -t smb".

Here is the latest and greates solution. Make sure you have "smbfs" package installed and do this :

% sudo mount -t cifs //192.168.1.102/share_name /media/my_share -o \
  username=theuser,password=thepass,iocharset=utf8,file_mode=0777,dir_mode=0777

With option "iocharset=utf8" it ought to handle international characters in all Windows NT-based systems. If you need to mount drives from Windows 95/98/Me, different translation options will be necessary.

see this page for more detailed explanations.

Labels: linux, Samba, Ubuntu, unicode, web-services, windows

# posted by Kostya @ 3:32 PM 0 comments

It seems almost incomprehensible how many problems emerge with simple WebDAV file hosting. While apache DAV server, and clients built-in into both Windows Shell and GNOME have been available fr many years, I still can't make it to work properly.

Here is a brief list of problems:

Apache 2 Server. Its problem is that it is bit difficult to make server work with both "resular" and SSL modes. This typically emerges as failure to "copy", "move" or "rename" file in non-SSL mode, while everything else seems to work.

Good explanation is available here:

The main issue was that the production system runs on https and there was a reverse proxy setup for this system. So, all the "https" requests were converted to "http" at this proxy level and forwarded to the main system. This was the main culprit.

Here is an example of HTTP request for the MOVE resource request for a WebDAV resource. For brevity I removed all unnecessary details.

MOVE contentLocation //request line, some https location, URI of webDAV resource
Destination:destinationLocation // this is the HTTP request header, should be absolute URI according to specifications.

Overwrite: "F" // this is also a HTTP request header

So, when the reverse proxy sees the request line, it knows that it has to convert this to HTTP request but the header Destination also contains an HTTPS request which would be ignored by the proxy. So, when the request reached the server, we are moving the resource from an URI which begins with a http to a URI which begins with an https. Server treats this request as a request to move a WebDAV resource from one server location to another server location.(Refer RFC: http://greenbytes.de/tech/webdav/rfc2518.html#METHOD_MOVE). This was the source of the main problem.

Linux/GNOME GVFS driver: Seems to work most of the time,but

• Fails to handle paths with user name "davs://username@server.com/path". "Top listing" is shown correctly, but an attempt to change directory fails. Reason unknown;
  
  
• When using drag-and-drop, or "cut/paste" interface, it attempts to "copy" file and then "delete" the original. I could not find any way to invoke "MOVE" command.
  

Windows (old driver). This is "old" Windows-2000 implementation, which identifies itself as "Microsoft Data Access Internet Publishing Provider DAV". This seems almost the most reliable implementation, and it worked for me for a while, but now it shows directory "Temp" as "temp" (all other directories similarly named are OK), and obviously fails to chdir to it. Reason unknown.

Windows (new driver, XP and newer). Identifies itself as  "Microsoft-WebDAV-MiniRedir/5.2.3790". Apparently only intended to work with Microsoft IIS, not compatible with Apache. This pages gives a consice overview:

For implementation of WebDAV on Windows XP and later , MSFT made it's own interpretation of the standard to work best with the Windows IIS servers. The problem due to this is three fold:

1. Windows XP authenticates users using the format "domain\username" by the mechanism of "Microsoft-WebDAV-MiniRedir/5.1.2600". Whereas Windows 98SE/2000 authenticates users as "username" using the mechanism of "Microsoft Data Access Internet Publishing Provider DAV 1.1".
2. The problem lies with the implementation of "Microsoft-WebDAV-MiniRedir/5.1.2600". If authentication is sent as "domain\username" then it would be received as "usernamedomain" or "usernamehostname" by the Web server and not as "username".
3. Also as per "Microsoft Knowledge Base, Article ID: 841215" Windows XP disables "Basic Auth" in its "Microsoft-WebDAV-MiniRedir/5.1.2600" mechanism by default for security reasons. But WebDAV expects "Basic Auth".

There are hundreds pages how to trick it to invoke "old" implementation, including some on the same page mentioned above; there is also a separate discussion regarding Windows Vista, where "old" implementation has to be separately installed. Windows 7 status is unknown.

Platform-independent clients.

1. SkunkDAV Java-based client is simple and reliable, but unfortunately I could not make it support SSL-based access;
2. Cadaver is CLI-based tool. It seems to work fine. 

Update (12-May-09). "Official" subversion book has a list of DAV clients. Among some known clients, some of them mentioned above, there is another Jaba-based client "DAV Explorer", it has been last updated in 2005 and looks a bit better than SkunkDAV, though I can't say there is a big difference.

Also, its help file says that in order to enable SSL in Java one has to run Java with -Dssl=true, and only from version 1.4 on; for earlier versions, one has to download special Java Secure Socket Extensions. It is likely with if run with these options, SkunkDAV will work ok with SSL DAV.

Labels: GNOME, linux, server, Ubuntu, web-services, windows

# posted by Kostya @ 10:42 PM 0 comments

Well-respected and popular blog TechCrunch recently published a review of new version of Zimbra desktop e-mail client; as blog authors put it, Zimbra Releases Version 3 Of Open Source Email Client, And It’s Awesome.

OK, I tried playing a little bit with Zimbra and I think that the concept is OK, application is responsive and useful. However, there are a few things that must be fixed or improved before it could indeed outplace Thunderbird on my desktop. In no particular order, these are:

1. Account setup is strange. Software insists that SMTP server must be specified and available at the time of the setup, or else it refuses to allow you to proceed. Why is that? What's wrong with using Zimbra to simply read e-mail from one of my IMAP servers, or e.g. setup SMTP server later?
2. It refuses to show you any information at all about a folder unless it is "synchronized", that is, completely copied to the local machine and indexed. Without doing that (which can take hours and gigabytes in traffic) you can't even (in Zimbra) figure out if there is anything in this folder which is worth your time (you can of course start synchronization and then cancel it);
3. Spellchecker options are nowhere no be found, but this does not actually mean that spellchecking is completely unavailable: since Zimbra UI is really nothing more than a "headless" Mozilla browser (they call this technology "Mozilla Prism") and Mozilla now has built-in spellchecker, you will see incorrectly spelled words underlined in red, except that you can neither see list of corrections (since Zimbra overrides RMB) nor change the language (for the same reason);
4. Zimbra is trying to use (by default) "conversation" mode, which is to say, combine together in the list of messages deemed (based on subject) to belong to the same thread. This is actually done very nicely from the UI standpoint (although GMail is still much better), but background engine which generates "conversations" is faulty; it constantly fails to realize that messages with subjects "foo" and "Re: foo" belong to the same conversation;
5. Zimbra is allowing custom-made tags, but for some reason only one tag per message. Why?
   Update. This is incorrect, you can assigned as many tags as you want to a message.
6. When retrieving message headers from IMAP server, in the message list Zimbra shows column "Received" which is actually the time when IMAP server registered the message, and not date of the message itself. Most people probably won't notice the difference, but since I am using my IMAP server as a backup for various other sources (e.g., GMail) and copy email there only occasionally, it is highly inconvenient for me;
7. I really like the concept of searches and conversations working across the IMAP folders. However, why not across separate "accounts"?
8. Unlike Thunderbird, "search" folders and (probably using same internal implementation) "tag" folders do not display number of unread messages inside them in the folder tree; I have to manually click on each of them to see if there are any new messages;
9. Also regarding search folders, and also unlike Thunderbird, there is no way to edit existing search folder, you can only remove it and re-create;
10. Working with contacts and address books is painful. Zimbra does not support any kind of LDAP queries and uploading complete corporate address book is buggy; even after I wrote a special tool to re-order and re-name properly fields in CSV file, ZImbra import missed about half of e-mails (why only e-mails?) from the import. Worse, I could not manually edit these contacts to add missing emails; the only possibility was to delete these contacts and then manually add (identical) new ones. This kind of worked....
11. For the life of me, I can't figure out where sent messages disappear to; I can't see anything in "Sent" folder (and I am not even mentioning a capability to push sent messages to server). I guess the only available workaround is to always sent a copy to myself...
12. And last but not the least, after a while Zimbra simply stopped updating some of my folder as new messages continued to show up on server. Till now, I couldn't say what specifically triggered this behavior.

Honestly, I can probably live with all these bugs and shortcomings - on a technical side, that it. But there is a bigger issue here of a psychological conformability. When I select a IMAP folder without any filters or adjustment - I expect my mail client to show me just that, folder, precisely as IMAP server reports it to be. If, instead, my e-mail client inserts some intermediate logical layers between me and my folder, even if these layers are supposed to be completely transparent, I know that in practice I can never be really sure that what I see is indeed my folder "as is" and not some internally generated skewed view of it. And that's the problem - for me, anyway.

P.S. Some clarification on the last problem of Zimbra not auto-updating some or all folders.

First, problem was likely triggered by an attempt to synchronize with some "weird" Microsoft Exchange folders which did not really have "messages" as such;

Second, you can get dome useful debugging info (mostly Java tracebacks) from <...>/zimbra/zdesktop/log/mailbox.log, (especially) if you checked "enable debug trace" in account setup.

Labels: mail, web-services, yahoo

# posted by Kostya @ 12:50 PM 0 comments

There is an interesting startup company Streamload that implements files, pictures and various media hosting in a way that could be potentially correct.

Namely, they allow you to both store files with a simple web-based file manager and selectively open access to these files for the public. The way it is done, essentially two completely different (Web) UIs are implemented; as of today, internal UI isn't bad (though does need a lot of polishing) while external UI is outright terrible.

Their pricing model is based on providing "unlimited" storage while restricting bandwidth which is also a sound business model. Storage space nowadays is so cheap that it practically does not count, while various other services guaranteeing "unlimited access" must compensate for it somehow (one popular "solution" is of course to limit your storage in a hope that it will indirectly limit bandwidth which is outright stupid). As for me personally, everything I might ever want publish online will be accessed very rarely so I am absolutely happy with bandwidth restriction.

That said, number of problems facing this service (and similar) is enormous; the most important question of course being whether it will survive in the more and more competitive marketplace. But, the good ideas outlined above make it worth checking periodically for the progress...

At the end, a few links of interest. I originally came across this site from Slashdot article referencing a PC World review of a few online file storage services. On the Internet, you can also enjoy older (February) review from PCMag, demo.com business overview of the company, both old and new blogs of company programmer(s), and perhaps most invaluably example of a user publishing many pictures and other media with this service.

Finally, one unrelated remark that could be more important than all of of the above: there are unconfirmed, but reliable reports that Google is preparing its own Web hosting platform.

UPD. Streamload is reviewed on digg.com.

UPD (10-Dec-06). Another interesting attempt at file hosting, www.bluetie.com .

Labels: web-services

# posted by Kostya @ 2:05 AM 0 comments