My Tech Notes

There is now a "standard" (free, BSD-style) image manipulation library ImageMagick. As it is pretty customary these days, it provides bindings for many popular higher-level languages, including Java, .NET, PHP, Perl, Python and Ruby. Ruby binding is known as RMagick.

One problem with ImageMagick is that it is notoriously difficult to install. The root of this problem may be that library insists on integration with X-server libraries (and indeed provides some X-client capabilities, though interestingly enough not supported on Windows).

(As a result, people are coming up with various ways to make a smaller library with restricted functionality, e.g. mini-magick. Unfortunately, there seems to be no documentation available, other than this tutorial, so I didn't want to spend time researching this)

Cygwin does include ImageMagic (you'll need at least 2 cygwin packages for ImageMagic: image-magick and libMagick-devel and various dependents for some reason not picked up by cygwin setup utility, e.g. libXft-devel and libbz2-devel), but all my attempts to first install ImageMagic and then RMagic failed (perhaps bundled version was too old, though judging from version numbers it was ok). ImageMagick web site includes pre-built version of all libraries for cygwin, but it appears to be very poorly built, includes many hardcoded paths from original build machine, etc.

So, the only option was to install ImageMagick from the sources, and it succeeded, after I explicitly excluded Perl bindings

./configure --without-perl

(be aware that built could take several hours to complete). After that, RMagick can be installed with command as simple as:

gem install rmagick

To try it out, you can use this program; try to uncomment line "canvas.display" near the end and look at the result. This didn't work well under Cygwin at all (though displaying same image just read from file worked fine).

Library itself is very powerful indeed. Documentation is good and detailed. One interesting thing that I discovered is that library has no less than 3 different methods to resize an image: "to sample", "to scale", and "to resize". You can compare all three by picking some random picture (sufficiently large) and running on it program like that:

#! /usr/local/bin/ruby -w

require 'rubygems'
require 'RMagick'
include Magick

imgfile = ARGV[0]
img = (Image.read imgfile)[0]

factor = 0.25
osize = File.size imgfile
puts "Original file #{imgfile} : #{osize} bytes, factor=#{factor}"

[:sample, :resize, :scale].each { |resize_alg|
  t = Time.now
  img_resized = img.method(resize_alg).call factor
  dt = Time.now - t
  fname = "test_#{resize_alg}.jpg"
  img_resized.write( fname );
  size = File.size fname
  puts "#{sprintf("%-6s",resize_alg)}: #{dt.to_f} secs, #{size} bytes =\
 #{sprintf("%.2f",100*size.to_f/osize)}%"
}

On my machine, it generated this output:

sample: 0.012 secs, 258985 bytes= 11.08%
resize: 1.287 secs, 227424 bytes= 9.73%
scale : 0.469 secs, 222147 bytes= 9.51%

Which is mathematically interesting: apparently, "resize" and "scale" provide some smoothing, therefore reducing size of the compressed image; "sample" is by far the fastest way to resize, but by just picking some more or less random pixels from the original image it generated result which is not only visually worse, but is harder to compress.

Labels: cygwin, images, ruby

Ruby FAQ which seems to constantly move around and is currently not available (Google cache is though) has a very useful section 10.6 Should I feel uneasy if I don't close a file :

There are at least four good ways of ensuring that you do close a file:

1. Use close (remembering to catch exceptions).

   f = open "file"
   begin
     f.each {|l| print l}
   ensure
     f.close
   end
   

2. Use the block form of open

   File.open("file") do |f|
     f.readlines.each { |l| print l }
   end
   

3. Use foreach

   IO.foreach("file") {|l| print l}
   

4. Use readlines.

   IO.readlines("file").each {|l| print l}
   

Labels: ruby

I am trying to get comfortable using Ruby (having already written a fair amount of code in Perl, Python, and PHP). It is apparent that one of the serious problems with Ruby is lack of support for Unicode (in the current 1.8.x Ruby version). Class String in Ruby is just a byte string and nothing else.

A few things of interest:

1. Standard Ruby distribution does support Iconv library. Fox example, to convert a string of text from UTF-16LE (used by Windows) to UTF8, one can use:

   require 'iconv'
   utf8_str = Iconv.iconv( "UTF-8", "UTF-16LE", win_str ).join

   (obviously, this is merely a wrapper around "standard" GNU iconv library. To see full list of currently installed encodings, use `iconv -l')
2. Also, there is a special option 'U' to standard String 'unpack' function, which effectively treats string as having UTF-8 encoding and then splits it into array of Unicode integers;
3. Using the above, there is an attempt made on rubyforge to create well-behaved Unicode String class purely in Ruby (the only way to install above seems to be to download the only source file directly). However, this project was obviously not finished and is barely used, given that last modification occurred 18 months ago and there seems to be no activity on project public forums. While many features of "regular" strings are supported and others could be added, it is not clear whether for example full regular expression support is feasible;
4. Interestingly, Ruby, being originally developed in Japan, was designed to deal with non-ascii encodings from its day one. In particular, Ruby supports the notion of "default encoding", which corresponds to "global variable" $KCODE . It can be set with command line option -K, e.g. `-Ku' for UTF-8. This variable can be assigned to at any moment to overwrite default value. The only trouble is, in my (Cygwin) Ruby and iconv installation, actual value of $KCODE which corresponds to utf8 is "UTF8", whereas iconv only has "UTF-8" (note the dash) and refuses to understand "UTF8". Assigning explicitly `$KCODE = "UTF-8"' does not help, as Ruby still resets $KCODE to "UTF8". It means that to use Ruby unicode library mentioned above I had to make changes to have it pass to Iconv "UTF-8" whenever previously it wanted to pass "UTF8";
5. In this e-mail thread, someone explains why he thinks Ruby is fine without any (more) Unicode support;
6. Here some Unicode-related changes planned for Ruby 2.0 are described;
7. However, here is the detailed description of what has already been done in Ruby 1.9, with an understanding that 2.0 will be a subset of 1.9; I cannot see Unicode ever mentioned there;

In a nutshell, all of the above means that at least before Ruby 2.0 is released, the best option to deal with Unicode strings is to make all String's (=byte arrays) used share same Unicode-compatible encoding, e.g. UTF-16LE . Then, we will need to always keep that in mind while dealing with such strings. Fox example, to split TAB-separated string, we would use:

words = line.split( "\t\0" )

Appendix 1. Some reference material on Ruby:

1. Two similarly brief introductions to Ruby: Programming Ruby: The Pragmatic Programmer's Guide and Ruby User's Guide (by language creator);
2. As part of Pragmatic Guide mentioned above there is a library reference and built-in reference; also, seemingly same documents are also available from here; I am not why if there is any difference or not;
3. The Ruby Language FAQ;
4. Ruby Cheatsheet;
5. Ruby-doc.org server features, among other things, standard Ruby documentation and customized Google search;
6. Ruby QuickRef;
7. Book Programming Ruby (2-nd edition) is not available for free (you can buy it as a regular book or download PDF file for $25), but number of useful excerpts are available, and judging from these excerpts book appears to be well-done and useful;
8. Server rubylearning.com offers some on-line tutorial, as well as (huge) PDF file download (link is only provided via e-mail, but see this for book and this for accompanying Ruby programs)
9. One more PDF book from the Scribd collection.

Appendix 2. On main Ruby web site, there is a special section Ruby From Other Languages. Especially interesting to us are possible confusion and incompatibilities compared to Python and Perl. A few things, however, are missing from the list of incompatibilities.

1. Ranges in Ruby are inclusive while in Python they are not; that is, in Python `range(2,10)' consists of all numbers from 2 to 9 while in Ruby similar n spirit notation `2..10' is used for all numbers from 2 to 10 inclusively. As a result, to, for example, drop the last character from a string in Python, we use `str[:-1]' while in Ruby this translates to `str[0..-2]';
2. Including function from a file in Ruby does not imply any consequences in terms on namespace, whatsoever. To take advantage of namespaces, you must use `module' explicitly;
3. Apparently Ruby does not use popular Perl regular expression library, so its regular expression syntax, while very similar to that from Perl, might not be fully compatible. E.g., Ruby does not support look-behind zero-width assertions;
4. On a positive side, Ruby does implement Python-style '%' operator for parameters substitution, though it is not used anywhere in the standard documentation; instead, Ruby-own parameter substitution ("x = '#{my_var + 57}'") is used. I prefer Python style, besides, it allows for formatted output;

(Happy Ruby programming! :-)

UPD [18-Jun-07]. Slashdot reviews new book "Practical Ruby Gems"

Labels: ruby, unicode