Sunday, 16 March 2014

Starting with Lisp

Just for some fun and relaxation, I decided to start doing some Lisp. I had set up SLIME a few months backs while I was starting some book but I had not done much. I decided to use SBCL because I had read that is usually fast. The reason I was using SLIME is that the back arrow key does not work when I use the SBCL REPL and that is very annoying to me because I type both opening and closing parentheses together and type the rest of the code in between. To setup SLIME, a lot of documentation is available online so I wont re-iterate. Just for reference, the SLIME section of the .emacs file is as follows:

(setq inferior-lisp-program "/usr/bin/sbcl")
(add-to-list 'load-path "/usr/share/emacs/site-lisp/slime/")
(require 'slime)

Once I was in SLIME REPL, I started writing a few s-expression to get a hang of it.

(+ 1 2)

(print 'hello)

(format t "hello world")

After that, I started writing a simple function that returns the nth number in the Fibonacci sequence. I came up with the following.

(defun foo (n)
     ((= n 1) 0)
     ((= n 2) (+ 1 (foo (- n 1))))
     (+ (foo (- n 1)) (foo (- n 2)))))

And I started testing it.

(foo 1)

(foo 2)
(foo 3)

The result for n = 3 was wrong. The reason lay in the last section of cond. I intended it to be default but I had not specified the condition for it. So, I corrected it as follows.

(defun foo (n)
     ((= n 1) 0)
     ((= n 2) (+ 1 (foo (- n 1))))
     (t (+ (foo (- n 1)) (foo (- n 2))))))

Now, the method was returning correct values. Clearly, it is a bad implementation. So, I thought the calls to foo should be memoized. I could go ahead and try writing my own memoization but I wanted to see what Common Lisp had to offer. However, before doing that I wanted to know whether memoization will be beneficial. So, I wanted to benchmark the memoized and the plain versions. Our good friend Google helped me out. I was able to start profiling using SBCL's built-in package sb-prof.

(in-package :cl-user)
(require :sb-sprof)
(declaim (optimize speed))
(sb-sprof:with-profiling (:max-samples 1000
                          :report :flat
                          :loop nil)
  (foo 100))

I was a little doubtful that I should not be trying for the 100th Fibonacci number but still I went ahead with it and even after some minutes, it was going on. So, I now I decided to kill the profiling. Ctrl+C Ctrl+C came in handy. I started low this time. From 10, 15, 25, I reached up to 40 at which value it took around 6 seconds.

(sb-sprof:with-profiling (:max-samples 1000
                          :report :flat
                          :loop nil)
  (foo 40))

Now, I was ready to test a memoized version of the function and see how much benefit I can get. I remembered reading some code by Peter Norvig in Python which used a decorator to achieve generic memoization. So, I thought Lisp ought to have something similar. I found a nice memoization API. However, I had to install the package and I did not want to delve into that because I did not find any packages for Arch. So, I decided to settle for a less robust implementation.

(defun Basic-Memo (Function)
  "Takes a normal function object and returns an `equivalent' memoized one"
  (let ((Hash-Table (make-hash-table)))
    #'(lambda (Arg)
    (multiple-value-bind (Value Foundp)
        (gethash Arg Hash-Table)
        (setf (gethash Arg Hash-Table) (funcall Function Arg))))) ))

(defun Basic-Memoize (Function-Name)
  "Memoize function associated with Function-Name. Simplified version"
  (setf (symbol-function Function-Name)
    (Basic-Memo (symbol-function Function-Name))))

(Basic-Memoize 'foo)
Now, the 1000th Fibonacci number was also easily calculated. When I tried to the above profiling code for (foo 1000), I was getting error for the run being too short. Trying for 10000th Fibonacci number, I got 0.01 seconds.

Friday, 14 March 2014

Speeding up RubyMine on linux

I use RubyMine for development on Ruby on Rails platform. A lot of my friends and colleagues using it often have the opinion that it is a resource hog and slows down the system. I have also observed a little slowness. Of late, the effect increased and crossed my tolerable limit. So, I decided to dig in and find out what was happening. Using top, I could see that for some reason a process jbd2 was being triggerred every 2 seconds. Reading about it, I found out that it was the process updating access time. I am using ext4 filesystem and I had not specified the noatime option in fstab for the partition. So, access times for files were being updated in real time. I do not need that at all. Access times are updated even for page cache hits, which is of no use to me. So, I turned it off by changing the mount options for the partition.

Now, it was time to look into RubyMine in particular. I was using 64-bit version of it and it is well known that the 64-bit version has deteriorated performance initially because the default heap size is set at the same value as the 32-bit version. I had already taken care of it by changing the heap size from 512 mb to 1536 mb. To do this, you can edit bin/rubymine64.vmoptions and edit it to reflect the following.


Another important property that I had not set was the ReservedCodeCacheSize. It was also having the same value as the 32-bit version, i.e. 64mb. Changing it as follows, improved performance.


This ensured a good deal of code could be cached in memory making RubyMine's operations faster.

Friday, 21 February 2014

Private class methods in Ruby

In Ruby instance methods can be marked as private by declaring them below the private section. However, the same does not work for class methods.

class SomeClass

  def method1
    # do something


  def method2
    # do something els


The above works fine. Instances (objects) of the class can not call method2 directly.

class SomeOtherClass

  def self.method1
    # do something


  def self.method2
    # do something else


In this case however, one can easily call method2 as follows.


To mark class methods as private,  we will need to use private_class_method as shown below.

class SomeOtherClass

  def self.method1
    # do something

  private_class_method :method2 :method3

  def self.method2
    # do something else

  def self.method3
    # do some other things here


Tuesday, 18 February 2014

Check if a MySql server is read-only

Sometimes, when you are not sure whether a MySQL server you connected to, is read-only or not and a write query is too risky, it is easy to find out the read-only option setting using the following query.

SELECT @@global.read_only;

Sunday, 19 January 2014

Combining multiple PDF documents into one

A number of times I have faced the requirement of combining multiple PDF files into a single file. Doing that is fairly simple actually. It can be done using a number of tools. Two ways of doing it using tools very common on linux systems are as follows.
A lot many systems have imagemagick installed.  Using it is so simple that I use it as my default tool for this purpose.

convert file1.pdf /path/to/file2.pdf /destination/path/store.pdf

It takes a lot of options. For PDFs containing images, it is better to specify a quality parameter.

convert -quality 100 mine1.pdf mine2.pdf merged.pdf
Ghostscript is also very commonly found package in linux systems. Its usage is a little more obscure. However, it is very fast. I use it when I need to get PDF files of reduced size.

gs -sDEVICE=pdfwrite -dCompatibilityLevel=1.4 -dPDFSETTINGS=/screen -dNOPAUSE -dQUIET -dBATCH -sOutputFile=output.pdf input1.pdf input2.pdf

The above however reduces image quality badly. To get decent image quality with a little larger PDFs, we can use the following.

gs -sDEVICE=pdfwrite -dCompatibilityLevel=1.4 -dNOPAUSE -dQUIET -dBATCH -sOutputFile=output.pdf input1.pdf input2.pdf

gs -dBATCH -dNOPAUSE -q -sDEVICE=pdfwrite -dPDFSETTINGS=/prepress -sOutputFile=output.pdf input1.pdf input2.pdf

Tuesday, 7 January 2014

Disable compression in Net::HTTP

Recently while upgrading to Ruby 2, I found a JSON API call was failing intermittently. The code was failing when I was trying to parse the response using JSON.parse. Investigating further, I found that the call was successful; but for some calls the response was different than the response for others. Looking at the response, I found it was not JSON at all. It looked like the following.


I had no clue as to what to make out of such a response. Reading up on it, I found out that the above might be a compressed response. It appeared that the server had the liberty of choosing whether to compress the response or not and whenever the response was uncompressed, my code was working fine. Looking into improvements introduced in Ruby 2, I found that Net::HTTP now automatically requests gzip and deflate compression by default. All I needed to do was to stop doing that and ask for uncompressed response. As my response was rather small, it would hardly matter. To request uncompressed response, I just needed to add the following header to my requests.

'Accept-Encoding' => 'identity'

Sunday, 5 January 2014

Counting words on the command line

Recently, I have been writing a number of essays with various word limits. So, I have been finding myself in need of doing frequent word counts. Getting a word count for a file is easy using the wc utility.

wc -w /path/to/file

However, sometimes I just write a piece in a browser text box, which don't have word count. So, I want to get a word count from the console rather than saving it to a file and working with it. It turned out to be quite easy actually. I just had to cat the entire text and pipe that to wc.

cat << EOF | wc -w
>Your text here
>More here.

Saturday, 21 December 2013

Spellcheck in Emacs

Recently I am using Emacs for writing a few essays. While writing, I started missing spellcheck feature of Microsoft Word. The fact that feature is also provided my many browser based tools, like in GMail has made me used to it. So, I did a quick lookup to find the shortcut for triggering spellcheck in the current Emacs buffer. The answer was to simply use M-x ispell. Trying that I got the following error in the Messages buffer below the main buffer.

No word lists can be found for the language "en_US"

The shortcut mentioned above uses aspell which I knew is installed on my system. So, looking for aspell errors, I figured I needed to install the package aspell-en to get it fixed. On Arch linux, it can be done using the following.

pacman -S aspell-en

Sunday, 15 December 2013

Gem installation fails due to packaged gems

Rake 0.9.6 comes bundled with ruby 2. However, I needed to build ruby-debug-ide 0.4.17.beta16 against rake 0.8.7. I tried uninstalling rake 0.9.6 but there is no way to do it. Using the following command does not list 0.9.6 as an option.

gem uninstall rake

So, now I had to find a way to tell the gem utility to use rake 0.8.7 instead of 0.9.6. A little reading showed that setting the RAKE environment variable could do this.

RAKE=/usr/lib/ruby/gems/2.0.0/gems/rake-0.8.7/bin/rake gem install ruby-debug-ide -v 0.4.17.beta16 --pre

The above  did not work for some reason. After that I tried the following and it worked.

RAKE=`bundle exec rake` gem install ruby-debug-ide -v 0.4.17.beta16 --pre